1 About

1.1 Report circulation:

Public - this report is intended to accompany the data release.

1.2 License

This work is made available under the Creative Commons Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) License.

This means you are free to:

Share — copy and redistribute the material in any medium or format
Adapt — remix, transform, and build upon the material for any purpose, even commercially.

Under the following terms:

Attribution — You must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use.
ShareAlike — If you remix, transform, or build upon the material, you must distribute your contributions under the same license as the original.
No additional restrictions — You may not apply legal terms or technological measures that legally restrict others from doing anything the license permits.

Notices:

You do not have to comply with the license for elements of the material in the public domain or where your use is permitted by an applicable exception or limitation.
No warranties are given. The license may not give you all of the permissions necessary for your intended use. For example, other rights such as publicity, privacy, or moral rights may limit how you use the material. #YMMV

For the avoidance of doubt and explanation of terms please refer to the full license notice and legal code.

1.3 Citation

If you wish to use any of the material from this report please cite as:

Anderson, B. (2019) GREENGrid Household Electricity Demand Data Test: Imputed total power demand using: circuitsToSum_v1.1. Centre for Sustainability, University of Otago: Dunedin.

1.4 History

Report history

1.5 Requirements:

This report uses the safe version of the grid spy 1 minute data which has been processed using the code in https://github.com/CfSOtago/GREENGridData/tree/master/dataProcessing/gridSpy.

1.6 Support

This work was supported by:

The University of Otago;
The University of Southampton;
The New Zealand Ministry of Business, Innovation and Employment (MBIE) through the NZ GREEN Grid grant (Contract ID: UOCX1203);
SPATIALEC - a Marie Skłodowska-Curie Global Fellowship based at the University of Otago’s Centre for Sustainability (2017-2019) & the University of Southampton’s Sustainable Energy Research Group (2019-202).

2 Introduction

The NZ GREEN Grid household electricity demand study recruited a sample of c 25 households in each of two regions of New Zealand (Stephenson et al. 2017). The first sample was recruited in early 2014 and the second in early 2015. Research data includes:

1 minute electricity power (W) data was collected for each dwelling circuit using GridSpy monitors on each power circuit (and the incoming power). The power values represent mean(W) over the minute preceeding the observation timestamp;
Dwelling & appliance surveys;
Occupant time-use diaries (focused on energy use).

NB: Version 1 of the data package does not include the time-use diaries.

This report provides summary analysis of the imputed total demand for each household. The imputation was done using:

https://github.com/CfSOtago/GREENGridData/blob/master/dataProcessing/gridSpy/imputeTotalPower.R
circuitsToSum_v1.1

3 Load data

## Date in ISO8601 format; converting timezone from UTC to "Pacific/Auckland".

Table 3.1: First few rows of grid spy data
hhID	linkID	dateTime_orig	TZ_orig	r_dateTime	circuit	powerW	r_dateTime_nz
rf_01	rf_01	2014-01-06 16:03	date NZ	2014-01-06T03:03:00Z	imputedTotalDemand_circuitsToSum_v1.1	474.17	2014-01-06 16:03:00
rf_01	rf_01	2014-01-06 16:04	date NZ	2014-01-06T03:04:00Z	imputedTotalDemand_circuitsToSum_v1.1	474.17	2014-01-06 16:04:00
rf_01	rf_01	2014-03-07 15:56	date NZ	2014-03-07T02:56:00Z	imputedTotalDemand_circuitsToSum_v1.1	477.02	2014-03-07 15:56:00
rf_01	rf_01	2014-03-07 15:57	date NZ	2014-03-07T02:57:00Z	imputedTotalDemand_circuitsToSum_v1.1	482.72	2014-03-07 15:57:00
rf_01	rf_01	2014-03-07 15:58	date NZ	2014-03-07T02:58:00Z	imputedTotalDemand_circuitsToSum_v1.1	567.70	2014-03-07 15:58:00
rf_01	rf_01	2014-03-07 15:59	date NZ	2014-03-07T02:59:00Z	imputedTotalDemand_circuitsToSum_v1.1	570.54	2014-03-07 15:59:00

Table 3.1 shows the first few rows of the Grid Spy 1 minute power data. Table 3.2 shows a summary of the Grid Spy 1 minute power data (for more detail see Section 7.1).

Table 3.2: Summary of grid spy data
hhID	linkID	dateTime_orig	TZ_orig	r_dateTime	circuit	powerW	r_dateTime_nz	obsTimeHMS	obsTimeHalfHour
Length:53528631	Length:53528631	Length:53528631	Length:53528631	Length:53528631	Length:53528631	Min. :-12407.5	Min. :2014-01-06 16:03:00	Length:53528631	Length:53528631
Class :character	Class :character	Class :character	Class :character	Class :character	Class :character	1st Qu.: 209.3	1st Qu.:2015-06-12 21:04:00	Class1:hms	Class1:hms
Mode :character	Mode :character	Mode :character	Mode :character	Mode :character	Mode :character	Median : 417.1	Median :2016-03-18 19:23:00	Class2:difftime	Class2:difftime
NA	NA	NA	NA	NA	NA	Mean : 873.0	Mean :2016-05-03 08:31:09	Mode :numeric	Mode :numeric
NA	NA	NA	NA	NA	NA	3rd Qu.: 1139.0	3rd Qu.:2017-03-17 18:38:00	NA	NA
NA	NA	NA	NA	NA	NA	Max. : 15593.2	Max. :2018-08-01 23:59:00	NA	NA

Note that:

r_dateTime_orig is the correct dateTime of each observation in original recorded time
r_dateTime is stored as UTC and so you may get strange results until you use lubridate to tell R to use tz = “Pacific/Auckland” with this variable;
there can be 0 or negative Wh observations.

Table 3.3 shows the summaries for each household as imputed using circuitsToSum_v1.1. Note that the presence of a PV inverter may only be known from the appliance summary and/or the household data.

The number of households excluded will depend on the circuits file used:

v1.0:
- no PV
- no -ve values (?)
v1.1:
- no exclusions

Table 3.3: Summary of monitored households (NA indicates data not included in circuitsToSum_v1.1)
linkID	hasHhData	hasApplianceSummary	pvInverter	energyStorage	notes	nObs	minW	maxW	Circuit 1	Circuit 2	Circuit 3
rf_01	No	No	Not known	Not known		851716	0.00	13560.50	mains$1634
rf_02	No	No	Not known	Not known		581156	-330.38	10204.00	Mains$1575
rf_06	Yes	No	Not known	Not known		1883074	-1436.21	10804.38	Hot Water - Controlled$2248	Incomer - Uncontrolled$2249
rf_07	Yes	No	Not known	Not known		2010201	-263.00	9508.46	Incomer 1 - Uncontrolled$2726	Incomer 2 - Uncontrolled$2725
rf_08	Yes	No	Not known	Not known		1548193	35.14	11973.94	Incomer - Uncontrolled$2093	Hot Water - Controlled$2094
rf_09	Yes	No	Not known	Not known		527883	-16.00	8586.00	Incomer 1 - Uncont - Inc Hob$2729	Incomer 2 - Uncont - Inc Oven$2730
rf_10	Yes	Yes	No	No		1822053	27.68	9860.26	Incomer - All$2599
rf_11	Yes	Yes	No	No		2127141	14.00	11855.49	Incomer - Uncontrolled$2585	Hot Water Cpbd Heater- Cont$2586	Spa - Uncontrolled$2587
rf_12	Yes	No	Not known	Not known		398134	-406.00	10439.03	Incomer 1 - Hot Water - Cont$2626	Incomer 2 - Uncontrolled$2625	Incomer 3 - Uncontrolled$2627
rf_13	Yes	Yes	No	No		2179995	92.46	11580.60	Incomer - Uncontrolled$2209
rf_14	Yes	Yes	No	No		1783337	-1758.08	9894.88	Incomer 1 - Uncont inc Stove$2718	Incomer 2 - Uncont inc Oven$2717	Hot Water - Controlled$2719
rf_15a	Yes	No	Not known	Not known	Disconnected 15/01/2015. Re-used	NA	NA	NA	NA	NA	NA
rf_15b	Yes	No	Not known	Not known	Re-user. Then disconnected 02/04/2016	390846	-1062.00	10536.10	Incomer 1$3956	Incomer 2$3955
rf_16	Yes	No	Not known	Not known		372306	-262.00	8691.89	Incomer 1 - Uncont inc Oven$2681	Incomer 2 - Uncont inc Stove$2680	Hot Water - Controlled$2679
rf_17a	Yes	No	Not known	Not known	Unusual & specialist energy tech configuration. Disconnected 28/03/2016. Re-used.	959929	-1084.72	9041.53	Incomer 1 - Uncont - inc Hob$2152	Incomer 2 - Uncont - inc Oven$2151	Hot Water - Controlled$2150
rf_17b	Yes	No	Not known	Not known	Re-user	272150	32.17	6495.37	Incomer 1 - inc Top Oven$5620	Incomer 2 - inc Bottom Oven$5621
rf_18	Yes	No	Not known	Not known		540446	-1842.66	13079.07	Incomer 1 - Uncontrolled$2128	Incomer 2 - Uncontrolled$2130	Hot Water - Controlled$2129
rf_19	Yes	Yes	yes	No		2109122	-12407.52	7363.92	Incomer 1 - All$2738	Incomer 2 - All$2737
rf_20	Yes	No	Not known	Not known		543382	-1273.19	9739.43	Incomer 1 - Uncontrolled$2112	Incomer 2 - Uncontrolled$2111	Hot Water - Controlled$2110
rf_21	Yes	Yes	No	No		1010258	31.00	7990.10	Incomer - All$2748
rf_22	Yes	Yes	No	No		1885333	47.92	11535.40	Incomer - Uncontrolled$2237	Hot Water - Controlled$2236
rf_23	Yes	Yes	yes	yes		2181070	-1871.83	11015.80	Incomer - Uncontrolled$2082	Hot Water - Controlled (HEMS)$2081
rf_24	Yes	Yes	yes	No		2107577	-7542.61	9929.11	Incomer - Uncontrolled$2101	Hot Water - Controlled$2102
rf_25	Yes	Yes	No	No		706148	0.00	11814.74	Incomer 1 - Uncontrolled $2763	Incomer 2 - Uncontrolled $2762
rf_26	Yes	Yes	No	No		1938549	-482.36	9366.50	Incomer 1 - All$2703	Incomer 2 - All$2704
rf_27	Yes	Yes	No	No		908658	58.00	9569.40	Incomer - Uncontrolled$2824	Hot Water - Controlled$2825
rf_28	Yes	Yes	yes	No		86311	-3366.52	7960.34	Incomer - All$4221
rf_29	Yes	Yes	No	No		1749695	76.85	12011.87	Incomer - Uncontrolled$4181	Hot Water - Controlled$4184
rf_30	Yes	Yes	No	No		743770	30.00	8706.07	Incomer - All$4239
rf_31	Yes	Yes	No	No		1760227	24.00	10547.60	Incomer - All$4199
rf_32	Yes	Yes	No	No		541109	0.00	8274.91	Incomer - All$4193
rf_33	Yes	Yes	No	No		1601314	63.47	9654.25	Incomer - Uncontrolled$4143	Hot Water - Controlled$4144
rf_34	Yes	Yes	No	No		730613	0.00	13043.50	Incomer - All$4225
rf_35	Yes	Yes	No	No		781977	-622.00	10373.54	Incomer - Uncontrolled$4126	Hot Water - Uncontrolled$4125
rf_36	Yes	Yes	No	No		1605647	0.00	14826.00	Incomer - All$4148
rf_37	Yes	Yes	No	No		1763330	51.00	9365.54	Incomer -Uncontrolled$4136	Hot Water - Controlled$4135
rf_38	Yes	Yes	No	No	very large number of circuits	886482	-179.00	9455.19	Incomer - Uncontrolled$4177	Hot Water - Controlled$4178
rf_39	Yes	Yes	No	No		1537301	-199.00	15593.20	Incomer - Uncontrolled$4248	Hot Water (2 elements)$4247
rf_40	Yes	Yes	No	No		348217	24.00	12527.77	Incomer - Uncontrolled$4168	Hot Water - Controlled$4167
rf_41	Yes	Yes	No	No		1372961	54.83	12857.00	Incomer - All$4192
rf_42	Yes	Yes	No	No		975198	33.61	12383.70	Incomer - All$4132
rf_43	Yes	No	Not known	Not known		296190	0.00	6687.53	Incomer - All$4213
rf_44	Yes	Yes	No	No		1762049	32.00	12165.12	Incomer - Uncontrolled$4156	Hot Water - Controlled$4155
rf_45	Yes	Yes	No	No		819601	23.90	10385.13	Incomer - Uncontrolled$4157	Hot Water - Controlled$4158
rf_46	Yes	Yes	No	No		1939634	92.00	12979.28	Incomer - Uncontrolled$4230	Hot Water - Controlled$4231
rf_47	Yes	Yes	No	No		588348	21.00	11171.30	Incomer - All$4170

4 Test 0 and -ve power value incidence

Figure 4.1 shows the incidence of -ve overall imputed power values by linkID (household) and date. As we can see they tend to be concentrated in a few households.

## Warning: Removed 21 rows containing missing values (geom_tile).

Figure 4.1: Incidence of -ve values by household ID, date and presence of PV (if known)

Table 4.1: Number of observations of 0 or -ve total W by presence of PV
pvInverter	nObs	nHouseholds
No	135919	27
Not known	189814	15
yes	1602350	4

Table 4.1 (-ve and 0 observations by PV inverter presence) shows whether or not this is likely to be a PV related problem in the dataset defined by . This is confirmed by Table @(tab:testPV) which shows the summary data from Table 4.1 but for just those we know to have PV inverters. If this version of the circuits file has excluded them then the power data will be all NA.

Table 4.2: Summary of monitored households (NA indicates data not included in circuitsToSum_v1.1)
linkID	pvInverter	energyStorage	nObs	minW	maxW	Circuit 1	Circuit 2
rf_19	yes	No	2109122	-12407.52	7363.92	Incomer 1 - All$2738	Incomer 2 - All$2737
rf_23	yes	yes	2181070	-1871.83	11015.80	Incomer - Uncontrolled$2082	Hot Water - Controlled (HEMS)$2081
rf_24	yes	No	2107577	-7542.61	9929.11	Incomer - Uncontrolled$2101	Hot Water - Controlled$2102
rf_28	yes	No	86311	-3366.52	7960.34	Incomer - All$4221

Figure 4.2 shows the mean overall imputed power values by linkID (household) and date where power <= 0. As we can see they tend to be concentrated in a few households with some reporting persistent -ve values even though they do not appear to have PV.

## Warning: Removed 21 rows containing missing values (geom_tile).

Figure 4.2: Mean of -ve values by household ID and date and presence of PV (if known)

5 Test power profiles

Next we test the power profiles by season to see if there is a seasonal pattern to the negatuive values which might indicate unreported PV (for example).

First we create a Southern Hemisphere season variable. We have a function to do this in the GREENGridData package. We print a check table to ensure we are all happy with the coding of season.

powerDT <- GREENGridData::addNZSeason(powerDT)
table(lubridate::month(powerDT$r_dateTime, label = TRUE), powerDT$season, useNA = "always")

##       
##         Spring  Summer  Autumn  Winter    <NA>
##   Jan        0 4126543       0       0       0
##   Feb        0 3854551       0       0       0
##   Mar        0       0 4374364       0       0
##   Apr        0       0 4687004       0       0
##   May        0       0 4693791       0       0
##   Jun        0       0       0 4863269       0
##   Jul        0       0       0 5178801       0
##   Aug        0       0       0 4721953       0
##   Sep  4420524       0       0       0       0
##   Oct  4486492       0       0       0       0
##   Nov  4106835       0       0       0       0
##   Dec        0 4014504       0       0       0
##   <NA>       0       0       0       0       0

Figure 5.1 plots overall mean power per minute by season and household id.

Figure 5.1: Mean total demand profile plot

If a household has PV this should be clearly visible as a -ve curve centered on ~ 12:00 - 13:00.

Figure 5.2: Minimum total demand profile plot

Figure 5.2 repeats this analysis but for minimum power per minute by season and household id and plots only those values that are <= 0 to clarify time of day effects on -ve values.

6 Implications

When using the ‘total demand’ derived data we suggest data users:

v1.0:
- excluding household (link) IDs: 25,26,43;
- excluding other -ve values on a per-value basis;
v1.1:
- excluding household (link) IDs: 14,25,26,43;
- excluding other -ve values on a per-value basis;
- only including households known to have PV if analysis of demand during non-daylight hours is being undertaken (c.f. Figures 5.1 and 5.2).

Seperately, for reasons explained elsewhere we recommend removing rf_46 from the data prior to analysis.

In all cases we recommend that users check the data carefully before analysis and document any filtering they apply!

7 Statistical Annex

7.1 Power data

## Skim summary statistics
##  n obs: 53528631 
##  n variables: 11 
## 
## ── Variable type:character ────────────────────────────────────────────────────────────────────────────────────────
##       variable missing complete        n min max empty n_unique
##        circuit       0 53528631 53528631  37  37     0        1
##  dateTime_orig       0 53528631 53528631  12  16     0  4138777
##           hhID       0 53528631 53528631   5   5     0       44
##         linkID       0 53528631 53528631   5   6     0       45
##     r_dateTime       0 53528631 53528631  20  20     0  2320295
##        TZ_orig       0 53528631 53528631   7   8     0        2
## 
## ── Variable type:difftime ─────────────────────────────────────────────────────────────────────────────────────────
##         variable missing complete        n    min        max   median
##  obsTimeHalfHour       0 53528631 53528631 0 secs 84600 secs 12:00:00
##       obsTimeHMS       0 53528631 53528631 0 secs 86340 secs 12:00:00
##  n_unique
##        48
##      1440
## 
## ── Variable type:factor ───────────────────────────────────────────────────────────────────────────────────────────
##  variable missing complete        n n_unique
##    season       0 53528631 53528631        4
##                                                  top_counts ordered
##  Win: 14764023, Aut: 13755159, Spr: 13013851, Sum: 11995598   FALSE
## 
## ── Variable type:numeric ──────────────────────────────────────────────────────────────────────────────────────────
##  variable missing complete        n   mean      sd        p0    p25    p50
##    powerW       0 53528631 53528631 872.99 1315.62 -12407.52 209.33 417.14
##      p75    p100     hist
##  1138.97 15593.2 ▁▁▁▇▂▁▁▁
## 
## ── Variable type:POSIXct ──────────────────────────────────────────────────────────────────────────────────────────
##       variable missing complete        n        min        max     median
##  r_dateTime_nz       0 53528631 53528631 2014-01-06 2018-08-01 2016-03-18
##  n_unique
##   2320295

7.2 Household data

## Skim summary statistics
##  n obs: 44 
##  n variables: 110 
## 
## ── Variable type:character ────────────────────────────────────────────────────────────────────────────────────────
##             variable missing complete  n min max empty n_unique
##        Clothes Dryer      23       21 44   3   3     0        1
##           Dishwasher      15       29 44   3   3     0        1
##      Electric heater      30       14 44   3   3     0        1
##       Energy Storage      43        1 44   3   3     0        1
##   Fridge / Freezer 1      13       31 44   3   3     0        1
##   Fridge / Freezer 2      24       20 44   3   3     0        1
##   Fridge / Freezer 3      38        6 44   3   3     0        1
##  hasApplianceSummary      13       31 44   3   3     0        1
##        hasLongSurvey      15       29 44   3   3     0        1
##       hasShortSurvey      31       13 44   3   3     0        1
##   Heated towel rails      23       21 44   3   3     0        1
##                 hhID       0       44 44   5   5     0       42
##   Hot water cylinder      16       28 44   3   3     0        1
##               linkID       0       44 44   5   6     0       44
##             Location       0       44 44  10  12     0        2
##            Microwave      14       30 44   3   3     0        1
##                notes      39        5 44   7  81     0        5
##      Other Appliance      30       14 44   3  42     0        8
##                 Oven      14       30 44   3   3     0        1
##          PV Inverter      40        4 44   3   3     0        1
##           r_stopDate      41        3 44  10  10     0        3
##            StartDate       2       42 44  16  16     0       42
##      surveyStartDate       2       42 44  20  20     0       42
##      Washing Machine      13       31 44   3   3     0        1
## 
## ── Variable type:integer ──────────────────────────────────────────────────────────────────────────────────────────
##          variable missing complete  n    mean     sd p0    p25   p50
##  Heat pump number      19       25 44   1.16    0.55  1   1      1  
##           nAdults       1       43 44   1.93    0.51  1   2      2  
##     nChildren0_12       2       42 44   1.02    1.02  0   0      1  
##   nTeenagers13_18       2       42 44   0.26    0.54  0   0      0  
##    Q10#1_1_1_TEXT      15       29 44   3.38    0.9   2   3      3  
##    Q10#1_1_2_TEXT      16       28 44   1.57    1.4   0   0.75   1.5
##    Q10#1_2_1_TEXT      15       29 44   1.38    0.49  1   1      1  
##    Q10#1_2_2_TEXT      16       28 44   1.32    0.67  1   1      1  
##    Q10#1_3_1_TEXT      15       29 44   1       0     1   1      1  
##    Q10#1_3_2_TEXT      18       26 44   0.92    0.27  0   1      1  
##    Q10#1_4_1_TEXT      15       29 44   0.14    0.35  0   0      0  
##    Q10#1_4_2_TEXT      26       18 44   0.11    0.32  0   0      0  
##    Q10#1_5_1_TEXT      15       29 44   0.34    0.48  0   0      0  
##    Q10#1_5_2_TEXT      24       20 44   0       0     0   0      0  
##    Q10#1_6_1_TEXT      15       29 44   1.62    0.62  1   1      2  
##    Q10#1_6_2_TEXT      23       21 44   0.43    0.87  0   0      0  
##    Q10#1_7_1_TEXT      15       29 44   0.69    0.54  0   0      1  
##    Q10#1_7_2_TEXT      26       18 44   0.056   0.24  0   0      0  
##    Q10#1_8_1_TEXT      15       29 44   0.83    0.38  0   1      1  
##    Q10#1_8_2_TEXT      25       19 44   0.053   0.23  0   0      0  
##             Q11_1      15       29 44   7.76    1.83  4   7      8  
##             Q15_1      15       29 44  14.03    9.76  3   7     10  
##             Q17_1      28       16 44  20.31    1.58 18  19     20  
##             Q18_1      15       29 44  20.69    1.58 18  20     20  
##             Q19_1      28       16 44   1       0     1   1      1  
##            Q19_12      37        7 44   1       0     1   1      1  
##            Q19_13      42        2 44   1       0     1   1      1  
##            Q19_14      43        1 44   1      NA     1   1      1  
##            Q19_15      42        2 44   1       0     1   1      1  
##            Q19_16      40        4 44   1       0     1   1      1  
##             Q19_3      31       13 44   1       0     1   1      1  
##             Q19_4      37        7 44   1       0     1   1      1  
##             Q19_6      29       15 44   1       0     1   1      1  
##             Q19_7      40        4 44   1       0     1   1      1  
##             Q19_8      43        1 44   1      NA     1   1      1  
##             Q30_1      36        8 44  19       2.07 16  18     18  
##             Q33_1      28       16 44  57.06    4.57 50  54     56.5
##                Q4       2       42 44  14.45    3.83  1  15     15  
##             Q40_1      15       29 44   1.31    0.54  1   1      1  
##            Q40_10      15       29 44   2.55    0.74  1   2      3  
##            Q40_11      15       29 44   2       0.93  1   1      2  
##            Q40_12      15       29 44   1.97    1.02  1   1      1  
##            Q40_13      15       29 44   2.86    0.52  1   3      3  
##            Q40_14      15       29 44   2.28    0.84  1   2      3  
##            Q40_15      15       29 44   2.1     0.9   1   1      2  
##            Q40_16      15       29 44   1       0     1   1      1  
##            Q40_17      15       29 44   1.76    0.99  1   1      1  
##            Q40_18      15       29 44   2.24    0.99  1   1      3  
##            Q40_19      15       29 44   3       0     3   3      3  
##             Q40_2      15       29 44   2.79    0.62  1   3      3  
##            Q40_20      15       29 44   2.93    0.37  1   3      3  
##            Q40_21      15       29 44   1.93    0.96  1   1      2  
##             Q40_3      15       29 44   1.93    1     1   1      1  
##            Q40_38      15       29 44   1.24    0.64  1   1      1  
##             Q40_4      15       29 44   3       0     3   3      3  
##             Q40_5      15       29 44   2.31    0.85  1   2      3  
##             Q40_6      15       29 44   1.9     1.01  1   1      1  
##             Q40_7      15       29 44   2.55    0.83  1   3      3  
##             Q40_9      15       29 44   2.14    0.95  1   1      3  
##                Q5      15       29 44   9.66    0.77  8  10     10  
##             Q53_1       2       42 44 157.24   78.38 50 100.5  140  
##             Q53_2      25       19 44  31.11   23.06  0   9.5   40  
##             Q53_3      37        7 44   0       0     0   0      0  
##             Q53_4      34       10 44   0       0     0   0      0  
##             Q53_5      28       16 44 224.38  178.13  0  75    226  
##             Q53_6      36        8 44  16.62   47.02  0   0      0  
##             Q53_7      36        8 44   0       0     0   0      0  
##             Q54_1       2       42 44 213.62  100.9  60 146.25 200  
##             Q54_2      24       20 44  56.55   51.54  0  19.75  50  
##             Q54_3      37        7 44   0       0     0   0      0  
##             Q54_4      28       16 44  87.38  115.96  0  18.75  56.5
##             Q54_5      29       15 44 241.8   172.44  0 115    252  
##             Q54_6      35        9 44  15.56   46.67  0   0      0  
##             Q54_7      37        7 44   0       0     0   0      0  
##               Q55       2       42 44  12.88    4.55  4  10     12.5
##               Q57       2       42 44   3.17    1.34  1   2      3  
##           Q58#2_1       2       42 44  42.07    9.29 28  34.25  41.5
##                Q7       2       42 44   2       1.1   1   1      2  
##     p75 p100     hist
##    1       3 ▇▁▁▁▁▁▁▁
##    2       3 ▂▁▁▇▁▁▁▁
##    2       3 ▇▁▅▁▁▅▁▂
##    0       2 ▇▁▁▂▁▁▁▁
##    4       6 ▂▇▁▆▁▁▁▁
##    2       6 ▆▆▇▂▁▁▁▁
##    2       2 ▇▁▁▁▁▁▁▅
##    1.25    4 ▇▁▂▁▁▁▁▁
##    1       1 ▁▁▁▇▁▁▁▁
##    1       1 ▁▁▁▁▁▁▁▇
##    0       1 ▇▁▁▁▁▁▁▁
##    0       1 ▇▁▁▁▁▁▁▁
##    1       1 ▇▁▁▁▁▁▁▅
##    0       0 ▁▁▁▇▁▁▁▁
##    2       3 ▇▁▁▇▁▁▁▁
##    0       3 ▇▁▁▁▁▁▁▁
##    1       2 ▅▁▁▇▁▁▁▁
##    0       1 ▇▁▁▁▁▁▁▁
##    1       1 ▂▁▁▁▁▁▁▇
##    0       1 ▇▁▁▁▁▁▁▁
##    9      10 ▂▁▂▅▁▅▇▃
##   20      44 ▇▇▃▃▁▂▁▁
##   21      24 ▃▆▇▇▁▃▁▂
##   22      23 ▂▂▁▇▃▁▅▅
##    1       1 ▁▁▁▇▁▁▁▁
##    1       1 ▁▁▁▇▁▁▁▁
##    1       1 ▁▁▁▇▁▁▁▁
##    1       1 ▁▁▁▇▁▁▁▁
##    1       1 ▁▁▁▇▁▁▁▁
##    1       1 ▁▁▁▇▁▁▁▁
##    1       1 ▁▁▁▇▁▁▁▁
##    1       1 ▁▁▁▇▁▁▁▁
##    1       1 ▁▁▁▇▁▁▁▁
##    1       1 ▁▁▁▇▁▁▁▁
##    1       1 ▁▁▁▇▁▁▁▁
##   21      22 ▂▁▇▁▁▁▃▂
##   60      65 ▃▂▇▃▂▇▁▃
##   16      17 ▁▁▁▁▁▁▇▆
##    2       3 ▇▁▁▂▁▁▁▁
##    3       3 ▂▁▁▂▁▁▁▇
##    3       3 ▇▁▁▃▁▁▁▇
##    3       3 ▇▁▁▁▁▁▁▇
##    3       3 ▁▁▁▁▁▁▁▇
##    3       3 ▃▁▁▃▁▁▁▇
##    3       3 ▆▁▁▃▁▁▁▇
##    1       1 ▁▁▁▇▁▁▁▁
##    3       3 ▇▁▁▁▁▁▁▅
##    3       3 ▅▁▁▁▁▁▁▇
##    3       3 ▁▁▁▇▁▁▁▁
##    3       3 ▁▁▁▁▁▁▁▇
##    3       3 ▁▁▁▁▁▁▁▇
##    3       3 ▇▁▁▂▁▁▁▇
##    3       3 ▇▁▁▁▁▁▁▇
##    1       3 ▇▁▁▁▁▁▁▁
##    3       3 ▁▁▁▇▁▁▁▁
##    3       3 ▃▁▁▃▁▁▁▇
##    3       3 ▇▁▁▁▁▁▁▆
##    3       3 ▂▁▁▁▁▁▁▇
##    3       3 ▆▁▁▂▁▁▁▇
##   10      10 ▂▁▁▁▁▁▁▇
##  197.5   429 ▆▇▇▃▂▁▁▁
##   50      60 ▇▁▅▂▁▃▇▅
##    0       0 ▁▁▁▇▁▁▁▁
##    0       0 ▁▁▁▇▁▁▁▁
##  352.5   500 ▇▃▂▂▆▂▁▇
##    0     133 ▇▁▁▁▁▁▁▁
##    0       0 ▁▁▁▇▁▁▁▁
##  258.75  485 ▃▇▇▆▃▂▁▂
##   80.75  180 ▇▂▇▃▂▁▁▂
##    0       0 ▁▁▁▇▁▁▁▁
##  100.75  451 ▇▆▁▁▁▁▁▁
##  385     500 ▆▃▂▂▆▂▁▇
##    0     140 ▇▁▁▁▁▁▁▁
##    0       0 ▁▁▁▇▁▁▁▁
##   16      26 ▂▃▇▅▃▅▁▁
##    4       6 ▃▇▁▇▇▁▃▂
##   49      64 ▆▇▃▅▇▂▁▂
##    3       4 ▇▁▅▁▁▃▁▂
## 
## ── Variable type:logical ──────────────────────────────────────────────────────────────────────────────────────────
##                 variable missing complete  n mean count
##  Other Generation Device      44        0 44  NaN    44
##                   Q19_10      44        0 44  NaN    44
##                 Q19_10.1      44        0 44  NaN    44
##                   Q19_17      44        0 44  NaN    44
##                    Q19_2      44        0 44  NaN    44
##                    Q19_5      44        0 44  NaN    44
##                    Q19_9      44        0 44  NaN    44
## 
## ── Variable type:numeric ──────────────────────────────────────────────────────────────────────────────────────────
##  variable missing complete  n mean   sd p0 p25 p50 p75 p100     hist
##     Q14_1      15       29 44 5.28 5.03  0   1 3.5   7   20 ▇▆▅▂▁▁▁▁

8 Runtime

Analysis completed in 942.64 seconds ( 15.71 minutes) using knitr in RStudio with R version 3.5.2 (2018-12-20) running on x86_64-apple-darwin15.6.0.

9 R environment

9.1 R packages used

base R (R Core Team 2016)
bookdown (Xie 2016a)
ggplot2 (Wickham 2009)
here (Müller 2017)
GREENGridData (Anderson and Eyers 2018) which depends on:
- data.table (Dowle et al. 2015)
- dplyr (Wickham and Francois 2016)
- hms (Müller 2018)
- lubridate (Grolemund and Wickham 2011)
- progress (Csárdi and FitzJohn 2016)
- readr (Wickham, Hester, and Francois 2016)
- readxl (Wickham and Bryan 2017)
- reshape2 (Wickham 2007)
kableExtra (Zhu 2018)
knitr (Xie 2016b)
rmarkdown (Allaire et al. 2018)

9.2 Session info

## R version 3.5.2 (2018-12-20)
## Platform: x86_64-apple-darwin15.6.0 (64-bit)
## Running under: macOS High Sierra 10.13.6
## 
## Matrix products: default
## BLAS: /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib
## LAPACK: /Library/Frameworks/R.framework/Versions/3.5/Resources/lib/libRlapack.dylib
## 
## locale:
## [1] en_NZ.UTF-8/en_NZ.UTF-8/en_NZ.UTF-8/C/en_NZ.UTF-8/en_NZ.UTF-8
## 
## attached base packages:
## [1] stats     graphics  grDevices utils     datasets  methods   base     
## 
## other attached packages:
##  [1] viridis_0.5.1     viridisLite_0.3.0 skimr_1.0.5      
##  [4] kableExtra_1.1.0  ggplot2_3.1.1     dplyr_0.8.1      
##  [7] tidyselect_0.2.5  here_0.1          drake_7.2.0      
## [10] data.table_1.12.2 lubridate_1.7.4  
## 
## loaded via a namespace (and not attached):
##  [1] Rcpp_1.0.1        tidyr_0.8.3       prettyunits_1.0.2
##  [4] visNetwork_2.0.6  assertthat_0.2.1  zeallot_0.1.0    
##  [7] rprojroot_1.3-2   digest_0.6.19     packrat_0.5.0    
## [10] utf8_1.1.4        R6_2.4.0          cellranger_1.1.0 
## [13] plyr_1.8.4        backports_1.1.4   evaluate_0.13    
## [16] httr_1.4.0        highr_0.8         pillar_1.4.1     
## [19] rlang_0.3.4       progress_1.2.2    lazyeval_0.2.2   
## [22] readxl_1.3.1      rstudioapi_0.10   rmarkdown_1.13   
## [25] labeling_0.3      webshot_0.5.1     readr_1.3.1      
## [28] stringr_1.4.0     htmlwidgets_1.3   igraph_1.2.4.1   
## [31] munsell_0.5.0     compiler_3.5.2    xfun_0.7         
## [34] pkgconfig_2.0.2   htmltools_0.3.6   tibble_2.1.2     
## [37] gridExtra_2.3     bookdown_0.10     fansi_0.4.0      
## [40] crayon_1.3.4      withr_2.1.2       grid_3.5.2       
## [43] GREENGridData_1.0 jsonlite_1.6      gtable_0.3.0     
## [46] magrittr_1.5      storr_1.2.1       scales_1.0.0     
## [49] cli_1.1.0         stringi_1.4.3     reshape2_1.4.3   
## [52] xml2_1.2.0        vctrs_0.1.0       tools_3.5.2      
## [55] glue_1.3.1        purrr_0.3.2       hms_0.4.2        
## [58] yaml_2.2.0        colorspace_1.4-1  base64url_1.4    
## [61] rvest_0.3.3       knitr_1.23

References

Allaire, JJ, Yihui Xie, Jonathan McPherson, Javier Luraschi, Kevin Ushey, Aron Atkins, Hadley Wickham, Joe Cheng, and Winston Chang. 2018. Rmarkdown: Dynamic Documents for R. https://CRAN.R-project.org/package=rmarkdown.

Anderson, Ben, and David Eyers. 2018. GREENGridData: Processing Nz Green Grid Project Data to Create a ’Safe’ Version for Data Archiving and Re-Use. https://github.com/CfSOtago/GREENGridData.

Csárdi, Gábor, and Rich FitzJohn. 2016. Progress: Terminal Progress Bars. https://CRAN.R-project.org/package=progress.

Dowle, M, A Srinivasan, T Short, S Lianoglou with contributions from R Saporta, and E Antonyan. 2015. Data.table: Extension of Data.frame. https://CRAN.R-project.org/package=data.table.

Grolemund, Garrett, and Hadley Wickham. 2011. “Dates and Times Made Easy with lubridate.” Journal of Statistical Software 40 (3): 1–25. http://www.jstatsoft.org/v40/i03/.

Müller, Kirill. 2017. Here: A Simpler Way to Find Your Files. https://CRAN.R-project.org/package=here.

———. 2018. Hms: Pretty Time of Day. https://CRAN.R-project.org/package=hms.

R Core Team. 2016. R: A Language and Environment for Statistical Computing. Vienna, Austria: R Foundation for Statistical Computing. https://www.R-project.org/.

Stephenson, Janet, Rebecca Ford, Nirmal-Kumar Nair, Neville Watson, Alan Wood, and Allan Miller. 2017. “Smart Grid Research in New Zealand–A Review from the GREEN Grid Research Programme.” Renewable and Sustainable Energy Reviews 82 (1): 1636–45. https://doi.org/10.1016/j.rser.2017.07.010.

Wickham, Hadley. 2007. “Reshaping Data with the reshape Package.” Journal of Statistical Software 21 (12): 1–20. http://www.jstatsoft.org/v21/i12/.

———. 2009. Ggplot2: Elegant Graphics for Data Analysis. Springer-Verlag New York. http://ggplot2.org.

Wickham, Hadley, and Jennifer Bryan. 2017. Readxl: Read Excel Files. https://CRAN.R-project.org/package=readxl.

Wickham, Hadley, and Romain Francois. 2016. Dplyr: A Grammar of Data Manipulation. https://CRAN.R-project.org/package=dplyr.

Wickham, Hadley, Jim Hester, and Romain Francois. 2016. Readr: Read Tabular Data. https://CRAN.R-project.org/package=readr.

Xie, Yihui. 2016a. Bookdown: Authoring Books and Technical Documents with R Markdown. Boca Raton, Florida: Chapman; Hall/CRC. https://github.com/rstudio/bookdown.

———. 2016b. Knitr: A General-Purpose Package for Dynamic Report Generation in R. https://CRAN.R-project.org/package=knitr.

Zhu, Hao. 2018. KableExtra: Construct Complex Table with ’Kable’ and Pipe Syntax. https://CRAN.R-project.org/package=kableExtra.

GREENGrid Household Electricity Demand Data Test

Imputed total power demand using: circuitsToSum_v1.1

Ben Anderson (b.anderson@soton.ac.uk, `@dataknut`)

Last run at: 2019-08-16 16:46:42

1 About

1.1 Report circulation:

1.2 License

1.3 Citation

1.4 History

1.5 Requirements:

1.6 Support

2 Introduction

3 Load data

4 Test 0 and -ve power value incidence

5 Test power profiles

6 Implications

7 Statistical Annex

7.1 Power data

7.2 Household data

8 Runtime

9 R environment

9.1 R packages used

9.2 Session info

References

GREENGrid Household Electricity Demand Data Test

Imputed total power demand using: circuitsToSum_v1.1

Ben Anderson (b.anderson@soton.ac.uk, @dataknut)

Last run at: 2019-08-16 16:46:42

1 About

1.1 Report circulation:

1.2 License

1.3 Citation

1.4 History

1.5 Requirements:

1.6 Support

2 Introduction

3 Load data

4 Test 0 and -ve power value incidence

5 Test power profiles

6 Implications

7 Statistical Annex

7.1 Power data

7.2 Household data

8 Runtime

9 R environment

9.1 R packages used

9.2 Session info

References

Ben Anderson (b.anderson@soton.ac.uk, `@dataknut`)