1.1 Report circulation:

• Public – this report is intended to accompany the data release.

1.2 License

This work is made available under the Creative Commons Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) License.

This means you are free to:

• Share — copy and redistribute the material in any medium or format
• Adapt — remix, transform, and build upon the material for any purpose, even commercially.

Under the following terms:

• Attribution — You must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use.
• ShareAlike — If you remix, transform, or build upon the material, you must distribute your contributions under the same license as the original.
• No additional restrictions — You may not apply legal terms or technological measures that legally restrict others from doing anything the license permits.

Notices:

• You do not have to comply with the license for elements of the material in the public domain or where your use is permitted by an applicable exception or limitation.
• No warranties are given. The license may not give you all of the permissions necessary for your intended use. For example, other rights such as publicity, privacy, or moral rights may limit how you use the material. #YMMV

For the avoidance of doubt and explanation of terms please refer to the full license notice and legal code.

1.3 Citation

If you wish to use any of the material from this report please cite as:

• Anderson, B., Eyers, D., Ford, R., Giraldo Ocampo, D., Peniamina, R., Stephenson, J., Suomalainen, K., Wilcocks, L. and Jack, M. (2019) NZ GREEN Grid Household Electricity Demand Study: 1 minute circuit level electricity power outlier report (version 1.0) , Centre for Sustainability, University of Otago: Dunedin.

This work is (c) 2019 the authors.

1.4 History

You may not be reading the most recent version of this report. Please check:

• the overall package documentation;

• this report’s edit history

1.5 Support

This work was supported by:

• The University of Otago;
• The University of Southampton;
• The New Zealand Ministry of Business, Innovation and Employment (MBIE) through the NZ GREEN Grid grant (Contract ID: UOCX1203);
• SPATIALEC - a Marie Skłodowska-Curie Global Fellowship based at the University of Otago’s Centre for Sustainability (2017-2019) & the University of Southampton’s Sustainable Energy Research Group (2019-202).

4.1 Negative power: Heat Pump

As Figure 4.1 shows negative power observations for Heat Pumps are spread across a number of households. It also shows that there is at least one very large outlier reading for rf_27.

## Warning: The following named parsers don't match the column names:
## dateTime_orig, TZ_orig

##      hhID              linkID            r_dateTime                 
##  Length:14250284    Length:14250284    Min.   :2015-04-01 00:00:00  
##  Class :character   Class :character   1st Qu.:2015-06-22 12:39:00  
##  Mode  :character   Mode  :character   Median :2015-09-16 13:12:00  
##                                        Mean   :2015-09-21 08:00:39  
##                                        3rd Qu.:2015-12-17 17:52:00  
##                                        Max.   :2016-03-31 23:59:00  
##    circuit              powerW        
##  Length:14250284    Min.   : -655.00  
##  Class :character   1st Qu.:    0.00  
##  Mode  :character   Median :    0.00  
##                     Mean   :  147.92  
##                     3rd Qu.:   61.29  
##                     Max.   :27759.00

Figure 4.1: Boxplot of Heat Pump power values per household

As we can see from Table 4.1 there is only 1 value where power > 10kW. We recommend that this value be excluded from analysis.

Figure 4.2: Number of negative power observations per hour (max = 60 * 60) by household over time

Turning to the negative values, Figure 4.2 shows that the vast majority of negative power values are due to one household (rf_46, which does not have PV). The others appear on isolated days and are most likely to be due to instrument installation error.

Table 4.2 lists the households where any negative heat pump power values are observed while Figure 4.3 shows a density plot for each household. The latter (together with Figure 4.2) suggests that the negative values continually observed for rf_46 may constitute some form of intermittent error since positive values were also recorded (c.f. Table 4.2). The other households on the other hand report a few negative observations suggesting instrument installation error that was quickly corrected.

## `stat_bin()` using `bins = 30`. Pick better value with `binwidth`.

Figure 4.3: Histogram of negative power values only

We therefore recommend that when analysing heat pump data:

• rf_46 is fixed - see Section 4.5;
• this value is removed: rf_27 2015-08-22 10:33:00 Heat Pump$2826 27759
• any day with negative power values for any household is also removed.

4.2 Negative power: Hot Water

As Figure 4.4 shows negative power observations for Hot Water are spread across a number of households. In this case though, there are no very large positive outlier readings.

## Warning: The following named parsers don't match the column names:
## dateTime_orig, TZ_orig

##      hhID              linkID            r_dateTime                 
##  Length:14493877    Length:14493877    Min.   :2015-04-01 00:00:00  
##  Class :character   Class :character   1st Qu.:2015-06-22 16:46:00  
##  Mode  :character   Mode  :character   Median :2015-09-18 21:32:00  
##                                        Mean   :2015-09-22 19:59:55  
##                                        3rd Qu.:2015-12-20 17:19:00  
##                                        Max.   :2016-03-31 23:59:00  
##    circuit              powerW       
##  Length:14493877    Min.   :-1110.0  
##  Class :character   1st Qu.:    0.0  
##  Mode  :character   Median :    0.0  
##                     Mean   :  283.8  
##                     3rd Qu.:    0.0  
##                     Max.   : 4076.0

Figure 4.4: Boxplot of Hot Water power values per household

Figure 4.5: Number of negative power observations per hour (max = 60 * 60) by household over time

Turning to the negative values, Figure 4.5 shows that the vast majority of negative power values are again due to one household (rf_46, which does not have PV) but with the addition of a number from rf_14. The others appear on isolated days and as before are most likely to be due to instrument installation error.

Table 4.3 lists the households where any negative hot water power values are observed while Figure 4.6 shows a density plot for each household. Note that these households do not necessarily match to those reported above for heat pumps partly because not all households have heat pumps. The latter (together with Figure 4.5) suggests that the negative values continually observed for rf_46 may constitute some form of intermittent error since positive values were also recorded (c.f. Table 4.3). The other households on the other hand (including rf_14) report negative observations on specific days suggesting instrument installation error that was quickly corrected.

Figure 4.6: Histogram of negative power values only

We therefore recommend that when analysing hot water data:

• rf_46 is fixed - see Section 4.5;
• any day with negative power values is also removed (including for rf_14).

4.3 Lighting

As Figure 4.7 shows negative power observations for Lighting are found in only one household. There are also some relatively large values across at least 2 or 3 households which may indicate that the lighting circuits did not just power lights.

## Warning: The following named parsers don't match the column names:
## dateTime_orig, TZ_orig

##      hhID              linkID            r_dateTime                 
##  Length:14493877    Length:14493877    Min.   :2015-04-01 00:00:00  
##  Class :character   Class :character   1st Qu.:2015-06-22 16:46:00  
##  Mode  :character   Mode  :character   Median :2015-09-18 21:32:00  
##                                        Mean   :2015-09-22 19:59:55  
##                                        3rd Qu.:2015-12-20 17:19:00  
##                                        Max.   :2016-03-31 23:59:00  
##    circuit              powerW       
##  Length:14493877    Min.   :-1110.0  
##  Class :character   1st Qu.:    0.0  
##  Mode  :character   Median :    0.0  
##                     Mean   :  283.8  
##                     3rd Qu.:    0.0  
##                     Max.   : 4076.0

Figure 4.7: Boxplot of Lighting power values per household

This is clear when we consider the distribution of power by household ID and circuit name (Table 4.4). Analysis of ‘lighting’ for these households should therefore be done with care.

Figure 4.8: Number of negative power observations per hour (max = 60 * 60) by household over time

Turning to the negative values, Figure 4.8 shows that all negative power values are again due to one household (rf_46, which does not have PV).

Table 4.5 lists the households where any negative lighting power values are observed while Figure 4.9 shows a density plot for each household. The latter (together with Figure 4.8) again suggests that the negative values continually observed for rf_46 may constitute some form of intermittent error since positive values were also recorded (c.f. Table 4.5).

Figure 4.9: Histogram of negative power values only

We therefore recommend that when analysing lighting data:

• rf_46 is fixed - see Section 4.5

4.4 Overall patterns of negative observations

Table 4.6 shows that there is only one household where negative values are recorded on multiple non-PV circuits. This confirms our view that data for rf_46 should be treated seperately (see Section 4.5) but that other data should be removed on a day by day/household by household basis.

4.5 The mysterious case of rf_46

19th August 2019: updated - see https://github.com/CfSOtago/GREENGridData/issues/1

The analysis above has identified this household’s data as being very strange. We detected a lot of negative power values over a long time but they seem to be interspersed with positive values.

We know that rf_46 did not have PV nor anything else of note. Let’s take a look at rf_46 a bit more closely…

Figure 4.10 shows the max and min power per half-hour for each circuit. This suggests that most circuits had two monitors of which only one was recording the ‘power demand’ we are interested in. The second monitor in each pair appears to have been measuring something which was an order of magnitude smaller (see y scales) than the power demand we are interested in and which fluctuated between relatively small positive and negative values.

Figure 4.10: Max and min power in kW per hour for each circuit

This is confirmed by 4.8 which indicates which of each pair of circuit labels should be excluded from any analysis that uses this household.

19th August 2019: updated - see https://github.com/CfSOtago/GREENGridData/issues/1 - we recommend not using rf_46 data in any analysis at the present time.

6.1 R packages used

• base R (R Core Team 2016)
• bookdown (Xie 2016a)
• GREENGridData (Anderson and Eyers 2018) which depends on:
  • data.table (Dowle et al. 2015)
  • dplyr (Wickham and Francois 2016)
  • hms (Müller 2018)
  • lubridate (Grolemund and Wickham 2011)
  • progress (Csárdi and FitzJohn 2016)
  • readr (Wickham, Hester, and Francois 2016)
  • readxl (Wickham and Bryan 2017)
  • reshape2 (Wickham 2007)
• ggplot2 (Wickham 2009)
• kableExtra (Zhu 2018)
• knitr (Xie 2016b)
• rmarkdown (Allaire et al. 2018)

6.2 Session info

## R version 3.5.2 (2018-12-20)
## Platform: x86_64-apple-darwin15.6.0 (64-bit)
## Running under: macOS High Sierra 10.13.6
## 
## Matrix products: default
## BLAS: /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib
## LAPACK: /Library/Frameworks/R.framework/Versions/3.5/Resources/lib/libRlapack.dylib
## 
## locale:
## [1] en_NZ.UTF-8/en_NZ.UTF-8/en_NZ.UTF-8/C/en_NZ.UTF-8/en_NZ.UTF-8
## 
## attached base packages:
## [1] stats     graphics  grDevices utils     datasets  methods   base     
## 
## other attached packages:
##  [1] tidyselect_0.2.5  drake_7.2.0       kableExtra_1.1.0 
##  [4] lubridate_1.7.4   ggplot2_3.1.1     bookdown_0.10    
##  [7] rmarkdown_1.13    here_0.1          skimr_1.0.5      
## [10] readxl_1.3.1      readr_1.3.1       dplyr_0.8.1      
## [13] data.table_1.12.2 GREENGridData_1.0
## 
## loaded via a namespace (and not attached):
##  [1] storr_1.2.1       progress_1.2.2    xfun_0.7         
##  [4] purrr_0.3.2       reshape2_1.4.3    colorspace_1.4-1 
##  [7] htmltools_0.3.6   viridisLite_0.3.0 yaml_2.2.0       
## [10] rlang_0.3.4       R.oo_1.22.0       pillar_1.4.1     
## [13] R.utils_2.8.0     glue_1.3.1        withr_2.1.2      
## [16] plyr_1.8.4        stringr_1.4.0     munsell_0.5.0    
## [19] gtable_0.3.0      cellranger_1.1.0  R.methodsS3_1.7.1
## [22] rvest_0.3.3       visNetwork_2.0.6  htmlwidgets_1.3  
## [25] evaluate_0.13     labeling_0.3      knitr_1.23       
## [28] highr_0.8         Rcpp_1.0.1        backports_1.1.4  
## [31] scales_1.0.0      jsonlite_1.6      webshot_0.5.1    
## [34] hms_0.4.2         digest_0.6.19     packrat_0.5.0    
## [37] stringi_1.4.3     grid_3.5.2        rprojroot_1.3-2  
## [40] cli_1.1.0         tools_3.5.2       magrittr_1.5     
## [43] base64url_1.4     lazyeval_0.2.2    tibble_2.1.2     
## [46] tidyr_0.8.3       crayon_1.3.4      pkgconfig_2.0.2  
## [49] xml2_1.2.0        prettyunits_1.0.2 assertthat_0.2.1 
## [52] httr_1.4.0        rstudioapi_0.10   R6_2.4.0         
## [55] igraph_1.2.4.1    compiler_3.5.2