Home Energy Scoring Tool accuracy

The accuracy of operational energy use estimates by the system underlying Home Energy Score has been established (Parker et al., 2012). Defining the expectations for accuracy of an asset-based modeling protocol is more nuanced, given that behavioral factors are normatively held constant and standardized defaults are applied to many appliances, and some loads that may be present in a real home (well pumps, workshops, pools, etc.) are assumed not to be present. Thus, significant differences can be expected between measured and predicted energy use for a given home, especially if that home is in any way non-average. These caveats notwithstanding, an asset-based tool would ideally produce estimates near the average bill for a large, diversified set of actual homes. This is indeed the case for Home Energy Score (see Figure below), which summarizes analysis conducted by the National Renewable Energy Laboratory (Roberts et al., 2012, and summarized by Bourassa et al., 2012) achieved excellent agreement with actual consumption among accuracy testing of three asset analysis tools. The analysis is based on the audit and billing data of 885 occupied homes in Oregon, Wisconsin, Minnesota, North Carolina and Texas.

  Comparison of HEScore, SIMPLE, and REM/Rate Model

Energy Estimates vs Billing Data

In the case of asset tools that map estimated energy use to a discrete score, it is important that assignments of such scores are largely in agreement with assignment of scores using model estimates. Given a 10-bin scoring scale (described below), assessor error (data entry, engineering assumptions, etc.) the analysis suggests that the Home Energy Score process will assign the correct score (i.e., within +/- 1 bin) 90% of the time. However, when all conceivable modeling uncertainties (assuming accurate assessor inputs) are included this decreases to +/- 1 bin 67% of the time. Though not relevant for determining the accuracy of the asset score, the study reassuringly notes that after including even the considerable uncertainties introduced by occupancy behavior effects, the estimated score will be correct in not less than 50% of the cases. In this analysis, field data were translated from REM/Rate inputs into HES inputs within NREL’s Field Data Repository process (Roberts, et al. 2012 - Appendix C).

The National Renewable Energy Lab conducted an Independent accuracy assessment of a BETA version of the tool (used in a National Pilot Study conducted between late 2011 and early 2012). Appendix A of that study documents additional accuracy gains, as of July 27, but omits further improvements made prior to the full National Launch of the tool in 2012 (reflected in the chart above).

The summary of mean and median errors for the three tools (per the NREL assessment) is as follows: