Oregon EPS Study

A third-party project attempted to validate HES against actual data applied HES, along with two other tools (REM/Rate and SIMPLE) deemed among the best on the market to 190 homes. That study (which has been widely misunderstood and misreported) in fact found HES to generate lower median errors than the other two tools against which it was compared (see figures), with a median error of 21.8% from normalized annual energy consumption (versus 24.0% for SIMPLE and 31.1% for REM/Rate). [Whether the differences are even material was not addressed in the study.]  HES was also the top performer by many other measures, including all-gas homes, newer homes, larger homes, and for non-space-conditioning electricity use. It should also be noted that the other tools exhibited strong tendencies to either over-predict actual energy use (REM/Rate) or to under-predict it (SIMPLE) for more energy-intensive homes. The authors' retained many outliers (likely from end-uses, such as wood heat, portable electric heaters, etc., not specified to the models). A more extensive critique is posted here.

The charts below show some of the raw data. The fit-lines and error bars are LBNL estimates, superimposed on the original charts for perspective.

Note that in the four years since this study was conducted, extensive updates have been made to the default values and associated modeling within HES. Together with more rigorous quality control on the input data, we found even higher accuracy and less "scatter" in the data for HES.

The National Renewable Energy Laboratory has meanwhile completed an assessment analogous to what was attempted in the original Oregon EPS study, i.e., under a special use case in which HES, REM/Rate, and SIMPLE are utilized in an "asset rating" mode, which holds occupant behavior so as to isolate the effects of building envelope and equipment. They found that HES predictions agreed almost perfectly with actual bills, on average, across as sample of 885 occupied homes in Oregon (same cohort as the EPS study), Wisconsin, Minnesota, North Carolina and Texas.