Energy Performance Score 2008 Pilot Report

The goal of the Energy Performance Score (EPS) 2008 Pilot was to find an effective, accurate, and cost effective method and set of tools to calculate and report a home's energy performance. The report was written by Eric Storm and Beth Meredith of Living Spaces for Earth Advantage and Energy Trust of Oregon. You can read the Executive Summary and download the full report and its attachments below.

                                             Full Report:

• EPS 2008 Pilot Report

                                             Attachments:

• EAI Organizational Overview
• UK Energy Performance Certificate
• EPS Field Technician Input Form
• EPS Homeowner Input Form
• EPS Interim Report
• EPS Score Sheet for Existing Homes
• EPS Score Sheet for New Homes
• EPS Score Sheet w/Letters
• EPS Energy Analysis Report
• EPS Auditing Protocol
• EPS Database Screen Shot
• EPS Web Tender Portal Screen Shot

EXECUTIVE SUMMARY
According to the Pew Center on Global Climate Change, energy used in residential, commercial, and industrial buildings produces approximately 43% of U.S. carbon dioxide (CO2) emissions, 21% of which is from the residential sector alone. The median home in the United States is responsible for 6.2 tons per year of CO2 emissions from electricity use. Given the magnitude of this contribution, it is essential that efforts to control climate change include an explicit focus on the building sector.

Historically, it has been difficult for homeowners to know how to evaluate the energy performance of different types of homes or the performance implications of the plethora of green building programs. What has been absent is a miles per gallon (MPG) rating for the built environment. By providing consumers with information on homes that do better than the minimum building requirements, standards and labeling can help overcome obstacles and advance building efficiency. Homeowners, municipal governments, utilities, and the real estate industry need an easy-to-understand means of comparing the performance and impact on climate change of homes in both the new and existing home sectors.

The goal of the Energy Performance Score (EPS) 2008 Pilot was to find an effective, accurate, and cost effective method and set of tools to calculate and report on a home's energy performance. The pilot drew upon the United Kingdom's Energy Performance Certificate (EPC) program that measures and reports on home energy use and associated carbon emissions. The EPC includes a score sheet graphically depicting energy use and carbon emissions and a multipage report describing energy improvements that can be made to lower energy use and costs.

In addition, the pilot considered how an MPG-type metric might apply to home energy use and sought out a metric that would meet the following criteria:

- Easily understood by the general public.
- Meaningful in different contexts to respective stakeholders.
- Applicable to new and existing homes so comparisons can be made between homes.
- Useful for indicating progress toward individual and community energy goals.
- Helpful to homeowners as a baseline against which to evaluate their own energy use.
- Consistent over time.

The Energy Performance Score for a home would use normalized values for occupancy and behavior in order to represent the annual energy use of a home under typical conditions with the caveat "actual use may vary" as is the case with MPG for cars. In this way, the EPS would be an asset rating which excludes all variability in occupant behavior from the assessment.

In order to find an accurate and cost effective method for evaluating, calculating, and scoring the energy use and carbon emissions of new and existing homes, the pilot reviewed home energy modeling software programs. While other approaches to scoring a home exist, energy modeling was determined to be the most appropriate for the goals of the EPS. Some other approaches use utility data as a starting point or for calibration, but these approaches would be hard to test for accuracy since no other generally available house-specific empirical data exists. An approach that relies on utility data would be unable to score homes that lack utility data, such as new homes, unoccupied homes, and homes with extremely unusual energy use.

Over 100 software tools were surveyed, and 4 were selected: REM/Rate, SIMPLE, and two versions of Home Energy Saver (HES). REM/Rate was selected since it was an accredited Home Energy Rating System (HERS) software and one of the best known and widely used. By contrast, SIMPLE was a newly developed program that required fewer data points, less testing, and less technical operator knowledge than REM/Rate. HES was selected on the basis of its reported accuracy and because it was an online modeling tool designed for homeowners. Of the three levels of analysis possible with HES, the pilot decided to test the middle and most complete level, which were dubbed HES-Mid and HES-Full, respectively.

It was determined that the best method for measuring the accuracy of modeling software was to compare each software's energy predictions to weather normalized utility billing data for each home. Since utility billing data includes the behavioral factors of each home, behavioral factors were also included in the modeled calculations. This means that while the EPS is an asset rating, the software testing done in the pilot was based on a comparison of operational ratings which includes occupant behaviors.

The analysis of the software program results was based on a sample of 190 homes. This sample was a subset of the 302 homes from which data was collected in Portland and Bend, Oregon, in the fall and winter of 2008. Of the 302 homes, 112 were excluded from analysis for a variety of reasons. The largest segment of those excluded (75 homes) was due to lack of complete actual use utility data. Each software tool was evaluated in terms of accuracy of energy use prediction, audit time, and ease of use.

Comparing the errors of each program, SIMPLE generally produced the most accurate results. When predicting total energy use, SIMPLE had a mean absolute percent error of 25.1% compared to HES-Full with 33.4%, REM/Rate with 43.7%, and HES-Mid with 96.6%. In other words, SIMPLE predicted energy use on average within plus or minus 25.1% of actual use.

Another means of evaluating accuracy was to look at the percentage of homes for which a program predicted well and the percentage for which it predicted poorly. Predicting well was defined as over-predicting or under-predicting by less than 25% (an absolute percent error of less than 25%). Predicting poorly was defined as over-predicting or under-predicting by more than 50%. Using these criteria, SIMPLE had essentially the same number of good estimates as HES-Full, but it also had approximately one-third as many poor results. REM/Rate had 10% fewer good predictions, but 24% more poor predictions compared to SIMPLE. In terms of offering reliable results, SIMPLE performed the best with the lowest frequency of large errors.

An in-depth review of the sample data does indicate trends whereby HES-Full and REM/Rate had a greater number of good predictions than SIMPLE for a type of energy within a subset of the sample. In nearly every cohort and for every fuel type, SIMPLE produced the fewest significant errors. This finding was somewhat surprising considering how many data points are required by each software program: HES-Full with 185 and REM/Rate with approximately 100 compared to SIMPLE with 32. Overall, SIMPLE offered consistently more accurate results, especially with older homes. Any of the software programs tested could be changed to meet the proposed standards. It is the hope of the pilot team that the EPS will spur the development or improvement of many tools as outlined in this report. However, in terms of quickly developing a tool to EPS standards, SIMPLE appears to be the most likely candidate at this time.

SIMPLE also showed the most promise for audit time and ease of use. Since data was collected for all the programs during the home audits, time comparisons for each software tool were based on the estimates of the field technicians who collected the data and entered it into each program. Based on the technicians' estimates, SIMPLE would require 1 hour to audit a home. HES-Mid was estimated to require almost 1.5 hours; HES-Full, 2 hours; and REM/Rate, almost 2.5 hours to audit the same home. Likewise, the technician's estimated data entry for SIMPLE at 14 minutes, only 3 minutes more than HES-Mid, and one-third the time of REM/Rate at 45 minutes and HES-Full at 47 minutes. These time estimates need to be tested in the field as the audit protocol for SIMPLE becomes better defined. One significant issue to be decided is the importance of blower door and duct pressurization testing both for accuracy in reporting the energy use and for determining where energy improvements need to be made for existing homes.

Calculating carbon emissions is an integral part of the EPS and completes the picture of energy use in the home by indicating the impact on greenhouse gas emissions of different fuels. The carbon score balances the fuel efficiency bias that would result from only using an energy score. In this way the carbon score helps the EPS to be a more fuel neutral approach. Homeowners were also very interested in knowing the carbon emissions associated with home energy use. On a broader level, the ability to track carbon emissions is also central to emerging energy policy at the local, state, and national levels. For these reasons, the pilot developed a methodology for calculating greenhouse gas emissions (referred to in this report as carbon emissions) based on the fuel types and amounts of the energy use in a home.

In addition, Earth Advantage Institute conducted a series of stakeholder surveys over the course of the pilot. These surveys solicited feedback on a range of topics, including attitudes toward home energy efficiency, the usefulness of energy auditing and analysis, comprehension of energy-related terms, score sheet preferences, and feedback from those who had a home audit during the pilot. The responses to these surveys guided the conceptual development of the EPS and were incorporated into the EPS score sheet, EPS energy analysis report, and program recommendations. The findings included the following:

- The EPS concept has considerable appeal with consumers, home improvement contractors, builders, and real estate professionals.
- The ideal price of an EPS would be $100 with a cap of $200.
- Cost is a major issue and the common language for understanding energy.
- The EPS must be presented in a clear and objective manner from a trustworthy source.
- Carbon emissions are relevant and very important to homeowners.
- Homeowners are most familiar with energy use in terms of watts and kilowatt hours.
- Homeowners want information on energy performance and where to make improvements.
- Home energy audits are perceived as useful by homeowners for a variety of reasons.
- Homeowners thought that their homes were more energy efficient than preliminary results indicated.
- Home energy auditing helps highlight the need for air and duct sealing.
- Financial incentives (from Energy Trust of Oregon in the state of Oregon) are important to making home energy upgrades about half of the time.

Based on the software analysis, research, and survey findings, the pilot team offered the following recommendations for the development of an Energy Performance Score:

1. The Energy Performance Score should be developed along two tracks: the EPS with energy and carbon scores and a performance profile of energy related elements of the home (EPS score only), and the EPS that additionally includes recommendations for energy upgrades (EPS w/ upgrades). Official EPS auditing for both tracks should be performed by trained and certified third-party auditors.

2. In order to offer a credible level of accuracy, EPS certified software programs should be able to predict energy use within 25% for 70% of homes and within 50% for 90% when compared to actual use. To this end, software should be developed to meet all the EPS requirements for accuracy and reporting. Given the analysis results, SIMPLE would be a good candidate for further refinements for a second phase of development. This should include field testing of the auditing protocol with varying levels of diagnostic testing.

3. The EPS energy score should be expressed as the total annual energy required for the house under normal conditions and be expressed in kilowatt hours per year. This score should be illustrated on a scale that also indicates the energy use of different fuels, relevant local comparisons, energy upgrades for existing homes and built to code comparisons for new homes, and community energy goals.

4. The EPS should include a carbon score that reflects the greenhouse gas emissions associated with the home's energy use. Comparisons on the carbon scale should include those listed for energy, as well as the predicted emissions if the homeowner used the most commonly subscribed renewable energy option through his or her utility or fuel provider.

5. The EPS should include an energy analysis report that includes an accounting of the annual estimated energy use and fuel costs for heating, cooling, water heating, and lights and appliances in the home, as well as the performance of the various energy-related elements in the home (e.g., walls, heating ducts, appliances). For existing homes, the report may also include recommendations for energy upgrades and the associated costs and predicted savings.

6. Ideally, the EPS will be a coordinated effort to ensure consistency of the core elements, including the EPS name and branding, and the standards for auditing, software modeling, auditor training, and reporting. There is also the need for a central database to serve as a clearinghouse for EPS scores for homeowners and homebuyers, as well as a Web tender portal through which contractors can offer estimates on energy upgrades to interested homeowners.

Energy Trust of Oregon and Earth Advantage Institute are moving ahead with the refinement of SIMPLE, as well as working with recent relevant legislation and efforts at home energy auditing in Oregon to develop and further test the EPS concept in different climate zones.