Project Comments

The following chart contains comments that were submitted by steering committee members during the 2011 review process. Contact Chuck Kurnik for additional protocol comments.

These comments have been grouped into different topics including: Measures, Project Questions, Baselines, and Net-to-Gross. If you have a question or comment that you would like to submit, please send via email to Chuck Kurnik.

Date	Steering Committee Member Comment or Question	Project Team Response
Measures
9/28/11	While "measures" might work for some things, some other taxonomy of grouped measure types might be better — such as whole house retrofits, or process measures, etc.	Indeed, whole house retrofit was selected as one of the "measures" to address in Phase 1.
10/13/11	1. HVAC system 2. Whole-house retrofit 3. Commercial lighting 4. Residential lighting These are from the perspective of what causes the most electricity use in a high demand day, which is when ozone levels are at their worst. Anything that keeps houses cooled more efficiently helps since then the houses are using less electricity which means less air pollution (thus picking 1 and 2). Lighting may contribute, thus 3 and 4.	Thank you for your comment.
10/18/11	Residential Lighting — Should lighting differentiate between lighting technologies, installation locations (operating hours differ - closet vs. living room). Geographic location influences runtime. Will this include outdoor lighting too? Will it include different control strategies (photocell, time, motion, remote)? How will baseline tech be determined? How will multiple technologies in a single location be addressed?	Excellent questions that will be shared with the technical experts in the Technical Advisory Group.
	Commercial Lighting — While many delivery channels and rebates are based on $/kW is important, there are other qualities to promote, such as energy savings, productivity, reduced maintenance cost, and enhanced control. For example, replacing High-intensity discharge lamps (HID lamps) with fluorescent provides demand and energy savings plus multi-level lighting and motion detector control.	Excellent points that will be shared with the technical experts in the Technical Advisory Group.
	Refrigerator Recycling-Rebates — Are bounty programs rebates? Most co-ops pay bounty for retirement plus fee to dispose of refrigerator. Some also pay different amounts if consumers purchase an ENERGY STAR^® refrigerator to replace it. The goal can be to remove old, under-utilized second or third refrigerators or to promote ENERGY STAR, or both.	We did use the term "rebate" to include bounty programs, for this measure.
	HVAC — This program is important to cooperatives because G&Ts (wholesale generation and transmission cooperatives) often promote HVAC replacement for demand control and load shape. Distribution co-ops may have more interest in energy impacts including revenue loss.	Thank you for your comment.
	Water Heaters — Under M&V approach, why do gas water heaters only include key parameters and electric ones include all parameters? Also, shouldn't ambient conditions be included as a parameter? Would load control—demand or time-of-use—be covered? Temperatures and water-use profiles matter. Also, we need to include total water heating load, including standby, distribution, and transient losses. Need to include total energy input - some gas water heaters use electricity and gas. How will cooling "output", dehumidification capacity of a heat-pump water heater be accounted for and valued?	The difference for gas and electric metering is related to the ease of measuring gas flow versus electricity usage. The methodology will be refined by the technical experts in the Technical Advisory Group during Phase 2.
	Whole House-Rebates — Important, but not the only delivery method, what about on-bill financing, loans, tax credits?	Thank you for your comments, we will discuss with the Technical Advisory Group how these alternate delivery channels may influence the EM&V approach.
	#1 and #2 Residential and Comm lighting accounts for 80% of savings, methods are similar for residential and commercial. #3 Lighting Controls are becoming more and more important, but harder to do in both new construction and retrofit. The key is pre-metering. #4 Wholehouse retrofit true root mean square (TRMS) don't currently address wholehouse retrofits. Instead, focus on individual measures with a growing program design approach. 5 High efficiency water heaters 6 Refrigeration Systems Ranking depends - need to narrow down, be more specific (e.g., evaporator fans in commercial refrigerators) 7 HVAC system (heating, cooling) Ranking depends, narrow down - chillers, unitary, heating equipment, economizers, commercial vs. residential? 8 Furnace ECM: high efficiency fans — decent amount of operating hours 9 Refrigeration Recycling: too complicated. 10 Appliances: re there enough CW rebates or is market close to being transformed? 11 Low flow shower heads	Thank you for your comments, we are working with the Technical Advisory Group to narrow the definition of HVAC Systems for Phase 1, and it will also be discussed for Refrigeration Systems for Phase 2. In addition, as we get to Phase 2, we can discuss whether ENERGY STAR appliances (e.g. clothes washers) should be included.
10/19/11	Refrigeration systems are important because the compounds they may emit have high greenhouse gas (GHG) potentials, and therefore measuring even small amounts released consistently around the country could improve GHG inventories and tracking in many areas. Similarly furnace systems (I confess I don't know what ECM means?) could be important in that they may have widespread national applicability and if they are replaced or upgraded in many places it would be good to be consistent in the application of EE measurement and verification.	Thank you for your comment. ECM stands for electronically commutated motor, which is a more efficient motor for the air handling system within a furnace.
Project Questions
10/3/11	Understanding that the Steering Committee members are being asked to serve as "ambassadors and promoters" of the final DOE guidelines, I have to build a process up on my end to ensure that the input I provide on various key project elements/milestones get vetted appropriately within my group. Such a process would require allowing for a reasonable lead time to review draft materials (e.g., 2 weeks +). I am a little concerned about review process, given we have a call this Wednesday with expectation to decide on priority measures yet we don't have list yet to review.	We appreciate your concerns about timeline and will be flexible with the schedule to allow for enough time for review.
10/3/11	In addition to participating as a steering committee member, an important way to help build comfort on draft materials developed by the technical group is to have representation from an expert in the Northeast region to help inform the Usability Technical Group. I would recommend one person in particular (a former National Grid evaluation director) who has extensive experience working on evaluation in New York, Massachusetts, Rhode Island, and New Hampshire. Might it be possible for DOE to add such an expert to your Usability Technical Group, given there is no utility representation? This may be important to bring perspective of integrating EM&V methods and associated data collection into utility/program administrator tracking systems.	Members of the Technical Advisory Groups are affiliated with national consulting firms working in jurisdictions across the country. We will, nevertheless, ensure that we will have experts from the Northeast region participating as Technical Experts.
10/4/11	The Template — Slide 13: I think we need to first discuss if this is a plan for a specific measure, collection of measures, or a program (and what delivery mechanism). Then whether it is an M&V plan or an evaluation (research plan). I think we are looking for evaluation research plans in which case there might not be anything on M&V (but in some cases just deemed savings or quasi-experimental design - large scale data analysis). In any event, I think you may need additional sections on applicability conditions; verification; ex-ante forecasts; 'evaluability' assessment (threats to certainty, bias); baselines; budgets; resource requirements; monitoring plan; persistence analysis; and perhaps a section on program theory.	We believe the proposed template covers many of these recommended sections.
10/19/11	Was the program/measure category of "informational/behavioral" programs considered for a focus under this project? That type of "program" is becoming much more prevalent, and is having some serious impacts on EE budget and 'savings' allocations in a number of states, and is fraught with significant EM&V questions and issues. I'm just wondering if that was considered, and if not, should it be?	Behavioral measures will be treated in Phase 2, utilitizing the results from SEE Action Group's work on this topic as well.
Baselines
10/17/11	Any uniform EM&V Methods issued by DOE should contain a thorough discussion of the relationship between net savings adjustments and the principles/practices used to establish baselines for derivation of gross savings.	Thank you for your comment. We agree baseline savings need to be clearly defined and is intertwined with net-to-gross. If baseline is not code/standard, the data required and methods for analyzing them to determine the baseline will need to be clearly described within the method.
10/17/11	In the case of refrigerator early retirement/recycling programs, is the appropriate baseline for estimating gross savings the energy use of the refrigerator being recycled or should the baseline account for the fact that some households normally replace their refrigerators before the end of their useful lives. The common use of the term "net savings", are those that would not have happened absent the program. Therefore, should the baseline for determining "gross savings" from a refrigerator recycling program recognize that baseline practice and that some program participants would have replaced their units independent of the program?	Excellent questions that will be shared with the Technical Advisory Group/Technical Experts.
Net-to-Gross
9/28/11	Just gross is very limiting; does not look forward for next stage of EE and EM&V; only useful for states in the 1990s stage of EE.	In response to concerns around net-to-gross (NTG), we are forming N-TAG (the NTG Technical Advisory Group), composed of interested steering committee members, TAG members, and possibly external experts. The goal of N-TAG will be to discuss how to treat NTG within the context of this project.
9/28/11	I thought the question on what our product was expected to be was particularly helpful and I think we need to discuss that more, particularly given the discussion of whether to look at net-to-gross. For example, how will we characterize our work to the outside world, where others are debating net-to-gross and what they might be expecting us to do? If we see ourselves taking a small but meaningful step, how will we discuss what we expect the next steps to be and who will continue in those efforts?	We expect that characterizing our work to the outside world with respect to NTG will become clearer as the NTG Technical Advisory Group addresses the methods. In terms of discussing next steps, we are developing a communications plan, which will articulate key audiences and messages. Again, as the methods develop, the "size" of the step we are taking will become more obvious, and this can be worked into outreach efforts.
10/17/11	My preference would be to look at methods to get to gross savings, adjusted gross savings, and net savings in a step-wise fashion.	In general, we think this approach will work well. However, given the concerns of some Steering Committee members, we plan to approach the determination of net savings in a parallel track to determination of adjusted gross. It is important to note that in some cases, such when billing analysis is the appropriate EM&V approach (e.g. whole-house retrofit), calculation of net and gross savings can be addressed simultaneously in the context of the research design. In other cases, the determination of baseline is intrinsically related to net savings.