More often than not, energy-efficiency and resource conservation programs are assessed retrospectively in conjunction with other analyses. After-the-fact accounts of customer satisfaction, free-ridership, spillover, and the like may give program administrators insights, but they do not allow them to respond to the market in real time.
In 2009, Energy Trust of Oregon enlisted Research Into Action to help them tackle this issue. They sought to establish a rigorous process for collecting ongoing, continuous feedback, which would empower their programs to be more agile and pivot in tandem with market forces.
What we did
To accomplish this objective, we designed and piloted an experimental test of a rapid response (“Fast Feedback”) process, helped roll-out the process, and later conducted a longitudinal analysis of the first three years. This process involved developing and refining the survey questions and determining what tracking data would trigger survey implementation. We also developed alternative formatting and delivery approaches; conducted an experiment to determine highest response rates; explored potential nonresponse bias; and developed a reporting format for incremental and cumulative findings. The pilot sought an effective method to achieve the industry-standard of 10% precision, with 90% confidence.
The surveys provided insights regarding program satisfaction, indicators of free-ridership, future intentions to work with Energy Trust, and additional services desired from Energy Trust. This allowed the organization to assess net effects on a quarterly basis (rather than waiting two or more years) and share findings with implementers, thus facilitating a process of continuous improvement.
After the roll-out phase, Energy Trust contracted with a survey firm for regular data collection, which they analyze and provide to internal audiences each quarter and to the public annually. In 2014, Energy Trust asked us to analyze nearly four years’ worth of quantitative and qualitative data. The pilot project final report can be accessed here.