Over the years, we have borrowed many tools from other fields when conducting process evaluations of clean energy programs or assessing market opportunities, response, and needs. For example, recently we developed a customer journey map (a most often used in the marketing realm) to explain how customers were engaging with the multiple facets of a marketing campaign in two cities in California.
That being said, techniques like journey mapping, likewise borrow from research methods long in use. This recent effort reminded me of a series of workshops I did with program managers in the early 1990s at Sacramento Municipal Utility District. Each workshop lasted about three hours, so we did one program before and one after lunch – which was always in this fabulous outdoor sandwich place a short drive from SMUD offices.
Any time I did workshops like this, my first question would always be, “So how does the customer first learn about the program?” The program team and the two evaluation team members in the room would then examine every entry point to the program to ensure it was clearly defined. Once they’d completed that exercise, I’d ask, “Okay, so then what does the customer do?”
As we walked through each program, we could identify opportunities to shore up the program design to improve the customer engagement process – though that wasn’t what we called it back in the ‘90s – and we would also identify which aspects of the program should take priority in the evaluation. In all, I ended up conducting nearly 10 of these workshops over a six-month period.
A couple of years later, I took a week-long course in service quality to examine program optimization from a customer service, which explored yet another way of thinking about the customer journey. This course used the theater as a metaphor, casting the customer and the service provider in the leading roles. Other potential customers comprise the audience, which gets the word if the service is excellent or not. Backstage is the program administrator and implementation contractor and other internal touchpoints, which make sure the service is delivered. Since taking this course, I have used this approach to both analyze programs and develop a method for a rapid process evaluation focused on improving customer service side of the programs.
It was quite a bit later that I started using program theory and logic models to help programs think about whether they had the most effective theory of what the program was trying to do, whether there were any missing holes in the theory, and, of course, how to track progress by seeing whether the various steps in the program logic resulted in the expected changes in behavior, attitudes, and knowledge.
While these techniques do not represent traditional research or evaluation techniques, they have been experiencing a renaissance as both implementers and evaluators alike are recognizing the insights they can provide. In the end, whether its journey mapping, a “theater”, or a logic model, these tools provide insight into program functioning. None are necessarily a silver bullet to ensure programs are running efficiently and effectively, but each has a time and purpose that can help shed new light on a program.