Guest blog post by Jerome Stiller:
If you're like me, you have to be vigilant to be sure that your waistline does not keep growing as you get older, like rings on a tree. It’s tough for me because I consider pastry, ice cream, and beer to be three of the four essential food groups (the other being coffee). So I exercise and try to watch my diet. But how do I know if my efforts are successful (or not)? Of course I could step on a scale every day or every week, and compare my weight to some ideal weight. I could do something similar by measuring my body fat percentage and/or BMI. Or I could use a tape to measure the circumference of my waist just below my belly button (don’t ask).
I've tried each of the methods above and don’t care for any of them. Weighing myself every day or every week means that I have to remember to do it, and it can be downright depressing because I am always heavier than I think I am. And what does “ideal” weight (or body fat or BMI) mean anyhow? Ideal for whom? Maybe not for me.What I do to measure my progress towards my weight goals is:
"I do not buy pants with a waist larger than 30 inches."
I wore size 30 pants in college, and I continue to do so today. When my pants are too tight, its time to increase my efforts, and when they fit quite well I know I’m doing a good job of managing my weight. In fact, discomfort from tight pants is an excellent and constant reminder that I need to do better (i.e., put down the cheese danish!) to meet my goal.
The example above takes a basic evaluation question (how am I/we doing?) and highlights some considerations of indicators and benchmarks.
An indicator is proxy measurement providing information to help answer the evaluation question. A benchmark is an explicit statement that provides a point or target for comparison.
And there is always a choice of benchmarks. For example, I could have chosen to compare my weight to some external standard (the “ideal” weight) , some self-determined target (to weigh 10 lbs less or to weigh 150 lbs); however these benchmarks would not be relevant to the tight-pants indicator. Notice that the indicator(s) you choose is crucial in choosing the appropriate benchmarks, and vice versa.
So, I have an indicator (how tight or loose my pants are) and a benchmark (a not-larger-than 30 inch waist) to help me assess my success in managing my weight. My evaluation strategy is to monitor the way my pants fit me and to use that data to inform my eating and exercise habits – looser means ice cream; tighter means no ice cream.
I chose this indicator because I am able to derive useful information from it (too tight -> need to workout and diet more), and it requires minimal time and effort because its something that I do already. While it doesn't measure my weight directly, it is easy and provides the information necessary to manage both my short-term and long term goals.
I hope you've found this useful in understanding some basic design considerations in program evaluation. Most importantly, remember that:
a) there are a wide range of options in determining evaluation indicators and benchmarks (many of which are not costly or even free); and
b) that your program evaluation will be successful if you choose indicators and benchmarks that are clearly aligned with your evaluation goals.
I chose an evaluation strategy that:
a) gave me meaningful information relevant to my evaluation question (are my exercise and diet efforts successful?) and
b) used minimal resources.
In designing and implementing a successful program evaluation, you can choose from a variety of strategies and their appropriate indicators and benchmarks to meet your evaluation needs. Me? I’m going to get some ice cream.
This Hip Tip is provided as a guest post from Jerome Stiller, Senior Data Analyst and expert in program development and evaluation. Jerome believes that data can and should be used to paint a compelling picture that will guide program and organizational development and provide persuasive evidence of impact. At Questus Strategies, we measure performance to guide development and improvement, and we measure impact for demonstrating success in achieving organizational goals. In today's highly competitive funding community, these data-driven demonstrations are increasingly important to funding sources.