FSG has an important post on evaluating how a problem is articulated when measuring impact:
Microcredit is a similar story. The problem was wrongly articulated as “not enough low income households can access loans.” Voila – the measure of success becomes number of loans or volume of loans dispersed. Easy to count and easy to celebrate as a win. We all know what happened in this case. Instead, the problem should have been defined as “not enough entrepreneurs can access capital to help grow their businesses or farms”. In this case, the measure of success would have been growth in asset base and cashflow of borrowers over time. And loans would only have been given to those who could put them to productive use. Again, by settling for a proxy (number of loans disbursed) impact was in many places quite negative.
Similarly, the US mortgage crisis comes to mind. The problem was unfortunately defined as “not enough low income households own their own home” rather than “not enough low income households are building a long-term asset base through home ownership”. Subtle difference in articulation, but major implication on what counts as success.
In all of these examples, focusing on increasing outputs of the (wrongly defined) problem actually ended up having a negative effect on impact!
As you think about your own theories of change, ask yourself if you’ve correctly defined the ultimate problem you’re trying to solve and if any of the indicators you’re tracking along the way may in the end be counterproductive.