ROI - what can I say Beth is a genius (Part 3 of 11 on Managing Tech 2 Meet Mission)

Let’s play a game.

Sure what is the game called?

I don’t know, it doesn’t have a name.

So you start to play the game, but as you play the game you and your friends make up the rules as you go. “You can only step on the yellow bricks” or “you cant get me when I’m touching the house”. Some rules work, some seem to ruin the game. But no one ever seems to win either, the game just sorta ends when Mom yells time for dinner.

Mom asks, what were you kids playing? The answer of course, “nothing”.

That nothing means more when it isn’t a game anymore. As kids we didn’t need to set goals or have to measure the success of our game. We just want to have fun. We look back at it and ask, “was that fun and should we play again? And if we play again what do we change?”

I get the feeling that many people approach some of their technology projects in the same way. They decide they want to try something, but they don’t set goals, don’t know the rules and aren’t really sure when they are done. So even though the technology may have worked and stuff got done, they aren’t really sure what they did.

Beth Kanter reigns supreme when it comes to measuring the ROI of a project, especially social media. Anything in the post I probably learned from Beth. OK, yeah, not probably, I did learn it from Beth. Her work on listening, metrics, and ROI is the best and most respected that I have seen in the nonprofit sector. So when it came time for me to write a blog entry about her chapter in the NTEN book, I was worried about what I could add to an expert like that. But then I remembered something Beth had talked about that didn’t seem to be as obvious in the book as I had expected.

It is a simple concept, ROI is best measured when the goals and metrics are established before you start.

However, as my pastor says simple is not the same as easy.

I cant tell you the number of times that as a project is finishing or in the middle of it, we get asked about the outcomes or the ROI. So we go back, dig through the data (if we have any) to look for evidence. But this evidence gathering should not be like hide and seek, where everything is hiding and you don’t know what you are looking for. It should be like a scavenger hunt, where you have a list before you start and gather the pieces as you go.

But why is it important to set the goals and ROI first. Can’t we just experiment and see what happens? Well for some things yes, you can. Beth does a great job in the chapter giving advice on how to decide how deep the ROI analysis should be. But measuring a project when it is done doesn’t determine if you accomplished your goal in the most effective way. You end up reporting outcomes, not reporting if you met your goals.

“What gets measured gets done” that is what a colleague of mine used to say all the time. So let me try to explain my thoughts on establishing ROI before you start in a different way.

Many organizations spend time at the end of the year reporting statistics of # of people served, communities impacted, new programs launched, etc. So if you budget and year end reporting are your only tools to measure success then the goal of your organization drifts away from accomplishing the mission and begins to be driven by meeting the numbers. So if you only measure the fiscal amounts and the number of people served, then that is everyone’s focus.

Metrics has to include tangible and intangible goals that are set and tied to ROI metrics before the efforts are started. Otherwise what will you review to gage progress? Yes, the results and outcomes may be very different and even much better than the metrics, which is awesome. So then you may have to adjust your metrics, but at least you have them to adjust.

Anyway, again those are my random but slightly related thoughts. So lets not set our ROI to make a real difference and stop making up the rules as we go.

Here are some of my favorite posts from Beth on ROI:

ROI - Fun read - Good ideas - makes ya think

ROI (Results on Insights): Nonprofit Examples of How Listening Returns Value

SXSW: Social Media Nonprofit ROI Poetry Slam - Slides, Links, and Poems (long)

Over 11 weeks I am doing a themed series of blog posts. Each week I will write about a chapter of the book called Managing Technology to meet your Mission. This week is on the 3rd chapter about ROI). Also dont forget that NTEN is also running an AWESOME 2 day online conference! You should totally buy the book and sign up. (In case you are wondering, I am volunteering to do this, I am not getting paid or in any other way reimbursed for this. I just love NTEN and their events.)