You Didnt Use Brainstorming to Select Your Measures, Did You?

Introduction

When Alex Osborn invented the creativity technique called brainstorming, I wonder if he had any idea just how extensively business would apply it. Almost every meeting employs some kind of brainstorming event, but there's one meeting that really should leave it off the agenda: the performance measure selection meeting.

There are 5 common ways people select performance measures

The selection of performance measures has never really been treated as anything more than a trivial, and often pesky, decision brought around by the annual business planning workshop. Usually people will take the fastest route to finalising a list of performance indicators in the KPI column of their business plan, and depending on your organisation, the fastest routes are usually some combination of the following:

- brainstorming, where participants just list as many potential measures as they can think of and then do some kind of short-listing

- benchmarking, or some other version of adoption (copying) measures from other organisations

- using existing data or measures, to save the costs of measuring something new, and having to collect the data

- measuring what stakeholders tell us to measure

- listening to what the experts in our industry have to say - what they "know" we should measure

Each of these methods certainly has some great strengths, but we often forget to examine the drawbacks. This article was written to open up the drawback discussion and offer a different way of thinking about measure design. but these common ways are limited!

brainstorming seems quick, but is really very hit-and-miss

Probably the most common approach taken to decide what to measure, brainstorming is the easy way out of an activity many people dread. Quality in equals quality out. A process that was designed for creativity and not measure design will not produce useful and usable measures.

pros:

- Seems quick.

- Lots of ideas for measures can be generated rapidly.

- Collaborative ideas - two heads are better than one.

- Easy to do, no special knowledge or skill is required.

- Engages people to be part of the measure selection process.

- A known/accepted approach, so the process doesn't get in the way of doing the activity.

All ideas are considered/accepted, which helps people willingly participate.

cons:

- Not really finished after the brainstorming is over - how to get a final selection of measures is vague.

- There is more to measurement than just selecting measures - thought about how to bring the measures to life is also needed.

- Too much information is produced, therefore too many measures often results.

- Ideas are not vetted or tested, our thinking is not challenged.

- We often are brainstorming against different understandings of the same objective/goal we want to measure.

- The bigger picture is not taken into account e.g. unintended consequences, relationships to other objectives/goals, silo thinking.

- Often what is brainstormed is not really a measure at all ? instead it is an action, a milestone, a piece of data, a vague fluffy concept.

- What is brainstormed is often expressed so vaguely no-one can remember what it meant later on.

Measure design needs to produce a few measures that have been thoroughly tested for their relevance or strength in tracking the goal or result they are selected for, and are supported by the people that they will affect.

benchmarking is convenient, but ignores strategic uniqueness

Benchmarking is about finding out what another organisation is doing, and this almost always involves or is based on some comparison of performance measures. If organisations share the same measures, then benchmarking is certainly easier to do, but there are consequences of adopting a "bolt on" set of performance measures.

pros:- - We feel safe & secure because others have gone before us.

- Others have (we assume) already put a lot of thought into those measures

? why reinvent the wheel?

- We can compare our performance with the performance of other similar organisations.

- We get a feeling of how good (or better) we are compared to others.

- It's easy ? just have to look and ask.

- Easier to justify to others why we are measuring what we are measuring.

- Widely accepted approach.

cons:- - There is more to measurement than just selecting measures.

- Not always collaborative - so little buy-in by people who will produce and use the measures.

- Not always like for like (apples with apples) - in fact, probably never is to the extent we assume.

- Isn't driven by the decisions we need to make and the information we need for those decisions.

- Doesn't challenge our thinking.

- It makes us bring some other organisation's strategy to life, not what is right for us (aren't we unique?).

- The goal posts change more frequently than the benchmarking process occurs.

- Our bigger picture is not taken into account, such as how this area of performance affects others in our organisation.

- Selecting measures against different understandings of the 'outcome' to measure.

Measure design needs to produce measures that encourage learning and sharing of knowledge, but not at the expense of discovering and focusing on the unique business strategy that best suits the organisation.

data availability makes it cheap, but focuses on yester-year's strategy

What data do we have? What have we measured in the past? What are we already measuring? All questions that are symptomatic of an organisation that is not open to challenging whether the data they are collecting really is capable of telling them what they need to know about where they are going. Measure what you have always measured, get what you have always gotten. What's strategic about that?

pros:

- Very easy, very quick.

- Known data sources mean low cost in data collection/capture ? already have systems to support it.

- People more likely share a common understanding of measures already.

- Consistency in information over time, to have valid comparisons over time.

- Have historic data available for trend analysis.

cons:

- We only bring yesterday's (or yester-year's) strategy to life.

- Rarely challenge the measure itself, so no better measures are explored (and therefore no better data will ever be collected to manage emerging strategic risks and opportunities).

- Not collaborative, because it is from previous thinkers, not today's doers.

- Bigger picture is not taken into account.

- Parts of our strategy that are new will go un-measured.

Measure design needs to produce measures that are cost-effective and have some historic data before too long, but must produce measures that will take the organisation toward its vision, not drag it back into its past.

stakeholders need information, but that's not the same as performance measurement

What's imposed on an organisation by regulators, shareholders, government, industry bodies and other stakeholders is often considered a constraint on the measures it can use to manage its performance. They struggle to retrofit the stakeholder-chosen measures to their strategy, or renegotiate the stakeholder-chosen measures. But these aren't the only two options. This method of measure selection is not really measure selection at all.

pros:

- We get told what to measure, and don't have to do the hard work ourselves.

- We give them what they want and thus we won't get into trouble.

- Often can be negotiated resourcing by the government group or stakeholder imposing it.

- Get higher management commitment to the need (and therefore to data collection and reporting) - it will get done.

- Our governance requirements are more likely to be met (assuming we report these measures properly).

cons:

- There is more to measurement than just selecting measures.

- Encourages an autocratic/ patriarchal management style.

- The imposers don't understand our strategic direction/don't trust that we do.

- Isn't driven by the decisions we need to make and the information we need for those decisions.

- Lack of ownership by us of those measures (and the results they track). - Bigger picture is not taken into account.

- The focus may not be the right focus or the only focus that matters.

- Parent-child (instead of partner) relationship with stakeholders could become the norm.

- Assumes that the stakeholders have robust methods of designing meaningful measures.

Measure design needs to produce measures that are relevant to the organisation's strategic direction, a direction that is understood supported by its key stakeholders. But the measure design process can also be used to design reports to stakeholders that are largely separate to its organisational performance measures.

experts have experience, but can be locked into one-size-fits-all

Industry experts, consultants, people with years of experience or self-nominated experts all carry a mystique of knowledge and wisdom that can make their ideas about what to measure sound more like truth than suggestion.

pros:

- We get told, and don't have to do the hard work.

- A focus is quickly clear.

- Experts can bring new ideas and experience we may not have.

- Have approaches that have worked for other organisations.

cons:

- The focus may not be the right focus or the only focus that matters for us.

- We usually don't challenge experts even though we have intimate knowledge of our unique business.

- Experts can assume we don't need to know their thinking behind the measures, so we don't learn how to think more wisely about the measures for ourselves.

- We may not really understand the measures that are recommended to us.

- Experts often cost a lot of money.

- Experts may not understand our organisation enough to know our uniqueness (the one size fits all problem).

- Experts may not take account of how feasible it might be (or not) to bring those measures to life in our organisation.

- Measure design needs to produce measures that we understand and have ownership of, and be a process that allows us to continue refining and refreshing our selection of measures as our performance and strategic direction changes.

Measure design needs a better methodology

The reason that the quick and easy methods above are used to select measures is the same reason that performance measurement is a dreaded event: people have no idea that measure design is a process of dialogue around the goals or objectives they want to measure. It is a way for them to engage in a deeper understanding of the results they are really trying to achieve, and how they would be convinced as to the degree to which they have achieved those results.

And that's why we've created an alternative method to measure design. A method that produces measures which:

- are few, not prolific

- have been thoroughly tested for their relevance or strength in tracking the goal or result they are selected for

- are supported by the people that they will affect

- encourage learning and sharing of knowledge

- assist the discovering and focusing on the unique business strategy that best suits the organisation

- are cost-effective

- have as much historic data as feasible

- will take the organisation toward its vision, not drag it back into its past

- have resulted in people (including stakeholders) having a deeper and richer understanding of what results the organisation is really trying to create

- people understand and have ownership of

This method is based around five simple but deliberate steps:

1. Write down the goal or objective or result you want to achieve (and thus measure). Stay focused on this particular result while you are designing measures for it (don't let your attention wander to other results).

2. Describe this result in a lot more detail, explaining what you and others would see, hear, feel, be doing (or even taste and smell!) if that result were happening now, explain the differences you would notice. This is listing the 'sensory descriptions' of your result. Do this until you have agreed what the result really means and what differences you will most likely notice between now and when the result is achieved.

3. Check if there are any unintended consequences of achieving this result, either positive or negative. Make sure it is a result you really do want to create before you bother measuring it.

4. Go back to the list you created at step 2 and list the potential things you could count or measure that would give you evidence of how much each of those sensory descriptions was actually occurring. For each potential measure you identify, give it a high, medium or low rating for how relevant it is to your result and a high, medium or low rating for how feasible it would be to measure it.

5. Use the high, medium and low ratings for your potential measures to shortlist to between 1 and 3 measures for this result. This method, just like all the traditional methods, has its own pros and cons as well:

pros:

- It produces very few and very meaningful measures.

- Thinking about strategy and measures is challenged and tested.

- The feasibility of measures is assessed.

- It is a conscious and deliberate approach that people have control over.

- Measures strongly link to the specific strategic outcomes that are to track.

- The bigger picture is taken into account ? and links and relationships with other measures are automatically identified (eg cause-effect, companion, etc?).

- Very collaborative because it is dialogue based, therefore high ownership results.

- Measures are not vague ideas but very evidence based.

- The measure is designed from the context of how it will be used.

- Can deliberately test the authenticity of each measure relative to its goal/objective.

- The type of language used to design the measures promotes a common and shared understanding of the result to be created (and this makes it easier to communicate strategy to all staff).

cons:

- It's not easy the first few times through - it's new and can potentially distract people if they forget what step they are doing and why (until they have been through it once or twice).

- It takes time - it needs good quality dialogue to build a deeply shared understanding of what is being measured.

- It will need training or resources to teach people how to do it and why to do it.

- Sometimes strategy needs to be altered (as this approach often deepens understanding of implications or misunderstandings of chosen strategy/wording of).

So, you no longer need to feel compelled to take the easy way out of selecting performance measures, and in five simple steps, you can not only design useful and usable measures, but also deeper your understanding of the results you are trying to achieve!

About the Author
Stacey Barr is a specialist in organisational performance measurement, helping people get the kind of data and information that tells them how well their business is performing, and how to make it perform better. Visit Stacey's website to sign up for her free newsletter: http://www.staceybarr.com

---

Google News Updated : Fri, 05 Sep 2008 14:05:39 GMT Jobless rate soars to 6.1% - CNNMoney.com Macon Area Online Jobless rate soars to 6.1% CNNMoney.com - 59 minutes ago Unemployment surges to 5-year high as employers cut workers for eighth straight month, bringing '08 job losses to 605000. By Chris Isidore, CNNMoney. Unemployment Rate Hits 5-Year High As Economy Continues to Shed Jobs Wall Street Journal Unemployment Rate Rises to 6.1% New York Times TheStreet.com - Forex Pros - Reuters - RTT News all 36 news articles Publ.Date : Fri, 05 Sep 2008 13:05:45 GMT State falling way behind No Child Left Behind - San Francisco Chronicle Modesto Bee State falling way behind No Child Left Behind San Francisco Chronicle - 6 hours ago California schools, required by the federal No Child Left Behind Act to lift more students over a higher academic hurdle this year, instead stumbled and slipped back, as nearly 1400 fewer schools met test-score targets. Only 48% of California high schools meet federal standards, even ... Los Angeles Times Middle schools lead county's API scores The Salinas Californian Visalia Times-Delta - Record-Searchlight - Modesto Bee - Register Pajaronian all 106 news articles Publ.Date : Fri, 05 Sep 2008 07:41:11 GMT Haiti doubles hurricane death toll; US extends storm watches and ... - International Herald Tribune Boston Globe Haiti doubles hurricane death toll; US extends storm watches and ... International Herald Tribune - 51 minutes ago AP WILMINGTON, North Carolina: Tropical storm watches and warnings in the United States were extended from Georgia to areas just south of New York City on Friday as states along the Atlantic braced for Tropical Storm Hanna, which killed at least 137 ... Southeast states brace for Tropical Storm Hanna The Associated Press Hanna heads to Carolinas; Ike not far behind CNN International guardian.co.uk - StarNewsOnline.com - Greenville News - New York Times all 2,807 news articles Publ.Date : Fri, 05 Sep 2008 13:14:26 GMT So What If Sarah Palin Used a Speechwriter? - FOXNews BBC News So What If Sarah Palin Used a Speechwriter? FOXNews - 1 hour ago By Betsy Newmark Disdaining Sarah Palin’s speech last night because she had a speechwriter is one of the lamest criticisms of her performance last night. Video: Inside the Tent: Reactions to Sarah Palin's speech Part 4 ReutersVideo Panel Reviews Palin, Previews McCain RealClearPolitics Baltimore Sun - Detroit Free Press - New York Times - Los Angeles Times all 4,224 news articles Publ.Date : Fri, 05 Sep 2008 13:03:46 GMT Partisan Reception Greets Palin as VP Pick - ABC News CTV.ca Partisan Reception Greets Palin as VP Pick ABC News - 2 hours ago Sarah Palin is receiving a highly partisan reception on the national political stage, with significant public doubts about her readiness to serve as president, yet majority approval of both her selection by John McCain and her willingness to join the ... Video: Joe Biden On Palin's Speech CBS Obama Camp Turns to Clinton to Counter Palin New York Times Boston Globe - AFP - Evening Bulletin - USA Today all 1,899 news articles Publ.Date : Fri, 05 Sep 2008 11:18:50 GMT McCain honors NH soldier in convention speech - Boston Globe BBC News McCain honors NH soldier in convention speech Boston Globe - 1 hour ago ST. PAUL, Minn.—In his speech to the Republican National Convention, presidential candidate John McCain paid tribute to a New Hampshire soldier killed in Iraq. Video: "Change is coming," McCain tells convention - 05 Sept 2008 AlJazeeraEnglish US media mixed on McCain speech BBC News Marin Independent-Journal - Memphis Commercial Appeal - Today's TMJ4 - Detroit Free Press all 1,008 news articles Publ.Date : Fri, 05 Sep 2008 12:43:54 GMT Thai PM buys time with referendum plan - Reuters BBC News Thai PM buys time with referendum plan Reuters - 4 hours ago By Darren Schuettler BANGKOK (Reuters) - Prime Minister Samak Sundaravej has bought some time with a referendum aimed at defusing street protests, but it will do nothing to resolve Thailand's fundamental political conflict, analysts say. Video: Locking horns over politics in Thailand -- 05 September 2008 AlJazeeraEnglish ANALYSIS: Thai demonstration raises democracy issues Monsters and Critics.com AFP - Times Online - Aljazeera.net - Inter Press Service all 3,754 news articles Publ.Date : Fri, 05 Sep 2008 09:33:48 GMT Add Data Feed Content to Your Website Amazon Associate Feed

PARLOT::Ebooks, Scripts, Websites, and more...

Adsense websites

Your Company Without Training - Any Questions?

Okay, be honest!Are you guilty of sticking in a few... Read More

10 Fool Proof Ways To Intensify Your Profits

1. Create benefit intensifiers for your list of ad copy... Read More

Communicating When A Crisis Strikes

How would you handle communication if your business or practice... Read More

Business Opportunity - Leverage Your Employees!

As a business asset, they don't sit well on the... Read More

Knowledge Management - Creating a Sustainable Yellow Pages System

How can I "know who knows" None of us can... Read More

Learn to Assert Yourself

Pinpoint your own blocks to assertiveness: fear of disapproval, need... Read More

The Seven Cs: Partnership Danger Signs - Communication Breakdown

An ongoing series of articles exploring the seven critical areas... Read More

Birds of a Feather May Be Turkeys

Birds of a Feather May Be TurkeysBy Gene Griessman, PhD... Read More

Innovation Management ? Flexibility

Creativity can be defined as problem identification and idea generation... Read More

Stomp Out Leeches In Your Company

Recently, I felt appalled to see a political candidate running... Read More

Success: A Wholistic Perspective

The world has always been mixed with the... Read More

Coaching Employees in the Workplace

After a full week of training, you are still a... Read More

Factoring Financing: How to Grow Your Business Without Debt or Loans

What is factoring?Accounts receivable financing, also known as factoring, is... Read More

Why Business Owners Need Security

The main reason is to stop any potential lawsuits from... Read More

Joint Accountability: Another Key for Your Effectiveness

I once was part of a group of management professors... Read More

Your Organization Is Only as Good as Your People

Let's begin by singing the jingle from an old US... Read More

Why Half of All Mergers Fail After the Honeymoon Ends

Marriages and corporate mergers in America have at least one... Read More

Creativity and Innovation Directors, Consultants and Managers

Creativity and Innovation is essential for competitive advantage, yet the... Read More

Keeping Team Meetings Vibrant

Meetings can be the life-blood of an organisation or the... Read More

Thoughts on Empowerment

People are empowered when they are given the authority and... Read More

Understanding How Change Has Changed

There seems to be an assumption that change is a... Read More

Setting a Pitiful Example: Twenty-six Warnings to Heed *

TO: All Parents [and Employers and Managers]From: Your Child and/or... Read More

How Your Business Can Pick A Software Developer

Eventually, your business is going to need to have some... Read More

Introducing the 15 Frameworks of Successful Self-Employed Professionals

A framework is a way of thinking, a point-of-view, a... Read More

How to Save Money on Training

1) Use a live instructor. Adults learn best by doing,... Read More