Archive for the ‘measurement’ Category

Complex business problems need diagnosis, not packaged solutions

Monday, January 14th, 2008

Dave Snowden, whose posts are always interesting and instructive, says this in a post today:

What I am finding is that the more accurately you can describe the situation, the less you need formal intervention methods. For example if I can show a statistically valid trend, supported by narrative then most people in leadership or management positions can work out what they need to do.

In other words: if you fully understand the problem, you don’t need complex managerial methodologies to solve it. Over-relying on “five practices” or “seven habits” or “four steps” amounts to short-cutting the real difficulty of understanding complex situations. There are lots of reasons, which Dave describes well in his post, why best practices and case studies are not good guidance for action in most circumstances.

And it is difficult to know what the problem is in a complex environment. Standard assessments & surveys demonstrate this. Asking a thousand employees, “How innovative are we on a scale of one to five?” produces, at best, a pretty-looking chart that signifies nothing. At worst, it can point you in a completely-wrong direction. But everyone wants a short cut.

Exhibit A: the current romance with performance dashboards. In my experience, at best dashboards point you to a situation that you need to understand more deeply (”Is this drop in region 5 sales an anomaly, or is there something substantial behind it?”). In no circumstances is dashboard information enough to act on.

So the need is more for problem-understanding skills, and less for problem-solving skills–meaning managers will need to get more in tune with narrative sense-making. Here are two examples where intelligent executives looked beyond the data, into the narrative mess behind a problem or dilemma, and used the story that emerged to guide their actions:

1) A.G. Lafley of Procter and Gamble, as described in the book “The Opposable Mind“:

While trying to decide to whether to roll out highly-concentrated laundry soap, Lafley faced a dilemma. Merchants loved the idea, but consumers, as measured by P&G’s quantitative research, were neutral to negative. In most cases, this would answer the question: don’t roll out a product that consumers didn’t love. But Lafley didn’t accept the data at face value:


…[He] decided to dive into the voluntary comments that some of the consumers added to their quantitative research forms…. Lafley took many evening and weekend hours to pore over more than four hundred handwritten voluntaries. He came to the conclusion that while consumers weren’t wildly enthusiastic about compact detergents, few were actively hostile….


And, as a result, Lafley decided that the product could be a success. And, of course, it was. (Stay tuned for a review of “The Opposable Mind” later this week.)

2) A Fortune 500 VP of Human Resources, evaluating a new performance appraisal system, as described in this blog:

Last year we had a pilot of a new performance management system for our employees. The trial group was 4000 people. We had spent a lot of time on the pilot and gathered a lot of data. At the end of the trial, the VP of Human Resources printed out all the comments that had been received on the survey forms. He took them home one night and read every single one. Then he came in the next day and said, “We can’t roll this system out.” And that was it. The trial was very expensive. We’d gathered lots of data, lots of numbers, but the final determinant was what he read in those comments.

One observation about the above two incidents is that reading narratives and evaluating qualitative data, if done at all, are relegated to nights and weekends.

This is perhaps a sign of some of the difficulties we face in the corporate world–you can spend all day looking at charts and spreadsheets, but read stories on your own time!

, , , ,

Sports Analogy week day 2: no mulligans

Tuesday, August 14th, 2007

I played golf with my brother-in-law a few years ago. He hadn’t played much and was struggling. But no matter where his drive went, or where he found his ball, he would not take a mulligan or improve his lie. “Go ahead and move the ball,” I told him. “Why suffer?”

“No thanks,” was his reply. And he hacked the ball back into the fairway.

That round made a big impression on me. For one thing, I was a poor-to-fair golfer and I’d take mulligans from time to time, or take a favorable drop, or the like. But what my brother-in-law taught me that day was that I would never know precisely how well I was doing at golf if I didn’t follow the rules.

And if I didn’t know how well I was doing, it would be hard to improve. I had to confront the brutal facts (as stated in “Good to Great”–wait, didn’t I discredit this book just last week??) before I could move forward.

I tried to take that lesson into my business life as well. How were things going, really? Was I using excuses to mask weaknesses I myself needed to address?

In short, to be great at business you can’t take mulligans. You’ve got to face reality and try to improve.

It was a very valuable day on the golf course. After I cut down on the mulligans and played the ball where it lay, my scores, predictably, got worse. Over time they’ve come down. And, best of all, they’re real.

(Photo: “Tee Time 3″ from Garrison Photography via stock.xchng–note: NOT my brother-in-law)

Most Significant Change – an update

Sunday, July 15th, 2007

A couple of months ago, I posted on Most Significant Change, a qualitative method to assess complex initiatives. Read here and here for some background.

Rick Davies, the developer of the technique, was kind enough to post a comment on the original post with some resources for those interested in MSC. Here it is in its entirety:

Hi John,

You and your readers might also like to know that there is an international email list, of 750+ people, who share information about the use of MSC. The members are mainly those using MSC in Africa and Asia, but there are also people from Europe, Australia and USA. People can join the list via this link:http://groups.yahoo.com/group/MostSignificantChanges/

I have also recently posted details about a related method of constructing stories, using a participatory process, called “Evolving storylines: A participatory design process?” at http://mandenews.blogspot.com/2007/05/evolving-project-designs-participatory.html

regards, rick davies

, , , , ,

More Most Significant Change (MSC) – how to use for business?

Thursday, April 19th, 2007

In an earlier post, I introduced the concept of Most Significant Change, a monitoring and evaluation technique that was developed for international aid programs. In short, MSC gathers stories and anecdotes on the impact a program has made for its clients, then sorts and selects them through several levels until a single story is selected that represents the most significant change brought on by that program.

So, in business, how would you use it? Let’s take the example of the new performance evaluation program that we discussed in the earlier post. Let’s say that the company used MSC to monitor this program. HR folks at each division collect stories, then via dialogue they select the most significant in their division; it’s passed up the chain, and a further selection is done, and so on, until the executive team selects one story most significant for the whole program. In our example, here’s the story they chose:

The most significant change for me, I guess, is that my boss never used to talk to me about performance until review time. Then, you know, I was usually in for a surprise-good or bad. Between reviews, nothing. So, then, after the new system was put in place, it took a few weeks, but all of a sudden she started talking to me about how I was doing as part of our regular one-on-one meeting. Things I was doing well, other things I should do differently. So I could fix what I was doing right away–and not get slammed in a review months later. More than that, a month or so after the program started, she came up to me and said, “Jim, this project plan you sent me to approve, it looks OK, but when you do your project plans, it would be more effective if you put the name of the responsible person alongside each task. That way, if there’s a schedule slip, I know who I might need to follow up with.” That’s a pretty big change compared to what had happened before.

What does this story say about the program? It says that, at least in one case, it spurred a manager to provide feedback on a timely basis. (Disclosure: like the vast majority of technology managers, I am not very good at this aspect of personnel management.)

The story doesn’t say that the program spurred this improvement for every manager–or even more than one. However, its selection as the most significant story tells everyone that this is the kind of behavior that the program was intended to foster. Which has three big benefits:

  1. Clear communication – one very important program outcome is communicated to everyone in a very tangible, usable way.
  2. Enables action – Management can now monitor whether this desirable change is a trend or an isolated case, and intervene accordingly.
  3. “More like this” – The people who select the most significant stories are cued to look for stories like this as they do their selections.

And it’s repeated on a regular basis, say every quarter or half-year. Over time, it shapes the organization’s view of the project and its objectives, far better than a bland statement (”We intend to improve the effectiveness of management in the areas of personnel appraisal and feedback, blah blah blah”). And it brings the management teams together regularly for intense dialogue about important issues, and causes them to make a joint decision (which is rare for lots of management teams).

Pretty useful, eh?

(Illustration from the MSC Guide by Rick Davies and Jess Dart (c) 2004, via Inforesources.)

, , , , ,

An anecdote on the power of business narrative analysis

Friday, April 13th, 2007

I had lunch with a former colleague yesterday, whom I hadn’t seen in ten years. She related to me this story:

Last year we had a pilot of a new performance management system for our employees. The trial group was 4000 people. We had spent a lot of time on the pilot and gathered a lot of data. At the end of the trial, the VP of Human Resources printed out all the comments that had been received on the survey forms. He took them home one night and read every single one. Then he came in the next day and said, “We can’t roll this system out.” And that was it. The trial was very expensive. We’d gathered lots of data, lots of numbers, but the final determinant was what he read in those comments.

I had been talking to my friend about collecting and analyzing business narrative to assess organizational change using Anecdote’s methods. I told her, “He didn’t use a system, and comments aren’t necessarily stories. But, in essence, your VP was using the narrative technique.”

, , ,

What in hell is Most Significant Change?

Thursday, April 12th, 2007

Let’s first pose a problem. You’ve put in place a new performance evaluation system and spent a year conducting reviews using it.

How’s it working out for you?

Your program will have target objectives like improved employee satisfaction, perhaps, or increased personnel retention. But the first indicator is difficult to baseline and measure, and the second requires a long time to discern a change.

Complex initiatives…vague indicators of success…the hunger to know whether something costly and labor-intensive was worthwhile.

International aid programs face this problem every day, and from that domain has emerged a new method for monitoring these types of programs.

It’s called Most Significant Change. (Here’s the official guide, written by Rick Davies, the creator of the approach, and Jess Dart.) At its most basic, this deceptively simple approach asks field workers (or first-line managers in the business context) to elicit anecdotes from the people affected, focusing on what most significant change has occurred as the result of the initiative, and why they think that change occurred. These dozens or hundreds of stories are passed up the chain and winnowed down to the most significant, as determined by each management layer, until finally one story is selected.

It’s not numbers, or graphs, or ROIs. It’s a story, with an explanation, and behind it a collection of other significant stories. A story that describes a real experience, reviewed, defended, and selected by the people charged with the success of the program.

Why can this monitoring method work for programs like our performance evaluation system? I’ll discuss that in another post.

, , , , ,