Tag

measurement

Is the impact of Agile just a Hawthorne Effect ?

By | Agile, Communication, Strategy | One Comment

L1001888The Hawthorne Effect is a human behavioural theory drawn by various social scientists from trials undertaken between 1927 and 1932 at Western Electric’s Hawthorne Works.

The trials found that improvements in behaviour, and productivity changes observed after adjusting a workplace, were in fact largely just the results of a placebo-type effect – any change seemed to cause improvement.

There were many changes trialled – the first (and possibly most important) were changes in lighting levels (‘the illumination experiment’) in a factory environment. The objective of the experiment was to find the optimum lighting level for productivity – the results showed that more was at work than just changing lumens, every change (up or down in brightness) caused an increase in performance.

Most theorists since have concluded that the improvements are caused by the act of measuring and engaging subjects.  If you missed studying this in your management 101 course, then read this article or take the slightly incomplete wikipedia catch up lesson and then continue reading.  Lets deconstruct the impact of this theory into its parts:

Measurement

We’ve already written about the impact of measurement before.  It’s also pretty well understood that humans perform against whatever KPI you give them, the way they do it may however be surprising or unintended.

Confirmation Bias

Then there is Confirmation Bias, basically the likelihood that you will both find what you’re looking for, and ignore stuff that you’re not.  Elegantly illustrated by this group of Radiologists who can’t find a Gorilla in their scans.  If a team expects a new process to be better, then their perception will likely match their expectation.

Empowered Staff

Finally the staff in the later trials, more focused on workplace layout and reward were engaged, they were part of the process.  They were made to feel special.  We’ve talked about this a fair bit too – though Dan Pink is more fun. Smart people are motivated by Autonomy, Mastery and Purpose. Pink was not the first to the conclusion that money was directly linked to productivity.

In summary, if you measure stuff, look for a change, and then make people feel special – surprise, surprise, you’re going to see a change!  Does this mean that some of the impact of Agile, the dramatic increase in staff engagement and useful productivity are perhaps just caused by the visual changes in the offices and agile rituals, but, in fact are just a ‘Hawthorne Effect’ ? In my heart, I have to say nervously, yes, at least to some extent.

Is this bad ? No.

Ok, but is the change real and sustainable? Yes, the changes and improvement in your team’s performance and culture are real.  Like all placebos, the source of the change might not be real, but the impact is.  If we can come up with better or different ways to achieve the same improvement then we should try them… any good agilista will tell you that anyway.

In the mean time I’ll leave you thinking about a sobering equation Nigel sent me as we’ve been debating this recently.

Hawthorne > Maslow + Deming

hawthorne effect workplace

“You can’t control what you can’t measure” – Tom DeMarco recants on his famous principle.

By | Agile | 2 Comments

Tom DeMarco

Governance and funding of business investment using agile development methods are two hotly debated and misunderstood topics among the agile community.

With governance principles originally borrowed from the exacting fields of construction engineering and accounting, where things can often be calculated to 2 decimal places, it seems weird that we more often than not disappoint our customers by delivering a different result to that which was blueprinted or planned in software ‘engineering’.

Fred Brook’s 1975 treatise The Mythical Man Month was probably the first to hint at the

core differences between engineering a bridge versus the knowledge work of solving a problem by creating code. In the 20th anniversary edition he added an extra commentary entitled ‘No Silver Bullet’ in which he further reflected on the shortcomings of applying a mechanical engineering metaphor to software development. It is a must-read book for everyone involved in agile or lean development – and if you’ve read Clay Shirky‘s 2008 book Here Comes Everybody you’ll recognise some early sources of Clay’s work on complexity of organisations.

By the way Clay Shirky also has my favourite definition of governance – ‘rules for losing‘.

Fred aside, one of my greatest heroes of the software world is Tom DeMarco, who wrote the seminal text on project and engineering control called Controlling Software Projects: Management, Measurement, and Estimation (Prentice Hall/Yourdon Press, 1982). But this one is a book you probably do not want to own. Why? Well, approaching 70, Tom gives us the benefit of his time to reflect in the newsletter of the IEEE Computer Society in 2009, downloadable here:

http://www2.computer.org/cms/Computer.org/ComputingNow/homepage/2009/0709/rW_SO_Viewpoints.pdf

My favourite quote, which describes in a nutshell why measurement and management aren’t as closely linked as he surmised 27 years earlier:

“Imagine you’re trying to control a teenager’s upbringing. The very idea of controlling your child ought to make you at least a little bit queasy.

Yet the stakes for control couldn’t be higher… now apply “You can’t control what you can’t measure” to the teenager. Most things that really matter–honor, dignity, discipline, personality, grace under pressure, values, ethics, resourcefulness, loyalty, humor, kindness–aren’t measurable.”

And a lot of product deliveries and projects I’ve been involved with acted way more like moody, irrational, changeable teenagers than respectable septuagenarians.

You are what you measure

By | Agile, Development | One Comment

When you put a volt meter into an electrical circuit to measure the voltage difference between two points the internal resistance of the meter itself changes the circuit and impacts the reading. The best volt meters are designed to have very very high internal resistance, but in all circuits the resistance of the volt meter changes the circuit and impacts the measurement (circular yes!); in sensitive low voltage circuits where accuracy is most important the impact is greatest.

One of the agile development teams I’ve worked with decided that they would use the number of cards as their measure for planning, velocity and by implication productivity. Each iteration the cards from the last release were proudly pinned to the wall and counted. An interesting pattern emerged over a number of iterations: the number of bug cards completed went up, and up and up.

We spent a great deal of time trying to work out where our development, testing and release process was going wrong that we were seeing so many old bugs. Were we finding old issues with more thorough testing? Were the developers being lazy? Or was the work we were tackling just much tougher than previous projects? It turns out that the explanation was in fact very simple. Previously developers found and fixed bugs as part of their development tasks (this is quite normal), now they were writing these up as cards and running them across the board through the full prioritisation and QA process. Consciously or subconsciously the developers had been sent a signal that the number of cards was being measured, and thus they responded to and optimised their process to meet that benchmark. Issues which would have been fixed quickly while the developer was deep in their context now became tasks of their own, a much more expensive and unnecessary process.

Perhaps a better title for this post might be ‘Only measure what you truly wish for’. This is a topic which we’ll surely return to again and again, but I’ll leave you with a quote I write into the front of every one of my notebooks to remind me of a valuable truth.

“Not everything that counts can be counted, and not everything that can be counted counts.” – Albert Einstein.

Subscribe to the Luna Newsletter

Quote of the week

The new competitive advantage is the ability to anticipate, respond and adapt to change.

Recent Luna Posts

Become Remarkable.