Google+ Followers

Tuesday, September 02, 2014

Agile outside of software

Later this week I’m doing a presentation at Agile On The Beach entitled: “Agile outside of software development”. (I resisted the temptation to call it “Agile Beyond Software”). The presentation will attempt to answer a question which is often asked at, and around, Agile On The Beach: “Is Agile only for Software Development?”

In truth it is not just at Agile On The Beach (AOTB) that this question is asked, I hear it more and more often elsewhere but the origins of AOTB, and target audience for AOTB, mean that the question is very much on topic.

I’ll post the presentation online after I have delivered it - until then I’ll be tweaking it - but right now I’d like to (kind of by way of rehearsal) run down my main argument, so here goes….

Before the term Agile Software Development was coined there was “Agile Manufacturing”. This was this term that inspired the software folks so perhaps the first answer is: “Yes! Of course Agile works outside of software because that is where it came from.”

But, Agile Software Development has far and away surpassed Agile Manufacturing in writing and mindshare. So much so that others in the wider business community have looked at Agile Software Development as a model for other types of working. In a way, “Agile” has come full circle.

Perhaps it is worth pausing for a moment and asking: “What do we mean by Agile?” or “What do we want from an Agile company?”

This could be a big big debate, ultimately it is for each company, each team, to decide what Agile means to them. Rather than have that discussion let me quote MIT Professor Michael A Cusumano:

“I can’t think of anything more important than building an agile company, because the world changes so quickly and unpredictably…

[Agility] comes in different forms, but basically it’s the ability to quickly adapt to or even anticipate and lead change. Agility in the broadest form affects strategic thinking, operations, technology innovation and the ability to innovate in products, processes and business models.” (MIT Sloan Management Review Summer 2011 interview with Hopkins)

Lets add to that a little bit by defining Agility from three views:

  • Agile Strategy: Adaptability, listen to customer, leading the market, use change competitively
  • Agile Tactically: Experimenting, “Expeditionary Marketing”, live in the now while preparing for the future
  • Agile Operations: Deliver fast, deliver quality, deliver value

We could go on but thats enough for now. Now we have an approximate understanding of what we want we can go back to the original question: “Is Agile only for Software Development?”

There are three parts to this answer: Practices, Roots and Case Studies.

Practices

Lots of the practices associated with Agile actually come from elsewhere. Examples of stand-up meeting proliferate - bars, healthcare, the military, Japanese local Government and so on. Many companies operate regular status or planning meetings, Agile just elevates this practice. WIP limits are well established in manufacturing.

Agile picks up some practices directly - visual boards (“information radiators”) are nothing new.

Some practices it picks up and changes - Retrospectives have long existed as “Lessons Learned” or “After Action Reviews.”

Some practices Agile (might have) invented - Planning poker but this is itself version of wide band delphi.

And some Agile plays back to the business - TDD and BDD are writ large in Lean Startup (which is itself an extreme version of Expeditionary Marketing from 1991, Hamel & Prahalad).

Roots

As already mentioned: Agile Software Development was inspired by Agile Manufacturing.

I’ve described my Agile Pyramid model before (Agile and Lean - the same but different, How do you make Lean Practical?) and my argument that “Agile is Lean thinking applied to software development” and I wrote whole book saying the software development, and Agile software development specifically, is an example of Organizational Learning.

Well, Agile Software Development is not the only application built on these foundations. Obviously the Toyota Production System is. And so too is its close cousin the Ford Production System. Look further afield and you will find Last Planner in construction. I could continue but I think my point is proved.

While not every Agile practice can be taken out of software development and used someplace else the roots of Agile mean that the principles, values and ideas which Agile is built on can be. In your domain Agile as now known might work quite well, but in someone else’s domain there may be more need to think deeper.

One caveat: I believe the more deeply you look at Agile and more Agile is applied outside of the software development the more it looks like Lean. Ultimately the distinction between Lean and Agile breaks down in my book.

Case Studies

Good news: There are case studies of teams using Agile outside of software - and if you know of any more please tell me! Add them in the comments on this blog post.

Bad news: There are not that many case studies. Unless you are a software team you probably can’t find one.

For the Agile on the Beach conference I have deliberately sought out Agile outside of software development in the last few years. This has resulted in two good examples.

Two years ago Kate Sullivan described how the Lonely Planet legal department adopted Agile working. In fact Kate said all of Lonely Planet adopted Agile.

Last year Martin Rowe talked about how he used Scrum to manage the Foundation Computer Science course at Plymouth University.

Elsewhere, six years ago the MIT Sloan Management Review carried a piece by Keith R. McFarland entitled “Should you build strategy like you build software?” in which he described how Shamrock Foods of Arizona used Agile to plan and execute their business strategy.

Myself, I have seen the marketing team at Sullivan Cuff Software Ltd use an Agile like approach.

And last year I helped the GSMA (the people responsible for SIM cards) use Agile on a project writing a document for cellphone manufacturers, mobile network operators and umpteen suppliers and partners to allow mobile phones to be used for loyalty coupons.

So, back to the original question: “Is Agile only for Software Development?”

Answer: No

Question: Will Agile work outside software development?

Answer: Yes

But, the detail may vary.

Finally, and please excuse the pull. As a result of this discussion my company has adapted its successful Foundations of Agile Software Development 2-day course for companies outside software. Please take a look at Agile Kick-Off (for non-software teams) and let me know what you think.

Wednesday, August 27, 2014

Book: Scaling up Excellence

In the early days of this blog I used to regular blog book reviews. As I found ideas of my own to blog about - and found fewer books I wanted to read - so book reviews ceased. But right now I want to blog about a book because I want to recommend this book.

The book is: Scaling Up Excellence by Robert Sutton and Huggy Rao.

I’ve been a fan of Sutton’s work for a while now - specifically The Knowing Doing Gap. In this book he continues some of that theme (“Why don’t we do what we know is better?”) but he looks specifically at scaling.

Now scaling is a hot topic in software development (I wrote a little about this myself, Scaling Agile Heuristics - SAFe or not!) so its a book which should find a ready audience in the software community. Although I sometimes feel this is a book about change and on the whole I’m no longer a fan of books about change (there lies a long story for another blog). As a book about change what makes this book worth reading is that it is particularly focused on scaling up changes.

I hope this book becomes more widely known in the software development community, and particularly those concerned with “Agile” and “Scaling Agile.” For now here are some of the key points I want to remember from the book and I hope will wet your appetite:

  • Sutton and Roa suggest “the problem of scaling” can also be defined as “the problem of more”. For example: how do we get more teams working in this excellent fashion? Or, how do we get this excellent team to be even more excellent?
  • “In order to scale excellence you need some excellence to scale”: as I said in my post last year: If you can’t do it in the small then you can’t do it in the large.
  • They introduce the idea of Buddhist and Catholic change models: in the Buddhist model the aim is to help every one/team come to their own understanding of improvement while in the Catholic model the aim is to help every one/team replicate an existing example of excellent. They don’t suggest either one is inherently better but they do suggest that sometimes one model if the context better.
  • Continuing on the Catholic theme, they also spend a lot of time discussing standardisation. This is something the has troubled me in the past - I don’t like standardisation, I naturally lean towards the Buddhist model. But I also feel I need to think more about this topic, this book has given me more to think about, watch out for a future blog.
  • There is good discussion of the role of hierarchies
  • Scaling is a marathon, not a spring

They are just a few bullets from the first chapter or two but I could go on through the whole book. My (e)book copy is covered in highlights and comments.

Let me add a story of my own.

Early on in the book they give three reasons why scaling initiates often fail. I think these three are worth considering in the Agile domain:

  • Illusion: leaders believe scaling will happen more easily than the facts suggest
  • Impatience: lenders scale too soon
  • Incompetence: leaders start scaling without really understanding the thing themselves

Arguably these three things are intertwined and overlap. The third one reminds me of some bank executives I heard speaking a few months ago. The 4 or 5 senior bankers were talking about why they loved Agile and why they wanted more people in the bank to adopt Agile. They were good. They came across as honest. Then they took questions.

The first question was: “Agile says collocated teams work best, how do we reconcile this with a bank policy of remote working and offshore development?”

Rather than say: “I understand your concern, this is something we are working on and we need your help, we might need to change policy or find new ways or working” the executive replied: “With modern tools this isn’t such a big problem any more…” Immediately it seemed that not only did the banker not understand Agile and why Agile said this but he set out to show that as a banker he understood more about modern technology than the technologist asking the question (who was wrong.)

Shortly afterwards someone said: “How do we reconcile regulation with Agile?” and a senior managers then started to wrap themselves in knots about allowing teams to choose Waterfall or Agile during which time one of them said: “Look, as long as your documentation is in order you can do what you like.” Again they demonstrated a lack of understanding.

With those two answers the managers not only shot themselves in both feet but destroyed all the credibility they had spent 30 minutes gaining.

Rather than continue, let me just say, if scaling (anything) interests you then read it - Scaling Up Excellence.

Monday, August 04, 2014

Bug Strategies & Xanpan volume 2

The summer lull has finally allowed me to get a lot done. In particular I’m very pleased to have finished a piece of writing I began some months ago: Bugs - Management Strategies.

This is an important piece for two reasons. Firstly I’ve been wanting to write this essay for a few years but never found the time. It has taken about four months in total and it runs to about 30 pages, now I know why it took so long.

Secondly, this is one of the corner stones for Xanpan volume 2. Volume 2 was going to be subtitled “Management Heuristics” but I’m now thinking it will deal more with organising so the sub-title might become “Management Heuristics and Organisation” or “Heuristics for organising software development” or some combination.

When I have a couple more chapters of volume 2 in place I’ll make it available on LeanPub. Until then you can register your interest in volume 2 right there.

In the meantime please read “Bugs - Management Strategies” and let me have your feedback. At this point I’m really posting it to get some feedback.

And if you haven’t already bought and read Xanpan volume 1: Team Centric Agile Software Development please do today!

(And if you prefer the a hard copy then get Xanpan at Lulu rather than LeanPub.)

Sunday, August 03, 2014

Introducing the Agile Basics video series

In the last year I’ve had a lot of people ask whether I do online training, and I’ve had several of the partner organizations I work with suggest I should do more video training courses. I’ve even heard stories that some people can make good money doing this.

My immediate reaction is doubt. Doubts because so much of what I do is interactive, exercises, following questions and I don’t know how to translate that into pre-recorded videos or other online material. If my courses were just me talking to the audience then maybe, but thats not what I do in my Agile training courses.

But really: I just don’t know. I don’t know what works and what doesn’t online. I don’t know how to approach videos, how much work is required? How much editing? Should I get a professional involved?

So I decided the best thing to do was to experiment. To find out for myself. O yes I could read a lot about what works and what doesn’t online but until I’ve actually tried its all a bit abstract.

Back in April I posted a “Xanpan Board Tour” video.

And in May ran a webinar with DevelopMentor which was recorded: Agile Basics - 40 minutes plus 20 minutes Q&A.

I learned from both of those, I got more confident. So I can now announce the results of my summer project: The Agile Basics video series.

I took the webinar I did in May, broke it into separate pieces and added some more material. The result is five episodes (quality, iterations, visualise, work in the small and teams) each of which has short video, 10 minutes approximately, plus an introduction and conclusion video - about 4 minutes each.

I hope people will find these useful, its just possible that some people having seen my recordings will decide to hire my services. More likely I imagine people who have been on my courses can use these to revisit some of the topics. After all, YouTube is full of similar “Agile Basics” videos.

But the person who has got most out of this is: Me.

I’ve learned a lot more.

I’ve learned that videos require work. Just like working in another medium - specifically writing - they require preparation, they require time to do the work and they require time to edit and change the work. If I am going to continue working in video then I need to practice these skills and learn more. And just perhaps I need to start working with more experiences, even professional, people.

Anyway, if anyone is interested the videos are available to watch for free either from the Software Strategy website or directly on YouTube.

And if you have a view on whether I should work more in video, or whether these videos have value, or suggestions for how to work better then please, as always, let me know. Getting feedback is hard!

Monday, July 28, 2014

Nightmare on Agile Street

I’m awake. I’m lying in my bed. I’m sweating but I’m cold.

Its the small hours of the morning and the dream is as vivid as it is horrid....

I’m standing in a clients offices, I’ve been here before, I know whats happening.

They are building an website. Quite a complex one, this will be the primary purchasing venue for many customers. This will project the company image - and with the right bits it can up-sell to customers - it can even help reduce costs by servicing the customers after the sale. All good stuff.

But it is atrociously “behind” schedule, someone said it would be finished in a year, that was three years ago before any code was written. Now its two years to completion but in my dream people say 2+3=2.

How can that be?

I can’t say it but the only way out I can see is cancellation. If I was suddenly in charge of the client I’d cancel the thing. I’d salvage what I could and I’d launch a new, smaller, initiative to replace the website.

But its too big to fail, even the board knows how much money they are spending. Who’s going to walk in there and say: “Scrap it.”

Saying “Scrap it” would be to admit one failure and invite a messenger shooting.

And if I was the head of the supplier I’d say the same thing. I’d say to my customer: “I know I’m earning oodles of cash out of this, I know its a high profile feather in our cap but really its out of control you really shouldn’t continue.”

But of course they won’t.

Forget the money they’d lose, they weren’t hired to answer back - like my tailor friend.

And of course I’m neither of those. I’m just the guy having the nightmare and in the nightmare I’m the consultant who is trying to fix, it. In the nightmare I’m not fixing it I’m providing cover, while I’m there its Agile, while its Agile its good, Agile is a good drug and I’m the pusher.

“You can’t cancel it because all the competitors have one and so we must have one” tells me a ghostly apparition.

“We must be best in class” says another apparition.

“We must be head-and-shoulders above the opposition” says third - aren’t the opposition seven times the size? And don’t the competition buy large parts of their solution off the shelf?

But every time I look the work seems to grow. Every discussion ends in more stories. Not just stories, epics, super-stories, sub-epics, mezzanine-stories.

But its OK, this is Agile.

The business keeps throwing new requests at it which are just accepted - because they are Agile!

Some of these are quite big. But that’s OK because the team are Agile. And Agile means the team do what the business want right?

I watch the Analysts work over the stories in the backlog, as they do each grows and replicates like an alien parasite. The Analysts find more edge cases, extra detail which need to be included, more scenarios which need to be catered for. Each becomes a story itself.

But that’s OK because the team are Agile.

And those damn competitors don’t stop adding and improving their site which mean the client must add that too.

But that’s OK because the team are Agile.

And the points…. points are the new hours, in the dream I have a book “The Mythical Man Point”. The backlog is measured in thousands of points. The burn-down charts go down - but only if you look at the sprint burn-down, hunt around Jira and you can find a project wide burn-down, O my god, no….. its full of stories!

This is not a burn-down chart carrying us to a safe landing, its a fast climbing interceptor…

The backlog is a demon… its… its… undead.

The faces of those who’ve seen the chart are prematurely aged. Open Jira and show someone the chart and…. their hair turns grey, the wrinkles appear, in moments they are….

One man is immune. As the points grow his power grows, he is… he is... The Product Owner.

He introduces himself: “Snape is the name, Severus Snape” - I knew I’d seen him somewhere before.

In the planning meeting, he sees the poker cards pulled out, he focuses on the developer with the highest score, there is a ray of cutting sarcasm… he withers. The developers submit, the numbers are lowered. The Product Owner chuckles to himself - no over estimating on his watch!

One of the developers suggest “Maybe we should wait until we finish the current work”

Snape sneers: “I thought you were Agile boy?”

“If you can’t handle it I have some friends in Transylvanian who are really Agile…. do you want to lose the contract boy? … Off-shore is so so cheap…”

There is a reality distortion field around the Product Owner. Show him a burn-down chart and it looks good, his presentations to the steering committee always show perfect burn-down.

I’m in my pyjamas standing outside the building at night: a sinister looking figure is breaking and entering, he sneaks into the building, he opens Jira and … inserts stories! His mask falls, it is….The Product Owner! Of course, without stories in the backlog he would cease to exist, his power comes from the size of the backlog, more stories more power.

Ever since his boss came down with a rare form of chronic flu a link in the reporting chain has been missing. Made worse when the next man up was dismissed for inappropriate behaviour in the canteen. Since when the Product Owner reports to the COO, a COO who doesn’t really have time for him and only has a shaky understanding of any IT related topic.

I do the maths. The backlog isn’t so much a backlog as a mortgage, and the team are under water! The payments they make against the mortgage aren’t even covering the growth in stories. The backlog growth is an interest rate they can’t pay.

It takes months for stories to progress through the backlog and reach developers. When work finally gets to developers they too uncover more edge cases, more details, more scenarios, more of just about everything. Why didn’t the Analysts find these? Did they find them and then lose them?

Then there is a stream of bugs coming in - oozing through the walls. The technical practices aren’t solid, they are… custard! Bugs get caught but more get through!

Bugs can’t be fixed because: “bugs are OpEx and we are funded from CapEx.”

Someone has slain the Bug Fixing Fairy, her body is found slumped in the corner, a nice your girl straight out of college. They are hiring another fresh young graduate to send to the slaughter, fortunately Bug Fixing Fairies are Plug Compatible with one another.

Release dates can’t be honoured.

Woody Allen and Anne Hall walk in - since when did Woody Allen do horror films?

‘two elderly women are at a Catskill mountain resort, and one of 'em says, "Boy, the food at this place is really terrible." The other one says, "Yeah, I know; and such small portions.”’

I have X-Ray vision: I can see WIP where it lies, there are piles of it on the floor. Its stacked up like beer barrels in a brewery. But the beer isn’t drinkable. Its a fiendish plan. If anyone drinks that beer, if the WIP is shipped, they will discover…. its full of holes! Quality control is… offshore.

Why is there so much WIP lying around? Why is the WIP rising?

Because they are Agile comes the reply… the business can change their mind at anytime, and they do.

I’m drowning in WIP.

WHIP to the left of me, WHIP to the right of me.

The developers are half way through a piece of work and the team are told to put it to one side and do something else. Nothing gets delivered, everything is half baked. WHIP - work hopefully in process that is.

When, and IF, the team return to a piece of WHIP things have changed, the team members might have changed, so picking it up isn’t easy.

WHIP goes off, the stench of slowly rotting software.

But that’s OK because the team are Agile.

Arhhh, the developers are clones, they are plug compatible, you can switch them into and out as you like… but they have no memory….

It gets worse, the client has cunningly outsourced their network ops to another supplier, and their support desk to another one, and the data-centre to the another… no one contractor has more than one contract. Its a perverse form of WIP limit, no supplier is allowed more than one contract.

O my god, I’m flying through the data centre, the data centre supplier has lost control, the are creepers everywhere, each server is patched in a different way, there is a stack of change configuration requests in a dark office, I approach the clerk, its its…. Terry Gilliam, the data centre is in Brazil….

Even when the business doesn’t change its mind the development team get stuck. They have dependencies on other teams and on other some other sub-contractor. So work gets put to one side again, more WIP.

All roads lead to Dounreay in Scotland, a really good place if you want to build something really dangerous, but why does this project require a fast breeder nuclear reactor?

But that’s OK because the team are Agile.

The supplier is desperate to keep their people busy, if The Product Owner sees a programmer who’s fingers are not moving on the keyboard he turns them to stone. The team manager is desperate to save his people, he rummages in the backlog and finds… a piece of work they can do.

(With a backlog that large you can always find something even if the business value is negative - and there are plenty of them.)

You can’t blame the development team, they need to look busy, they need to justify themselves so they work on what they can.

But that’s OK because the team are Agile.

Get me out of here!!!!!

I’m in my kitchen.

My hands are wrapped around a hot-chocolate, I need a fresh pair of dry pyjamas but that can wait while I calm down. I’ve wrapped a blanket around me and have the shivers under control.

Are they Agile? Undoubtedly it was sold as Agile. It certainty ain’t pretty but it is called Agile.

They have iterations. They have planing meeting. They have burn-downs. They have a Scrum Master and they have Jira. They have User Stories. They have some, slow, automated acceptance tests, some developers are even writing automated unit tests. How could it have gone so wrong?

Sure the development team could be better. You could boost the supply curve.

But that would be like administering morphine. The pain would be relieved for a while but the fever would return and it would be worse. The real problem is elsewhere. The real problem is rampant demand. The real problem is poor client management. The real problem is a client who isn’t looking at the feedback.

The real problems are multiple, thats what is so scary about the dream. They are all interconnected.

In the wee-small hours I see no way of winning, its a quagmire: To save this project we need to destroy this project. But we all know what happened in Vietnam.

What is to be done? - I can’t go back to sleep until I have an answer.

Would the team be better off doing Waterfall?

The business would still change its mind, project management would put a change request process in place and the propagation delay would be worse. There would probably be more bugs - testing would be postponed. Releases would be held back. This would look better for a few months until they came to actually test and release.

If they did waterfall, if they did a big requirements exercise, a big specification, a big design, a big estimation and a big plan they might not choose to do it. But frankly Agile is telling them clearly this will never be done. In fact its telling them with a lot more certainty because they are several years in and have several years of data to look at.

Agile is the cover. Because they are Agile they are getting more rope to hang themselves with.

But all this is a dream, a horrid dream, none of this ever happened.

Saturday, July 12, 2014

What we forget about the Scientific Method

I get fed up hearing other Agile evangelists champion The Scientific Method, I don’t disagree with them, I use it myself but I think they are sometimes guilty of overlooking how the scientific method is actually practiced by scientists and researchers. Too often the scientific approach is made to sound simple, it isn’t.

First lets define the scientific method. Perhaps rather than call it “scientific method” it is better called “experimentation.” What the Agile Evangelists of my experience are often advocating is running an experiment - perhaps several experiments in parallel but more likely in sequence. The steps are something like this:

  1. Propose a hypothesis, e.g. undertaking monthly software releases instead of bi-monthly will result in a lower percentage of release problems
  2. Examine the current position: e.g. find the current figures for release frequency and problems, record these
  3. Decide how long you want to run the experiment for, e.g. 6 months
  4. Introduce the change and reserve any judgement until the end of the experiment period
  5. Examine the results: recalculate the figures and compare these with the original figures
  6. Draw a conclusion based on observation and data

I agree with all of this, I think its great, but…

Lets leave aside problems of measurement, problems of formulating the hypothesis, problems of making changes and propagation problems (i.e. the time it takes for changes to work though the system). These are not minor problems and they do make me wonder about applying the scientific method in the messy world of software and business but lets leave them to one side for the moment.

Lets also leave aside the so-called Hawthorne Effect - the tendency for people to change and improve their behaviour because know they are in an experiment. Although the original Hawthorne experiments were shown to be flawed some time ago the effect might still be real. And the flaws found in the Hawthorne experiments should remind us that there may be other factors at work which we have not considered.

Even with all these caveats I’m still prepared to accept an experimental approach to work has value. Sometimes the only way to know whether A or B is the best answer is to actually do A and do B and compare the results.

But, this is where my objections start…. There are two important elements missing from the way Agile Evangelists talk about the scientific method. When real scientists - and I include social-scientists here - do an experiment there is more to the scientific method than the experiment and so there should be in work too.

#1: Literature review - standing on the shoulders of others

Before any experiment is planned scientists start by reviewing what has gone before. They go to the library, sometimes books but journals are probably more up to date and often benefit from stricter peer review. They read what others have found, they read what others have done before, the experiments and theories devised to explain the results.

True your business, your environment, your team are all unique and what other teams find might not apply to you. And true you might be able to find flaws in their research and their experiments. But that does not mean you should automatically discount what has gone before.

If other teams have found consistent results with an approach then it is possible yours will too. The more examples of something working the more likely it will work for you. Why run an experiment if others have already found the result?

Now I’m happy to agree that the state of research on software development is pitiful. Many of those who should be helping the industry here, “Computer Science” and “Software Engineering” departments in Universities don’t produce what the industry needs. (Ian Sommerville’s recent critique on this subject is well worth reading “The (ir)relevance of academic software engineering research”). But there is research out there. Some from University departments and some from industry.

Plus there is a lot of research that is relevant but is outside the computing and software departments. For example I have dug up a lot of relevant research in business literature, and specifically on time estimation in psychology journals (see my Notes on Estimation and Retrospective Estimation and More notes on Estimation Research.)

As people used to dealing with binary software people might demand a simple “Yes this works” or “No it doesn’t” and those suffering from physics envy may demand rigorous experimental research but little of the research of this type exists in software engineering. Much of software engineering is closer to psychology, you can’t conduct the experiments that would give these answers. You have to use statistics and other techniques and look at probabilities. (Notice I’ve separated computer science from software engineering here. Much of computer science theory (e.g. sort algorithm efficiency, P and NP problems, etc.) can stand up with physics theory but does not address many of the problems practicing software engineers face.)

#2: Clean up the lab

I’m sure most of my readers did some science at school. Think back to those experiments, particularly the chemistry experiments. Part of the preparation was to check the equipment, clean any that might be contaminated with the remains of the last experiment, ensure the workspace was clear and so on.

I’m one of those people who doesn’t (usually) start cooking until they have tidies the kitchen. I need space to chop vegetables, I need to be able to see what I’m doing and I don’t want messy plates getting in the way. There is a word for this: Mise en place, its a French expression which according to Wikipedia means:

“is a French phrase which means "putting in place", as in set up. It is used in professional kitchens to refer to organizing and arranging the ingredients (e.g., cuts of meat, relishes, sauces, par-cooked items, spices, freshly chopped vegetables, and other components) that a cook will require for the menu items that are expected to be prepared during a shift.”

(Many thanks to Ed Sykes for telling me a great term.)

And when you are done with the chemistry experiment, or cooking, you need to tidy up. Experiments need to include set-up and clean-up time. If you leave the lab a mess after every experiment you will make it more difficult for yourself and others next time.

I see the same thing when I visit software companies. There is no point in doing experiments if the work environment is a mess - both physically and metaphorically. And if people leave a mess around when they have finished their work then things will only get harder over time. There are many experiments you simply can’t run until you have done the correct preparation.

An awful lot of the initial advice I give to companies is simply about cleaning up the work environment and getting them into a state where they can do experiments. Much of that is informed by reference to past literature and experiments. For example:

  • Putting effective source code control and build systems in place
  • Operating in two week iterations: planning out two weeks of work, reviewing what was done and repeating
  • Putting up a team board and using it as a shared to-do list
  • Creating basic measurement tools, whether they be burn-down charts, cumulative flow diagrams or even more basic measurements

You get the idea?

Simply tidying up the work environment and putting a basic process in place, one based on past experience, one which better matches the way work actually happens can alone bring a lot of benefit to organizations. Some organizations don’t need to get into experiments, they just need to tidy up.

And, perhaps unfortunately, that is where is stops for some teams. Simply doing the basics better, simply tidying up, removes a lot of the problems they had. It might be a shame that these teams don’t go further, try more but that might be good enough for them.

Imagine a restaurant that is just breaking even, the food is poor, customers sometimes refuse to pay, the service shoddy so tips are small, staff don’t stay long which makes the whole problem worse, a vicious circle develops. In an effort to cut costs managers keep staffing low so food arrives late and cold.

Finally one of the customers is poisoned and the local health inspector comes in. The restaurant has to do something. They were staggering on with the old ways until now but a crisis means something must be done.

They clean the kitchen, they buy some new equipment, they let the chef buy the ingredients he wants rather than the cheapest, they rewrite the menu to simplify their offering. They don’t have to do much and suddenly the customers are happier: the food is warmer and better, the staff are happier, a virtuous circle replaces a vicious circle.

How far the restaurant owners want to push this is up to them. If they want a Michelin star they will go a long way, but if this is the local greasy spoon cafe what is the point? - It is their decision.

They don’t need experiments, they only need the opening of the scientific method, the bit that is too often overlooked. Some might call it “Brilliant Basics” but you don’t need to be brilliant, just “Good Basics.” (Remember my In Search of Mediocracy post?).

I think the scientific method is sometimes, rightly or wrongly, used as a backdoor in an attempt to introduce change. To lower resistance and get individuals and teams to try something new: “Lets try X for a month and then decide if it works.” Thats can be a legitimate approach. But dressing it up in the language of science feels dishonest.

Lets have less talk about “The Scientific Method” and more talk about “Tidying up the Kitchen” - or is it better in French? Mise en place…. come to think of it, don’t the Lean community have Japanese word for this? Pika pika.

Thursday, July 03, 2014

A footnote of a book: Agile Reader, 3rd edition

AgileReader3-2014-07-3-17-59.jpg

If you have had the pleasure to see me present in recent months - either in public or on a training course - you might have noticed that I have taken to describing myself as “a writer who pays the bills by consulting and providing training.”

Partly thats because I always seem to come back to writing, I seem to have this insatiable need to express myself and explain my thoughts. Hence two traditional, old fashioned books Changing Software Development and Business Patterns. When I say traditional I mean full on professional publishers (Wiley): an editor, proper copy editing, professional layout, print runs that run to thousands and so on. A book by anyone’s standard.

And more recently Xanpan: Team Centric Software Development. A book, two hundred pages of words, even printed on dead trees but really an e-book at heart. But no formal publisher; yes it has been professionally copy edited but layout, you don’t get many layout options in LeanPub. The print on demand version even has some really good reviews online (see Xanpan at Lulu).

I’m very happy with Xanpan’s success, and I have ideas for two more volumes. Although, as I keep pointing out: writing doesn’t pay the mortgage so I should do something more commercial with my time!

However, there is another book. A book I shy away from calling “a book” - I usually call it a mini-book. Perhaps pamphlet would be a better old-fashioned term. Surprisingly the sales on this lessor known book probably exceed any of the other three. (Actually I’m still trying to work out what a book actually is, these four books demonstrate the dilemma very well.)

And I’ve just released a third edition. This time I’ve moved it all over to the LeanPub system so you can buy a cheap e-book Agile Reader for you Kindle (or whatever) but I’m also making Agile Reader available via Lulu print on demand as a printed book. (As with the printed Xanpan, all printed books include a token to obtain a free e-book version by the way)

I don’t make a big fuss about Agile Reader because historically little in it is original, its just pulls existing material together. But much to my amazement two people have already bought the third edition it from LeanPub - before I’ve even told anyone it exists!

Agile Reader is a collection of writing I give attendees on my training courses as further reading. I don’t believe slides make for interesting reading so I provide a book. I believe my Agile course should be about giving people Agile experience in the classroom so they can just do it. Agile Reader exists to fill in everything else.

The funny thing is, because everyone on my courses for the last 3 years have received a free copy Agile Reader 2012 (the second edition) it is one of the top selling (top 100) books on Lulu globally! But I felt it needed a refresh….

Originally Agile Reader was articles I had published elsewhere. In the second edition, Agile Reader 2012, there was some new material and a contribution from Kevlin Henney. The third edition, Agile Reader (3rd edition), retains Kevlin’s contribution and has more original material. In addition a lot of the other material has been refreshed to bring it up to date.

In fact, Agile Reader 3 contains so much revised material on requirements that I think its almost a Xanpan volume 3 prototype. But now I’ve refreshed Agile Reader I really want to write Xanpan volume 2, I’m just bursting with things I want to say so volume 3 will have to wait!

Just a foot note really.

Thursday, June 26, 2014

How do I make testing faster?

Earlier this week I was the guest of a large bank in the City, OK Canary Wharf actually. They had their own little internal Agile conference. As well as myself some of the usual suspects were on parade as well as some internal speakers. It was very enjoyable and as usual the C-List speakers were some of the most interesting.

Later in the day I found myself in conversation with two people concerned about software testing. They posed the question: “How do we make testing faster?” Specifically, how do we make SIT (“System Integration Testing”) and UAT (“User Acceptance Testing”) faster?

Now of these solutions are far from unique to this particular bank. Indeed after listening to the question and situation I recalled several discussions I’d had, problems I’d seen and solutions I’d suggested before. (OK bad consultant me, I should have enquired into the problem, understood the context fully, coached her to find her own solutions etc. etc. But we only had 30 minutes and I wasn’t being paid for a full consulting session - nor am I charging you for reading this! - so lets not invent the wheel, lets fill in with assumptions and given some general answers.)

The solutions we came up with were:

  1. More testing environments: it is shocking how often I hear testers say they do not have enough test environments. They have a limited number of environment and they have to time-share or co-habit. There are two answers: either increase the number of test environments or reduce the number of test and testers. I’m sorry I can’t give you a magic sentence that persuades the money people to buy you more environments. Now we can virtualise machines there is very little excuse for not creating more test environments. The usual one is cost. But even here the costs are not that high. (Unless that is you need to buy a new mainframe.)
  2. Testers integrated & co-located with developers: if you want to go faster I’ve yet to find anything that beats face-to-face direct verbal communications. If you want/need people distributed then you should accept you will not be as fast as you could be. I’ve said it before and I’ll keep saying it: Testers are first class citizens, treat them as you would developers.
  3. Automate SIT: simple really, automate the hell out of testing. Well easy to say, harder to do but far from impossible. Since most UAT/Beta testing is exploratory in nature it is largely SIT that need automation. And please please please, don’t spend money on tool test tools.
  4. Make UAT just UAT: Recognise that some of what passes as UAT is actually SIT and automate it. In large corporates an awful lot of what is called “User Acceptance Testing” isn’t “User Testing” - although it may well be “Acceptance Testing.” Look carefully at what happens in this phase: that which is genuine Acceptance Test can - indeed should - be specified in advance (of the coding), can be moved into the SIT phase and automated. The true User bit is going to be much more explorative in nature.
  5. Improve development quality with TDD and do it test first: one reason for long SIT and UAT phases is that they find bugs. One way to significantly reduce the number of bugs is to have developers practice Test Driven Development, i.e. build in quality earlier. This may mean coding takes longer in the first place but it will speed up the wider process. TDD is still quite new to many developers so you can speed up the process by giving them some help, specifically TDD coaching from someone who has done lots of this before. Until developers make the jump from “post coding automated unit tests” to “pre-coding test first development” you will not maximise the benefits of TDD, you will have more code in your system than you need and you will have a poorer overall design, and both of those make for more tests later on and potentially more problems. When tests are written before code TDD becomes a design activity.
  6. Reduce batch size: i.e. reduce the size of the release, and probably do more releases. Do less, do a better job, push it out more often, reduce risk. Think dis-economies of scale. Many large corporations use a large batch size simply because any release is so painful. This is completely the wrong approach, rather than running scared of releases take them head on.
  7. Done means done and tested in the iteration - at the end of the iteration it is releasable: when you reach the end of an iteration and declare work “Done” that means “Done ready for release (to customers).” Well, that should be your aim anyway and you should be working towards that point. If “Done” currently means “Done development, ready for SIT” then you should be working towards bringing SIT inside the iteration. If that looks hard right now then look again at items 1 to 6 in this list. And when you’ve moved SIT inside the iteration start on UAT. Until both those phases (and any other phases in the tail) are inside the iteration they can throw up problems (bugs) which will disrupt a future iteration which implies the work you said was Done is not Done.

Now the catch. Money.

Some of these things require money to be spent. In a rational world that would not be a problem because there is plenty of evidence that increased software quality (by which I mean specifically few bugs and higher maintainability) leads to shorter schedules and therefore reduced costs. And as a plus you will get more predictability. What’s not to like?

So rationally its a case of spend money to save money. Quality is free as Philip Crosby used to say.

But unfortunately rational engineers often find it difficult to see rationality inside big corporation operating annual budgeting cycles. One of the reoccurring themes this week - although not one I think was given official time - was the constraints of the budgeting cycle. The rational world of the engineer and the rational world of the account and budgeteer seem a long way apart.

There is a solution but it is a big solution which will scare many people. It is Beyond Budgeting, and that is why I was so keen to invite Bjarte Bogsnes - the author of Implementing Beyond Budgetting - to talk about Beyond Budgeting at this years Agile on the Beach conference. And its probably why Bjarte recently tweeted “Invitations to speak about Beyond Budgeting at Agile and HR conferences are now as frequent as Finance conference invitations. Wonderful!”

Sunday, June 22, 2014

Do you know the way to Falmouth? (Agile on the Beach)

The Agile on the Beach 2014 conference is fast approaching and there is every sign that it will, again, be sold out. Which means, in the days and weeks before the conference I’m going to have several people asking me for my advice on how to get to Falmouth. So here is some advice on getting to Falmouth and the conference, not everything you could know, but the main points I tell people when they ask. (If you want a quick summary skip to the end.)

Firstly even some English people are stumped when they stop and consider how to get to Falmouth. Its in Cornwall, its not as far west as it could be (thats Penzance) but it is part of the country that many people have never been to. And if you are coming from outside the UK its even more likely that you need to stop and think.

So the first piece of advice is: plan in advance. Don’t leave it to a few days before hand. You probably need a day just to get the Falmouth.

And that leads to the second piece of advice: if you can plan to extend your stay it is well worth it. The conference is Thursday and Friday, if you can plan to stay longer. I know speakers and attendees who have brought their significant other with them, or arranged for the significant person to arrive on the Friday.

Falmouth is a great little town to spend a day or two in. Better still St Ives is close by, I have been known to go all the way to St Ives just for an exhibition at The Tate gallery there. Then there is the rest of Cornwall to explore. And, if you look carefully there are great places to eat.

Lets start with the most common case…

Falmouth by car

You can drive from London to Falmouth, its about 300 miles and five hours driving - before you take any breaks. Personally I don’t recommend it. I’ve done it a couple of times and its not my idea of fun. Especially once you leave the M5, the A30 seems to go on and on and on… If you do drive allow plenty of time and plan on a couple of breaks.

Also, you are likely to be charged for car parking at the conference itself. The conference is held at Falmouth University and despite lots of talks with the University people they refuse to remove the car parking fee. As I recall this was £3 a day but I might be wrong. Hopefully they will relent but there you go.

Flying - from London or elsewhere

You can also fly. Actually you fly to Newquay (NQY) airport from where you need to get the 25 miles or so to Falmouth. You are probably looking at a taxi which adds to the cost. Also adding to the cost is the third-world like £5 “Airport Improvement Taxi” which you need to pay on departure.

I don’t like NQY. Of all the airports I’ve been to I find the security staff the most intrusive. I know they have a job to do. I know they have to screen you. I know there are bad people in the world. But… of all the airports in the world I find the security here to be the biggest infringers of my personal space.

I once saw the security staff send once send a lady back to the checkin desk because she had a boarding pass issued by a code-share partner of the airline. The demanded a genuine FlyBe boarding pass and not a self-printed BA boarding pass. Maybe this is common but I’ve never seen it elsewhere.

Anyhow nobody flies for fun these days so maybe I’m being an grumpy old man.

If you want to fly you probably want FlyBe from Gatwick, although FlyBe also have daily flights from Manchester. FlyBe also have some flights from other places but these don’t happen everyday. At the time of writing EasyJet also list a flight from Liverpool and have operated from Southend before now but again its not everyday.

Internationally

If you are travelling internationally you may well be getting on a plane anyhow. Although Lufthansa has an occasional flight to Newquary you are probably going to have to connect somewhere. Gatwick is the obvious place but Manchester is just as good an option.

One thing to be aware of: if you are flying into Heathrow and need to get from there to Falmouth I recommend you switch to trains. Transferring between Heathrow and Gatwick is better avoided. On the other hand, getting from Heathrow to Paddington and picking up a train is really easy.

One minute please, I’ll talk about trains in a moment.

Another option is Exeter airport - Exeter also has an airport with some flights to Amsterdam and other locations in the UK and Europe. Since Amsterdam is a hub this is a viable option for anyone coming from further a field.

From Exeter airport you could continue by train. I’m not sure how you get from the airport to Exeter train station, bus or taxi is my guess.

Last year some Dutch speakers flew to Exeter and hired a car to drive the second leg.

Train

Personally I always take the train. I live in London so this means getting to London Paddington and picking up a train to Penzance. But you don’t ride all the way, a few stops before Penzance is Truro where you need to change.

Paddington to Truro is about 4.5 hours, then change for Falmouth at Truro for a little train but get off at Penryn station (Penryn live departure boards), from there it is a short walk up the hill to the University campus.

I always take the train. Once I’m on the train at Paddington I can work, read, sleep, what ever I like. Unlike the plane where you are queuing to get through security, queuing for coffee, queuing to get on the plan, squeeze in flight for 40 minutes, queuing to get off the plane you sit on the train.

If you are coming from somewhere else in the country you can pick up the Cross Country train that also goes to Penzance or pick up the train from Paddington at Reading.

If you’ve never done the journey before pay special attention to the Dawlish section after you leave Exeter, the scenery is beautiful - and yes it has been rebuilt after last winter.

My big piece of advice here is: book your train tickets in advance. If you leave it until the last minute you could end up paying £300 for Second Class. If you book in advance you can get First Class for as little as £100. The extra’s in First Class aren’t up to much but you do get free water, tea and coffee (compared to first class on East Coast Mainline or Virgin West Coast the FGW First Class is poor).

Note also: FGW don’t have wifi onboard yet and 3G reception is poor after Exeter. The further west you go the weaker it is.

Some of the First Great Western trains have a “Travelling Chef” who does a good bacon sandwich. But, determining which trains have travelling Chef’s is hard, FGW’s website makes you work hard. Second: sometimes the Chef doesn’t turn up. I’ve planned on a bacon butty before now only to find the Chef didn’t turn up for work that day.

Next advice: going home look at the sleeper - especially if you live in London. You leave Cornwall about 10pm and get to Paddington about 5am the next morning. I’ve done it for the last three year - although I’m probably not doing it this year. Again FGW don’t make the sleeper that easy to find or book (no Cross Country this time).

The sleeper isn’t really that much more expensive - I think it cost me £100 last year - and it is fun. You don’t sleep all the way but you will sleep. Its an experience.

A warning: The same trains which have the sleeper cars have regular cars where you can sit -airline style - and try and sleep. If you don’t explicitly book the sleeper you have a seat not a bed.

There you go, that is pretty much the advice I give to anyone who asks: How do I get to Agile on the Beach?

  1. Avoid driving
  2. Only fly if you are coming from international (or Scotland) and then try to connection somewhere other than Heathrow
  3. Take the train if you possibly can
  4. Book early and pay the extra for first class
  5. Think about getting the sleeper back

Wednesday, June 18, 2014

Tailor or an image consultant?

“Gentlemen, you are getting married. You go to the best tailor in town, and you say: ‘I’m getting married, my bride is beautiful, make me look one million dollars, I don’t care what it costs.’

The tailor measures you very carefully then he stands back. He rubs his chin, he is clearly troubled by something. ‘Tell me’ you say, ‘Please tell me.’

‘Sir’, he replies, ‘I don’t think it is my place to say’

‘Tell me’ you beg, ‘I must looks fantastic for my girl’

‘Well Sir, …’ he talks slowly and cautiously, ‘. if you really want to look $1,000,000 can I suggest you lose 5kg?’

What do you say?”

This is the dilemma many a consultant find themselves in. More importantly it is the dilemma that I find again and again in corporate IT:

  • Is IT a service department which is expected to build what is requested without answering back?

Or

  • Is are the IT departments the experts in maximising benefit and value out IT investment?

Is the tailor a master of clothes who just makes beautiful clothes? Or are they, because of their long experience making people look beautiful in clothes, an image consultant who can help you look beautiful?

Is your IT department a tailor who is expected to produce something brilliant no matter what the request is? Or are they people who, because they work with information technology every day, have experience and knowledge to help their (internal) customers maximise their benefit? (Even if that means the original request should be reconsidered.)

I distinctly remember a senior IT manager at an American bank telling me that his department was most definitely a service department, it wasn’t his job to answer back to business requests, his job was to build what was asked for. The implication being that even if his people knew the (internal) client was approaching the issue from the wrong point, and even asking for the wrong system, then his people should not correct them. They should just build what was asked for.

Perhaps this dilemma is most accurately felt by Business Analysts who are sometimes expected to write down the orders that someone else wants to give to IT even if they know better. Fortunately some Business Analysts can go further and advise their “customers” on what the bigger questions.

The original metaphor come from Benjamin Mitchell. As I recall it was part of a very deep conversation we had in Cornwall back in 2011 and which also led to the “Abandon hope all ye who enter Agile” blog post. I’ve embellished and elaborated the story over the years.

I reminded Benjamin of this story in January - he’d forgotten about it, thats how brilliant he is, he has so many great ideas he can’t keep track of them all. Once I’d reminded him he immediately made it better.

“How would you know?” he asked, meaning, “How can you determine what is expected of you?”

For me the story is about missed opportunities, about frustration and about what you might do to change the position. For Benjamin it is about asking “Do we all expect the same thing? - If you think they consider you a service group how can you confirm this?”

Either way the core problem is about a mismatch of expectations. Your IT department might be a service department, that approach could work, its how many big corporations operate (I my experience).

Or the IT department could be the experts in technology, how to apply technology, how to frame problems so technology can help and how to maximise the return from technology.

Both are valid options.

But the real problem is when someone in the department thinks the department is should be doing more than the rest of the company does; or when the rest of the company expects more from the department but the department sees itself as a service group.

Finally, go back to my series on the economics of software development, the supply and demand curve, and apply this story. If the IT department is seen as a tailor who shouldn’t answer back they will find it very difficult, if not impossible, to change the demand curve. Such a tailor can only work on the supply side.