Patterns of Success - Andy Hunt

Last year, as Chair of our IEEE Computer Society Chapter I put together a meeting on "Practical Software Development". Karen Smiley did a great job of documenting the meeting on her blog. It was not until that meeting that I had realized that one of the signers of the Agile Manifesto lived in my back yard. Since that time, I have heard Andy speak on a couple of other occasions. I have been impressed by his no-nonsense approach (very pragmatic) to software development.
I am on a quest to interview as many of the signers of the Agile Manifesto as I can.











John - Andy, tell me what is going on with you lately.

Andy - Lots of new and exciting things in the works. At any given time we probably have 50-80 new titles that are being worked on. We keep coming out with what we think is interesting. That is the metric we use when considering a new title... will it be something that we would want to read. Unfortunately, what we see with a lot of other publishers is that the content may be interesting in an academic sense, but it is not helpful. If it does not help you with your job or you becoming a better person then it is a waste.

John - I have found a few archetypal books in my career. In fact remember last year at the IEEE session we met Fred Brooks. His Mythical Man Month was such a book for me. Other books like, Software Engineering Econimics or Applied Software Measurement, or Structured Analysis And System Specification or...

Andy - You know, all those books have a timeless quality. Take Brooks' book for example and substitute the IBM 360 / OS 360 with modern computers and operating systems and book would still be useful to us. That book addresses problems with people working together and that has not changed very much since the 60's. And that is the same issue that I have been focusing on in my recent work. We have better languages, we have better tools, we have better methodologies, but were still having the same old problems.

John - So for you and in your career what have been your archetypal books?

Andy - Its interesting because some of them have been off the beaten path. For me, when a book hits you is as important as what it says. For example Jim Coplilen's "Advanced C++ Programming Styles and Idioms". That was one of those things where you get to the top of the hill and see over and say "Oh! It could be like this!". Just one of those Ah ha moments. And with Brooks' book, I remember first reading it and thinking "Poor fools back in the day. We are so much better off today". I was a lot younger back then.

John - You said earlier that the thing that has not changed over the years is people.Their psychology, the politics, the cultural norms, have not changed much.

Andy - And that is the same all over, You know the one book that changed my life the most was "The Pragmatic Programmer". It is interesting how that came about. Dave and I did not set out to write a book. We were out doing the consulting game, advising them on how to develop software more effectively, and we we tell the client some anecdotal examples from other clients or what we had read on Usenet. Eventually, all this story telling was taking too much time, so we decided to write a little white paper, a pre-read for a client before we came on-board. A bullet list of ideas we had collected. And like many projects this one had some scope creep and ended up being the first draft of our book. And we knew nothing about publishing a book.We got some advice from a friend of ours who was trying to become a novelist. He said "Look on your bookshelf and pick the books that you really like and approach that publisher. So we contacted Addison-Wesley and we thought they would laugh us right out the door, but they sent us a contract. So we wrote the book, we typeset the book, we edited the book. From that experience we eventually realized that we could publish books ourselves, and in 2003 we started the publishing business.

John - One of the other people I interviewed for Patterns of Success is Ed Yourdon. He was also an author that started a publishing company (Yourdon Press). In fact he bought one of the first commercial Unix systems so that he could typeset the books. Of course today, with places like Lulu and Amazon it is easy for an author to self publish.

Andy - It is easy to go from a Word document to a published book but they are missing several key elements. We assign a development editor to each author.  We curate the proposals that come in. There is significant value added to the authors initial draft. For good books in the future that team of author and editor must still be in place somehow.

John - What about the marketing side of the business? A good publisher will get the title and synopsis in front of a broad audience to create demand.

Andy - I have never cared for the word "marketing". It sounds too much little convincing someone to buy something that they don't really want. With a really good book, all you need is some publicity to let people know the book exists. And that is what we do, we make sure people hear about it. For example, one of our recent books "Hello, Android" is just flying off the shelves because it is a really good book.

John - Its a really good book, its dealing with a subject that is of great interest, and Android developers are flocking to the platform to strike it rich. But it is a good example of how a publisher can keep a book current. Android is coming out with a new version every six months. Unlike Mythical Man Month which is timeless, the half-life of Hello, Android is about three months.

Andy - And we do offer e-books to our customers in addition to ink/paper books. And if a customer buys an e-book they will get automatic updates as the content is revised to keep it current.

John - So lets talk about some Patterns of Success. For the last few minutes we have been more at a meta-level of how you get great concepts out to the public, like your publishing efforts. But I would like to look at the actual ideas that you have tried with customers that have been the most successful for them.

Andy - The number one thing that I have seen projects adopt that has brought the most success are feedback mechanisms. When writing the book "Practices of an Agile Developer" with Venkat Subramaniam we needed to come up with a definition of Agile. So we said "Agile development uses feedback to make adjustments in a highly collaborative environment". The most successful project I was ever on had a highly technical user right next door, We got immediate answers to the issues that would pop up during development. They key is that it is all about feedback, and using the feedback. So at all levels of feedback, from pair programming to User Acceptance Testing, there are two parties: The producer of the feedback and the consumer of the feedback, and there are two activities: Identifying and clearly communicating the feedback, and hearing / taking action on the feedback. Feedback will not work unless both parties and both types of activities are in place.

John - So at one end of our feedback spectrum we have the classic waterfall scenario where development gets requirements from a customer and then goes off for several months until User Acceptance Test when the customer may or may not get what they needed. If we could quantify the feedback instances it would be a low number. At the other end of the spectrum is the nirvana that you are speaking of with rich and continuous feedback that is immediately acted on.
Is it possible to have too much feedback?

Andy - It is always possible to overdo it. If you get feedback that is not telling you anything then it becomes noise. However, since Agile encourages everything to be done is small increments, the feedback is not dumped all at once. It is not a torrent, it should be a gentle Spring shower.
An analogy I use for this is how you drive a car. It is a continuous series of small adjustments to the car, the traffic, the environment. This has taken on a whole new meaning as I am teaching my child how to drive.

John - I don't know about you, but the way I learned to drive, was my dad took me out in our subdivision, where there was no traffic, easy straight roads, and taught me the fundamentals of gas/brake/steering before he took me out on the highway.

Andy - That is a great extension of the analogy. Many companies are making a fundamental mistake in adopting Agile practices by trying too much, too quickly, with people fresh out of school and toss them on a mission critical project that is already under an aggressive schedule. And of course it blows up. At several companies I have recommended that they create an A list team, made up of the skilled, experienced people, to be used on difficult projects, and a farm team with the newbies and less skilled people, that is used on less critical projects. And they always tell me "What a great idea!" However, in most companies the seniority and implied expertise is correlated with how many years you have been out of school. And that is not a good correlation. Another bad gage of effectiveness is certifications. In my Wetware book, I quote a study where they proved that just acquiring knowledge in a subject area does not improve effectiveness. It is the experience of applying that knowledge to a real world problem that makes people effective. So certificate programs by themselves are useless.

John - It seems that in addition to individuals needing education and then experience, teams made up of these individuals need to work together to gain a level of team effectiveness. One of the other improvements that Agile fostered was the small, self governing team, using frequent retrospectives to improve their effectiveness.

Andy - I think that is another area where we can fault our education system. It places so much emphasis on individual learning and accomplishment. When people get together to solve a problem they call it cheating. So recent graduates have no experience with pair programming or continuous builds.
Another fundamental problem I see is that humans are not wired to do Agile. I wrote a blog recently on Why Johnny can't be Agile that points out some of the conflicts between recommended Agile Practices and the way our brain works.

John - So do you overcome these innate deficiencies through  process or checklists? How do teams learn to do things correctly?

Andy - Same way we do anything else: Through awareness and feedback. Awareness is a big thing. A lot goes by that we are oblivious to. Its funny, but as a consultant we go into a company and after a while we make some observation and suggestion, and the client is amazed how insightful we are. When it is more that they have become so used to the situation that it is invisible to them.
There is an undercurrent in most of the current literature on adopting Agile, that says you will not be successful on your own. You need an outside coach who can observe with a fresh pair of eyes.

John - Of the Agile Practices of which there are dozens, which do you think are the most challenging for a company to adopt?

Andy -  Pair programming is really hard, retrospectives (when done correctly) are difficult. At the other end of the spectrum there are some practices that are easy to adopt and have a tremendous bang for the buck. The daily 10 minute stand-up meeting is a great way to communicate status within a team and improve cohesiveness. Unit test based on TDD type thinking is another big win. Both unit test and stand-ups are a variation on the feedback success pattern.

John - So lets shift gears. You thought that the number one success factor was having feedback loops. Over your career when you have encouraged clients to embrace feedback loops, what have been some to the dramatic failures and why?

Andy - I think the primary reason is you get feedback that is politically undesirable.  People that are made aware of a problem choose to ignore it because they will have to admit they made a mistake and fear they will lose power, prestige, their job. All projects have symptoms and warning signs of failure. Teams always are aware of what the problems are but are reluctant to make hard decisions.  Sometimes a problem is known at the team level but kept hidden from management, sometimes the team reports the problem to immediate management but they hide it from upper management. The hierarchical reporting structures in companies often impede corrective actions.

John - I did a blog post a while back on Learning from Mistakes. There are some institutionalized practices including the Failed Product Review at companies like Toyota, or the US Army After Action Report. I compared them to Retrospectives. In all cases for the process to work the culture must have a degree of trust where people will not come to harm because they share their opinions.

Andy - We often tell clients fix the problem don't fix the blame. I think smaller, more entrepreneurial teams are more able to accomplish this because they share a common vision and have gelled as one team.

John - It does seem that the larger corporations have more difficulty in general with adopting these practices.

Andy - I think in industries with non-tangible products, we are seeing a shift towards smaller companies. The barrier of entry in Software is becoming less and less. So we see two-three person companies developing remarkable initial products.

John - Of the dozen or so interviews that I have done so far in Patterns of Success, I think you are the third or fourth person to mention this phenomena of micro-sized companies that quickly enter the market.

Andy - There is a great book by Paul Graham called Hackers and Painters who tells a story of how he built a store front generator with a small team using server side Lisp. They could envision and deploy a new feature in an afternoon, when their competitors were looking at a six month release cycle. They kicked butt and they got bought out and it became Yahoo Stores.
At one point after the Manifesto was signed, there were a bunch of us, Fowler, Cockburn, at a speakers table at a conference someplace, and the question was put to the table... "If you could do one thing for a team to improve their productivity what would it be?" And to a person the common answer was "Fire the bottom two thirds of the team".  Get a small number of really sharp people and you will be better off.

John - Sounds like a corollary to Brook's Law..."Adding more people to a late project makes it latter" BUT "Adding people to a project makes it later". Does not matter if you are late or not.

John - So lets move on to THE NEXT BIG THING. If we were doing this interview again in three years what would be the new idea taking hold?

Andy - I don't know if I could pick out something specific, but in general we see the acceleration of the fact that mobile rules. The desktop computer will become less relevant to how we work. So me are migrating from Desktop to Laptop to Tablet to Smartphone.

John - So in addition to becoming smaller, they are mobile. How will this mobility affect our work?

Andy - It makes a huge difference. I follow the music industry (Ed: Actually Andy does more then that. Check it out)  and look how that had changed from physical vinyl or cassette collection to archives of mp3 files being downloaded to devices, but still a personal collection. Now we are seeing, with always available high speed internet connections, a streaming mindset where you subscribe to a cloud based music universe of all recorded music. Thats a huge change. If you look at books or software delivery, lots of innovation based on being always connected to the cloud.
For Dave and I, we run our business off our laptops. We can do anything we need to do from that platform.

John - And pretty soon you will be able to do it off you iOS or Android platform.

Andy - I can almost run the business from my iPad now. There are just a few things that we need when editing and building books that are not on the iPad. Yet.
Look at what that will do to our society. Look at our transportation system based on commuting patterns, our housing based  on commuting patterns, taxation based on commuting patterns. When we can work when and where we want the social landscape will change.

John - Thanks for sharing your insights with us.