Nathan Wilson

A member of the Gartner Blog Network

Nathan Wilson
Principal Research Analyst
1 year at Gartner
27 years IT industry

Nathan Wilson is a principal research analyst in Gartner Research, where he focuses on agile development methodologies.Read Full Bio

The challenge for agile in 2014

by Nathan Wilson  |  January 10, 2014  |  Comments Off

Now that the holidays are over, it is time to get start the 2014 work year. My inquires in 2013 showed a continuing interest in agile. Changes included an increased interest in strategic agile development. While I still talk to development mangers wanting to convince their management that agile is a good idea, I had an increase I the number of managers telling me that they have been told to do agile asking me how to implement it.

This raises the question: Can agile be driven from above? Too often these directives result in either window dressing or worse, the route adoption of a cookie cutter framework that does not fit an organization. Frameworks and maturity models often follow into this trap. They become ever increasing lists to ensure that a the “correct” practice is being followed. This often results in a long checklist that inhibits effectiveness instead of fostering it.

The agile manifesto www.agilemanifesto.org was not constructed as a framework, it is a set of values and principles. The challenge facing agile in the coming year is to make sure that all agile initiatives are evaluate on these values and principles, not a set of prescribed activities.

Comments Off

Category: Uncategorized     Tags:

Phase Gated

by Nathan Wilson  |  October 25, 2013  |  Comments Off

Phase gates provide a structure to large project. The give the impression of being able to control a project or at least track it’s progress. The reality is that each phase cannot be verified until the next is complete and verified. This means that design defects are commonly discovered in final test. This tendency for phase gated projects to hide bad news is the source of the expression that 90% of all software projects are 90% done 90% of the time.

This trend was on display yesterday when congress was investigating the issues on healthcare.gov. Officals from QSSI and CGI both stated the end-to-end testing started too late. It is difficult to gain much insight from any post-mortem blame fest, much less one held in the national spotlight. In addition, I have no non-public knowledge about what happened during the project. Despite this, it would appear that phase gating was a major contributor of this.

Why would you start end-to-end testing too late? Because it is the final phase of testing and testing is the final phase of development. With a firm deadline, every slip in earlier phases of the project took time away from the most critical testing phase.

The solution to this problem is to define “done”. If each piece of functionality was end-to-end tested as it was developed, there would not have been any critical crunch of testing at the end. In addition, if the functionality was built starting with the most important features first, then the most important functionality (such as account creation) would have been working a long time ago.

The approach that I have described is agile development, we have been doing it for over twelve years now and it works. Of course there is no silver bullet in software development and it is not clear that functional but incomplete solution would have been acceptable, but it would have been better that what we got.

Comments Off

Category: Uncategorized     Tags:

Agile and Lean – What do they really mean?

by Nathan Wilson  |  October 18, 2013  |  Comments Off

As hard as it is for a nerd to admit, words matter. One of the challenges of talking to clients about methodology is that two of the most important words have different meanings to IT and the wider organization.

First is the word agile itself. The authors of the Agile Manifesto did a great job of picking a term with universal appeal.  Who would want to be less agile? Nobody arrives at work wanting to make development less responsive.  The problem with this is that often when the business wants IT to be more agile, development will assume that they are being told to adopt an agile methodology. Most business leaders are not looking for IT to DO agile, they are looking for IT to BE agile. Scrum, Kanban, and XP are just implementation details.

The second word is Lean. When the business wants IT to adopt lean principles, they at least understand some of the agile goals. Lean principles of small batch sizes, shorter cycle times and reducing waste are a pretty good fit with MVP, continuous deployment, and collaborative development.  The problem is that when the business asks IT to be lean, IT often assumes that they are being told to implement the Lean/Kanban methodology. The reality is that all of the agile methodologies are techniques to apply lean principles to software development.

When the business asks for lean or agile, they are asking you to be more responsive to their changing needs. This desire for responsiveness can be a great way to start  the conversation about how your relationship with the business will need to change. It is really hard to be responsive to a business while following a detailed yearly plan.

Comments Off

Category: Uncategorized     Tags:

Back to school (and work)

by Nathan Wilson  |  August 30, 2013  |  Comments Off

Well another summer is over and the kids are back in school. My to-do list for this fall is to finish up research on agile and architecture, update research on technical debt and prepare for our AADI conference in early December.
I still talk to a lot of organizations that want IT to be more responsive but they cannot get past the command and control model. They still want to make sure that they are pushing their teams hard enough and cannot trust them to be truly self organizing and self pacing. Given some of the research that went into the Tribal Leadership book (http://www.triballeadership.net/book), it looks like only about 25% of companies are culturally ready for agile.
Does this leave the other 75% in a permanent “have not” status for IT? Some of them will be able to move the cultural maturity forward, but what will happen to the rest? We know that the long term waterfall project is not really viable, but can a command and control organization even manage the conscious uncertainly that comes with iterative development?
I should be an interesting fall as more and more organizations attempting to do agile are not culturally ready for it.

Comments Off

Category: Uncategorized     Tags:

Agile 2013

by Nathan Wilson  |  August 15, 2013  |  1 Comment

I am still catching up after spending a week at Agile 2013. After almost two years at Gartner it was nice to get back to my geek roots again. It was also nice to go to a conference in Nashville instead of the usual Orlando or Vegas. Of course there was too much happening to cover in one blog post, but a couple of themes caught my attention.
First the US Government is getting serious about agile. Despite the fact that most funding models and governance processes are built around waterfall phases, there are a lot of IT folks in the Government making agile work.
The second recurring theme was the growth of Lean/DevOps. Agile has always been about applying lean manufacturing principles to software development. In many organizations, this results in pushing more work on an overloaded operations team. My only concern with the term DevOps is that it focuses on the implementation details. Better flow from development, through test and into operations is important, but it may not be the bottleneck in the cycle from a consumers need for a solution to that consumer being able to use the solution. Lean IT is probably a better name, but that also can be interpreted as a specific agile methodology instead of the context in which agile development is operating in. Gene Kim (@RealGeneKim) covered the three ways of flow in his closing keynote, providing a great description of the context in which agile team members work.

Finally, I had a great talk with Woody Zuill (@WoodyZuill) and others about Mob Programming. We understand that pair programming results in better code and improved net velocity. Woody and his team are coding with an entire team and on (big) monitor. Given the fact that it is still hard to get organizations to understand that pair programming is not twice as expensive as solo work, it will probably be a while before many organizations are willing to support this model. On the other hand, mob programming is worth watching.

1 Comment »

Category: Uncategorized     Tags:

Apocalypse Now?

by Nathan Wilson  |  June 13, 2013  |  Comments Off

Apocalypse is often thought of as the end of the world in popular movies and literature. This is not it’s only meaning. Apocalypse is the Greek work for uncovering or revealing what has been hidden.  In ancient texts including the bible, it often referrers to the end of one age and the beginning of the next.

To carry this metaphor to software development I think that I have seen at least three horsemen of the apocalypse in the last year:

  • The Microsoft VisualStudio 2012 launch event where Microsoft talked about agile as much as their products.
  • Gartner publishes “The End of the Waterfall as We Know It” (subscription required) declaring that the long term waterfall project model does not work.
  • IBM’s Innovate conference this month where the focus was on DevOps which they defined as continuous development and delivery.

Agile is now mainstream and it has shifted from a grassroots revolution of developers to a management driven push to make IT departments more responsive. Those of us who have been involved in agile for a while can be forgiven for wondering if agile can survive the shift.

The way forward is to leverage this high level attention to push for the changes outside of the development teams that agile requires. Businesses know that they need IT to change, now is the time to explain how small short projects and better engagement between development and their customers can enable the responsiveness that the business needs.

Comments Off

Category: Uncategorized     Tags:

Agile and Telecommuting

by Nathan Wilson  |  March 15, 2013  |  1 Comment

It is a long held principle of Agile development that face to face communication is the gold standard for interactions. I have been thinking a lot about this while following the debate on telecommuting following Yahoo’s elimination of working from home.
The truth is that the Nexus forces of Mobile and Social have changed the way that we communicate in a fundamental way. Like many recent parents of teenagers, I have witnessed conversations via text message even when my boys are in the same room as their friends. My nephew picked someone that he knew from online gaming to be his roommate at college. We are all accustomed to keeping up with our friends lives with Facebook.
Does this shift in how we communicate change the requirement for co-located agile teams? While I am not yet willing to give up on the co-located team as the gold standard for agile development, It does appear that the cost of separating a team across distance is dropping. Recent moves by Agile ALM vendors to add social capabilities to their offerings reflects this desire to harness social media to support agile teams.

1 Comment »

Category: Uncategorized     Tags:

Catching Up

by Nathan Wilson  |  February 1, 2013  |  Comments Off

The transition to agile requires major changes to many roles outside of the development group. One of the bigger impacts is on how projects are managed.  I came across an example of this week. An agile team did not complete all of the stories for a sprint.  The reaction of the program manager was to ask the development team to add that story to the next sprint without taking out any of the planned stories. The goal was to “catch up” to the original schedule, and the inevitable result will be that the development team will miss the target for the next sprint.

Of course the desire to “make up” for lost time is pretty universal in software projects, it is one of the reasons that Tim Cargill ninety-ninety rule “The first 90 percent of the code accounts for the first 90 percent of the development time. The remaining 10 percent of the code accounts for the other 90 percent of the development time” still holds true.

The reality is that if one of the stories in the backlog had a estimate that was too optimistic, others will also have the same issue. By using past progress as an accurate estimate of future progress, agile development highlights bad news early in the process. The hard part is to trust your burn down charts, accept the bad news and adapt to the new reality.

Comments Off

Category: Uncategorized     Tags:

Welcome to the post-Waterfall era!

by Nathan Wilson  |  January 2, 2013  |  1 Comment

Happy new year! It is 2013 and the Mayan apocalypse did not happen after all. We have also avoided the “fiscal cliff”, at least for a couple of months.  However the IT world is going through a form of “end times”. It is now clear what we are coming to the end of the traditional long-duration Waterfall project. During 2012, Gartner announced the “The End of the Waterfall as We Know It” and placed Waterfall at the “dusk of obsolesce” in the IT market clock, indicating that it is time to move to iterative and agile development methodologies.
It is not clear what the post-Waterfall era of IT will look like. Agile development is now common and successfully used by many organizations. On the other hand, in increase in agile adoption as a top down initiative will result in some spectacular failures as organizations continue to attempt large complex and long term agile projects with little knowledge and experience in the methodology. The result is more accurately described as “WaterSCRUMFall” than agile, and success is far from guaranteed.
Iterative development techniques have been lost in the battle between Waterfall and Agile. Over 2013, we should start to see if they will undergo a resurgence in the post Waterfall era, or continue to fade away.
My resolution for 2013 is to continue to provide the best impartial advice I can through this blog and my formal research.
Welcome to the post-Waterfall era!

1 Comment »

Category: Uncategorized     Tags:

One year later…

by Nathan Wilson  |  November 9, 2012  |  Comments Off

This week marks my one year anniversary as a Gartner analyst. I have authored 11 pieces of research. Most are advise for those transitioning to agile, but there are several aimed at more advanced agile practitioners. I have also co-authored several more, including the Application Development Market clock where I added incremental and waterfall to the chart for the first time.
The best part of this job has been the opportunity to talk about agile with over 100 Gartner clients over the last year. I have noticed a change in my calls over the year. Last year at this time there were a lot of “how do I get started in agile” calls. While I still get those calls, I am getting more and more calls from organizations that have successfully run a pilot project and are looking for advice on next steps. I look forward to continuing the conversation in future years
The hardest part has been to go “cold turkey” on coding for an entire year. I admit that I did some Excel VB hacking a couple of weeks ago for old time’s sake.

Comments Off

Category: Uncategorized     Tags: