Nathan Wilson
Principal Research Analyst
1 year at Gartner
27 years IT industry
Nathan Wilson is a principal research analyst in Gartner Research, where he focuses on agile development methodologies.Read Full Bio
by Nathan Wilson | June 13, 2013 | Submit a Comment
Apocalypse is often thought of as the end of the world in popular movies and literature. This is not it’s only meaning. Apocalypse is the Greek work for uncovering or revealing what has been hidden. In ancient texts including the bible, it often referrers to the end of one age and the beginning of the next.
To carry this metaphor to software development I think that I have seen at least three horsemen of the apocalypse in the last year:
- The Microsoft VisualStudio 2012 launch event where Microsoft talked about agile as much as their products.
- Gartner publishes “The End of the Waterfall as We Know It” (subscription required) declaring that the long term waterfall project model does not work.
- IBM’s Innovate conference this month where the focus was on DevOps which they defined as continuous development and delivery.
Agile is now mainstream and it has shifted from a grassroots revolution of developers to a management driven push to make IT departments more responsive. Those of us who have been involved in agile for a while can be forgiven for wondering if agile can survive the shift.
The way forward is to leverage this high level attention to push for the changes outside of the development teams that agile requires. Businesses know that they need IT to change, now is the time to explain how small short projects and better engagement between development and their customers can enable the responsiveness that the business needs.
Category: Uncategorized Tags:
by Nathan Wilson | March 15, 2013 | 1 Comment
It is a long held principle of Agile development that face to face communication is the gold standard for interactions. I have been thinking a lot about this while following the debate on telecommuting following Yahoo’s elimination of working from home.
The truth is that the Nexus forces of Mobile and Social have changed the way that we communicate in a fundamental way. Like many recent parents of teenagers, I have witnessed conversations via text message even when my boys are in the same room as their friends. My nephew picked someone that he knew from online gaming to be his roommate at college. We are all accustomed to keeping up with our friends lives with Facebook.
Does this shift in how we communicate change the requirement for co-located agile teams? While I am not yet willing to give up on the co-located team as the gold standard for agile development, It does appear that the cost of separating a team across distance is dropping. Recent moves by Agile ALM vendors to add social capabilities to their offerings reflects this desire to harness social media to support agile teams.
Category: Uncategorized Tags:
by Nathan Wilson | February 1, 2013 | Comments Off
The transition to agile requires major changes to many roles outside of the development group. One of the bigger impacts is on how projects are managed. I came across an example of this week. An agile team did not complete all of the stories for a sprint. The reaction of the program manager was to ask the development team to add that story to the next sprint without taking out any of the planned stories. The goal was to “catch up” to the original schedule, and the inevitable result will be that the development team will miss the target for the next sprint.
Of course the desire to “make up” for lost time is pretty universal in software projects, it is one of the reasons that Tim Cargill ninety-ninety rule “The first 90 percent of the code accounts for the first 90 percent of the development time. The remaining 10 percent of the code accounts for the other 90 percent of the development time” still holds true.
The reality is that if one of the stories in the backlog had a estimate that was too optimistic, others will also have the same issue. By using past progress as an accurate estimate of future progress, agile development highlights bad news early in the process. The hard part is to trust your burn down charts, accept the bad news and adapt to the new reality.
Category: Uncategorized Tags:
by Nathan Wilson | January 2, 2013 | 1 Comment
Happy new year! It is 2013 and the Mayan apocalypse did not happen after all. We have also avoided the “fiscal cliff”, at least for a couple of months. However the IT world is going through a form of “end times”. It is now clear what we are coming to the end of the traditional long-duration Waterfall project. During 2012, Gartner announced the “The End of the Waterfall as We Know It” and placed Waterfall at the “dusk of obsolesce” in the IT market clock, indicating that it is time to move to iterative and agile development methodologies.
It is not clear what the post-Waterfall era of IT will look like. Agile development is now common and successfully used by many organizations. On the other hand, in increase in agile adoption as a top down initiative will result in some spectacular failures as organizations continue to attempt large complex and long term agile projects with little knowledge and experience in the methodology. The result is more accurately described as “WaterSCRUMFall” than agile, and success is far from guaranteed.
Iterative development techniques have been lost in the battle between Waterfall and Agile. Over 2013, we should start to see if they will undergo a resurgence in the post Waterfall era, or continue to fade away.
My resolution for 2013 is to continue to provide the best impartial advice I can through this blog and my formal research.
Welcome to the post-Waterfall era!
Category: Uncategorized Tags:
by Nathan Wilson | November 9, 2012 | Comments Off
This week marks my one year anniversary as a Gartner analyst. I have authored 11 pieces of research. Most are advise for those transitioning to agile, but there are several aimed at more advanced agile practitioners. I have also co-authored several more, including the Application Development Market clock where I added incremental and waterfall to the chart for the first time.
The best part of this job has been the opportunity to talk about agile with over 100 Gartner clients over the last year. I have noticed a change in my calls over the year. Last year at this time there were a lot of “how do I get started in agile” calls. While I still get those calls, I am getting more and more calls from organizations that have successfully run a pilot project and are looking for advice on next steps. I look forward to continuing the conversation in future years
The hardest part has been to go “cold turkey” on coding for an entire year. I admit that I did some Excel VB hacking a couple of weeks ago for old time’s sake.
Category: Uncategorized Tags:
by Nathan Wilson | September 14, 2012 | Comments Off
I just returned from the Microsoft Visual Studio launch. Having been out of the loop with Microsoft for the last few years, the biggest surprise for me was that most of the Keynote was on how Visual Studio enables Agility. While many vendors talk about agile these days, the most interesting detail for me was that the SaaS version of Team Foundation Server is now being updated every 3 weeks. Given the “eat your own dogfood” culture at Microsoft, this means that the people developing TFS are using TFS do agile development. I expect this to have a big impact on the quality of the agile experience on TFS.
This is also another example of a large team successfully using agile techniques to develop a major software product. I am not convinced that we are over the tipping point to agile, but this brings us a whole lot closer to it.
Category: Uncategorized Tags:
by Nathan Wilson | August 23, 2012 | Comments Off
One of the unfortunate side effects of the tension between agile and waterfall development methodologies has been a polarization of views on the subject. The reality is that many companies were starting to become more iterative in their development methodologies before the signing of the agile manifesto in 2001. This has pushed many traditional shops to focus on Waterfall as the “correct” way to develop software.
Matt Hotle, Dave Norton and I have just published a new research article titled “The End of Waterfall as We Know It” . This research explains the growing consensus that long duration Waterfall projects are the highest risk ways to deliver software. Despite Waterfall being an established methodology with over 50 years of usage, an alarming number of projects are delivered late or worse deliver software that falls far short of meeting the business need.
The classic “long tail” software project presents the business with a Sofie’s choice. After investing many months and millions of dollars in a project, the business is told that the project is behind schedule and will not be completed on time. Given the phased development approach, there is no software that is complete enough to be deployed. The business can either abandon the project and lose the sunk cost, or pour more money into the project.
Once the project is delivered, it is commonly found to not meet the business need due to a lack of understanding of the business problem, or changing business conditions. Many years of improving the process of requirements gathering have failed to eliminate this scenario.
This does not mean that all IT shops need to become agile. Iterative development will provide for more frequent feedback and avoid working for too long on the wrong solution. A focus on automated testing and continuous integration will ensure that there is something worth deploying on the scheduled release date. Agile will tighten the feedback loop even more.
Category: Uncategorized Tags:
by Nathan Wilson | July 27, 2012 | Comments Off
We are working on the Hype Cycle for Application Development here and Project Level Agile is pretty close to the trough of disillusionment. While this can seem like the end of agile, it is a normal part of any IT trend that is going mainstream. The early days of any trend are full of promise, followed by a level of hype that the trend is going to be a silver bullet that will solve all problems.
Of course no new trend can meet these expectations, and the trough of disillusionment follows when people realize that this is not a silver bullet. Agile is following this pattern. A few years ago agile was going to be the revolution that solved all of IT’s problems.
There is a common description of agile that can be paraphrased as “Agile is like exercise, most people who claim to be doing it but most are not and most of the ones that are doing it are doing it wrong” (I think the original and potentially inappropriate version of this metaphor appeared first on http://www.thehackerchickblog.com). The people who claim to do agile without understand agile and the ones that are doing agile wrong will continue to provide plenty of ammunition for the agile critics.
The reality is agile works very well in some organizations and has challenges in organizations where the corporate culture does not support it. The long term trend is agile is working well in more and more companies, so the future of agile is still promising.
Category: Uncategorized Tags: agile, hype cycle
by Nathan Wilson | July 6, 2012 | Comments Off
I was a speaker at a testing conference last week. A lot of the conversations were about the divide that exists between QA and the rest of development. I guess that this is to be expected in a traditional “throw software over the wall to QA” project, but I am surprised on how often it is an issue in experienced agile shops.
TDD, BDD and continuous integration are all pretty worthless as long as there is an adversarial relationship between QA and the rest of development. As many times that we say that quality is everyone’s job and not just QA’s problem, we still seem to have a “QA vs. Development” mindset.
I spent a day in Boston this weekend touring the sailing ships that were there to commemorate the war of 1812. During that war, Admiral Perry famously sent a message to headquarters that “We have met the enemy and they are ours” after the battle of lake Erie. In the war between development and QA a common parody of Perry’s quote is more appropriate: “We have met the enemy and they are us!”
Category: Uncategorized Tags:
by Nathan Wilson | June 1, 2012 | Comments Off
The agile movement started, or more accurately named 11 years ago and sometimes I think that that was the last time that agile developers and traditional developers talked. Over the last 11-13 years, I have seen many new coding practices mature. For example, Test Driven Development (and it’s close cousins Behavior Driven Development and Acceptance Test Driven Development) now have a long track record and robust tools to support the practice.
I am working on some research on what traditional projects can learn from agile. It seems that WaterSRUMfall is the wrong approach in that it partially implements the process changes, resulting in a confused process that is neither fish or fowl. Adopting some of the practices of agile without the process seems like a better start.
Like many in the agile community, I have not been closely involved in any traditional project for a while. I do wonder what we could learn from their advances over the last decade.
Category: Uncategorized Tags:
