———————————————————————————————————-
Independent verification and validation (IV&V) – the process of ensuring that a system fulfills all specifications and its intended purpose – is critical to the earliest stages of the development process and essential to minimizing an IT project’s exposure to failure. Many CIOs see the cost of IV&V as a great investment to minimize risks in large and complex projects such as supply-chain software upgrades, building/moving a data center or doing a major system modernization.
———————————————————————————————————-

Recent discussions with numerous CIOs reveal that identifying and mitigating project risks are critical steps, especially when cost overruns are no longer tolerated. Cost over-runs and outright project failures represent millions of lost investment dollars and opportunity costs that are incurred far too often. Businesses can no longer afford failed projects.

CEOs receive a range of staff responses when they ask, “Why is the project failing?”
• The selected technology did not work as expected and is complicated.
• User requirements and expectations were not managed well or clearly defined.
• Resources were not managed to achieve the desired business value.
• Too many project changes resulted in an overly complex system that was hard to test.
• We have vendor delivery problems; or, we had internal management challenges.

Two CIOs recently stated that during cost cutting, “We eliminated several high-risk projects and implemented an IV&V process to ensure that the remaining projects were managed better and would deliver the needed value.”

A common theme of identifying problems earlier and using an IV&V process helped several organizations modify project plans or stop projects due to technical or managerial issues. According to one state CIO, “We previously viewed IV&V as a luxury, but the cost is such a small percentage of the project budget and it helps ensure the delivery of projected business value. The result is that we’re now mandating it on many of our large and high-risk projects.”

A government CIO stated: ”We have since implemented an IV&V process on our major projects in the planning and development phases. We will not undertake another large-scale project without an IV&V process.”

The CIO of a Fortune 100 company stated that the enterprise is much better off using an IV&V contractor and process because they “identified several major technical problems the prime contractor did not recognize.”

A healthcare CIO stated that the IV&V process helped deliver a $20 million patient administration and records project on time in a situation where there was no room for overruns or mistakes.

CIO CALL TO ACTION
The following best practices will help ensure that both business leaders and the IT organization reap strategic value from IV&V:
• Find a reputable, independent and experienced third party; internal resources are not independent.
• Determine the scope of responsibilities for the IV&V contractor that best fits the project.
• Develop an IV&V management plan that is approved by business executives.
• Ensure that IV&V addresses both managerial and technical approaches, requirements and ability to deliver the business value as stated in the project charter.
• Do not treat IV&V as an overhead expense; it is a critical risk management component of the project cost. If the business does not want to pay for IV&V, it must accept the additional risk and conduct more periodic technical and management reviews.
• Recognize the complexity of a project and ensure that the IV&V process provides actionable information for timely decision making.
• Ensure that the appropriate roles, responsibilities, authority and reporting lines are clear, and develop a set of comprehensive, business-oriented IV&V performance metrics.

BOTTOM LINE
By implementing upfront planning and ongoing due diligence, IV&V is an excellent process to ensure better project oversight and to reduce project cost and schedule over-runs. A secondary and often overlooked benefit is reduced post-implementation support, which can save considerable longer-term costs.

Business Impact:
IT project failures and projects that produce suboptimum results will increasingly be less tolerated by the business. Organizations can no longer afford the additional costs or the potential loss of promised agility or growth when projects do not achieve the business case for which funding was approved.

Additional Insights:
1. “Driving Quality Upstream by Using Unit Testing” (Research)
2. “Lack of Goals and Specifications Dooms FBI’s Virtual Case File Management System” (Research)
3. “Case Study: Grants.gov Is a Step Toward Streamlining Grants in the U.S. Federal Government” (Research)
4. “Market Overview of Application Testing-Service Providers” (Research)
5. “Testing Services: Guidelines for Understanding and Using Testing Service Key Terms and Definitions” (Research)

Please e-mail the authors with your comments and suggestions. We also invite you to participate in a case study.

Amos Auringer: amos.auringer@gartner.com
Jonathan Settle: jonathan.settle@gartner.com

———————————————————————————————————-
In recent IT Odyssey interviews, many CIOs say that a more activist government posture requires them to look beyond input from just their own businesses, consultants and vendors. A new “awareness trinity” is evolving, made up of activist board members, academic counselors/advisors and government regulators (often pushing internal auditors); and it is placing new and different demands on CIOs and their staffs.
———————————————————————————————————-

Since 2003, in over 700 IT Odyssey visits with IT leaders, less than 10% of CIOs have mentioned what they learned from an academic or government affiliation. If any comment was made, it reflected a critique that the outside or advisory board members were “unfamiliar with IT or IT issues,” and that academics were either “not dealing with real IT management issues” or “weren’t properly preparing their graduates for real IT jobs.”

Other than in the financial services, utilities and biomed/healthcare industries, or related to SOX 404 compliance, regulators were seldom mentioned. CIOs did constantly note what they had learned from their vendors, other IT executives, research providers or consultants.

Since March of this year, over 60% of CIOs have described how a more activist government/regulatory regime is not only impacting what IT is being asked to do, but also what CIOs are expected to know or include in their thinking processes. This isn’t just about the financial services and automotive industries. It has emerged in virtually every industry visited: healthcare/pharma, chemicals, logistics, entertainment, retail, travel/hospitality, energy, consumer goods, manufacturing, defense, technology, professional services, state and local government and insurance.

IS THIS THE NEW NORMAL?
Private and public sector CIOs all mention that they are being asked to provide more internal control, activity transparency, regulatory reporting and detailed analysis to meet increased scrutiny. One CIO said, “We’re being asked not only for historical information we’ve never kept, but also to project the information we don’t have into the future. In one month, we were asked for detailed employment analysis, environmental impacts, energy uses, breakdowns of activity by zip code, detailed shipping records, contractor records, detailed financial analysis related to our business with government contractors, research project analysis, foreign taxes paid by product and e-business sales.”

Many CIOs described new or more active “scrutiny boards” that have emerged to examine or help with the management of their businesses. These boards often include academics. Several CIOs described being ambushed in meetings with questions such as “How can you be qualified to be a CIO in this industry if you aren’t familiar with our research on that?” In other cases, CIOs were asked why they were not participating in relationships with, or had formed IT advisory boards that included, outsiders such as academics, pundits and governmental counterparts.

Public sector executives and educational communities are being asked a corresponding set of questions. Non-government organization (NGO), federal, state and local CIOs have been asked why they aren’t reaching out to the business community. One noted professor said he was told his grant might not be renewed if he didn’t have an advisory board of appropriate business/IT executives. Another college was told the effectiveness of its business and M.B.A. programs would be subject to evaluation by business executives. The new dean strongly believes “that it will be strong business, governmental and university working partnerships that will differentiate academic institutions over the next decade.” The CIO of an NGO was told her management systems would now be reviewed by a panel of outside business/IT executives.

In the 1970s, there were often strong ties between IT executives and academic communities, but since the 1990s, the two areas have grown apart – not only appearing to avoid each other, but often disparaging each other. The new trinity that is emerging is seeking an active partnership, and IT and academia are being encouraged to become more aware of each other.

CIO CALL TO ACTION:
Several of the CIOs interviewed suggested proactive efforts:
• Budget a reserve or contingency: Resources are tight and already fully allocated for next year. But governmental/regulatory and transparency/disclosure/compliance needs will certainly increase.
• Assign academic coverage: Have members of the IT leadership team, perhaps in league with specific user executives, follow the work of academics that may influence or benefit a course of business. Use the alumni ties of the IT leadership team.
• Build academic and governmental alliances: Participate in academic or governmental groups that can benefit the organization or create a constructive working relationship.
• Strengthen certain ties: Assign appropriate IT leaders to develop a working relationship with the enterprise people who handle governmental liaisons.
• Outside boards: Have the IT leadership team use their networks to seek opportunities to be on outside oversight and advisory boards.

BOTTOM LINE:
CIOs must better prepare their teams to engage and respond to activist boards, academic advisors and more government/regulatory scrutiny. More transparency will also be demanded. This will require time, effort and some additional funds.

Business Impact:
New expectations, in an era of more activist governments/regulators, boards and shareholders will impact virtually all C-suite leaders. Along with new technologies, alternative IT delivery vehicles, new business models and new IT application sets, these changed expectations are reshaping where CIOs, in particular, must allot their time and focus.

*IT Odyssey: Each year, Bruce J. Rogow independently conducts face–to- face visits with more than 120 IT executives under the banner of his IT Odyssey. As a Gartner Executive Programs affiliate, he provides summary observations of what he is hearing and thinking.

———————————————————————————————————-
Some government organizations have already implemented open-source solutions or are conducting open-source trials in an attempt to reduce licensing costs for end-user computing. But some of our Executive Programs members have reported that hidden costs and technical incompatibility issues with collaboration, mobility suites, as well as existing software, will generally not yet favor the adoption of open-source solutions.
———————————————————————————————————-

Our discussions with government CIOs show that these executives are increasingly examining open-source solutions as a way to reduce licensing costs at the desktop and laptop levels. This follows the decision of several countries’ central governments to encourage more open-source utilization. However, many CIOs come to the conclusion that the promise of cost savings doesn’t yet justify a full rollout of open-source solutions.

One local government organization moved its MS Office suite to open source a few years ago to reduce licensing costs and cope with a tighter IT budget. The CIO has reported that the organization is currently in the process of developing a more collaborative relationship with other local government agencies and businesses. But most of the technology tools that enable quicker, more integrated communication and collaboration (such as unified communications or team collaboration) do not fully support open-source formats. Therefore, the CIO is considering moving away from Open Office, unless the central government’s strong support for open-source solutions drives an increased vendor commitment to developing full interoperability with communications and collaboration offerings.

Another central government agency is testing Open Office on mobile devices to enable field workers to update and exchange documents anywhere, anytime, thus increasing their productivity and fostering greater collaboration with external parties. But early testing has not shown positive results, and the project is likely to be abandoned for now. According to the CIO, “Most of mobility solutions do not fully support open-source formats, and as a result, the inability to properly view and edit open-source documents via mobile devices may jeopardize our investment in a multiyear mobility strategy and undermine its promise to deliver business agility.”

A third local government organization is planning to move a number of users to Open Office, following the national government’s renewed focus on open source. The users who have been selected across several departments are more likely to exchange documents with other government agencies or interact with citizens. From a pure licensing-cost perspective, the open-source solution would be less expensive, but the CIO reports that the following factors have deterred the organization from moving ahead with the Open Office pilot:
• Migration costs and training requirements.
• Interoperability issues caused by the different versions of Office documents being exchanged and edited
• Costs due to lack of interoperability such as higher help desk support on technical issues
• User complaints due to difficulties in properly viewing/reading/editing documents

CIO CALL TO ACTION
Government CIOs who are considering a migration to open source for office productivity tools should do the following:
• Carefully consider the big picture, and do not just focus on pure licensing cost savings. Factor in costs for migration, integration and testing, training, upgrades and maintenance, as well as user perception.
• Do not consider open-source projects as isolated initiatives; put the open-source project into a broader perspective. When planning to leverage technologies that enable real-time communication and deliver greater employee productivity and collaboration with external parties, be aware that a wide range of open-source formats may not be supported by vendors’ solutions.
• Thoroughly investigate and understand governments’ positions on open source and how committed they are to ensuring that vendors develop solutions that can be easily and effectively integrated with open source.
• Understand all the situations in which the organization uses and shares documents, especially with other departments/agencies and constituents. Account for mobile device access to ensure that Office Suite or file format changes do not break an existing process.
• Continue to monitor the progress of open-source office productivity tool suites. Gartner estimates these will be more mature in two to three years.

BOTTOM LINE:
CIOs should not only focus their attention on licensing savings when running an open-source pilot but also take into account their collaboration and mobility strategies. Integration issues with collaboration and mobility suites, as well as existing software, are likely to be major hurdles to the success of the pilots.

Business Impact:
The benefits of open-source solutions are often negated by the loss of productivity, increased costs of interoperability and the increased difficulty in linking to suppliers and customers.

Additional Insight:
1.  The U.K. Government Beefs Up Its Open-Source Policy (Research)
2.  Findings: Open-Office Formats May Impede Collaboration Among Mobile-Device Users (Research)
3.  Hype Cycle for Open-Source Software, 2008 (Research)