———————————————————————————————————–
At a recent Business Continuity Management / Disaster Recovery Planning (BCM/DRP) workshop, IT Executives and program leaders expressed concerns about the usefulness of recovery “testing” to prove readiness. While the consensus of the group considered testing a best practice, many of the practioners concluded, “Testing approaches alone do not generate complete confidence” their systems and processes could recover successfully in a real world disaster. Their experience indicated that even when systems passed unit, system, integration, performance, volume and stress testing, “real world” stresses often created new issues and unforeseen problems.
———————————————————————————————————–
When asked why they had such low levels of confidence, the executive group listed these reasons from their experiences:

• Testing was often approached as a ‘check-off’ step, rather than an integral and important part of their culture – tests were often simplified and were therefore of limited utility.
• Tests may not reflect realistic, high-probability, or high impact operating conditions or failure scenarios – so an un(der)-tested high-probability function could result in major problems.
• Use of ‘sample data’, ‘conference room conditions’ and non-production-matched infrastructure create test conditions that are not realistic.
• Their own testing managers/teams are often not fully confident after passing tests but due to time and resource constraints discontinue testing.

Another concern indicated is the responsible team perceives testing as an ‘evaluation of performance’ instead of a process to ensure preparedness and successful implementation. One CIO stated he “agreed that testing was not the best approach but he felt it would be extremely difficult to change this situation, given constant and growing pressures on financials and delivery of technology.”

There was a consensus that organizations could be more successful and prepared if they focused on running ‘exercises’ rather than just executing tests. They defined an exercise as practicing real world scenarios, based on high probability and impact conditions, to see what works and what does not. In an exercise, the team does not expect everything to work perfectly and as one client leader noted, “in fact, failure is a measure of success in an exercise.” Failure provides the team with needed, useful information on how to improve the system, process or procedure. Most importantly, in an exercise, participants do not feel the same pressure to “pass” and can focus on improving performance iteratively.

CIO CALL TO ACTION
CIOs must apply training, exercise and selective application of testing if they want to be confident their systems and processes will meet real world business continuity needs. Best practice CIOs recommend that their peers ask the following questions to assess and optimize their business continuity readiness procedures:
 How well do the tests represent real world situations?
 Do the tests focus on high probability and high impact areas?
 Has the team simplified the tests or process so they can pass and keep work moving?
 Does successfully passing the tests generate high confidence of system owners?
 How can an exercise approach increase promote learning, successful execution and increase confidence?
 Who can serve as an effective coach to change culture, lead, guide and encourage the process?
 How can the results of exercises be used to promote open exchange on what needs to be improved, avoiding pass or fail assessments?

BOTTOM LINE
Traditional software design and functionality testing and DRP testing often do not generate confidence among CIOs, business leads or system owners. A paradigm shift from pass/fail testing to running “exercises” can change enterprise culture and result in better systems design and implementation. Market-leading CIOs develop confidence in their systems’ ability to meet business requirements, and recover effectively and efficiently in disasters, by using an exercise driven testing approach.

Business Impact:
Successful BCM/DRP programs are a high priority for the board and senior management. Testing results alone are not sufficient measures to demonstrate readiness and identify recovery gaps. Recovering seamlessly from IT system failures is increasingly a brand reputation issue and thus a risk management imperative. Regular “exercises” and reporting results to the Board (probabilities, insurance, costs, etc) should be part of regular CIO reviews.

Additional Insights:
“Business Continuity Management Defined” (Research)

“How to Conduct a Disaster Recovery Management Self Assessment” (Research)

“How to Organize for Disaster Recovery Management” (Research)
“Key Issues for Software Quality and Testing” (Research)

“Standardize Definitions and Expectations for Testing Activities” (Research)

Please e-mail the authors with your comments and suggestions. We also invite you to participate in a case study.

Irving Tyler: irving.tyler@gartner.com
Marc Andonian: marc.andonian@gartner.com

————————————————————————————————————————-
In recent conversations with CIOs, portfolio management has been a recurring topic of interest, as IT organizations (ITOs) seek to optimize resource allocation and subsequently maximize return on investment. This also comes on the back of a recent CIO-CFO roundtable where the attending CFOs identified “transparency” as one of their greatest IT headaches. Visibility into the portfolio of work that consumes a large portion of IT resources helps facilitate that transparency and informed decision making.
————————————————————————————————————————-
Portfolio management has increasingly become a popular IT executive conversation topic, as ITOs attempt to yield the greatest value from their scarce resources (i.e., money and people).

There are two fronts to this discussion:
Project portfolio. Less mature ITOs are formalizing the discipline of project/program management (PPM) and building out capability to reduce scope creep and deliverable slippage, which also provides the basis for reporting a consolidated view of IT projects. One CIO of a manufacturing company in the Midwestern U.S. is initiating PPM in response to the CEO’s demands for “a consolidated view of work being performed by IT across the organization.”

Those with a formal program management office (PMO) are attempting to integrate the informational output of the PMO with IT governance arrangements, thereby placing a magnifying glass on resource consumption and providing the basis for educated decision making. As the CIO of a Midwestern U.S. agricultural products company put it, “Help to separate demand from supply by adding baseline information when selecting new work.”

Application portfolio. This comprises the analysis of the application set for risk, value and architectural relevance, and has been especially evident in large, complex organizations with historically decentralized or loosely defined IT accountability. In one example, the head of IT strategy and architecture for an insurance company in Chicago had responsibility for reviewing the application portfolio against the above criteria and then thinning out the portfolio where possible. He noted that “the issue is now one of IT governance because the tough decisions about application retirement must now be sold to the business users and implemented.”

In both examples of portfolio management, the recent economic slowdown has provided the impetus to move more forcefully to target immediate benefits through the reduction of risk, duplication and cost. While this is the equivalent of gathering evidence to present a case, most CIOs are not viewing these activities as a one-off, and are instead assuming that they will foster a permanent change in behavior toward IT resources. Beyond that, there are varying views of longer-term architectural-based benefits associated with cost, compliance and agility.

CIO CALL TO ACTION
• Assign review and recommendation responsibility for both project and application portfolios to a PMO to increase its role in overall IT governance.
• Perform a detailed audit of applications and rate for value vs. risk (cost and consequences), using consistent and balanced criteria. See the September 2006 Executive Programs report “High Value, High Risk: Managing Legacy Portfolios.”
• Develop application life-cycle scenarios that require increased attention to retire, replace or remediate, and seek input from architecture/standards groups.
• Consider involving the architecture group to help identify underlying architectural services that can be leveraged for common infrastructure or integration.
• Inject portfolio information into annual budgeting and planning cycles and key IT governance arrangements. The real value from either an application or project portfolio is not just the information it provides but the actions taken.
• Anticipate backlash, as enterprisewide portfolio management challenges feudal or functional power bases, and be prepared to allocate and adjust resources based on governance style.
• Begin portfolio management using simple tools (e.g., expanded spreadsheets) to quickly illuminate the key financial and risk information before exploiting more sophisticated models.

BOTTOM LINE
Both project and application portfolio management are powerful tools that can help facilitate rational discussion around the usage of IT resources. They also assist in advancing architectural objectives and in increasing agility. But the real value of a portfolio exercise emerges from the organizational actions taken; and thus inclusion with balanced IT governance arrangements is vital.

Business Impact:
Successfully implementing portfolio management – whether it be project, program or application – provides both short- and long-term benefits in better managing costs, including avoidance and investments (via a more architected approach), as well as outcomes.

Additional Insights
• “Management Update: Managing a Federated Architecture,” Burke, B., December 2007 (Research)
• “Toolkit Presentation: Applications Portfolio Management Starter Kit,” Duggan, J. et al, April 2007 (Research)
• “Make IT Demand Governance Easier: Use Project Portfolio Management,” Gerrard, M., October 2006 (Research)
• “Portfolio Management; CIO Desk Reference Chapter 10,” Handler, R., October 2009 (EXP Research)
• “High Value, High Risk: Managing the Legacy Portfolio,” Hunter, R., Aron, D., September 2006 (EXP Research)
• ““How the CIO Can Increase the Value of the Application Portfolio,” Kyte, A., February 2009 (Research)

———————————————————————————————————————-
During a recent Gartner meeting with several members, the turnaround-style CIO members discussed that they are not afraid to take some risk, and doing so has paid off with improved service and a modernized IT environment. In 12 months, one CIO was able to “rip and replace” all infrastructure systems, and, in the process, over 90% of all enterprise services were upgraded, replaced or retired.
———————————————————————————————————————-

One of the executives we engaged is the CIO of a $1 billion federal R&D organization specializing in energy research, supporting 4,000 staff, 10,500 systems and 6,000 scientific users per year. He recently executed a three-year turnaround (i.e., IT modernization program) and shared with us the nature of the initial challenge, approach, success factors and business impact.

The CIO noted that his business colleagues were extremely supportive: “As a team, we had an opportunity to make a major impact on how they conduct business and how they work with other organizations. The leadership wanted to embark on transforming their organization in numerous areas within a very short period of time.”

The CIO further stated that “there were many challenges to overcome, both from a culture standpoint and a technology perspective. One of the most difficult decisions was moving forward on a ‘single-stack’ IT solution versus implementing a ‘best-in-breed’ IT solution for each process or IT challenge, and in doing so, there was no shortage of challenges.”

Nature of the initial challenge:
• Unknown number of computers and unknown IT costs
• Hundreds of applications written in a variety of languages and technologies
• Disjointed end-user application experience and communication
• Unknown number of IT staff spread over approximately 20 IT departments
• Concerns around the growing challenge of attracting and retaining the brightest staff and scientists
• Weak performance monitoring
• Inconsistent communication and collaboration among staff

How did the organization do IT?
The CIO told us that he “directed the IT staff and vendor partners to develop small project teams of skilled experts that were isolated to modernize a discrete, manageable handful of highly visible enterprise applications one at a time and often in parallel. These small teams of experts depended on each other, and trust was established very quickly as a result. The decision to standardize to a single-stack proved to make a considerable difference in the ability to deploy rapidly, and the technologies worked much more seamlessly.”

There were multiple critical success factor (CSF) outcomes:
• Consolidated all IT staff into a single IT organization
• Consolidated hundreds of applications based on a standard methodology
• Standardized computers and enterprisewide technical, business and data architecture
• Created a standard and documented approach for application development
• Modernized the IT environment to improve knowledge sharing, and automated processes for efficiency through:
o Collaboration tools: instant messaging, unified communication, desktop VTCs
o Knowledge management tools: portals, blogs, visual enterprise search engines, expertise location
o Email services: improved IT services by providing 3GB email inboxes with integrated voicemail

BOTTOM LINE:
CIOs must make significant IT changes to transform the way data is stored, how knowledge is captured, and how it is shared and communicated internally (to staff and LOB clients) and externally to business partners and customers.

Business Impact:
By modernizing and transforming IT, cross-disciplinary collaboration is at an all-time high and people are able to rapidly identify who has specific domain knowledge and how to partner to address mutual interests and improve efficiency and effectiveness (productivity, profits, revenue, customer retention/up-sell, market share and EBITA).

Additional Insights:
• “How the CIO Intersects With Other C Suite Roles” (Research)
• “Case Study: CIO Phil Pavitt’s Development as a Change Leader” (Research)
• “The Enterprise CIO’s Role in Business Process Improvement” (Research)

——————————————————————————————————————–
During the initial research case-study interviews, participating CIOs shared a consistent approach to centralizing and standardizing infrastructure support structures and processes wherever possible to achieve operational excellence and maximized efficiency. Yet, when selecting a structure for their business-facing functions, they chose different models to improve IT performance and effectiveness in contributing to business success. These structures are more aligned with business structures and processes.
——————————————————————————————————————–
Changes in economic and business conditions create challenges for the enterprise and its strategies and plans. CIOs need to evolve their organizational structure to continually align with the business, improve IT performance and contribute to business success.

How the IT organization is structured has implications for resource planning and management. Well-designed IT organizations focus on building IT capabilities that will maximize the value and contribution of IT to the business. This research will provide a collection of proven design options from high-performing IT organizations for CIOs to consider and will include guidance on how to determine what the right option is for their organization.

There is no one-size-fits-all solution to organizing IT. A variety of internal and external factors goes into determining what the right (and effective) structure is for the organization. Early indications show that the following factors impact the roles and functions needed in IT and how those roles and functions are structured:
• Business-operating-model alignment (e.g., centralized vs. federated)
• Business structure alignment (e.g., processes vs. lines of business)
• Complexity of businesses and geographic coverage
• Role of IT in the enterprise
• Organizational culture and leadership styles
• Current leadership capabilities and the need for development
• Span of control
• Technical architecture (e.g., a single instance of ERP implementation)
• Cost pressures
• Best practices identified through benchmarking with peers

Initial research shows that at a macro level, CIOs choose centralized or federated IT operating models to improve IT operational efficiency. This is reflected in the consolidation and standardization of non-differentiated IT services and processes (e.g., infrastructure support and business processes). Regardless of the macro structure selected, IT organizations must establish an appropriate alignment to the business they support, which is key to improving IT effectiveness.

The early indications from our research show that there are several basic approaches to business alignment. CIOs mix and match these approaches in structuring their IT organizations. The right structure needs the support of formal or informal governance processes, and structures in place to break down barriers between IT and the business, address frictions and drive synergies across the business.

Recommended actions include the following:
• Dedicate resources (e.g., business requirements analysis, portfolio managementand solution development) to each strategic business unit.
• Align resources by common business processes shared across multiple business units (e.g., HR, logistics, manufacturing, sourcing).
• Align resources by core IT services (e.g., application management, infrastructure management, project and portfolio management).

Discussions with the interviewed CIOs also indicate that they align staffing plans with the business plan (three- to five-year horizon) and track day-to-day demand to determine workload and staffing needs. Hands-on support services are typically co-located with users or close to the regions where customers reside and local language support is desired. Business solution development, architecture and project management resources can be centralized or virtualized where skills are available. Centralized service delivery groups (e.g., data center, network) are staffed and placed where cost-efficiency can be maximized.

CIO CALL TO ACTION
The CIOs interviewed in the case study research to date suggest taking the following actions when restructuring IT for success:
• Take time to understand how the business is structured and run, and then align the IT operating model and structure with that business model.
• Identify where you can standardize to drive efficiency and where you need to differentiate to meet business unit needs.
• Have a compelling reason for any structural change (value and efficiency); share and communicate the strategic agenda across IT and the business.
• Organize by competency and enable collaboration across geographic and business boundaries.
• There is no single structure that is right and perfect. When reorganizing to address structural and process frictions, focus on building capabilities vs. solving problems.
• Don’t consider outsourcing lightly.

BOTTOM LINE
Regardless of the macro structure selected (e.g., centralized vs. federated), there isn’t much variation in how CIOs structure their non-differentiated IT services (e.g., data center, network support). What’s different from one organization to another is the mix and match of different approaches used by CIOs to structure their business-interfacing functions (e.g., business analysis and planning, solution development) and governance functions (e.g., architecture, security, project and portfolio management) to improve IT effectiveness, which also reflects different roles IT can play in the enterprise.

Business Impact:
CIOs who design an IT organization best suited to meet business needs while providing IT efficiencies are better positioned to contribute to positive business results for the enterprise.

We invite your comments and suggestions, and encourage your participation in the research process for this topic. Please e-mail the authors with your comments and suggestions. We also invite you to participate in a case study.
Lily Mok: Lily.Mok@gartner.com
Heather Colella: Heather.Colella@gartner.com
Diane Berry: Diane.Berry@gartner.com

Additional Insights:
• “Principles and Options for Choosing the Location and Role of an IT Organization,” John Mahoney, 20 July 2009 (Research)
• “Role-Based Organization Design” Andrew Walker and Diane Berry (EXP HR – Q4 2008)
• “Role Definition and Organizational Structure: Program and Portfolio Management,” Donna Fitzgerald and Audrey L. Apfel, 17 April 2009 (Research)
• “Effective IT Organizations: Design Matters,” Patrick Meehan, David B. Pack, 1 July 2008 (Research)
• “IT Organization Trends: Survey Shows Multiple Ways Forward,” Ellen Kitzis and John Mahoney, 21 February 2008 (Research)