Paul Proctor

A member of the Gartner Blog Network

Paul Proctor
VP Distinguished Analyst
10 years at Gartner
28 years IT Industry

Paul Proctor is a vice president, distinguished analyst, and the chief of research for security and risk management. He helps organizations build mature risk and security programs that are aligned with business need. Read Full Bio

An Update on the Gartner GRC Reset

by Paul Proctor  |  July 1, 2014  |  1 Comment

This post is being updated periodically to address vendor categorization changes. Last update 9 July 2014

We are 6 months in to our GRC process reset and we have some progress to report. A quick disclaimer: This blog post contains no Gartner analysis, because to this point, our process has (mostly) been a self-selecting process. For a complete discussion of our reset process, read this post.

Here is a quick recap: GRC is one of the most flexible terms in the vendor lexicon, because most of them use it to describe whatever they are selling. Also, many of these products are shells that can be programmed to do whatever you need them to do, so we are putting the emphasis on production usage.

By the numbers
9 – Number of Gartner analysts in our GRC working group

1000 – Number of calls these 9 analysts take on GRC each year

6 – Number of common use cases

78 – Number of vendors we contacted

37 – Number of vendors who responded with references

15 – Number of vendors who have not yet produced references

16 – Number of vendors we dropped

600+ – Number of references produced by 37 vendors

4 – Number of vendors able to produce 5 references in all 6 use cases

 

The top 6 use cases:

-          ITRM – IT risk management

-          ORM – Operational risk management

-          VRM – Vendor risk management

-          BCMP – Business continuity management and planning

-          AM – Audit management

-          CCO – Corporate compliance & oversight

 

We asked 73 vendors to provide 5 references  for each of these use cases, and absent our rigorous research process, what they were willing (or able) to produce is interesting all by itself.

The following vendors produced references for specific use cases. THESE ARE NOT GARTNER RECOMMENDATIONS. Our reports in September will vet these and produce recommended vendors for each use case.

These are the vendors who produced one or more references for one or more of our defined use cases. The codes in parens are the first letter of the use cases for which they produced references.

  • ACL (A)
  • Agiliance (I, O, V, B)
  • Allgress (A, I, V, C)
  • BPS Resolver (O, A, C)
  • Brinqa (I, V)
  • CMO Compliance (O, V, C, A)
  • ControlCase (I)
  • Covalent Software (A, O)
  • DF Labs (O)
  • eGestalt (I, O, A, B, V, C)
  • Enablon (O, A, C)
  • Group Enode (O)
  • Happiest Minds (I)
  • IBM OpenPages (A, O, I, C)
  • Ideagen (I, A, B, V)
  • LockPath (I, O, A, B, V, C)
  • Magique Galileo (A, O)
  • Mega (A, O, C)
  • MetricStream (I, O, A, B, V, C)
  • MKinsight/Morgan Kai (A)
  • Modulo (I, O, B, V)
  • Nasdaq BWise (I, O, A, B, C)
  • ProcessGene (A, I, C)
  • Protivity (A, O, C)
  • Quantivate (O, V)
  • Rivo Software (O, A, C)
  • RSA/Archer (I, O, A, B, V, C)
  • Rsam (I, O, A, V, C)
  • SAP (I, O)
  • SAS (O)
  • Securimate (A, O, C)
  • Sword Group (A, O, C)
  • TeamMate (A)
  • Thomson Reuters (O, A, C)
  • Tripwire (O, I, C)
  • Wolters Kluwer (O, A, C)
  • Wynyard Group (O)
  • Xactium (O)

The following vendors are likely to be included in our analysis, but have not produced references.

  • ANX
  • Aruvio
  • Coalfire
  • Fusion Risk
  • Hiperos (recently acquired by Opus Global)
  • Prevari
  • ProcessUnity
  • SAI Global
  • SecondFloor
  • Software AG
  • The Network
  • TraceSecurity
  • RESI-Informatica

Notable vendors who do not address the Gartner definitions of the GRC use cases for ITRM, ORM, AM, BCMP, VRM, or CCO.

  • Symantec (No longer positions in the GRC market)
  • Hewlett Packard (Reported that they do not have any offerings that address our use cases)
  • McAfee (Has not responded to our inquiries)
  • Microsoft (Reported that our use cases do not fit their strategic direction)
  • NetIQ (Has not responded to our inquiries)
  • Qualys  (Has not responded to our inquiries)
  • Trustwave
  • Oracle GRC – A company is free to use any name they want for their product, but this product addresses SOD in ERP

Vendors who are on our radar, but do not meet our inclusion criteria for one reason or another.

  • 12feet
  • SDG TruOps
  • Happiest Minds
  • Optial

The next step in the process is to send a rigorous survey to each of the references and a survey to the vendors to complete our analysis process. We expect to publish in the fall.

NOTE: If you have used any of these products for a GRC project and would like to be a reference, please tweet me (@peproctor).

NOTE: If you are a vendor and have corrections to this post, please contact me at Gartner.

Follow me on Twitter (@peproctor)

1 Comment »

Category: Uncategorized     Tags:

Digital Business Forever Changes How Risk and Security Deliver Value

by Paul Proctor  |  June 2, 2014  |  2 Comments

Risk and security teams are going through a major transformation. Mobile, social and cloud move business data and processes move outside of the perimeter, and outside of traditional enterprise control. Plus, these are dynamic environments with no stability or predictability. Managing appropriate levels of risk in this environment will require a new approach.

Watch the video:  Digital Business Will Force You to Reset Your Risk and Security Program.

Non-Gartner clients can also access this video by registering.

 

2014-04-29_101922 (John and Paul)promo_security_role_webinar4

Unless you live under a rock, you have probably seen the term “digital business” popping up all over the place. The term “digital” has been around for decades, often in the context of technology marketing. However, only in the past three years has it really become a major driver associated with the progressive use of technology at the wider business strategy level.

Gartner’s definition of “digital business” is the creation of new business designs by blurring the physical and digital worlds. It’s not about getting your business on-line, it’s about dramatically shaking up business models. For example, the way Airbnb is shaking up the big hotel chains and the way Uber and Lyft are challenging traditional taxi services.

Digital Business (Final)

Digital business is fueled by the integration of technology and business models. This includes traditional IT environments, cloud, mobile, operational technology, telephony, audio, video, and IoT.

Digital risk and security refers to the management of risks on all forms of technology that use digital information. Connected products are the new and disruptive forces to which companies must adapt over the next three years. You need to be ready for the next generation of digital business decisions and risk planning.

A consistent, unified approach to digital risk has the potential to support cost efficiencies and greater assurance for business processes. But development of a digital risk capability will require the re-engineering of current organizational structures and responsibilities. We need to develop new capabilities in security and risk assessment, monitoring, analysis and control.

Buckle up Dorothy, because Kansas is going bye-bye.

Follow me on Twitter (@peproctor)

2 Comments »

Category: Uncategorized     Tags:

Security and Privacy Remain Doomed with the Rise of Digital Business

by Paul Proctor  |  February 20, 2014  |  1 Comment

You think Target was a big deal? Get ready for more of the same thanks to the attitudes and understanding of consumers and corporate leadership.

The cultural disconnect between business decision makers and technology risk remains epic. They still believe this is a technical problem, handled by technical people, buried in IT. You don’t need to look any further than this report by NPR on digital startups and security at CES by Aarti Shahani (@aarti411).

In the piece, one startup CEO for a wearable that is gathering sensitive health information says, “So what kind of security you will need with your own sleep and wellness data, which is stored in your own mobile device?”

I get the fact that he is focusing on feature and function because he has limited resources, but c’mon man! He was being interviewed by a national radio program and demonstrated complete ignorance of the security risk. This is not some thoughtful trade-off, or strategy, this is pure ignorance.

This is a CEO who, if he is successful with his company, will be wondering later why he has to invest in all this “security” stuff. Didn’t he hire people to take care of this? Did he hire the wrong people?

Or worse, after his wearable gadget leads to the compromise of millions of people’s personal health information, will he feel responsible, or just scramble to find a way to save his company?

Has he signed off on the privacy language on the website with equal lack of knowledge? The FTC loves to hold companies liable who carelessly promise security to consumers and fail to deliver.

With such wanton ignorance, will he skip any consideration of public-cloud related risk when he decides that’s a great new strategy that will benefit his customers and his shareholders?

In another story on NPR by Dan Charles (@nprdancharles) about sensitive information in the cloud, the focus is on both customer knowledge and corporate decision making. This is a story about farmers allowing the big agricultural companies like Monsanto and John Deere to gather sensitive, detailed information on their entire cycle of crop management. There are questions posed about cloud security, farmer knowledge of technical risks, and the use of the data to potentially manipulate markets.

The farmer says (I’m paraphrasing): “I don’t have a problem with them gathering this information. They’re my partner.” The companies say: “We will always protect the information and we would never share it or manipulate markets.” Does this tune sound familiar?

I’m explicitly not calling into question Monsanto or John Deere’s decision making because I have no knowledge of it, unlike the CEO above.  I’m calling attention to the issues that are swirling around us and a continued ignorance of consumers and many corporations. As digital business rises more failures will mount until understanding and decision making improves across the board.

Accepting risk is always a legitimate business decision, but it is only defensible if it follows a serious consideration of the attendant risks. You can’t just stay willfully ignorant and say “we accepted the risk.”

Security by heroes doesn’t work anymore. “IT risk is business risk” is not just a platitude anymore. Business decision makers can’t remain disconnected from a proactive consideration of technology and privacy risks as they engage in digital business.

I applaud NPR for their stories and I hope to see others in the press keep after these issues.

Follow me on Twitter (@peproctor)

1 Comment »

Category: Uncategorized     Tags:

Information Security Headlines are Misleading

by Paul Proctor  |  February 11, 2014  |  2 Comments

The headlines are schizophrenic. One day it is “Oh no! Oh no! We’re all gonna die!” and the next day it’s “What? Me worry?” The more dangerous of these are the headlines that suggest that we are all going to be fine, because the FUD may be annoying, but organizations are always seeking an excuse to ignore the problem.

While both extremes should be expected from a profit-driven, eye-ball grabbing press, there is a very practical reality somewhere in the middle that should not be ignored. Neither of the extremes really support appropriate levels of attention on  this critical topic. Here are a few examples from very recent headlines that mislead.

“Information Security? What Security?”

“Forget Prevention, says Venture Capitalist Ted Schlein, Focus on Limiting Damage” says the subtitle below the headline. This is a videotaped interview published in the Wall Street Journal on February 11, 2014 with some transcription that includes these gems, ripe for misinterpretation by a non-subject-matter-expert reader (AKA everyone reading the article).

“I’m a firm believer there are only two kinds of companies—those that have been breached and know it, and those that have been breached and don’t know it.” – Ironically, he has worked FUD into a commentary that most people will interpret as you are wasting your time with security.

“Most of what we do in security is around prevention, prevention, prevention. Great. Just know it won’t work. Know that they’re going to get in.” – This is one of those mixtures of fact and fiction that is completely misleading. “…just know it won’t work” is not at all true and gives a license to decision makers to reduce security investment in a very bad way. I believe he is trying to say that there are new and exciting technologies to address security that don’t focus exclusively on prevention. What it sounds like he’s saying is that prevention is a waste of time.

“You ought to be thinking, “Hey, I want to find out where they are as fast as humanly possible, contain it and remediate it.” – Then he follows up with a confirmation that yes, he did mean that prevention is a waste of time. So you can skip all that vulnerability patching! Just drop all the defenses and remediate compromised machines! Remember that this is coming from a VC who has likely invested in some of those companies that provide these capabilities.

“How Cyber-Security Laws are Outdated”

In this article also published in the Wall Street Journal on February 11, 2014 Mike McConnell, Former director of the National Security Agency and of National Intelligence talks about the details of the NSA’s surveillance program and tells how Edward Snowden stole classified information from the agency. Basically his focus on nation-state hacking and a focus on public private information sharing with a headline that suggests our cybersecurity laws are out of date just creates a jumble of information that would leave most people saying “what, the what?”

“Gartner: CIOs Deprioritize Security” and “CIOs Downgrade Cybersecurity”

In these two blog posts (that were highlighted in the Journal’s CIO Journal) from mid-January 2014, WSJ reporter Michael Hickins (@Michael_Curator) seizes upon Gartner’s own CIO study that discusses the relative priority of cybersecurity vs all the other priorities a CIO addresses. He notes: Security ranks at #8 on the list of strategic priorities ranked by CIOs; 10 years ago it ranked as the top priority…”

Full disclosure, he quotes Gartner’s own Dave Aron who was a co-lead on the study. But as Dave tells me, he never left the impression that security was not important to CIOs even though that is the impression the headline gives. Evidently Michael thought it was a noteworthy enough observation that he repeated it in a blog post the very next day.

The bottom line: Headlines are made to grab eye-balls, I get that, but information security and IT risk are critical topics that are going through a sensitive, and in many ways fragile, cultural shift. These misleading headlines are not helping.

The WSJ is not the only media source doing this. I just read the WSJ every day so this is where I pulled my examples.

Follow me on Twitter (@peproctor)

2 Comments »

Category: Uncategorized     Tags:

Think Sochi is a Cyber-War Zone? Try Your Local Library.

by Paul Proctor  |  February 6, 2014  |  10 Comments

Richard Engle and NBC News recently posted several reports from Sochi, Russia based on an “experiment” they did.

I applaud them for bringing attention to the critical condition of cybersecurity, but the report is misleading in two major respects. First, they have directly positioned this as just turning on your mobile device and computer will result in you being “hacked.” To quote Brian Williams introducing the story:

“If <travelers to Sochi> fire up their phones at baggage claim, it’s probably too late to save the integrity of their electronics and everything inside them. Visitors to Russia can expect to get hacked. It is not a matter of if, but when.” This is an overstatement and misleading.

Second, most everything they describe in the story is as equally true at your local Starbucks as it is in Sochi. Therein they miss the opportunity to present a more accurate picture of global security, as opposed to the “evil Russians.”

Researching this blog post, I wrote down a lot of the quotes from the story but I have decided it would bore all of you to walk through them. Here are a few that tell the tale:

    • “<after plugging in brand new machines> it doesn’t take long for someone to TRY to tap into your<device>” and “within a minute hackers were SNOOPING around TRYING to see how secure the machines were.” – So reconnaissance is confused with turning on your machine and becoming hacked.
    • “…and very soon I was sent a CUSTOMIZED PHISHING email.” – I have 3 customized phishing emails sitting in junk mail right now. So do you.
    • “We did a little browsing and almost immediately landed on a site that infected our brand new phone.” – So they had to surf to an infected site for the demonstration. Those pesky Russian hackers! I’m glad that website won’t infect me if I click on it from San Diego.
    • “<known as a> ‘honeypot’ an attractive target, left out in the open for hackers to come at.” – So they created an open and attractive target then reported that it attracted interest? Knock me over with a feather.

And here is the biggest mislead of all: none of this requires you (or the hackers) to be in Russia. In fact, I’ll bet the “Russians” they were hacked by were smart enough to route their traffic through compromised machines in China. That would have been an interesting and ironic twist, but it would detract from the story line.

Basically they had to make all these bad things happen, they were not location dependent, and they waited till the very end to mention that maybe you shouldn’t click on links you don’t recognize from people you don’t know.

Here’s the bottom line: NBC missed an opportunity to point out that you are not really “safe” anywhere and that your behavior is the deciding factor in your risk of being hacked regardless of location.

I would encourage Richard Engle and NBC News to repeat the same experiment in a local library anywhere. Those same “Russians” are right there, waiting for them… and you.

Follow me on Twitter (@peproctor)

10 Comments »

Category: Uncategorized     Tags:

Gartner Resets Approach to GRC

by Paul Proctor  |  February 3, 2014  |  3 Comments

Gartner’s new approach to GRC will deemphasize the presence and demonstrability of features and functions while increasing the weight of implementation and production use of GRC products against specific use cases.

Last year I blogged about Why I Hate the Term GRC and then about resetting our definition of IT GRC. This clearly did not go far enough, so in 2014 we are changing the game entirely. We are doing this for multiple reasons:

  • The term GRC is the most overused term in the risk and security lexicon. Vendors use it to describe whatever they are selling and our clients use it to describe whatever problem they have.
  • The delineation of IT-GRC vs EGRC is almost meaningless because all of the IT-GRC vendors claim to do everything the EGRC vendors do and vice versa.
  • When a client calls us and asks to discuss GRC vendors, our first question is always “what are you trying to accomplish?”
  • The features and functions of the vendors are becoming indistinguishable. For example, they all have a survey function and there is no material difference in implementation.
  • Every vendor demo looks exactly like every other vendor demo and they can all demonstrate excellent capability against a wide range of GRC requirements.

The bottom line is that our current approach to this market is not helping our clients match their needs to appropriate technologies. So we are going to change it.

There are good differentiating characteristics of vendors.

  • All basic functions look the same in demonstration, but not all function the same against specific requirements. For example, a survey function may work well for gathering data internally for product teams, but not function well at all trying to gather vendor risk data externally from 3rd parties.
  • Some vendors have subject matter expertise and implementation experience that makes them stronger for some requirements and not others.
  • Architecture issues may not be obvious in a marketecture diagram, but the limitations become obvious in certain environments.
  • Level of required customization, cost, and satisfaction with those services vary greatly across various implementations.

What all of these have in common is that differences are implementation and requirement dependent. Also, our clients start out asking for generic GRC software and then quickly narrow it to the use cases, requirements, and workflows they are actually going to go implement. Therefore…

Gartner’s new approach to GRC will deemphasize the presence and demonstrability of features and functions while increasing the weight of implementation and production use of GRC products against specific use cases.

Implementing our approach we have defined 6 use cases. We know there are more, but we need to start somewhere.

Use case 1: IT Risk Management (ITRM). The use of GRC tools for management, measurement, and reporting against IT risk. While this may include security operations data and processes, implementations that are primarily focused on security operations, analysis, and reporting will be considered “below the line” and not part of this use case. See this blog post for more information.

Use case 2: Operational risk management (ORM). The use of GRC tools for management, measurement, and reporting against operational risk. There is a bright line between the ORM and ITRM use cases which is beyond the scope of this blog post, but fundamentally ORM addresses IT and OTHER operational risks with deeper risk management capabilities like capital allocation, predictive analytics, and statistical modeling.

Notably, my colleague John Wheeler and I have decided to explicitly exclude enterprise risk management (ERM) as a use case because our definition of ERM includes credit and market risk, which we do not believe is currently a credible capability for any of the GRC vendors.

Use case 3: Audit management. Audit solutions used by internal audit teams that document and track phases of the audit cycle — audit planning, audit risk assessment, audit project management, time and expense management, issue tracking, audit work paper management, audit evidence management, and reporting. Implementations primarily for the benefit of non-audit functions are excluded.

Use case 4: Vendor risk management (VRM). The use of VRM tools for management, measurement, and reporting against vendor and third party related risk.  This will include capabilities to identify, classify, monitor, and recommend risk mitigation to support business operations and regulatory requirements.

Use case 5:  Business continuity management (BCM). Supporting the coordinating, facilitating and executing activities that ensure an enterprise’s effectiveness in identifying and mitigating operational risks that can lead to business disruptions, and recovering mission-critical business operations after a disruptive event turns into a disaster.

Use case 6: Corporate Compliance and Oversight. Compliance management and reporting associated with corporate governance codes, ethics, and financial reporting integrity regulations, such as Sarbanes-Oxley, Turnbull and others, and other regulations, standards and policies that materially affect the compliance posture of the overall enterprise.

The Process (for vendors and references)

Reference surveys:

Our research process is going to emphasize references (both those the vendors give us, and those that we speak to as Gartner clients). The reference process will be fully automated and only take 15 minutes to fill out by a knowledgeable person at the implementing organization.

If you have implemented any of the use cases we have identified with a GRC product, please tweet me @peproctor and we can arrange to send you a link to the survey.

Vendor surveys:

We are seeking to create a single unified vendor survey that will serve the entire process regardless of home many use cases a vendor supports. That means one survey, not six surveys.

The Deliverables

We are discontinuing the IT-GRC marketscope and EGRC Magic Quadrant. We are replacing these with seven new documents, one for each use case and a top down document. Some of these deliverables will include vendors who specialize in a use case but do not traditionally position as GRC vendors. For example, the business continuity management magic quadrant will only be made up of about 50% GRC vendors with the rest focusing exclusively on BCM.

  • Market Guide for Audit Management
  • Magic Quadrant for Operational Risk Management
  • Magic Quadrant for Security & IT Risk
  • Critical Capabilities for GRC
  • Magic Quadrant for Business Continuity Planning
  • Magic Quadrant for Vendor Risk Management
  • Market Guide for Corporate Compliance and Oversight

Timing for Vendors

These times are rough approximations:

-          Feb – Send out a cover letter to engage vendors

-          Mar – Request reference lists

-          Mar -May – Gather reference surveys

-          May-June – Vendor surveys

-          Fall – Documents publish for each use case.

Gartner Client Benefit

The single largest advantage to this change is that our clients will better be able to identify appropriate technologies to match their specific requirements.

The benefit to vendors is fewer surveys to fill out and a more accurate accounting of your strengths and weaknesses based on production deployments.

If you have implemented any of the use cases we have identified with a GRC product, please tweet me @peproctor and we can arrange to send you a link to the survey.

 

Follow me on Twitter (@peproctor)

3 Comments »

Category: Uncategorized     Tags:

Please Stop Asking Me for a List of Your Top Risks (aka, Everyone Wants a Pony)

by Paul Proctor  |  October 15, 2013  |  1 Comment

There is no list of risks that is relevant to every organization, but there is a list of risks that is relevant to you.

I always get the question “what are my top risks?” This comes in different flavors like top BYOD risks, top risks in my industry, cloud, social media, data security, etc. but it all amounts to me handing out a simple answer to a question that does not have a simple answer.

I understand why you ask. It would be nice to have a third party confirm the “most important” risks so you have a starting point and a basis around a standard of due care which all aids in defensibility. The problem is, I can give you a list of high level things that you should be worried about, but I haven’t delivered anything of value. Trust me, our clients sense lack of value before I’m even done answering the question.

I’m in a tough situation here (cue the violins). One of the most horrible things I can say to a client is “you’re asking the wrong question.” But you are. No one can tell you your risks, because each organization is unique. This is NOT a cop out by me! Asking the question is a cop out by YOU! You want me to tell you, so you don’t have to do the work to understand your organization. There, I said it.

It all comes back to why we do risk management. Good risk management should influence better business decision making. If it doesn’t, why are you bothering? A generic list of risks, disconnected from your organization, will not influence anything.

How to determine your list of top risks:

Start by identifying your desired business outcomes and the supporting business processes. Then identify supporting operational dependencies and risks that may impact the dependent business processes. Use a formal process and engage business stakeholders because they will better understand impacts on desired business outcomes. The most likely risks with the most impact on desired business outcomes are your top risks. There. Simple.

Example: Saying that sensitive information on a mobile device is a top risk is devoid of value until you integrate it with the business processes that involves both mobile and sensitive information. A hospital that deploys mobile devices with protected health information to manage patient care in select departments is an example of a top risk for this organization.

One way to look at this is that your top risk is that you have no way to identify your top risks.

Here’s the bottom line, I can teach you how to find a pony, but everyone wants me to just give them a pony. When I show clients a picture of a pony, they get upset because they already found that picture on the internet. Plus they already drew a picture of one that wasn’t half bad.

9467666-a-small-shetland-pony-standing-in-a-green-meadow-with-hawthorn-hedgerow-under-a-blue-sky

I get it, everyone wants a pony, but this is just one you’re going to have find on your own.

Follow me on Twitter (@peproctor)

1 Comment »

Category: Uncategorized     Tags:

No One Cares About Your Security Metrics and You are to Blame

by Paul Proctor  |  August 11, 2013  |  4 Comments

I live by the axiom “If risk management doesn’t influence better business decision making, why are you bothering?” This can be extended to all security metrics and dashboards to mean “If the metrics you deliver aren’t valuable to decision makers, why are you bothering?”

In my role as chief of research for risk and security at Gartner, I get to see dozens of examples of metrics programs and dashboards every year. Everyone is asking about them, and everyone is doing it wrong. I could sugarcoat this, but I don’t want to waste your time. The single biggest mistake is that no one cares about what you are delivering to them. Search your feelings Luke, you know this to be true.

The reason no one cares is because you know nothing about them and what matters to them. They (the decision makers) ask you for these metrics because they have a fiduciary duty to do so. Not because they care or even want to hear from you. They want this problem (security distracting them from their day jobs) to go away.

Very simply put, if you want them to care, you have to give them something that influences their decision making, something that matters to THEM. That means you have to know something about them AND be able to link your metrics to their issues. This also means that no one, NO ONE, can hand you a list of metrics that works well for everyone. More on that later.

First a quick model to understand the issue. If you’ve read any of my research, seen me speak, or spent any time drinking the finest Islay malts with me, then you know this model already, but let’s apply it to metrics specifically.

Above the line, below the line

As seen in the diagram, above the line are executive decision makers, and below the line are IT operations. Below the line are a number of operational metrics that benefit operational decision making, but everyone makes the same mistake. They try to identify the “best” below the line operational metrics to share with executive decision makers above the line. We now know this does not work…ever.

The correct approach is to abstract out all of the technology and operational metrics leaving only metrics with causal relationships to executive decision making. This is very, very difficult to do. Let me demonstrate with a very traditional example of vulnerability management.

  • Metric: Number of times we were “attacked” last month. <- Very common, completely worthless
  • Metric: Number of unpatched vulnerabilities. <- Very common. Has potential, but it is not specific enough to guide even operational decision making. If you report an improvement in this, you have only proven that you are doing your job.
  • Metric: Number of unpatched critical vulnerabilities against critical systems. Not as common as you think because most organizations do not have sufficient asset management to know which systems are critical. Potential for useful operational decision making when this number goes up or down, but the “number” of critical patches and even critical systems fluctuates, so a “number” can be meaningless.
  • Metric: Percentage of unpatched critical vulnerabilities against critical systems. Now we are getting somewhere. By normalizing the number we can compare it month over month. This is useful for operational decision making to guide resources. However, still worthless for executive decision makers.
  • Metric: Number of days it takes to patch critical systems with critical patches. This is even more useful because it further abstracts out the technology so it is understandable by a non-IT decision maker. It still doesn’t cross the threshold to be useful for non-IT decision making because it has no business context.
  • Metric: Number of days it takes to patch systems supporting the manufacturing line in Kuala Lumpur with critical patches. To appreciate this you need to know that the manufacturing line in KL has 3x the unscheduled outage time caused by IT of all the other facilities and it represents 40% of the company’s revenues based on new contracts in the last year. Reporting this to the CIO and the P&L business owner helps them address the unscheduled outage time and respond to a very pissed off CEO who wants to know why output dropped 3% last quarter at the most critical line of the company. Now THIS is a useful above the line metric because the technology is abstracted out, it has a business context, and it supports critical decision making all the way up to the CEO.

Good metrics don’t have to be complex. They just have to support real problems. You still need operational metrics, but keep them where they belong… in operations.

As you can see, the metrics that matter are the ones with a business context for YOUR executives. It is not possible to pick up a published list from any source that will be anything more than starting point for you to develop a good and unique set of metrics for your organization.

And this discussion is only the starting point for the real holy grail which is the integration of risk and security metrics integrated with corporate performance. Gartner clients can access our risk-adjusted value management methodology and our catalogs of more than 200 leading indicators of operational risk and business performance.

The Gartner Business Risk Model: A Framework for Integrating Risk and Performance (G00247513)
The Gartner Business Value Model: A Framework for Measuring Business Performance (G00249947)

We also just published workshop materials for clients to facilitate a conversation between IT and non-IT executives so they can determine the dependencies and performance linkages at their organization.

Toolkit: Risk-Adjusted Value Management Workshop (G00247503)

Finally, we also just published a formal methodology to tie risk metrics back to the corporate financials.

Implement Business Outcome Monetization as a Process for Increasing Project Success (G00249950)

You already know that creating meaningful metrics is hard. I’m sorry for not being the bluebird of happiness, but you aren’t going to find “the right set” of metrics sitting on the web. You need to roll up your sleeves, sit down with your decision makers, and figure out what matters to them.

Follow me on Twitter (@peproctor)

4 Comments »

Category: Uncategorized     Tags:

Warren Buffet is Wrong on Chief Risk Officers

by Paul Proctor  |  July 1, 2013  |  6 Comments

Warren Buffet is a very smart man, but he is leaving the wrong impression on the topic of risk management. A colleague forwarded me this interview from the Motley Fool Website titled: Buffett Says “Chief Risk Officers” Are a Terrible Mistake. That is a very sketchy statement so I had to dig in.

In a video clip from the 2013 Berkshire Hathaway shareholders meeting, Lawrence Cunningham, author of The Essays of Warren Buffett: Lessons for Corporate America says:

“A common response to the ‘08 crisis…was to have every company appoint a Chief Risk Officer… This whole new industry…within corporate governance has installed this new person to be in charge of all risk activities… Buffet just declares this an abdication of responsibility. And a terrible mistake. The CEO is the CRO… only [the CEO] can really get the whole picture. You can’t delegate risk to this manager and leave it there. It has to come to [the CEO’s] desk. [Buffet] is emphatic about that.”

This left my head spinning on right and wrong, so I purchased my own copy of the book and this is what Buffet says:

“I believe that a CEO must not delegate risk control.” “If Berkshire gets in trouble it will be my fault. It will not be because of misjudgments made by a Risk Committee or a Chief Risk Officer.”

Well Hallelujah. I don’t disagree with a word of that, so what’s my problem with Buffet? I decided to DuckDuckGo “Buffet risk management” to see if there are any clarification on his thoughts regarding the role of risk management departments and the appropriate role of a chief risk officer and I found this clip of Buffet from January 23, 2010. In it he says:

“When you have a company as large as Berkshire and all the obligations we have…I have to be the Chief Risk Officer. I should be the best person to do that because I have this overview of the whole operation and I understand risk …”

Buffet carries a lot of weight with his guidance and he is pushing back against the idea of an office that measures and reports on risk related information to executives. This is a very bad idea.

So there it is. This is where I disagree. Fundamentally, a CRO never makes decisions on behalf of executives. The role should be to facilitate a balance between the needs to protect the company and the needs to run the business.

I’m at odds here, because you have to read very carefully everything that is being said, and I agree with most of it. Here’s the breakdown of right and wrong (LC = Lawrence Cunningham commenting on Buffet’s views, WB = Warren Buffet):

WRONG: LC: “This whole new industry…within corporate governance has installed this new person to be in charge of all risk activities.” – Where this is happening, it is an inappropriate implementation of a CRO role.

RIGHT & WRONG: LC: “Buffet just declares this an abdication of responsibility. And a terrible mistake.” Where it has been inappropriately implemented he is absolutely right. He is wrong because he is stating this as the definition of a CRO. It is not.

RIGHT: LC: “You can’t delegate risk to this [CRO] and leave it there.” Of course you can’t. Anyone doing this, doesn’t have a CRO. They have a scapegoat.

RIGHT: LC: “[Risk information] has to come to [the CEO’s] desk. [Buffet] is emphatic about that.” And a good CRO does that. It is their job.

RIGHT & WRONG: WB: “I believe that a CEO must not delegate risk control.” – This is right because it is absolutely true. It is wrong, because it made in the context that a CRO is delegated to make risk decisions. They are not.

RIGHT: WB: “If Berkshire gets in trouble it will be my fault. It will not be because of misjudgments made by a Risk Committee or a Chief Risk Officer.” – Absolutely true. I don’t know anyone who would suggest otherwise.

WRONG: WB: “When you have a company as large as Berkshire and all the obligations we have…I have to be the Chief Risk Officer.” – This is just final confirmation that Buffet does not understand what a CRO does. Probably because he doesn’t have one and is offended by his own perception, so he has never interviewed a true risk professional.

WRONG. DEAD WRONG. WB: “I should be the best person to do that because I have this overview of the whole operation and I understand risk …” – This statement implies that all CEOs should be responsible for knowing every critical detail of their organization. It minimizes the idea that a risk department could gather information, weigh options, and make recommendations regarding risk. Well, I congratulate him for having this level of oversight, but I’m willing to guess most CEOs could use a little help.

So here’s the bottom line. I’ll bet you that Warren Buffet and I agree on every single point written here. This is speculation because I didn’t run this by him before publishing. I’ll bet he has teams of people who regularly gather and report information to help him make informed risk decisions. What I truly disagree with then is the way this all reads as he puts it out there in the marketplace of ideas.

Organizations are struggling because they do not have a good view of the risks facing them. They need organization of this information reported in a business context to support business decision making. I KNOW organizations need this because I see it every day.

I wish he wasn’t out there giving executives a reason to say they don’t need risk departments or CROs.

Follow me on Twitter (@peproctor)

6 Comments »

Category: Uncategorized     Tags:

Harvard Business Review Posts Terrible Advice for CEOs on Information Security

by Paul Proctor  |  June 19, 2013  |  9 Comments

 

Earlier this year I wrote about 2013 is the Year for Hard Change in the Risk and Security Profession.

Then I recently read this blog post from HBR with shock. Does Your CEO Really Get Data Security? This HBR guidance has so many things wrong with it, I don’t even know where to begin, but let me try.

  • If you follow this advice as an information security professional you will at best be completely unproductive and at worst get fired, immediately, and with prejudice.
  • It is based entirely on fear, uncertainty, and doubt (FUD) which hasn’t worked for 10 years. Although it is still very common and it can serve a limited purpose, ask any professional and they will tell you FUD is not productive.
  • And even the FUD is comical. “This is wartime,” “Hire NSA-style, Military-grade cryptanalysts,” and “The CSO is arguably a more valuable asset than the CFO.” Yeah, that’s going to resonate.

Gartner’s core guidance after tens of thousands of interactions with risk and security leaders across every industry and size of organization can be summed up as:

Risk and security officers must act as the facilitators of a balance between the needs to protect the organization and run the business.

This is not rocket science, see How to Get Funding for Your Security Program. Let me walk you through some of the HBR guidance and tell you why you should never, never do this.

  • He suggests that if your CEO pushes back over the criticality of the risk you should “Just laugh”. Now in all fairness, he sets this up as some dystopian fantasy where the security officer gets to run the company for 10 min. I get it, but what I don’t get is that, as professionals, we are working hard to help organizations understand how to effectively communicate to non-IT executives. This writer on the other hand thinks you should treat them like idiots. In his fantasy, you get to fire the CEO because he doesn’t appreciate the dire nature of the infosec situation.
  • “The CSO is arguably a more valuable asset than the CFO because breaches cost a lot of money; the ROI on security, as risk analyst Don Ulsch states, is "the value of your company." He goes on to reference the Sony breaches. Well, Sony is a pretty good example of a very expensive breach with material ramifications to their business, but I hardly think Sony’s business troubles would be resolved if the CEO dropped all the worthless attention on “consumer products strategy” or whatever, and focused on the real problem, information security.
  • “Remind the board that in a war, the company needs a warrior mentality. The CSO must make use of covert strategies and hire NSA-style military-grade cryptanalysts.” Seriously? This reminds me of the 2008 Hannaford grocery store chain when the CEO promised "military-grade security" at his grocery stores! I hope that investment bought customer loyalty because it sure as heck was overkill and overspend from a risk management perspective.
  • “Finally, the CSO must be given authority over people, processes, and technologies.” Folks, we tried this for the last decade and we know IT DOES NOT WORK! Technical IT people telling executives that can’t have iPads and marketing departments they can’t connect to social media is lunacy, not effective security.

This is really shockingly bad, bad advice for CEOs. I have written previously about security officer failures. This amounts to scaring the hell out of your executives and forcing them to invest heavily (and poorly) in information security.

Gartner clients can watch the global keynote from the Gartner Security and Risk Management Summit on June 10, in Washington DC on www.gartnereventsondemand.com. I’m seeking to get this posted on YouTube for those of you that are not Gartner clients because it is the most effective rebuttal I can suggest for this drivel.

The author suggests all kinds of career limiting moves that accomplish nothing but perpetuate outdated stereotypes that minimize the value a competent IT risk and security officer can bring to a company.

I’m not just pushing back against this HBR blog post, I’m pushing back against the existence of this type of thinking in the marketplace of ideas. It is NOT helping.

What do you think?

Follow me on Twitter (@peproctor)

9 Comments »

Category: Uncategorized     Tags: