Jay Heiser
Research VP
6 years at Gartner
24 years IT industry
Jay Heiser is a research vice president specializing in the areas of IT risk management and compliance, security policy and organization, forensics, and investigation. Current research areas include cloud and SaaS computing risk and control, technologies and processes for the secure sharing of data… Read Full Bio
by Jay Heiser | February 3, 2012 | 1 Comment
The dozens of petabytes of Megaupload data belonging to millions of Internet users is manifesting itself as a giant hot potato, currently burning a cash flow and PR hole into the bottom lines of several global hosting firms.
The Electronic Frontier Foundation has formerly requested that this hot potato be allowed to fester indefinitely, announcing yesterday “EFF formally requested the preservation of the data seized when the U.S. government shut down Megaupload.com and related sites in January of 2012, notifying the court and attorneys involved in the case that Megaupload’s innocent users deserve a fair process to control and retrieve their lawful material.”
I also agree that innocent users deserve a fair process, although it is difficult to envision what that could be. What I don’t agree with is the part about ‘data seized’. As far as I can tell, its still sitting in its original servers in multiple data centers belonging to Carpathia, Cogent, and some number of additional hosting firms. The DOJ did not seize it at all–they just took multiple steps to ensure that the service would be inaccessible:
- They took possession of Mega’s domain names, making it impossible for customers to access it.
- They froze Mega’s financial assets, making it impossible for them to pay the hosting providers.
- They arrested Mega leadership on criminal charges, ensuring that they would be focused on staying out of jail, instead of figuring out how to restore their file storage services.
Mega’s staff are under arrest at worst, and unpaid and looking for work at best. Mega’s hosting firms are stuck with thousands of idle servers, mostly filled with toxic digital waste of bootlegged movies and pornography. Carpathia has strongly suggested that they do not have administrative access to these servers (although they haven’t explicitly said so). It would be nice to think that any legal content would be provided to the 50,000,000 or so people to whom it belongs, but its difficult to envision the practicalities.
Without providing any public suggestion of how it should be done, in a letter to the DOJ on Feb 1, the EFF formally requested that the DOJ take possession of the poisonous potato. Described as a matter of fairness, with Constitutional overtones, this preservation step would presumably be a financial one, but not a physical one.
For the DOJ, theirs was a hugely visible act which immediately encouraged several Megaupload competitors to change their practices. It sent a clear message that ‘the USA will not tolerate Internet IP piracy.’ Given the huge level of citizen push back on SOPA and PIPA, its easy to envision growing pressure to change US policy.
For the hosters, this digital hot potato represents an immediate loss of income, and a potential PR disaster. Just leaving the Mega servers in place represents an ongoing expense, actually turning them on and serving their content would represent an even bigger expense. Coming up with a mechanism to allow ‘legitimate’ users to collect their data while excluding illegal content seems a practical and legal rat hole, with endless potential to attract lawyers from the DOJ, the EFF, foreign governments, and the entertainment industry. It isn’t difficult to envision that they would eventually be on the receiving end of some sort of class action lawsuit.
For the EFF, this is a PR gift, representing their biggest ever opportunity to play hero for millions of impacted Megausers. I don’t blame them for making hay in this sunshine. Cloud computing not only means that the criminals and innocent bystanders are sharing the same virtual premises, but the scale of cloud computing ensures an astounding amount of collateral damage. This isn’t the 1920s, and today’s digital G Men can’t shoot a bootlegger without also hitting an innocent bystander.
For the bootleggers and porn pushers, this probably represents no more than a minor setback.
For some number of individuals and small businesses, too naive to have understood the relative risks and benefits of the public cloud computing model, this probably represents a permanent loss. The EFF is actively soliciting the names and details from impacted users, and it will be interesting to see what data is provided on the number of individuals claiming that their only copy of their personal property is trapped in Megalimbo.
For me, this is an endlessly fascinating story, resulting in some of my best Gartner blog readership stats. Aside from sheer drama of the event, though, it raises important questions about the role of government within the Internet, the liabilities of a provisioning model that relies on a chain of providers, and whether the leverage of this computing model is creating monster sized services that are too big to allow to fail.
Category: Cloud risk management security Tags:
by Jay Heiser | February 1, 2012 | Submit a Comment
Last November, Gartner analyst Richard Hunter and I published research entitled ‘Black Swans’ Are Sure to Fly in the Public Cloud. Based on ideas popularized by Nassim Nicholas Taleb (The Black Swan: The Impact of the Highly Improbable, Random House, 2007), we strongly urged the users of cloud-based services to plan for the possibility of ”severe failure with significant operational and reputational consequences”. Taleb reminds us of the hapless 364-day old turkey who has no experience to tell him that Thanksgiving represents the end of the line. We also referred to quality guru W. Edwards Deming, who explained ”Experience by itself teaches nothing.”
It comes as no surprise to me that a public cloud provider can suffer an unforeseen incident that impacts millions of people. I admit to surprise over the form and scope of the Black Swan Event experienced by Megaupload last month. The extent of the enterprise use of their service also comes as a surprise.
In an interview yesterday with Bloomberg, Palo Alto Network’s Nir Zuk explains that Megaupload was the most widely used file sharing service within the enterprise. While this was a short and perhaps misleading interview, specific details on enterprise usage of cloud-based file sharing systems can be found in Palo Alto’s Application Usage Risk Report. My read of the latest version of this report is that Megaupload is indeed the largest bandwidth consumer within the enterprise (enterprise being those orgs that buy Next Gen Firewalls), with Dropbox being the #2 bandwidth hog. Megaupload traffic appears in 57% of enterprises, which is quite a lot, although 5 other vendors appear in a higher percentage of enterprises. Dropbox wins that race,with their traffic detected in 76% of enterprises.
In one fell swoop of the black swan, users in over half of enterprises suddenly lost all access to one of their highest bandwidth external services. While this particular case probably came as a relief to a lot of IT managers, not all of the Megaupload traffic represents bootlegged multi-media content. The service did have a reputation in the power user community for being fast and scalable, attracting people who had a legitimate need to share their own bulky content. Hopefully, most of the enterprise users savvy enough to recognize it for those advantages, were also prudent enough to avoid using it as their primary storage for important files.
Category: Cloud risk management security Tags:
by Jay Heiser | January 31, 2012 | Submit a Comment
Leverage and scale are two of the most fascinating aspects of Cloud Computing. In one fell swoop, the US Department of Justice burst Megaupload’s cloud, immediately sending a loud anti-piracy message across the entire globe.
If it is truly the case that Carpathia is hosting 25 Petabytes of Megaupload customer data, and that is only part of what has become inaccessible through the 19 January Department of Justice shutdown of Megaupload.com, then this represents the biggest cloud incident by orders of magnitude. (See my blog entry How much of your data is lost at Megaupload for the full story)
It isn’t easy to balance the desires of the entertainment industry against the baby pictures and small business records belonging to millions of naive Internet users inside and outside the USA. I’m entirely sympathetic to the frustrations of the DOJ, whose simple anti-piracy act resulted in such incalculably high levels of collateral damage.
In an interview with PCMag.com, a DOJ spokesperson explained ”It is important to note that Mega clearly warned users to keep copies of any files they uploaded..” Readers of this blog, Symposium and Summit attendees, and Gartner clients have heard me give this same advice far too many times: “do not store important data in somebody else’s cloud without keeping a copy somewhere else.” Yet individual users, just looking for a cheap and convenient place to park files, insist on not following this sound advice. The DOJ and I remain mystified at this widespread lack of policy compliance and good sense.
The good news is that the Electronic Frontier Foundation is also willing to leverage this incident for all its worth. Today they announced that with Carpathia’s support, they are going to think really, really hard about how to solve the difficult problem of separating out the huge amounts of illegal content, and getting what’s left back in the hands of several million users, only a few of whom actually paid for this wild ride.
Futher good news is that Megaupload’s US Attorney, yet another party who deeply feels the users’ pain, has said that both Cogent and Carpathia will maintain the material for at least 2 more weeks. Although they are apparently not obligated to do so, and almost certainly won’t get paid for it, Cogent and Carpathia might yet figure out a way to provide access to the non-pirated content, through a web-based front end that they apparently don’t control.
I’m going to go out on a limb and predict that we have not yet heard from all the parties who can leverage an incident like this. Its not a question of whether or not there will be a lawsuit–its a question of how many there will be. Maybe Congress can come up with a data amnesty bill.
My serious suggestion is that IT decision makers leverage all the media attention around this unfortunate incident and use it as an opportunity to figure out what their users are up to, and help their users recognize that a carefully selected and paid for service is better than something they are likely to dig up on their own. Gartner customers looking for advice on meeting the needs of users for external file storage should take a look at my latest research, How to Control File Synchronization Services and Prevent Corporate Data Leakage.
Category: Cloud security Tags:
by Jay Heiser | January 30, 2012 | 4 Comments
On the 19th of January, US authorities shut down a popular file sharing service, Megaupload.com, impacting millions of users. The whole sordid story, along with much of the backlash and legal discussion, can be found on Wikipedia, and short version in a press release from the US Attorney’s office. . A flurry of Jan 30 news reports suggest, probably erroneously, that this customer data will be deleted on Thursday.
Like many file sharing services, Megaupload was merely the top link in a Chain of Providers. According to AP ”A letter filed in the case Friday by the U.S. Attorney’s Office for the Eastern District of Virginia said storage companies Carpathia Hosting Inc. and Cogent Communications Group Inc. may begin deleting data Thursday…..The letter said the government copied some data from the servers but did not physically take them. It said that now that it has executed its search warrants, it has no right to access the data. The servers are controlled by Carpathia and Cogent and issues about the future of the data must be resolved with them, prosecutors said.” The letter, which is not indexed on either the DOJ or Federal Court web site, apparently allows the providers to delete the data, but does not necessarily require them to do so. Given that Megaupload’s financial assets are frozen, their hosters certainly have strong financial incentive to reclaim all that floor space (Carpathia is reportedly storing 25 Petabytes for Mega, and it comes as no surprise that the DOJ didn’t attempt to seize 1000 servers).
Without a running front end application, there’s no mechanism allowing customers to log in and access their data. How else could anyone make any sense of the millions of files stored at Carpathia and Cogent? Depending upon the support arrangement for the servers, hosting providers likely have no need to know what is stored or how to access it. This was made clear in a press release this morning ”Carpathia Hosting does not have, and has never had, access to the content on MegaUpload servers and has no mechanism for returning any content residing on such servers to MegaUpload’s customers. ” (They also explicitly denied awareness of any sort of instruction for a Feb 2 deletion)
I sincerely doubt that any Gartner clients have formally contracted with Megaupload (let alone some of their sleazier porn-related sites) as a cheap (no pun intended) form of collaboration or file backup. But I am certain that individuals within thousands of organizations, having decided that it was a useful service that their own IT departments refused to provide them, had uploaded corporate data into Megaupload. If that data wasn’t backed up, it is almost certainly gone for good. This is neither the first nor the last case in which a SaaS provider disappeared overnight, effectively taking all of its customer data with it, but it may well be the largest data loss from a SaaS provider. The fact that the data is still extent, yet inaccessible, must be especially frustrating to those who have just lost their sole copy of family photos or corporate documents.
I was going to say that as a best practice, companies that store significant amounts of pirated or otherwise illegal content should be avoided. Then I realized that this is virtually impossible. Carpathia and Cogent, like Amazon and any other hosting service provider, always have huge amounts of illegal and unsavory content within their infrastructures. At least in this case, it reportedly was stored on dedicated servers, not multi-tenanted ones. Let me be more precise and suggest avoiding multi-tenant SaaS offerings that are likely used by pirates. Freebie web sites that provide public file sharing are almost certainly chock full of unsavory content, and are obviously not suited for enterprise use.
This might be a good time to figure out if your users have been uploading your corporate data to Megaupload or some other freebie file sharing site. It should also serve as a reminder that accessibility to data within a SaaS provider is dependent upon the ongoing viability and competence of that provider. If you have important data within a service provider, you need a contingency plan in case that provider disappears.
Category: Cloud security Tags:
by Jay Heiser | December 23, 2011 | 2 Comments
I’ve reviewed three different policies so far this month, all of which contained the a similar requirement that users not write down their password. How counterproductive is that?
It is impossible to impose any level of password complexity and regular expiration, without expecting that the password holders write the things down for reference at least until they’ve memorized the new password. Demanding that users not write down their passwords is a quarterly opportunity to send the message that security policy is a useless bureaucratic exercise.
One thing that I have not seen popping up in security policy is a requirement that users not use the same password for the multiple unsynchronized systems that are inevitable within an organization of any size. Including work-related eZines and a memory stick that disappeared years ago, I’ve got 30 different Gartner ’accounts’ and passwords stored in my personal password database. That database has over 200 secret authentication strings stored in it, which leads to the point of today’s blog, that corporate password policies should forbid the use of ‘home’ passwords on corporate systems.
The reuse of passwords across multiple accounts is a well-recognized phenomenon within the security community (even security specialists have stumbled into self-imposed cascading password exploit incidents). Multiple published studies indicate an unfortunately high level of password duplication. While most people intellectually recognize this as the case, I’m not sure how many have fully understood the implications. The use of the same password for multiple accounts represents a single point of failure, in which the (dare I say lazy?) actions of an individual can result in multiple simultaneous failures.
I recognize the impracticality of preventing the shared use of passwords on both personal and employer systems (I won’t suggest that corporate surveillance tools could easily identify the majority of personal passwords, given the number of people who access FaceBook and gmail from work). Even though it cannot be reliably enforced, I see nothing but advantage in encouraging employees to avoid reusing passwords.
PIN and password proliferation is one of the unfortunate and virtually insoluble challenges for the digitally active individual. I try to maintain unque passwords for several hundred systems, and my dependence upon several synchronized copies of an encrypted password store is only one of the several inconveniences, and I am not recommending it. I do recommend that until we have widespread availability of a stronger mechanism that scales to dozens of corporate sites, and hundreds of personal sites, individuals should try to perform a degree of password proliferation.
Enterprise policy writers must recognize that only a small number of humans have photographic memory, and the proliferation of accounts and passwords, and the frequency of changes, dramatically increases the number of random strings that need to be protected. Memorization is impossible. Instead of telling users not to write down their passwords, ask them to treat passwords as carefully as they treat their own money.
Category: Cloud IT Governance risk management security Tags: passwords, policy
by Jay Heiser | December 14, 2011 | Comments Off
I review a lot of corporate security policy material, and it is the rare organization that doesn’t have an ITsec policy statement amounting to “all employees must obey the law.” Well, yes. I reviewed one today that managed to get this point across 4 times during the first page and a half, a remarkable achievment for a 2 page document.
Its easy to imagine where this nearly universal policy compunction comes from. In a legalistic society, the Simon Says Rule, and its inverse, trumps all other considerations. When it comes time to fire someone, its just so much easier if they can’t use the Simon Didn’t Say Rule to claim “you didn’t tell me I wasn’t supposed to break the law.” It is ironic that the legal field puts such emphasis on the reasonable man standard, given that so much of what happens in law defies common sense. To loosely apply the reasonableness standard in this situation, would there not always be an expectation that a reasonable person would consciously obey the law?
Even worse is a policy statement such as “all employees must obey all applicable laws.” What reasonable person could disagree with that? I would. That is tantamount to telling your employees “we will not tell you what specific laws you must follow as part of your job, let alone how to follow them, but if you do not, then we will punish you.” If your own organizational lawyers can’t specifically state what individual legal responsibilities are, why should there be any expectation that non-lawyers will be able to do so?
Vague and unactionable policy, full of ambiguous legal language is a corporate cop out. If you want your people to do the right thing, you must give them sufficient guidance such that they know what that thing is. If you want people to avoid doing certain classes of act, then you must provide sufficient details such that they can reasonably know what they must not do. A policy requiring employees to “do everything we should have thought of but didn’t” is a weak policy that will have virtually no impact on reducing risk.
Category: IT Governance risk management security Tags: law, policy, security
by Jay Heiser | December 1, 2011 | 1 Comment
I heard a story on the radio today about an old man who had donated an overcoat to charity, forgetting that one of the pockets contained his life savings. It seems that he was one of those people who are uncomfortable with the idea of trusting their money to a bank. Its been suggested more than once that avoiding public cloud computing is tantamount to keeping your money in a mattress. Given what’s happened over the last 4 years, why would anyone automatically assume that the use of banks represents a low level of risk?
The history of banking has been checkered with countless failures. Many people and many institutions have been ruined when their financial institution failed. Today, in many parts of the world, your money is safer in a bank than it is in your mattress, but this is a recent and unique situation in the several millenia evolution of the banking sector. Banks were not really ‘safe’ until risk transference mechanisms were developed in the 20th century, and only national governments had the ability to underwrite the risks, insuring individual deposits and in some cases, intervening to ensure the economic viability of a major bank. Given the European debt crisis, this seems a particularly apt time to question the relative advantages of mattresses and banks.
Banks are highly-regulated institutions and their customers can rely on multiple forms of government intervention, and consumers can rely on government insurance to prevent the complete loss of deposits. Bank customers benefit from huge levels of transparency. In contrast, clouds are a brand new form of depository institution with minimal transparency, no risk transference mechanisms, and no expectation that national governments consider them too big to fail.
In practice, moderately-skilled private individuals, let alone small to medium business, are able to provide an in-house IT service that far exceeds the reliability of a mattress as a piggy bank. Individuals have no ability to create duplicate copies of cash, but data can be easily backed up and stored off site. Cash must be physically protected from theft, but data can be encrypted. The suggestion that the avoidance of public cloud computing is tantamount to keeping your life savings in a mattress is a deeply flawed one, if not an outright lie.
Category: Cloud risk management security Tags: backups, disaster recovery
by Jay Heiser | November 28, 2011 | 2 Comments
With the understanding that I am not a lawyer, and Gartner is not a law firm, here’s my brief summary of the contractual language dealing with SaaS security as provided by a prominent vendor:
- We believe that we obey the law. If there are any questions pertaining to how your data is handled within our system, it is YOUR problem.
- We won’t give your data to the police. Unless we do give it to the police.
- When this contract is over, you may have the ability to get your data back, but that is your problem, not ours.
- If one of your customers contacts us, we won’t give them anything. Unless we are forced to give them something.
- We will store the data in whatever country we want.
- We might have third parties help us with this, and they of course would be held to the same weak levels of standard as we contractually obligate ourselves to follow.
- You the customer are obligated to obey the law at all times, even if you have no idea what that may entail. (Guess what happens if there is a dispute with us and our lawyers can find some way to demonstrate that you didn’t completely follow the law.)
- We will follow appropriate security measures—as understood by us.
- We will back up your data at least once a week, we will review our procedures periodically, although this seems unnecessary, given that none of these procedures were knowingly designed to fail. If we have the slightest plan for testing our ability to recover, we are not sharing it with you and we hope that you won’t ask that question.
- If any of our support personal ever accesses your data, by definition, it is necessary access.
I make light of the sort of legalistic word games typical of SaaS contracts, but given the constant stream of complaints from Gartner clients about this form of service, who can blame me for being more than skeptical? If a potential buyer is looking for a contract that clearly protects their interests, and is presented with an inflexible document that, in highly obtuse language, spends far more time describing what the vendor will NOT do than what they intend to do, why shouldn’t buyers be frustrated? As long as enough buyers are willing to give money to a service provider on the basis of what amounts to that provider saying that they will try to do a good job, and without actually accepting any significant level of risk if they don’t, then who can blame them?
Category: Cloud Vendor Contracts risk management security Tags: disaster recovery
by Jay Heiser | November 17, 2011 | 2 Comments
In the Disney cartoon Dumbo, a misfit elephant discovers that his ears are so large that he has the ability to fly. Lacking confidence in this unusual capability, he is understandably reluctant to fully exploit it. His shrewed friends, a pack of crows, come up with an ingenious psychological ploy. They give him a single feather, explain that it has magical properties, and as long as he clutches it in his trunk, he will be able to fly. This ruse is wildly successful, enabling the young pachyderm to soar with the crows.
This story is compelling because it illustrates a common form of human behavior. Desiring to do something, and unfamiliar with the associated risks, we often latch onto talismans, superstitiously hoping that they will protect us from unforeseen disasters. We do the same thing in the IT world, vainly clutching contractual SLAs in order to gain the false confidence that it will enable us to safely fly in the public cloud.
A typical example is a security document I recently reviewed which explained how this particular enterprise could safely rely on a specific SaaS vendor to reliably protect, and if necessary recover, their data, because it was ‘protected by a high level of SLA’. AN SLA IS NO MORE THAN AN EXPRESSION OF INTENT; IT IS NOT EVIDENCE OF DELIVERABILITY.
I’m not saying that you should not seek very specific service level agreements in your contracts–by no means. What I am saying is that the mere fact that a vendor promises to do something does not mean that you as the buyer can rely on it to happen.
If your business depends upon having access to data or services that are hosted externally, then you need to either have a tested contingency plan that functions entirely independently of that provider, or you need specific evidence that the provider is not only reliably making offline backups that are consistent with your recovery point objectives, but also that they have a proven ability to restore your data within your time objectives.
I have to admit that I’m completely at a loss to come up with some sort of test which would reliably demonstrate that in the event that even a small part of their client data was lost, say a petabyte worth belonging to a couple hundred thousand digital tenants, that a service provider would be able to restore it within a few days. As a recent case in point for what was a minuscule disaster, earlier this year it took Google 4 days to restore what they described as .02% of their gmail users.
What form of evidence could demonstrate that a provider with hundreds of thousands of tenants, if not millions, could, after some sort of data-eating disaster, reliably restore that data from offline backups and link it back into the accounts and applications such that it could be used again by their customers? How long would it take for the highly-leveraged administrative staff of a cloud services provider to complete such an operation? Where in the recovery queue would you be?
An SLA from a public cloud service promising some sort of recoverability can be a crow feather, clutched in the trunk of the enterprise elephant, providing them the false courage to be willing to fly in the public cloud. I hope that this is not a lesson that your organization will have to learn the hard way.
Recent Gartner research on this topic includes ‘Black Swans’ Are Sure to Fly in the Public Cloud, and The Realities of Cloud Services Downtime: What You Must Know and Do.
Category: Cloud Vendor Contracts risk management Tags: continuity, contract, Dumbo, feather, recovery, SLA
by Jay Heiser | November 9, 2011 | 1 Comment
I have to wonder how many consumers truly understand that when they are paying money for rights-managed content that they have locked themselves into the future of a specific vendor. At best, this represents a lifetime relationship, and at worst, it represents the potential to lose years worth of music or books.
A perfect example would be this year’s business failure of the Borders book store chain. Borders offered an ebook service, and those who spent money to license this digital content now find that their supplier is missing. In the olden days, the business viability of your local book store had absolutely no impact on your ability to read whatever you might have bought from them. In the digital world, your continued ability to use rights-managed content, be it music, video, or books, is completely dependent upon the willingness and ability of a service to support it on your current device. You cannot access your purchase without the key to unlock it, and an application to consume it.
While most services download the key to whatever your initial consumption device is, be it a computer, a phone, a book reader, a tablet, or some other sort of player, there can be no reasonable expectation that this device will last for more than several years. Devices break, they are stolen, they become obsolete, and Windows requires periodic restoration. I’ve got 80 year old records that I can play, and 15 year old files that I cannot.
While many people are happy to only watch a movie or read a book once, I certainly am not the only consumer who has a large collection of books and movies that I expect to use for the remainder of my life (however long that may be). Books, LPs, and CDs are all relatively open, and in the unlikely eventuality that I can no longer purchase readers for either form of music disk, the migration path is simple, and entirely within my own control. In contrast, licensed digital media introduces two significant requirements for ongoing accessibility:
1) a license server willing and able to serve replacement keys to legacy customers
2) reading client software for new devices
Clearly, a bankrupt Borders is going to be unable to fulfill either of these requirements. In this case, the licensing agency had a minority stake in Kobo, which has apparently agreed to take on support for existing Borders customers (although they are actually being somewhat non-committal). In practice, this probably doesn’t represent a huge impact for Borders customers–at least for those who figure out how to transition to Kobo while the offer is still outstanding. What it does mean is that Kobo will be providing licensing and software support to some group of people who have never paid them for it, and in the future, may well never pay for any Kobo content.
I’ve experimented with both Microsoft’s media player and Apple’s iTunes, and found both to be quirky. Effective use of either means entering metadata, like ratings, that is probably not portable, so even without paying for content, the more you use one of these products, the greater the level of vendor lock-in you are creating for yourself. For those who’ve given up on CDs and LPs (yes, they still press these things), the lock-in stakes are much, much higher.
From my POV, managed content is a bit more convenient than going to the library, but its not something that can be counted on for indefinite availability. If I’m going to listen to a tune multiple times, or if I’m going to mark up a book with annotations and highlighting, I’m going to pay for an unlicensed copy that I can control.
Category: Applications Cloud risk management security Tags: DRM, rights management, vendor lockin
