Kyle HilgendorfI’m pretty vocal when it comes to challenging Cloud Service Providers (CSPs) regarding increasing the amount of public transparency they share with not only customers but with prospects. On a very regular basis, I take calls from Gartner clients about the challenges in evaluating CSPs and the frustration with the lack of published information that exists at most providers.
I’ve seen some CSPs make some very good strides lately in terms of improving websites and publishing architectural and security related information. One particular aspect where the industry has seen very little improvement is transparency with audits.
A common discussion for me at Gartner has centered on SAS 70 Type II audits, and now SSAE 16 / SOC 1 reports. The latter has replaced SAS 70 and having an SSAE 16 audit and SOC 1 report completed by an independent third party is table stakes for competing in the public cloud services market. There are many problems with the SSAE 16 audit, namely that CSPs still get to designate which control objectives an auditing agency verifies. If a CSP does a poor job at logical access security, they could choose not to have the third party audit them against that control. It seems unfair and a loophole. As such, customers actually do need to see the SOC 1 report and must sign a confidentiality agreement with each provider to do so. That does not scale well.
But why a confidentiality agreement? Why don’t CSPs simply publish their SOC 1 report online? I’ve spent the last month talking to a number of CSPs about this. I get the token response that it would divulge sensitive security configurations that if published would put the cloud service in jeopardy of being attacked/exposed. My response to that is, “Ok, but let’s get creative.” I have not been able to understand why a CSP cannot publish a summary report listing each of the controls that were audited and the relative findings for each objective. There is a stark difference in mentioning that a third party confirmed security surveillance cameras are in place versus actually listing each physical location of all individual cameras.
Well after having several in depth conversations with many providers, I believe our cross hairs need not focus on the CSPs as much as the auditing agencies. More than a few of the CSPs have apparently gone to their auditing agency and requested the right to publish the SOC 1 report publicly. All providers that have done this were denied that ability. The auditing agency holds the copyright to the report and the legal agreements of the audit restrict the CSP from publishing without auditor consent.
A few providers claim they have gone further and have asked the auditor if they can takes portions of the report and publish as an executive summary or FAQ to highlight for customers the controls and summarized results. Again, those providers were not able to obtain the rights to do so.
What are these auditing agencies / large consulting companies needing to hide? If they truly are independent, third parties, why can’t they stand behind their report publicly? If not the entire report, why not a summary of findings?
Providers are not 100% absolved of any responsibility here either. Even if the auditing agency refuses to release any information from the report, the provider should still publicly list the controls that the provider asked the auditor to look after. That would be a big step for many providers and would at least start to level-set the playing field for customer evaluations. Furthermore, the best CSPs will put more emphasis on obtaining ISO 27001 certification, which does provide a base standard for controls.
I would love to hear from you on this. Are you a customer that is tired of signing agreements simply to confirm controls? Are you a provider that wants to publish more information but are restricted by auditors? Are you an auditor that would like to have a deeper discussion? Please contact me.
Category: Cloud Evaluation Providers Tags: Audit, Cloud, Evaluation, SSAE 16
9 responses so far ↓
1 Doug Barbin February 3, 2012 at 11:38 pm
Kyle,
Very interesting and well-written article and brings up issues around this topic that are rarely discussed. I’m happy to provide my 2-cents as a CPA whose firm performs hundreds of SOC examinations annually as well as someone who previously ran a managed security service business and was a SAS 70 “auditee.” I also volunteer for the CSA participating in the CloudAudit and Cloud Controls Matrix (CCM) projects.
Some feedback on a couple of your statements:
1.-> “As such, customers actually do need to see the SOC 1 report and must sign a confidentiality agreement with each provider to do so.” – Nothing in professional standards (that I know of) dictates this. Any such arrangements are at the discretion of those parties. At my previous employer we had an NDA for customers or prospects just for the SAS 70 report which was pretty straight-forward. I do think that an NDA is generally a good practice as the people that actually take the time to read the reports usually wanted to have a follow-up discussion about them which we were always happy to have.
2.-> “I have not been able to understand why a CSP cannot publish a summary report listing each of the controls that were audited and the relative findings for each objective.” – That’s really up to the CSP not the audit firm. A SOC 1 or 2 is an attestation which means we (the CPA firm) opine on assertions from management. The CSP can publish whatever they are comfortable with doing so. The only thing I have ever told a client was that they could not share selected parts of the report as all of the components from the system description to test of controls are meant to be consumed together.
3.-> “The auditing agency holds the copyright to the report and the legal agreements of the audit restrict the CSP from publishing without auditor consent.” – This one was actually news to me, but when I took a look at a few reports from Big 4 firms that have switched to BrightLine I noticed those types of clauses in the footnotes. I also know that at least one firm has an online repository with a EULA that the reader has to accept. Like the confidentiality agreement, there is no industry requirement or professional standard that this be the case. The CSP has the ability and right to push back. In our agreements, the client retains the IP rights to the report.
4.-> “What are these auditing agencies / large consulting companies needing to hide? If they truly are independent, third parties, why can’t they stand behind their report publicly? If not the entire report, why not a summary of findings? – The standards say “restricted use.” Authorized users should rightfully receive it. Other than that, I have no argument with you.
As a former product manager, there is nothing that stops a CSP for marketing their controls as features in more detail. The Microsoft 365 document posted on the CSA STAR Registry is a very good example. https://cloudsecurityalliance.org/star-registrant/microsoft-office-365/ What I like about it in addition to the fact that it maps to the CSA Control Matrix, it also cross references to the company’s ISO 27001 certification, which I agree is a far better vehicle for communicating security.
Let me know if I can help any further.
Best Regards,
Doug
Douglas W. Barbin, CPA, CISSP, PCI QSA
BrightLine CPAs & Associates, Inc.
http://www.BrightLine.com
CPA Firm • PCI QSA • ISO 27001 Registrar
Blog: http://www.thepragmaticauditor.com
2 Chris Schellman February 4, 2012 at 1:31 am
The criteria used for SSAE 16 examinations are relevant only for the purpose of providing information about the service organization’s system, including controls, to those who have an understanding of how the system is used for financial reporting by user entities. Paragraph .A63 of SSAE 16 confirms that the audit firm is NOT responsible for controlling a service organization’s distribution of a service auditor’s report” but may advise the service organization that the SSAE 16 report is “not intended for distribution to parties other than the service organization, user entities of the service organization’s system during some or all of the period covered by the service auditor’s report, and their user auditors.” Accordingly, virtually every SSAE 16 opinion letter concludes with a “restricted use” statement that the report and the description of tests of controls are intended only for use by management of the service organization, customers of the service organization (“during some or all of the period covered by the report” for a type 2 report, and “as of the ending date of the period covered by the report” for a type 1 report), and their user auditors (i.e., the external financial statement auditors of those customers).
This doesn’t mean that the service organization is prohibited from sharing their report with other parties, but any such parties are not “intended users” of the report. At the risk of oversimplifying, these parties would use the report for informational purposes only since any such use is outside the intended purpose of an SSAE 16 examination and resulting report.
For risk management reasons, some firms will attempt to obligate their clients to report distribution restrictions far beyond the professional requirements. One firm is even well known for putting a “click wrap” EULA on their reports that is entirely oriented around protecting the audit firm. Of course, there is no requirement that any of this be done and informed CSPs could (and in my opinion, should) refuse to agree to these tactics during contract negotiations.
I would also note that audit firms do not have intellectual property rights to the contents of SSAE 16 reports any more than they have such rights to financial statement audit reports. Skim either type of report issued by any major vendor and you will not find copyright notices for the actual contents. I suppose a service organization could give up these rights during contract negotiations with an audit firm, but I think most organizations are savvy enough to understand that they own the work product of their third party vendors.
Regarding the “loophole” comment, no audit firm worth their salt would allow that hypothetical situation to occur without ramifications. If such logical access controls are relevant to the “system”, they must be included in the scope of the report. If included, the report is likely to be qualified for suitability of design and/or operating effectiveness issues. If management refuses to include the controls, the auditor is required to qualify the report for fairness of presentation, which would give the readers a full description of the issues. Either way, there is no place to hide for management assuming that the auditor properly executes the SSAE 16 audit.
The SSAE 16 standard is publicly posted here – http://tinyurl.com/74fdq74. Paragraphs .A61 – .A64 deals with the report distribution topic. Examples of opinion qualifications are found at .A69, but the SSAE 16 audit guide is a better source for further information on the topic.
Feel free to contact us anytime for clarifications on SOC reporting topics.
All the best,
Chris Schellman, CPA, CISSP, PCI QSA, ISO Lead Auditor
President
BrightLine
3 Auditors: A problem to CSP transparency | SSAE 16 | Scoop.it February 6, 2012 at 1:53 pm
[...] background-position: 50% 0px; background-color:#222222; background-repeat : no-repeat; } blogs.gartner.com – Today, 8:53 [...]
4 Kyle Hilgendorf February 6, 2012 at 2:40 pm
Chris,
Thank you very much for your comments. Both you and Doug have been very helpful and have provided great context and insight into the intention of the reports. Unfortunately, it seems that some auditing agencies are wrapping additional restrictions around the SOC 1 report that are not “required” by the standard. Perhaps that is the root of the issue and what I am trying to uncover.
Thanks,
Kyle
5 Kyle Hilgendorf February 6, 2012 at 2:42 pm
Doug,
Thank you very much for your comment. Very helpful and insightful. A few more comments from me in response to your replies:
1.) I’ve yet to speak to a customer who has seen a SAS 70 or SOC 1 report without signing an NDA. Furthermore, I’ve never spoken to a provider who publishes these reports publicly. That does not mean that situations do not exist, but I’ve spoken to a lot of customers and providers and it is a very large sample size.
2.) Providers have been telling me that their audit firms prohibit publishing any type of summary report that refers to anything within the report. You also mention that a CSP can not publish selected parts of the report. So I guess my question is: Is a summary report allowable or is it considered a selective part of the report? This can probably only be settled by individual agreements and lawyers.
3.) It may not be an industry requirement that the audit firm owns the copyright, yet it does appear to be quite common.
Thank you for pointing out the Office 365 information. Very helpful.
Regards,
Kyle
6 Doug Barbin February 6, 2012 at 11:31 pm
Kyle,
Thank you. Re: 1) the NDA, I concur with your assessment if that wasn’t clear. I generally also think having one is a good practice, not for the protection of the report but for the follow-up questions and discussions that happen afterwards. When I was at the MSSP, I always enjoyed discussing the report and supporting details behind it with clients who actually took time to read the report. Rare, but on occasion, I even got feedback that helped us improve our controls for subsequent years’ reports.
Re: 2) I think the issue here is publishing something that includes a statement that an auditor has bought-off on whatever the provider is saying when the auditor is really only allowed to buy-off on the report. Attestation reports generally have the following components:
-Opinion letter
-Management assertion
-System description
-Results of tests of controls
-Other information provided by the service organization (optional)
There is nothing prohibiting a service organization from publishing information about its system description or even about the controls that they assert to have in place. CSA/CCM/CloudAudit.org are also avenues to do this… From the auditors perspective if someone wants to see the results of an audit they need to look at the report in its entirety. Otherwise you risk certain information being taken out of context. Same would hold true for a company who publishes summary financial sales information on its website versus the financial statements themselves. The information is there but you would want to go to the audited source for more assurance.
Re: 3) it is common unfortunately especially when the bar is set by the Big 4 firms. I can tell you we do not and don’t plan to.
Best Regards,
Doug
7 Anwar Sadhe February 11, 2012 at 2:54 pm
Read was interesting, stay in touch……
[...] will take fairly long time for getting an improvement. One will need to have real patience [...]…
8 Orgreenic set February 14, 2012 at 12:42 am
Websites worth visiting…
[...] Blogpost: article .. stumbled upon this article with regards to [...]…
9 merchant cash advance info February 15, 2012 at 10:54 am
Best additions…
[...] yet still deliberating with an actual specific fashion that you influenced who [...]…