First, you should make sure that you are clear about what is most important to the consumer in terms of performance. For some it may be availability or system responsiveness, and others a timely back up schedule. I recommend listing the attributes of the service that are most important, and then figuring out how to measure those attributes in the most systemic and meaningful way possible. I had one agency tell me that a major problem came up because two vendors were using different sources of time. So you may even want SLAs on how time is used and what source it comes from.

Next, as it turns out, coming up with good metrics is one of the most difficult steps to establishing an SLA. Let's takes something as seemingly strait forward as availability. How do you measure it? Is it the availability of the infrastructure and network or the availability of the application? If it is the application, which applications are critical to the end-users? How do you handle scheduled down time? There is a twist to choosing metrics. In some cases metrics can drive unintended behavior. For example, we used to measure programmers by lines of code delivered (or bugs fixed). The natural result: very long, monolithic, inefficient code. Many of these unintended behaviors can be hard to predict. Make sure you have a way to adjust the SLA if it is creating unintended behavior or just not working for you.

Finally I would recommend that your SLAs have real teeth in them, aligning risk and reward appropriately. If the risks are outside of your immediate control, the SLA should address the risk and the consequences. If the SLA does make the provider a little uncomfortable, they may be more responsive and deliver quicker solutions. In either case, both parties must be involved in creating, monitoring, and fixing SLAs.

In summary, first focus on the attributes most important to you, remembering relationship is important. Next, build good metrics for those attributes. Monitor the SLAs to make sure they are working for you and change if necessary. Finally, make the consequences of failed SLAs painful enough to promote quick response.

For further information, please contact Gregg (Skip) Bailey at: gbailey@deloitte.com

Posted: September 23, 2010

1) An MOU won't cut it: Although it can express an obligation between the service provider and the agency, an MOU is not a strong enough document to govern the relationship. MOUs fall short of being an enforceable contract. Because the outsourced services may be mission critical and involve fundamental building blocks such as infrastructure or platforms, it is far more effective to leverage a contract that describes what services are outsourced, what the responsibilities of both parties are and what the performance characteristics will be.

2) SLAs and contracts are the same thing: Outsourcers have used a specialized version of an SLA called an underpinning contract (UPC) for years. This contract outlines the services the provider will deliver, the penalties and credits associated with under-and over-performance, and the metrics that will be used to describe how the contract will be delivered.

3) Measure performance, not just provisioning: Note that the SLA is a contract, not an operational performance characteristic such as "% uptime." Instead of SLAs or UPCs, these measures are "metrics," indicators of the cloud service's operational parameters. Because much of the external cloud market grew out of the virtualization space, many providers offer provisioning metrics, but steadfastly avoid performance metrics (% uptime, throughput, capacity, etc.). These cloud services are still fundamental building blocks for running your agency and must be protected operationally.

4) It is your data, and you are not doing the provider a favor: It's not uncommon for providers to keep performance metrics close to the vest. Many are trying to avoid obvious penalties and some go to great lengths to report performance only to the minimum required level. If a provider claims the performance data is proprietary, they are wrong. The agency needs to ensure that it has access to these metrics for a level of assurance against the obligations agreed upon in the contract itself.

5) Active -- not passive -- monitoring is key: Another way some cloud service providers might fail to report performance is to expect it of the agency). They leave it up to the customer to find the outage, report it, and collect the penalty. Instead agencies need to be proactive, either requiring the provider to implement a Service Level Management solution to actively monitor the agreement, or they need to do so remotely. In this way, both parties are able to agree to the performance of the contract.

For further information, please contact Erik Hille at: Erik.Hille@ca.com

Posted: September 24, 2010

Wrong! As much as we are (rightfully) intrigued by the technology of cloud, and as much as we impressed by the snappy and snazzy way cloud technology seems to respond to our needs, the real power of cloud is what happens around the technology. Clouds need technology for sure. But, trusted clouds need people, and process, and rules for operation, and governance, and accountability for outcomes even more. The technology of clouds is evolutionary. The consumption model for clouds is revolutionary. When those people and process and rules and accountabilities that are needed span organizations (internally or externally), then we inevitably must include some sort of agreed mechanisms for cloud service delivery, e.g., Service Level Agreements (SLAs), Memoranda of Understanding (MOUs), or contract terms and conditions (T&C's).

Okay, we're making progress. We know we need the important (revolutionary) mechanisms around sound (evolutionary) technology in order to generate and sustain trust in cloud operations and reap the payoffs that are promised to us. But, what should those mechanisms emphasize in order to capture the best payoff situation?

That’s the thrust of the question for September. And, as usual, the Government's desire to find the important characteristics of such mechanisms is not much different from that of industry.

The answer to that question lies in the recognition that:

In the cloud, 'security' begins with a 'T'.

Transparency is the single most important characteristic required to generate trust and capture payoffs. Beyond the standard characteristics of availability and incident response timeliness (for which SLAs are well known), additional SLAs, MOUs and/or T&C's should reinforce the characteristic of transparency in cloud service delivery. While the technology must support the delivery of transparency of service, it is the accompanying mechanisms of service definition that provide the real payoffs.

The Precis for the CloudTrust Protocol includes a list of the elements of transparency that can be the basis for an SLA that requires the measurement and reporting of such metrics. The CloudTrust Protocol is intended to provide a vehicle for such reporting. In addition, that same reference also describes a recommended SLA for self-reporting by cloud service providers (to reduce the chances of 'gaming' results). Whether in SLAs or MOUs or T&Cs, or even in standards and 'best practices' themselves, attention to the transparency of service is essential.

Just remember your spelling lesson for the cloud and payoffs can come your way.

See the full blog response at www.trustedcloudservices.com.

For further information, please contact Ron Knode at: rknode@csc.com

Posted: September 27, 2010

Organizations should make a cloud comparison, not against an ideal, but against legacy IT's reality. When organizations own and operate their own IT infrastructure, they assume the entire burden of providing reserve capacity and protection against interruption of capability. Backup storage, backup power, even entire backup data centers are routine expenses in the world of traditional IT. If these protections turn out to be inadequate, the user organization bears all costs of mission failure.

In contrast, cloud providers provide reserve capacity and redundant capabilities, such as backup power and backup connectivity, at a lower cost per customer due to massive economies of scale. Moreover, cloud providers have enormous incentive to assure that their customers do not experience service degradations that lead to unfavorable publicity. Further, each customer of a true multi-tenant cloud enjoys "sum of all fears" protection: the provider must satisfy all concerns of all customers, and in the process will generally address a superset of the concerns of any single customer.

If cloud providers price their services to cover worst-case consequential damages of any service limitation, as felt by their most demanding customer, the result will not be economically attractive to the vast majority of customers. Those customers will be better served when they make their own assessments of risk and cost, and mitigate those risks in a way that meets their own requirements.

Cloud data centers will be, for the first time, statistically similar enough to enable accurate pricing of risk: a vast improvement over the unique, unpredictable risk of a myriad of individual and uniquely configured on-premise data centers. We may therefore expect to see an efficient marketplace of risk that the IT professional has previously lacked.

Compare the cloud's risks to the risks that are experienced by the customers of a shipping service. Those customers make their own judgment of the reliability of that service, and of the consequences of any delay or damage or loss of a shipment. They purchase insurance, or keep extra inventory on hand, or take other measures to limit their exposure. In a similar way, the most risk-sensitive customers of cloud services will choose their own measures that are suited to their own particular circumstances.

A "service level agreement" does not give the customer what's really needed -- which is reliable, secure service that gracefully handles peak loads without the customer needing to own peak-load capacity. That's what the cloud is all about, and that's what cloud service customers are quickly learning to expect.

For further information, please contact Peter Coffee at: pcoffee@salesforce.com

Posted: September 29, 2010

But as cloud infrastructures have improved, access seems like a pretty low bar. If I'm a Public Sector CIO evaluating cloud computing options, I'm not willing to accept a significant decrease in access (uptime, availability, reliability) in order to gain the other benefits cloud offers (efficiency, scalability, cost reductions). A large part of my decision will be based on a cloud solution's ability to be there when I need it, and it shouldn't be much different the reliability of traditional IT infrastructures. Federal agencies can't afford regular, unexpected service interruptions. The data and the mission is too important. This is why data portability is essential. It gives agencies the ultimate option - to immediately relocate to another cloud provider if their service needs aren't being met. Agencies need the freedom to move their data to an environment they trust, and SLAs that include data portability language protect customers more effectively than any other metric or clause.

It's common for SLAs to include financial compensation for service outages, and that's an important start. Customers should be compensated for lost access, but if there are repeated, unscheduled breaks in service, that policy is failing to provide value. All enterprise organizations require consistent access to their computing resources, and when service needs aren't being met, data portability adds another layer of assurance beyond financial return.

It's true that service interruptions often occur because of network outages rather than issues with the cloud solution itself. Unfortunately, the result is the same for customers – lack of access. To limit these breaks in service, vendors should address minimum network connectivity requirements in the SLA. Network monitoring is a key component of a holistic cloud implementation, and vendors should continually and proactively work with network providers to ensure connectivity needs are being met. SLAs can address these issues at the outset, and can even outline network backup options like leveraging satellite connectivity.

Overall, SLAs are extremely important, but they are evolving as cloud offerings improve. Customers are rightly expecting more, and vendors must step up their game to deliver. Ensuring data portability in SLAs avoids vendor lock-in, promotes choice, increases competition and allows government enterprises to freely choose the best available solutions.

For more information, please see Teresa Carlson's FutureFed blog at: http://blogs.msdn.com/USPUBLICSector/

Posted: October 4, 2010

Other necessary and fundamental aspects of the relationship include:

• Control: Delineates the aspects of the service which are and are not under the control of the provider and is critical to writing an effective, enforceable SLA. The provider is held accountable for delivering a particular level of service which is agreed upon in the SLA; however, the provider should not be held accountable for failures that occur which are outside their control. The complexity of today's computing environments necessitate that the SLA clearly describe those aspects of the service which are and are not under their control. In general, descriptions such as this necessitate the inclusion of an architecture diagram to supplement the verbiage provided.



• Measurement: Ensures and demonstrates that the agreed upon level of service is delivered. Measurement encompasses measures and metrics. A measure is a value that is recorded as a result of a physical measurement such as a single instance of a response time. A metric is a quantitative measure of the degree to which a system, component, or process possess a given attribute. Metrics are the foundation of a well defined SLA; they must be objectively measureable or calculated from objectively defined measures. The lack of objectively measureable metrics may result in an SLA that is unenforceable.



• Transparency: Implies openness, communication, and accountability. A successful relationship is always based, in part, on trust and transparency is fundamental to trust. Transparency applies to many aspects of the SLA including the definition of unambiguous responsibilities and metrics as well as a clear understanding of the provider’s span of control. In addition, it is extremely important that the reporting process, scheduled reviews, and methods for computing incentives and penalties be completely transparent to all involved parties.

Taken together, responsibilities, control, measurement and transparency in an SLA can help to establish trust and facilitate a successful cloud computing relationship.

For further information please contact Lynn McPherson at cloudbloggers-list@lists.mitre.org

Posted: October 5, 2010