Comprehensive Guide to Hybrid Cloud Storage and Computing
What is a Hybrid Cloud?
Hybrid cloud is a relatively new approach to IT infrastructure that aims to bridge the gap between the two major ways of organising a business’s computing resources. Hybrid cloud combines a public and private cloud within the same organisation, with each cloud serving different applications and purposes: as the name suggests, some resources are run with in house, and others outsourced remotely. The idea is that an organisation can gain the best of both worlds by keeping certain key services on a private cloud but accessing the advantages of the public cloud for others that are more appropriate to this approach.
Public vs Private Clouds
Public and private clouds are similar in many ways but very different in certain critical aspects. In a private cloud, the storage and computing resources are maintained behind a corporate firewall, giving stronger security and greater control over the cloud and data stored within it. The downside is that a private cloud will typically be kept on the premises or in a single physical location, making it more susceptible to outages and damage from natural causes.
With a public cloud, the computing resources are supplied by a third party provider, offsite, on servers that also accommodate many other customers. There is less visibility and control over these servers. This can raise issues from a security and compliance point of view, but there are typically advantages of scalability and reliability, as well as disaster recovery.
The private server is not merely connected to the public cloud. In a hybrid cloud the private facility will be running cloud services, allowing for the full features and benefits of a cloud architecture. Adopting a hybrid cloud approach allows an organisation to access the benefits of both private and public clouds, without being limited by the downsides in each case. Public and private clouds are designed to be used independently of each other. Communication between the two takes place over an encrypted connection, so that data can be securely moved between the two as and when necessary.
Top 3 Hybrid Cloud Providers
There are a number of popular and well-regarded hybrid cloud providers, including Rackspace, Amazon Web Services and Google – with plenty of smaller companies gaining market share. When comparing different services, there are a number of factors to consider, including security, speed, and ease of set up and management.
1. Amazon Web Services
There are a couple of caveats. One is that Amazon have suffered some high profile outages, making some people think twice about purchasing their services. However, they are not alone in this respect and other major providers have had to contend with similar problems. The other issue they face is one of reputation. Quite simply, Amazon are an online giant who make enormous profits from both understanding their customers’ spending habits and being able to undercut the competition. Of all the hybrid cloud providers, Amazon is the one that business leaders express wariness about using, for that reason.
2. Rackspace (OpenStack)
One of the foremost of the smaller players behind the giant Amazon, Rackspace has carved out a niche in the hybrid cloud space thanks to its open source approach with OpenStack. Whilst it may not be able to compete on price or sheer breadth of offerings alone, Rackspace has committed to their services – especially customer services – and have appealed to enterprises in a way that Amazon has not yet been able to manage and likely never will. The open source architecture gives users greater freedom and flexibility and positions Rackspace firmly as the non-corporate version of Amazon. This has given them a stronger customer base among large enterprises – which account for over 50 percent of their customers.
It’s this more managed approach to infrastructure provision that is their key differentiator. Amazon sells services, and tech-savvy customers will be happy taking these ultra-low-cost services and turning them into something their organisation can use productively. Rackspace has anticipated a customer base that cannot afford or does not want to hire an expert to set up a hybrid cloud system and capitalised on this need.
Another giant of the internet, Google has the capacity to compete with Amazon in a way that other providers do not. The search firm already has a large number of data centres configured to allow rapid scaling, which arguably puts them at an advantage as far an infrastructure goes. The tech company’s background means it can go head-to-head with other high-volume, low-price outfits, and this is exactly what it has done: like Amazon, Google’s business model is about selling services without offering a great deal in the way of support. In fact, Google and Amazon have fought a brisk price war in a kind of race to the bottom for low-cost IaaS, leaving Rackspace to expand into the managed niche.
In January 2015, Google entered a partnership with VMware that would give its hybrid cloud offering a real boost, particularly to enterprise customers. VMware benefit from Google’s public cloud and services, whereas Google now has far more to offer in the hybrid sphere. That partnership allows them to compete favourably with other large providers, such that they can consolidate their position as the #2 hybrid cloud service.
Understanding ‘the Cloud’
Cloud computing is a frequently misunderstood term. Because data is saved ‘to the cloud’, it is easy to imagine an impersonal, decentralised system to which files are transferred, and that can be securely accessed at any time. The reality is that ‘the cloud’ is a physical location, with all the implications that brings. Your ‘cloud’ is actually a data centre somewhere in the world. It has real servers, located in a specific jurisdiction. It has real employees, who will have received a certain level of training and security clearance; it has real power requirements, is subject to certain environmental conditions, uses particular hardware, and so on.
These factors raise a number of questions around security and reliability. For this reason, many organisations will be wary of using the public cloud for some operations, even though it may offer advantages in terms of cost and scalability. In certain sectors, there may be very clear legal or ethical reasons why data cannot be outsourced to another provider. In other cases, it may not make practical sense to run critical services from a public cloud – whilst the same organisation might make use of the public cloud for other applications.
For example, an organisation might choose to maintain their operational customer data on a private cloud, perhaps located on their own premises. Non-essential data or material that is no longer used on a day-to-day basis may be archived, encrypted and stored on the public cloud. This is a scalable approach, but still ensures adequate security.
When setting up a hybrid cloud, it is possible to start from either an existing public cloud setup, or a private cloud/data centre setup, since it is a hybrid solution. However, depending on what you plan to achieve, one or other of these starting points might make more sense. Hybrid cloud systems can be created at least three separate ways:
1) Different cloud providers offering private and public clouds respectively work together to give an integrated service.
2) A given cloud provider offers both public and private cloud packages in single, integrated hybrid service.
3) A business or other organisation that already has its own private cloud service adds a public cloud offering, which is then integrated with their existing infrastructure.
Advantages of a Hybrid Cloud Setup
A hybrid cloud implementation has a number of considerable advantages over the alternatives of in-house services/private cloud or public cloud services alone. It will not be suitable for every application, but in many cases it does offer significant benefits of security, scalability, cost, speed and legal compliance.
Security. Because the private cloud is maintained behind the organisation’s firewall, it can be far more secure than a public cloud (assuming the correct precautions are taken). This allows an organisation to keep better control of critical data, whilst still accessing the conveniences of the public cloud for data that is less sensitive.
Scalability. Private clouds come in different forms and will often provide a limited level of flexibility. This will partly depend on whether the cloud is hosted internally, on your own servers, or externally in a data centre that may have some ability to increase storage and computing power if you need it. Nevertheless, public cloud services tend to be far larger and will be able to scale far more quickly and effortlessly. Companies like Amazon and Google serve thousands of large customers and have plenty of spare capacity, enabling rapid expansion of your own infrastructure if circumstances demand it. This gives the option of keeping critical services on your private cloud whilst moving everything else onto the public cloud, which can be increased at a moment’s notice if required.
Costs. You will need to price up the different options for storage. Data growth is a big problem for organisations today, with unstructured data (particularly Office documents and images) increasing at an exponential rate. This can rapidly fill in-house storage, necessitating one or other strategies to increase it – typically moving to the cloud, restructuring or using a compression/optimisation utility to free up space. Each of these have their own advantages and disadvantages. Moving to a public cloud solution is a straightforward way to access all the space you need – though for this reason, it can become expensive, especially if best practices are not maintained. It also avoids the downtime and up-front costs that restructuring entails. The nature of a public cloud is that the economies of scale it allows outweigh the upsides of expanding private cloud storage. However, you will still have your private storage available for the most sensitive data.
Speed. One often-overlooked implication of using the public cloud is that everything will need to be accessed through your internet connection. In most cases, this will not cause a problem and you may not even realise that this is going on: upload and download speeds are fast enough that you won’t experience much difference from accessing a private cloud or in-house server. However, depending on the nature of your organisation, this could pose a significant problem. Users may not notice the difference, but for high-speed computing applications, the latency involved in transferring data to and from the cloud could cut your competitive edge and outweigh the cost advantages of using a public cloud. In these circumstances, having a hybrid solution allows you to keep in-house computing resources at hand for the most intensive tasks, whilst leaving the public cloud for less critical data or to top up your resources at times of peak demand.
Connecting to your data via the public internet may have speed implications if you are located in certain areas of the world, or if you require particularly high bandwidth (for example, if you are dealing with the transfer of very large files). Aside from this, there are risks inherent in using a connection to a public cloud. It is not yet clear what effect new and proposed legislation will have (such as the concept of Net Neutrality, or a reduction in the number of Internet Service Providers, in the US). This introduces a level of risk that may not be necessary or wise for your most sensitive data and operations. Keeping this data in-house enables fast and reliable access at all times, regardless of the speed and status of your external internet connection.
Flexibility. Flexibility is built into a hybrid cloud offering. By its very nature, there is capacity to offload more data onto the public cloud or to keep it in the private cloud. The line between the two clouds is clear, but moving data between them is nevertheless straightforward. In a similar way, in-house computing power can be supplemented at peak times by outsourcing tasks to public cloud computing.
Regulatory compliance. This is a major factor for organisations that deal with sensitive data, or where there are particular restraints on how data is handled. Maintaining a private cloud provides a greater degree of security and control, which can also contribute to satisfying any legal or regulatory requirement for the storage of data. At the same time, the superior scalability of the public cloud can be drawn on for less sensitive data where there are not the same regulatory issues to contend with.
Downsides of a Hybrid Cloud Setup
Hybrid cloud has several clear advantages over an exclusively private or public cloud infrastructure. However, it’s also worth noting that a chain is only as strong as its weakest link, and each element of the hybrid cloud is subject to its own vulnerabilities (lower speed and security for any data held on the public cloud, for example, and poorer scalability for data storage that needs to be kept in-house or in a private data centre). In some cases, sending data across a network of any kind, where it is potentially subject to interception, poses an unacceptable risk. Beyond that, there will be a couple of major inherent drawbacks to a hybrid cloud system.
Cost. The benefit of a public cloud is that organisations can access the cost savings that come with being able to offer economies of scale: public cloud storage comes cheaply. It’s scalable on a granular level; organisations won’t have to purchase a whole new server, because they will – within reason – only pay for the storage they need and use. Those cost benefits are badly compromised by the need to purchase and maintain in-house servers or rent storage at a private data centre. At this point, you are looking at investing in racks of hardware that come with certain fixed and recurring costs. These are expensive, especially if you are dealing with any significant volume of storage. Smaller businesses and organisations with limited IT budgets are simply likely to find these too expensive to be worthwhile. If that’s the case, the only real alternative is to continue to use the public cloud alone.
Latency. Another issue that a hybrid cloud operation may raise is when data must be transported quickly across the network from both public and private clouds. For applications that are particularly time sensitive, the latency involved in moving data or carrying out computational work in the public cloud may preclude a hybrid solution outright. One example of this was given by Tatsuya Kimura, the head of international affairs at the Japan Meteorological Agency (JMA). Kimura has expressed doubt about whether prediction data for weather patterns could be uploaded to the public cloud fast enough to gain responses in a useful time frame. JMA currently uses a custom-built supercomputer to analyse data from earthquakes to predict whether they need to issue a tsunami warning. It is also used to predict earthquakes in certain regions of Japan. The speed with which these calculations need to be carried out, and the volume of data involved, means that moving the processes to the public cloud is not currently a workable solution.
Hybrid Cloud Use Cases
There is generally a fairly clear distinction between the kinds of services that can be maintained on a public cloud and the ones that should be kept on a private system. A private cloud will naturally be more suited to those applications for which security is paramount. Where cost is the key factor, a public cloud will be the default choice. For example, online merchants might choose to keep customer details and the backend of their store on a private cloud, due to the enhanced security and control this allows. At the same time, material that is less sensitive – information for help and support, product details, contact information and so on – can be uploaded to a public cloud, which will cost far less. Financial details of any kind will almost always be kept on a secure private cloud. There may be other documents, though, that are edited or used by many different people both within the core team and outside of it. In these instances, a public cloud offering will likely make more sense.
Healthcare. The healthcare industry is a prime use case for hybrid cloud IT infrastructure, due to the large quantity of confidential patient information that is transported between healthcare providers, insurers and other groups involved in the healthcare system. The Health Insurance Portability and Accountability Act (HIPAA) raises regulatory issues for the movement of this data, because compliance with the Act entails keeping protected information confidential – whilst there is, of course, still the need to communicate other information with third parties.
Retail and Big Data. In an age of big data, enormous amounts of information are collected, stored and analysed to understand patterns of sales and customer behaviour. The storage of this volume of data poses a problem for in-house or private systems, which cannot scale as well. At the same time, analysing the data is an undertaking that requires high levels of computing resources. It therefore makes sense to draw on both public and private clouds, depending on the specific needs. Many retail firms are also wary of using public cloud infrastructure from the larger providers; there is a scepticism about the security of these services, especially when the providers may be seen as direct competitors to the retailers themselves.
Law Firms. For regulatory and compliance reasons, law firms need to store client information on-site or in a secure cloud facility. Nevertheless, this can present a problem in that there is a single point of failure, and loss or damage of hardware would compromise this information without the chance of recovery. The solution is to encrypt it and store it on the public cloud, so that it can be regained if there is ever a problem with the local data store.
Finance. Latency is a particular issue in the finance sector, where even a few microseconds can make a difference between profit and loss. In these circumstances, physical proximity with the servers is key, and using a public cloud infrastructure is completely impractical. Because proprietary algorithms form such an important component of many trading firms’ strategy, there are also security considerations at play. However, there is also the need to store and analyse large volumes of financial data. Outsourcing this to the public cloud frees up space and computing power for the trading engines.
On-site Hardware. Lastly, organisations may have many physical devices and systems on their premises that are not suited to integration with public cloud services. Day-to-day hardware including printers, scanners, telephone and fax systems, as well as security devices such as cameras and smoke/fire detectors cannot easily be used with a public cloud approach. Using the private cloud for such so-called mission critical hardware whilst maintaining a public cloud offering for other data is a viable solution.
Although the Cloud is now a household term, the idea of the hybrid cloud has not yet gained such popular traction. ‘The cloud’ is typically used as a term for offsite storage, whether on an individual level – often for photos and other personal files – or for businesses. Whilst the public cloud is extremely useful in offering large amounts of storage and computing power scalably and at a low cost, it has its drawbacks in terms of security, control and latency.
The emergence of the hybrid cloud highlights these drawbacks and is a way of bridging the gap between the speed and security of the in-house, private cloud and the convenience of the public cloud. More and more public cloud providers are offering hybrid services. Whilst many of these simply allow enterprises to access low-cost infrastructure – Amazon and Google being the two largest players in this space – others have emerged with more specific propositions. Rackspace is the foremost alternative to this trend, with their managed hybrid cloud. They have also gained a good reputation through its use of the open source OpenStack platform.
Ultimately, the nature of the cloud your organisation uses – private, public or hybrid – as well as the configuration of that system, will be down to your specific needs. The good news is that with a large and growing number of providers, and due to the inherent flexibility of cloud services, you will almost certainly be able to find something that suits your requirements.
Sources and More Reading
Abramson, Steven, William Horka, and Leonard Wisniewski. Harvard University. A Hybrid Cloud Architecture for a Social Science Research Computing Data Center (n.d.): n. pag. 2 Sept. 2014. Web.
Rodriguez, Mario. Advantages and Disadvantages of Hybrid Cloud Computing in Business Mario Rodriguez University of North Texas
Breiter, Gerd, and Vijay Naik. “A Framework for Controlling and Managing Hybrid Cloud Service Integration.“N.p., n.d. Web. 17 Mar. 2015.