Cloud management is the set of processes and tools that an organization uses to monitor and control all of their cloud resources and services, including user profiles and permissions, data, and applications, whether they be from their own or from public cloud service providers (CSP). To help unify and simplify cloud management, cloud management platform (CMP) software allows IT administrators to manage multiple dynamic cloud environments and services from a dashboard environment, typically with advanced features, visualizations, and analytics.

Cloud management and cloud governance share overlapping concerns but they do address two different but complementary set of activities. Cloud management is concerned with the practical monitoring and implementation of cloud resources, e.g. automating analytics and cloud orchestration to uncover business insights and optimize resource utilization. Informed by those business insights and a company's business goals, cloud governance concerns itself with the policies that dictate how company cloud resources should be used by users and other systems, e.g. bandwidth limits on unessential web traffic, or prioritization of critical services over others to guarantee their availability and therefore honor service agreements. In short, cloud governance policies guide parameters around how cloud management does its job. And like many things in the cloud, cloud management and monitoring platforms often offer automation functionality that can help alert when policy violations occur and then take action to enforce policies.

The challenge solved by cloud management is to unify a single view and control over a multitude of cloud resources. While CMPs help to standardize a visualization of these resources, by pulling together data from disparate cloud services and combining them through analytics, the use of a larger framework is needed to ensure that all management bases are covered. The following general management domains outline the major areas of management.

  • Automation and Orchestration DomainCloud services typically provide a layer of automation to address repetitive, error-prone, and administrative tasks like setting configurations, backing up data regularly, or managing servers. Efficient cloud orchestration is dependent on automation techniques, and so is closely associated with it. Orchestration refers to the complex coordination of creating, managing, and destroying virtual machines and containers, as well as allocating appropriate shares of the resource pool on demand, a sophisticated coordination process potentially encompassing thousands of servers running even more VMs.
  • Security Domain — Using cloud services from the public cloud introduces new security issues. By shifting a larger percentage of company traffic through the Internet, companies are opening themselves up to a greater attack surface. The first step in reducing this is to know how your data traverses each service and on-premises infrastructure. Misunderstanding how data is handled can lead to vulnerabilities, think misconfigurations or benign shadow IT, that hackers can exploit to gain access to systems. Other threats also loom, like DoS attacks on third-party cloud vendors that can leave your company without critical services—setting up redundant vendors for critical services is a frequent mitigation measure for this scenario.
  • Governance and Compliance Domain — Personal and private data has become a hot issue, and will continue to garner greater attention in the future. Regulations like the European Union's General Data Protection Regulation (GDPR) and the California Consumer Privacy Act (CCPA) have introduced a number of compliance requirements for companies who retain private and personal data. Working with vendors, companies must consider how those cloud providers comply with regulations—if regulations are geo-locked, citizen’s private data must reside on a server within a national boundary, then how does the vendor ensure this? Also overlapping the security domain, compliance is not security, a cloud vendor can be compliant and still have low or no security.
  • Performance Monitoring Domain — Performance monitoring is an on-going management initiative that aims at optimizing the quality and availability of the cloud services working together. Service, network, or app baseline comparisons is the general approach to performance monitoring. The challenge arises as more cloud services are added, complicating the company’s cloud fabric. Manually tracking and understanding performance, let alone troubleshooting bottlenecks and slow downs, is simply not feasible in these types of configurations, and companies are advised to look into network performance monitoring software.
  • Cost Management Domain — A key goal in cloud management is to understand the costs of services. In the cloud many vendors offer pay-as-you-grow billing schemes which makes cost controls much easier on cloud consumers than on-premise consumers. Simply, the cloud allows users to offload their IT capital expenses, and pay a single price, with foreknowledge of additional costs for greater service. However, as companies add more vendors and cloud services to their cloud tech stack, costs can become unwieldy. CMP software with built-in costing analytics is a helpful solution for these consumers.

Adhering to a cloud management framework, and carefully understanding top-level goals and how each cloud service will contribute to achieving those goals is the first step in cloud management. This step aims to place context around all the components that must be managed, and understanding which are critical, and possibly which existing services are redundant. Only then can cloud management tools become relevant and useful, and policies can be shaped around practical aims.

In the context of how each cloud, or service, contributes to business goals, teams can begin to baseline their cloud deployments using cloud management and monitoring platforms. CMP software will gather and compile data from each service from private cloud, public cloud, hybrid cloud, or multi-cloud configurations, to use in visualizations and reporting. If an organization’s cloud “mix” becomes adequately complex, public cloud tools may fall short of the performance monitoring, security, and cost management functionality needed. Third-party or home-grown tools can provide more particular or advanced solutions.

Technically, a CMP is deployed into an existing cloud environment, likely into a VM of its own. Using application programming interfaces (API), the CMP connects to the various other virtual infrastructures, and relays that data to a performance analyzer. Cloud management platforms should offer the following functionality:

  • Monitoring and optimization of cloud resources in various cloud environments
  • Delivers analytics and visualization, self-service capabilities, and insights into consumption patterns
  • Offers managerial controls over cloud functionalities and cloud security
  • Offers governance and compliance controls

Cloud management informed by a quality cloud management platform ultimately benefits companies by making more visible the behavior of their cloud. Using automation and algorithms, networks can be made more secure, cost-reductions achieved through optimizations, and compliance with government data regulations can be ensured. Cloud management benefits organizations by giving them:

  • Enhanced visibility & control — More companies are moving to the cloud, and those already there are adding more cloud services to their mix that are spread across a wider geographic surface. Enhancing cloud visibility and control is the central empowering capability of cloud management platforms that brings tangible measures that allow teams to successfully manage their cloud configurations.
  • Heightened security — Migrating to the public cloud does expose companies to new security concerns. They should evaluate whether keeping company IT resources behind a firewall is worth forgoing the opportunity costs of the cloud, namely unfettered cloud resources. Today, cloud security can be as secure as on-premise protections. Companies that choose to move to the cloud will of course need to adopt new methods of securing their resources, namely a quality cloud management platform that monitors traffic, enforces best practices and policies, and alerts teams of impending or occurring attacks, but in practice, cloud service providers the potential of similar or better protection as on-premise with the advantages of easy redundancy and scalability.
  • Cost-savings — Using CMPs offers insights into traffic usage at a granular level, some even present visualizations such as heat maps, to emphasize high traffic areas. From this point, admins can leverage usage and cost information and optimize their networks to balance load more efficiently effectively and improve user experience while reducing costs. Cloud visibility leads to cost reductions.
  • Simplified compliance — Personal data is regulated under two important laws, the EU General Data Protection Regulation (GDPR) and the California Consumer Privacy Act (CCPA). And while these two sets of regulations cover EU and California, respectively, their impact reaches further than that. Many companies see these as the tip of the data regulations sword, providing a blueprint for how the rest of the world may follow, and so regardless of where they are, they are taking steps to comply with regulations. Companies can equip themselves with CMPs that have powerful tools to monitor public cloud configurations for offenses against these standards, which then can alert and respond to these violations.

Networks are complex today, with increasing numbers of connected devices, nodes, segments, cloud partners, and more adding further complexity every day. Businesses and organizations whose missions rely on these complex cloud fabrics are challenged with understanding how these networks are behaving and contributing to costs as well as their mission’s bottom line.

Network traffic is not inherently visible, in fact quite the opposite, and so full visibility and control over a company’s cloud ecosystem is a primary aim of cloud management. Cloud management platforms help to achieve fully visible networks through real-time cloud monitoring using automations that optimize the network and alert admins to bottlenecks as well as cyber-attacks.

While the top names in cloud management can accommodate many cloud configurations, it is always advisable to understand a vendor’s offerings before committing to a CMP. Some platforms cater to SMEs while others are better suited for the enterprise sized workloads. An initial concern is to choose between open-source platforms or a vendor’s proprietary cloud management platform. Often this is a choice of preference, with teams choosing the platforms they are most familiar with or experienced in.

Open-source Cloud Management Platforms

Open-source CMP options are abundant, and many are popular, like Apache CloudStack. Open-source platforms are exceptionally robust with reliable functionality.

The top pro for open-source: it’s free. Companies will be able to skip paying licenses. However, the top con for open-source, though there is much contention about this between camps, is that technical support is usually less accessible. Open-source communities provide a collaborative source for asking technical questions, but unlike proprietary software, they may not have a dedicated customer support service that can make timely and knowledgeable responses. This means that a company’s IT department will likely need to possess specific expertise to ensure that operations are not hampered due to critical issues. Though the community creates transparency, because code is open-source and many developers review it, many developers find this peer collaboration arrangement more comforting and reliable. Then again, with open-source there are infringement risks, and license restriction risks—for example, if open-source code is modified, and released to the public, any proprietary code modifications may then need to be opened-up for review, potentially exposing company IP.

The following are top open-source cloud management platforms.

  • Apache CloudStack
  • OpenStack
  • ManageIQ
  • Cloudify
  • Mist.io

Proprietary Cloud Management Platforms

Proprietary vendor cloud management platforms offer enterprise level cloud management capabilities underneath a license agreement, and are defined as “owned” software. Companies choose to go with proprietary platforms for many reasons, the main reason, several have become de facto industry standards for enterprise needs. As well, these platforms have dedicated development teams to ensure software is bug free and patches continuously address newly discovered ones. However, open-source proponents argue that the open-source community is vast enough to adequately address bug fixes and troubleshoot problems, and can point to users like Walmart, Blizzard Entertainment, and China Mobile which have incorporated open-source cloud management platforms into their tech stacks. But these organizations may also payroll large DevOps teams to ensure their technology, essentially bringing help desks inside their doors.

The following are some of the most popular proprietary cloud management platforms.

  • VMware vSphere
  • IBM Cloud Orchestrator
  • Flexera
  • BMC