The Docker hype is real, and cloud computing continues its blistering growth. That means fewer on-prem “monoliths” and more cross-cloud (micro)service-oriented architectures (m/SOA).
For savvy DevOps folks, this should raise alarm bells: What is deployed where? Is it on-prem or in a public cloud? In the latter case, in which region does it exist, and under whose policy? Answering these questions requires visibility into the services running on-prem as well as on private and public cloud servers.
Gaining visibility has traditionally been complex and costly, hence the dreaded “microservice premium.” But assuring visibility is critical for teams scaling to cloud infrastructure or microservice architectures.
Cross Cloud Visibility: Think of your brain, divided.
Monolithic software stacks offer easy visibility because they are relatively simple to inspect. Once the service is built, the monitoring tools are like the orange jelly bean in Martin Fowler’s diagram below. Each new monolith build includes a proportionate amount of orange jelly beans.
But in a distributed system, only services that need to scale are built out. That means you the orange jelly beans are not necessarily near the pink bowties, the magenta blobs, or anything else. New connections are made, and old connections are updated and rewired. This makes visibility difficult.
Linus Torvalds describes it this way (when describing the problems of microkernels):
“Think of your brain. Every single piece is simple, but the interactions between the pieces make for a highly complex system…If you take a problem and split it in half and say that the halves are half as complicated, you’re ignoring the fact that you have to add in the complication of communication between the two halves.
…
This is the kind of complexity you really, really don’t want to debug. It’s wonderful when it works, but part of why it’s so wonderful is the fact that it’s almost impossible to understand how it really works, and when things go wrong, you have a really hard time fixing them.”
Two Approaches to Visibility: PaaS vs. Agent Services Control
The PaaS Approach
Some teams have worked toward visibility solutions with creative, custom PaaS systems. These systems loosely combine tools into a trusted service layer, making for a kind of synthetic monolith of DevOps tools.
For example, PaaSTA, Yelp’s open-source PaaS, was built to assist in managing the decomposition of “yelp-main” into a hybrid cloud m/SOA. In the process, Yelp has open-sourced their distributed tracing, service aggregation and dependency graph tools, as well as helpful guides to the emerging ecosystem.
This ecosystem is a mix of extended and repurposed configuration management tools (Chef, Puppet, Ansible, HashiCorp et al), opinionated PaaS engines (e.g. Flynn, an Postgres/FOSS environment PaaS), and creative uses of container managers, primarily Kubernetes.
The Agent Services Approach
Another way to gain cross-cloud visibility is to rely on agent services. Since this approach does not require major changes to management tools, it is likely to be simpler for many organizations, especially ones that are not as large as Yelp and lack the in-house expertise to extend and repurpose complex tools like Chef or Puppet.
The agent services approach uses diverse tools to provide visibility and access through a cross-cloud pane of glass. Like the PaaS approach, it assumes that the existing tools—Chef/Puppet, SCOM, or Nagios/Splunk/Zabbix/SumoLogic—won’t go away. Instead of extending them or building them into an integration database, they are left as-is and managed by agents.
The Advantage of Containerized Agents
Plus, by using solutions like Jetpatch’s, those agents can run in containers, which simplifies the deployment process in many ways.
For one, containers simplify the installation of agents. Rather than having to install agents manually using RPMs, Debian packages or a similar tool on each server, you can deploy them as containers very quickly using an installation process that is uniform across your infrastructure. This approach also has the advantage of avoiding dependency issues related to package managers. And it makes it easy to uninstall an agent cleanly, without leaving behind traces on the system that are difficult to clean up completely. This is not usually the case within the context of traditional software installation and uninstallation.
Container-based agents also make your infrastructure more agile. For the reasons noted above, deploying the agents is faster with containers. In turn, faster deployment means a greater ability to modify your infrastructure as needed. That breeds agility, the sine qua non of DevOps.
Last but not least, container-based agents offer security and access-control advantages. That’s because they allow admins to specify, on a granular level, which system resources the container should be able to access. At the same time, because the agents runs in an isolated container environment, all other resources on the host are not available to the agent. This isolation helps to mitigate attack vectors without compromising the flexibility needed to give agents access to parts of the system on an as-needed basis.
Conclusion
Visualizing the management layer—as Jetpatch has done—provides the power-tool benefits with a controlled, throttled agent. This allows for troubleshooting without decommissioning, and visibility without writing a tool from scratch. It’s a sensible solution to a problem that will only get more difficult as mSOA proliferates.
