Is navigating cloud-native complexity worth the hassle?

Last month 7,000 developers traveled to Valencia to attend the combined KubeCon and CloudNativeCon Europe 2022 conference, the event for Kubernetes and cloud-native software development. A further 10,000 developers joined the conference online. The event is organized by the Cloud Native Computing Foundation (CNCF), part of the non-profit Linux Foundation. The CNCF supports and promotes over 1,200 projects, products and companies associated with developing and facilitating innovative cloud-native practices. All CNCF projects are open source – free and accessible to all – and its members work together to create scalable and adaptable applications.

The tools and services discussed at the event aim to solve the technical complexity of cloud-native practices, but there is a new complexity in the vast choice of tools and services now available. Organizations face a difficult choice when choosing which projects and products will best meet their needs and then designing an application using these tools to meet large-scale requirements. A fuller explanation of cloud-native principles is available in the Uptime Institute report Cloud scalability and resiliency from first principles.

Kubernetes — one of CNCF’s core projects — is a management platform for software containers. Containers, one of the key technologies of cloud-native IT, offer a logical packaging mechanism so that workloads can be abstracted from the physical venues in which they run.

A core value of containers is agility:

• Containers are small and can be created in seconds.
• Rather than needing to scale up a whole application, a single function can scale up rapidly through the addition of new containers to suit changing requirements.
• Containers can run on multiple operating systems, reducing lock-in and aiding portability.
• Containers are also updatable: rather than having to update and rebuild a whole application, individual containers can be updated with new code and patched independently from the rest of the application.

Containers underpin microservices architecture, whereby an application is decomposed into granular components (via containers) that can be managed independently, enabling applications to respond rapidly to changing business requirements. Cloud-native practices are the array of tools and protocols used to keep track of microservices and operate them efficiently, by managing communication, security, resiliency and performance.

Organizations developing cloud-native applications face two sets of challenges, both related to complexity. First, containers and microservices architectures increase the complexity of applications by decomposing them into more parts to track and manage. Thousands of containers may be running across hundreds of servers across multiple venues in a complex application. Keeping track of these containers is only the first step. They must be secured, connected, load balanced, distributed and optimized for application performance. Once operating, finding and fixing faults across such a large number of resources can be challenging. Many of the CNCFs projects relate to managing this complexity.

Ironically, the CNCF’s vast ecosystem of cloud-native projects, products and services creates the second set of challenges. The CNCF provides a forum and framework for developing open source, cloud-native libraries, of which there are now over 130. It does not, however, recommend which approach is best for what purpose.

Of the 1,200 projects, products, and companies associated with the CNCF, many are in direct competition. Although the common theme among all of them is open source, companies in the cloud-native ecosystem want to upsell support services or a curated set of integrated tools that elevate the free, open-source code to be a more robust, easy-to-use and customizable enabler of business value.

Not all these 1,200 projects, products and companies will survive – cloud-native is an emerging and nascent sector. The market will dictate which ones thrive and which ones fail. This means that users also face a challenge in choosing a partner that will still exist in the mid-term. The open-source nature of cloud-native projects means the user still can access and support the code, even if a vendor chooses not to – but it is far from ideal.

There is currently a lack of clarity on how to balance risk and reward and cost versus benefit. What projects and technologies should enterprises take a chance on? Which ones will be supported and developed in the future? How much benefit will be realized for all the overhead in managing this complexity? And what applications should be prioritized for cloud-native redevelopment?

Attendees at KubeCon appear confident that the value of cloud-native applications is worth the effort, even if quantifying the value is more complex. Companies are willing to invest time and money into cloud-native development as demonstrated by the fact that 65% of attendees had not attended a KubeCon conference previously and CNCF certifications have increased 216% since last year. It isn’t only IT companies driving cloud native: Boeing has been announced as a new Platinum sponsor of the CNCF at the event. The aerospace multinational said it wants to use cloud-native architectures for high-integrity, high-availability applications.

Cloud-native software development needs to be at the forefront of cloud architectures. But users shouldn’t rush: they should work with vendors and providers they already have a relationship with and focus on new applications rather than rebuilding old ones. Time can be well-spent on applications, where the value in scalability is clear.

The complexity of cloud native is worth navigating, but only for applications that are likely to grow and develop. For example, a retail website that continues to take orders during an outage will likely derive significant value — for an internal Wiki used by a team of five it probably isn’t worth the hassle.