Cloud Native Platform Engineering Community – Article

Blog: A Living Blueprint: Evolving the Platform Engineering Whitepaper and Maturity Model

Tue, 03 Mar 2026 12:00:00 +0000

A Living Blueprint: Evolving the Platform Engineering Whitepaper and Maturity Model

In Q1 2026, the CNCF Platform Engineering Technical Community Group (TCG) kicked off a refresh of two foundational community resources: the Platform as a Product Whitepaper and the Platform Engineering Maturity Model. Since their initial release, both of these documents have been crucial guidance for organisations beginning their internal developer platform journey, and now, two years later, it’s time to ensure they grow with the industry.

We launched two focused initiatives to not only make initial updates but also to define a clear, sustainable path forward. We believe these should be living documents that continuously reflect industry advancements, spawn new ideas, and serve as a durable, long-term resource for the community.

Our current initiative focuses on two main areas to meet the immediate needs of the platform engineering community and pave the way for the future. First, we are Defining Future Growth and Scope by leading a discussion to clearly articulate how both the Whitepaper and Maturity Model must evolve to meet the current and emerging needs of the cloud-native industry. This ensures a measured, strategic approach to larger future changes and includes integrating content on new essential areas, notably the secure, governed integration of Artificial Intelligence (AI) tools and prototypes into the software development lifecycle.

Second, we are pursuing Clarity and Usability Improvements through Language and Terminology Refinement, making concise, simple changes to update the core language across both documents, ensuring clarity and aligning them with established vocabulary. We are also focusing on Maturity Model Practicality by adding clearer, real-world examples and scenarios. This enhancement will empower teams to assess their current state more effectively and identify concrete next steps for advancement.

The collective goal of these initiatives is to have drafts of the Whitepaper and the Maturity Model available before KubeCon EU 2026, for formal release to the community at KubeCon! Accompanied by a feedback survey, this will provide an immediate opportunity for review, feedback, and further community contributions to steer future initiative work through 2026 and into 2027.

Get Involved Now: Your Expertise Is the Blueprint!

The success of these documents and their evolution depends on the collective expertise of the cloud-native community. We invite you to join the conversation, contribute your experience, and help us finalise this next iteration.

Connect With Us Online

Slack: Join the #platform-engineering channel on CNCF Slack. It’s the best place to ask questions, share ideas, get involved, and stay up to date on progress. A link to the drafts will be published in the channel as soon as they’re ready, and formally at Platform Engineering Day on Monday 23rd March 2026.
Website: Visit cloudnativeplatforms.com to learn more about our group and resources. The draft documents and survey will be available there.
LinkedIn: Follow our latest posts and updates on LinkedIn.
Survey: We’re launching a new version of our community survey at KubeCon Europe 2026! Your input will directly inform the final shape of the updated materials — keep an eye on the channels above for the link.

Meet Us at KubeCon EU

If you’re attending KubeCon + CloudNativeCon Europe 2026 in Amsterdam, find us! We’ll be hosting Platform Engineering Coffee meet-ups Monday-Thursday 07:00-8.30, lunchtime meet-ups 12:30-13:30, at Platform Engineering day all day Monday 23rd March. Or stop by the Platform Engineering TCG booth at the Project Pavilion on Wednesday, 25th March to chat with contributors, ask questions about the updated documents, and share your own platform engineering stories. Whether you’re just starting your platform journey or you’ve been building internal developer platforms for years, we’d love to hear from you.

We look forward to building the next chapter of these resources together.

Blog: Cloud Native Platform Engineering Japan Kickoff in Japan!

Wed, 14 Jan 2026 00:00:00 +0000

Hello, everyone. I’m Ken Komazawa, Chief Architect at Red Hat.

Today, I’d like to share details about the Cloud Native Platform Engineering Japan Kickoff held on December 8th. This marks the first event of its kind in Japan organized by the CNCF community focused on platform engineering.

This event is organized by Cloud Native Platform Engineering Japan, a SIG of Cloud Native Community Japan ( CNCJ), which is the Japanese chapter of the CNCF. The group’s mission is to “spread awareness about platform engineering from the CNCF’s perspective! Let practitioners gather to share insights, collaborate closely with upstream projects, and connect Japan with the world!”

I am also participating as one of the organizers. As shown in the slide below, it was announced at the LF Japan Community Days on October 21st.

The story for launching this community began with a conversation between Nakamura-san, Head of OSPO at Hitachi and a member of the CNCF Governing Board, and Vincent, APAC CTO at Red Hat. They discussed how platform engineering activities in Japan and Asia lagged behind those in Europe and the US, concluding that establishing a dedicated community in Japan would be beneficial. That’s how Nakamura-san and I connected.

Coinciding with the launch of the CNCF’s platform engineering certification, CNPA (Cloud Native Platform Engineering Associate), I approached Nakamura-san with the idea, “Why don’t we localize it into Japanese?” This sparked our communication, and this group came together to volunteer and drive forward such activities.

Hayakawa-san from LINE Yahoo has been involved in the CNCF Platform Engineering Technical Community Group from early on and also had connections to the domestic community in Japan (Platform Engineering Kaigi/Meetup), so he serves as the leader of this group.

Tabata-san from Hitachi is also a Keycloak committer and actively promotes the community in security fields like the CNCF TAG Security and Compliance. He joined this community, leveraging his operational experience. (He is also a CNCF Ambassador.)

Furthermore, Matsuda-san from Hitachi is a Kubestronaut (a superhuman holding all Kubernetes certifications) and enthusiastically joined, eager to contribute to this community! (He is the only Kubestronaut in the Chugoku region of Japan!)

Okabe-san is a university student and a Mercari intern, but he was the driving force behind this project’s planning and execution, pushing things forward energetically. Okabe-san is also a CNCF Ambassador and a leader with a strong presence in the OSS community.

And after this event, we even had Aoyama-san from CyberAgent join us! With Aoyama-san—who also organizes events like Kubernetes Meetup and Cloud Native Days—participating, our community operations team has become even stronger.

What should I do among such diverse and talented members!?

I serve as an architect at Red Hat. As an ambassador for Japan, I believe I can take on the role of promoting “ The Open Source Way"—the approach Red Hat has practiced to build a business around open source. The Open Source Way, a development model that drives developer innovation through open decision-making while enhancing transparency, should be highly compatible with platform engineering—which maximizes developer experience as an internal platform.

While working at Red Hat, this approach to organizational management and mindset can start to feel like second nature. Yet putting it into practice is extremely challenging, and much of it remains tacit knowledge, largely unknown outside the company. I hope to use this community to share how Red Hat has developed software and coexisted with the open source community.

That was a long introduction, but I’d like to briefly introduce the event’s content.

Keynote

Vincent delivered the keynote titled “Bridging the Gap”. His message was that platform engineering activities, grounded in both internal and external community initiatives, will shape the future of software development in Japan. The challenges of IT talent shortages and skill gaps within end-user companies are more severe compared to Europe and the US. To improve this, companies should advance internal “innersource” activities (activities that openly share knowledge within the company to build collective intelligence). As one key measure for this, the Internal Developer Platform (IDP) becomes crucial.

Platform Engineering from CNCF’s Perspective

Next, Hayakawa-san from LINE Yahoo introduced platform engineering from the CNCF’s perspective. CNCF provides resources such as the Platforms Whitepaper, which outlines the role, value, and measurement methods of platform engineering, along with how to establish platform teams; a Maturity Model to review platform outcomes and identify improvement opportunities; and a certification program for platform engineering. In this way, CNCF is a community that shares the foundational knowledge and theory for practicing platform engineering, supporting the formation of common understanding and best practices within the cloud-native field.

Hayakawa-san serves as the Product Owner for his company’s internal platform. Drawing on his in-house experience, he actively provides feedback to the CNCF and shares insights within the community. His discussions are highly persuasive, grounded in practical experience with mission-critical infrastructure at his company, including best practices for multi-tenant operations in Kubernetes and infrastructure automation.

Avoiding Pitfalls When Adopting Gateway API: Secure-by-Default Design Achieved with Kuadrant and Keycloak

Next, Matsuda-san from Hitachi gave a technical presentation titled “Avoiding Pitfalls When Adopting Gateway API: Secure-by-Default Design Achieved with Kuadrant and Keycloak”. He explained Kubernetes architecture by personifying it as a family structure, making it both easy to understand and enjoyable.

In increasingly complex platforms, it is crucial that security is built in from the design phase. How can we clearly communicate to engineers how the Kubernetes ecosystem addresses security challenges? I believe this is one of the most critical tasks within the domain of platform engineering.

Matsuda-san is a Kubestronaut (the only one in the Chugoku region of Japan). He provides such clear explanations through his blog and other channels, making him an incredibly valuable and reassuring team member.

The Evolution of Platform Engineering at Mercari

Finally, Rafael, Engineering Manager of the Engineering Enablement group at Mercari, gave a presentation on the evolution of platform engineering at Mercari.

He specifically outlined the challenges and countermeasures in platform engineering efforts from 2018 to the present. He explained that the platform team needed to become an enabler focused on developer success, not just a tool provider. To foster an inner-source culture, they shifted to a structure where domain expert teams lead the community, ensuring the platform team itself doesn’t become a bottleneck.

Mercari, where Okabe-san works, has achieved a culture distinct from typical Japanese organizations. This makes it a highly insightful experience for other companies to learn from.

Summary

This kickoff meeting covered the content described above.

For Japanese engineers, this kickoff aimed to pose the question: “What kind of impact can platform engineering activities have?”

As part of your company’s business activities, platform engineering can serve as a powerful driving force. Let’s not only accumulate this experience internally as inner-source activities, but also share it externally. Doing so creates a positive feedback loop: it contributes positively to the open-source community (outer-source), which is recognized as enhancing your corporate brand, ultimately benefiting your company.

We ask for your continued cooperation to keep energizing this community!

Bonus: A Thinking Framework for Platform Engineering

Below is my perspective on the overall landscape of platform engineering. I see three overlapping problem domains: “Platform as a Product”, which treats internal platforms as products; “Platform as Engineering”, which views them as an extension of software engineering; and “Platform as Organization & Architecture”, which considers them as organizational and architectural challenges. By framing it this way, I believe we can position platform teams as core functions within corporate strategy and design, enabling the planning of investments.

Below is a list of excellent books to guide you. The three logos at the bottom represent open-source practice communities based on Red Hat’s experience. If you can adapt and implement the ideas and practices they convey to suit your own company, you should be able to build a platform that maximizes your business value.

Let’s all practice platform engineering together!

Blog: Navigating Platform Engineering with the maturity model assessment

Mon, 15 Sep 2025 01:00:00 +0000

The CNCF Platform Engineering community has been working on an assessment to accompany the Platform Engineering Maturity Model. It’s now available as a preview for feedback, but it’s crucial to understand its purpose to know what it can do for you and what you need to put in once you get your score.

The assessment is a series of multiple-choice questions that takes around 20 minutes to complete. It runs in your browser and no data is shared. Once complete, you’ll get a report card that shows you where you sit within the Platform Engineering Maturity Model.

Help us out by trying out the assessment and giving us feedback.

Platform orienteering

Picture yourself at the edge of a vast forest. You know there’s a destination somewhere out there, but between you and your goal lies unfamiliar terrain filled with hills, valleys, streams, and dense woodland. This is orienteering, the sport of navigating unknown territory using only a map and compass.

You can take part in orienteering with minimal kit. If you have a map, you can work out the rest using landmarks. Perhaps you’ll spot that distinctive hill in the distance, follow the bend of a river, or use a church spire as a reference point. But it gets a whole lot easier if you have a compass to tell you which direction you’re facing.

That’s why we wanted to add an assessment tool to accompany the Platform Engineering maturity model. Just like orienteering requires both a map and a compass working together, a successful Platform Engineering transformation needs a clear picture of the landscape and a way to determine your current position.

The maturity model is the map

In orienteering, a map is your window into the landscape. It shows you the contour lines that reveal steep climbs and gentle slopes, the symbols that mark forests, clearings, and water features, and the network of paths that connect different areas. But here’s what’s crucial to understand: the map shows you the terrain as it exists, not where you are within that terrain.

The Platform Engineering maturity model works the same way. It’s a detailed map that describes the features of the Platform Engineering landscape. It has the organizational structures, the technical capabilities, the process maturity levels, and the cultural elements that define different stages of platform evolution. It shows you what “good” looks like at various levels of sophistication.

What a map doesn’t tell you is where you are right now, or where you want to go. When it comes to your destination, it’s your journey, so it’s for you to decide. We’re each heading somewhere different, and that’s as true for Platform Engineering maturity as it is for choosing whether to head to a peaceful lakeside clearing or push for the challenging summit.

Some organizations want to reach the beach: a stable, reliable platform that serves their current needs without unnecessary complexity. Others are drawn to the heady heights of mountains, pursuing cutting-edge capabilities and pushing the boundaries of what’s possible with Platform Engineering. Both destinations are valid; the map just helps you understand the terrain that lies between you and your next waypoint.

The assessment is a compass

Once you have the map and an idea of where you want to get to, the one thing standing in your way is knowing where you are right now. In orienteering, you might recognize that hill over there and that stream down below, but you can’t orient the map correctly without knowing which way is north. You could be looking at the landscape upside down for all you know.

You can’t plan a route until you know your starting point. Are you at the bottom of the valley looking up, or are you halfway up the slope? Are you even on the side of the right mountain? The difference determines everything about your next steps.

This is where the Platform Engineering assessment comes in. It functions as your compass, not the magnetic but the organizational kind. It helps you work out which way around to hold the map and where you are within the Platform Engineering landscape. The assessment asks targeted questions about your current practices, capabilities, and organizational structures, then plots your position on the maturity model map.

With both map and compass in hand, you can finally see the whole picture: here’s where you are, there’s where you want to go, and now you can begin to understand the terrain you’ll need to cross to get there.

Route planning

When we were gathering feedback on the assessment during its development, one question came up repeatedly: “This is great for showing me where I am and where I could go, but how do I actually get from here to there?”

It’s a natural question. In orienteering, once you know your position and your destination, the next step is route planning. Do you take the direct path that cuts straight across that steep hill, or do you follow the longer but easier route that curves around the valley? Should you push through the dense forest or stick to the clearer paths that might add distance but save time?

At the moment, detailed route planning isn’t part of the assessment. Think of our current tool as a reliable map and compass combination, but not a complete GPS navigation system yet. The “SatNav version” that provides turn-by-turn directions is a later evolution we’re considering.

The assessment might prompt you about your intended destination and offer some general recommendations, like suggesting that organizations aiming for advanced platform capabilities typically need to focus on developer experience metrics or that teams targeting rapid scaling should prioritize automation investments. But we deliberately don’t want to remove human expertise and judgment from the process.

When we add more detailed recommendations in future versions, those recommendations will remain general enough to require customization. There’s always space for the next steps to be tailored by people who understand your organization’s unique context, constraints, and goals. After all, no two orienteering courses are exactly alike, and neither are any two Platform Engineering transformations.

Prototype assessment feedback

We are open to feedback for all elements of the assessment. For example:

Are the questions and answer options clear?
Did the result match your expectation?
Are the visualizations helpful or should we try something else?

The assessment scores responses for each category in the maturity model. This information is shown in three ways in the assessment’s output.

Spider chart: Uses the average score in each category to highlight strengths and weaknesses.
Matrix: Plots answer density to show the spread within each category.
Table: Displays average scores per category.

Helping you navigate

The assessment gives you the essential foundation: a clear understanding of where you stand and the possible paths forward. From there, the art of navigation, whether through forests or platform transformations, remains a uniquely human skill.

You can help by trying the assessment and giving us feedback.

To learn more about what’s happening in the platform engineering community, join our Slack Channel and follow our LinkedIn page.

Blog: From Chaos to Clarity: Navigating Observability in the Platform Engineering Era (and a Dash of AI)

Wed, 20 Aug 2025 12:00:00 +0000

There was a great energy at KCD New York this year, and for Graziano Casto, a personal highlight was leading a roundtable on observability. It was a fascinating discussion that really got him thinking about the challenges we’re all facing in the platform engineering space. Here is Graziano’s recap of the key problems and promising ideas that came up.

Introduction

It was an absolute pleasure recently to moderate a roundtable at KCD New York, diving deep into the fascinating (and let’s be honest, sometimes frustrating) world where Platform Engineering meets Observability. As my first time moderating a roundtable, I was genuinely thrilled by the energy and candid participation from everyone in the room. A huge thank you to all the participants: Michel Murabito, Colin Lacy, Tiara Sykes, Andrew Espira, Mariia Rudenko, Aderianna Williams, Marino Wijay, and Maria Ashby whose insights made this discussion truly invaluable. We had an incredibly insightful exchange, and I walked away with some serious food for thought.

We kicked off by acknowledging a universal truth in today’s cloud-native landscape: managing a full observability stack often feels like trying to hit a moving target. The more we aim to observe, the more inherent complexity seems to creep in. We continuously pile on tools, data, and dashboards–be it metrics, traces, logs, or profiling – and suddenly, we’re swimming in a sea of cognitive load, entropy, and quite often, plain old confusion. So, instead of me listing the common headaches, I threw it open to the room: “When you think about managing a full observability stack, across logs, metrics, traces, and so on, what are your biggest pain points? If you had to name the biggest challenge your team is facing with observability right now, what would it be?”

The responses flowed freely, revealing a shared understanding that while observability promises clarity, its real-world implementation often introduces its own unique set of challenges. And, quite organically, our conversation drifted into the exciting (and slightly unsettling) realm of Generative AI, specifically discussing how Large Language Models (LLMs) can be synergistically integrated within platforms to serve as enablers in resolving some of these very challenges.

The Observability Headaches: Where Do We Feel the Pinch?

One of the loudest points of contention was the persistent struggle to correlate telemetry data with the actual services generating them, and the broader challenge of telemetry data correlation. It’s like having all the pieces of a complex puzzle but no clear idea how they fit together. You might spot a spike in CPU utilization – a metric – but then you’re left guessing which microservice is the culprit. Then begins the detective work: diving into logs to pinpoint an error, and finally tracing requests to understand the flow. The fundamental problem is that these critical data points often reside in disparate systems, use inconsistent identifiers, and demand a significant amount of manual effort and intuition to connect the dots. This fragmentation makes it incredibly difficult to quickly identify the root cause of a problem when seconds count.

Adding to this complexity is the sheer volume of alerts and the difficulty in correlating them with the actual underlying problem. We’ve all experienced it: a dozen alerts fire simultaneously, each pointing to a symptom, yet none clearly indicating the core issue. This leads to what’s known as alert fatigue, resulting in missed critical incidents, wasted time triaging false positives, and ultimately, a palpable loss of trust in the alerting system itself. The challenge isn’t merely about receiving notifications; it’s about receiving meaningful alerts that directly pinpoint the underlying problem, not just its outward manifestations.

Furthermore, a significant unspoken burden that often comes with observability is the cost of both creating and maintaining the entire observability stack. From licensing fees for proprietary tools to the infrastructure costs of storing massive volumes of telemetry data and the operational overhead of managing these complex systems, the financial outlay can be substantial. This constant investment of resources, both human and monetary, can become a major pain point, often weighing heavily on budget decisions and resource allocation.

Then there’s the pervasive issue of making insights accessible and visualizing them in a way that provides the right insight to the right person. Raw telemetry data, in its unadulterated form, is overwhelming. Different roles within an organization – SREs, developers, product managers – need distinct views and varying levels of detail. A developer might require granular trace data, while a product manager needs high-level business metrics. The constant battle involves creating and maintaining these tailored dashboards and ensuring everyone knows where to find what they need. This often leads to information silos and missed opportunities for proactive improvement.

A recurring theme throughout our discussion was the persistent problem of siloed teams and the resulting lack of standardization. When different teams adopt disparate observability tools, inconsistent naming conventions, or even varied logging formats, it inevitably creates a fragmented and chaotic landscape. This makes it incredibly challenging to gain a holistic view of the system, collaborate effectively during incidents, and leverage best practices across the entire organization. It’s a classic case of “everyone doing their own thing”, leading to pervasive inefficiencies and increased complexity.

Finally, a crucial point that resonated deeply was the importance of developer education. Observability isn’t merely about deploying tools; it’s about cultivating a specific mindset. Developers need to grasp why observability is vital, how to effectively instrument their code, how to interpret telemetry data, and critically, how to leverage observability tools to troubleshoot their applications. This knowledge gap can lead to poorly instrumented services, ignored alerts, and a general underutilization of the powerful observability stacks organizations invest heavily in.

Internal Developer Platforms: The Unified Solution

So, with these common headaches laid out, how do we begin to alleviate them? This is precisely where the concept of an Internal Developer Platform (IDP) steps in as a truly powerful solution, providing a cohesive answer to many of these challenges.

An IDP, at its core, inherently solves the problem of standardization. By providing clear standards and abstractions through “golden paths” for building and deploying applications, it ensures consistency across the organization. However, it’s crucial that these golden paths don’t become “golden cages”. A well-designed IDP empowers developers with the necessary autonomy to cover edge cases, allowing them to step outside the perimeter of the provided golden paths when needed for specific requirements. This balance is vital for both consistency and innovation.

Moving beyond standardization, IDPs also play a crucial role in addressing the challenges faced by siloed teams that might be working on different components of the same system and often lack a shared performance baseline. During our discussion, we introduced the concept of leveraging generative models within these platforms. Specifically, the role of Generative AI, particularly Large Language Models (LLMs), in the observability space emerged as a truly futuristic and exciting prospect. The idea is that LLMs can help close the gap between users and telemetry data. Imagine being able to ask natural language questions like, “Why is our checkout service slow right now?” and have an LLM sift through mountains of metrics, logs, and traces to provide a concise, actionable answer. Or, “What were the top 3 errors in our authentication service last night?” and get a summary, perhaps even with links to relevant log lines. These models can also be instrumental in enabling teams to define and compare their telemetry data against customized thresholds, ensuring that the entire system is monitored according to a collectively defined baseline, fostering a shared understanding of system health.

From here, we delved into how these models further enhance the transparency and clarity of insights. LLMs, integrated within the IDP, can analyze vast amounts of telemetry data and provide various stakeholders with personalized insights and alerts. This capability opens the door to entirely new interfaces beyond the traditional dashboards, making complex operational data more accessible and actionable for different roles. Unfortunately, we didn’t have the opportunity to delve deeper into the intricate topic of telemetry data correlation during the roundtable, but I have written an article that explores this topic further, which you can find here.

The Open Question: Balancing Trust and Cost with Benefits in the LLM Era

However, as with any powerful new technology, the discussion around LLMs quickly led to a critical open question for the community:

How do we effectively balance the significant benefits that LLMs bring – such as improved automation and deeper insights – against the inherent costs? These costs include not only the economic investment required for these models but also the crucial aspect of trust, both in the accuracy of the results and in entrusting our sensitive data to an LLM, particularly when utilized as a service.

This is a conversation that needs to continue. As we push the boundaries of what’s possible with AI in operations, we must collectively figure out how to build systems that are not only efficient and intelligent but also fundamentally secure, trustworthy, and cost-effective.

Wrapping Up

My first moderating experience at KCD New York was an absolute blast, and the insights from the roundtable on Platform Engineering and Observability were truly invaluable. It’s clear that while observability brings its own set of complexities, Internal Developer Platforms offer a robust framework for overcoming these challenges by promoting standardization, providing contextualized insights, and empowering developers. And looking ahead, the potential of LLMs to revolutionize how we interact with our telemetry data is incredibly exciting, even if it comes with some important questions we need to answer as a community.

What are your thoughts on these challenges and solutions? And how do you see the role of LLMs evolving in the observability space, especially concerning the trust and cost trade-offs? Let’s keep the conversation going!

Blog: Demystifying Composability on Platforms: extending Platforms to business needs

Wed, 29 Jan 2025 12:00:00 +0000

Introduction

Let’s start with what’s a Platform, according to the CNCF Platforms White Paper, “A platform for cloud-native computing is an integrated collection of capabilities defined and presented according to the needs of the platform’s users. The specific set of capabilities and scenarios supported by a platform should be determined by the needs of stakeholders and users. And while platforms provide these required capabilities, it’s critical to note that platform teams should not always implement them themselves.”

Composable platforms promote flexibility and adaptability in the configuration and setup of the Platform. These individual solutions are then combined to form a larger, more feature-rich Platform. This allows configuring different components to achieve various technical and business needs by providing specific components or allowing setup of this component. When looking for a platform, understanding the end goal – bringing value to the business – is critical for ensuring the right capabilities meet the organization’s needs.

Composability, then, is the characteristic of a platform that, through different user experiences and a broad range of components, expands to fulfill technical needs at the end, supporting business value and increasing competitiveness advantage.

As the platform supports and enables different personas, each persona will have distinct needs and usage of the platform and, accordingly, differing requirements and perspectives on what composable means. Platform consumers may leverage some or all of the composable capabilities, depending on their needs. For example, while one development team may choose to only make use of the platform’s full capabilities to build, test, scan, and store artifacts; another team may opt into the build and scan capabilities only, storing their artifacts in a different repository from the one provided by the platform team. The flexibility to consume only the parts that provide business value based on a team’s specific business deliverable, as opposed to an all-or-nothing approach, is a hallmark of composable platforms.

Fig 1. Platform Capabilities

It’s worth noting that this loose coupling does not preclude the benefits of interoperable components. While capabilities such as build and test, as well as artifact storage may be consumed separately, a composable platform can still provide benefits for those consumers who wish to use the two together, such as automatic image signing and SBOM generation.

A key concept in building platforms is the persona. We refer to a persona as a group of stakeholders within a platform. Note that a persona is not the same as a role, and is intended to apply to different roles in team topologies. Some individuals will fit into one or more persona - especially in smaller organizations where team members are expected to wear multiple hats.

Characteristics of a Composable Platform according to the personas

Builders (focus on platform and beneficiary experience): From being extensible and configurable, the platform should support a variety of underlying infrastructure on which the platform will run, from on-premise to cloud.
Enablers (focus on end-users and application capabilities): Provide availability to access different installation methods and configure the platform by allowing the custom configurations, extend the platform to additional infrastructure. Increase platform capabilities to either of the following two roles- Developers and Business Customers
Consumers - Platform Customers (Developers) components are the tools that add application capabilities, such as database, observability, and application log management to accomplish business goals. Decreasing lead time in development will increase developer productivity.
End-users (customers) - End users don’t interact directly with the platform but benefit from the increased productivity of the platform consumers, which reduces the time to market. Platform capabilities and application features will contribute to business use cases and add customer value. Customers can also measure the value of a strong platform through enhanced reliability and availability, building trust over time. Furthermore, the ability to support a variety of workloads helps the platform meet business requirements and provide value to the end users.

Learn more about the personas: https://tag-app-delivery.cncf.io/blog/paap-personas/

How do the different personas view a platform?

The following diagrams help us explore how a platform looks from the differing perspective of each persona and how the platform evolves with the new components.

From Builders:

The underlying infrastructure may not necessarily be part of the platform. However, the platform takes advantage of the underlying hardware resources and services. The following diagram shows an example of how a composable platform expands its components according to the organization’s setup and configuration requirements.

Once the infrastructure is defined, the platform can be set up; as a platform expanded using different configurations from networking and security configuration. Once the platform is up and running and accessible it can be handed over to the next team.

Fig 2. Platform Expands within an infrastructure

Once the infrastructure is defined, the platform can be set up as a platform expanded using different configurations from networking and security until the platform is running and accessible for the next team.

From Platform Enablers:

Growing a Platform by increasing platform and application capabilities, taking advantage of the platform components or new components selected by the organization.

Fig 3. Platform Expands within platform and application capabilities

After cluster installation and setup typically with a Kubernetes distribution, platform engineers embarked on Day 2 operation activities, ensuring relevant governance and security are in place. Next, the teams will work to ensure an efficient onboarding experience and a great user experience, allowing teams to access the platform directly or indirectly and bringing application capabilities closer to developer teams. It’s critical to meet the business’s scalability, resilience, and high availability needs before moving forward with production workloads.

From Developers (platform end-users):

The platform expands with each component, building out CI/CD pipelines, then moving on to tune resources to meet requirements, and ensuring the availability to run multiple and diverse workloads. The platform can be extensible enough to support different third-party tools for application and user needs, such as databases, interconnectivity, and security tools.

Fig 4. Platform Expands within new workloads, tools and resources.

End users of the platform, such as Developers, MLOps, and Software Engineers, will benefit from the platform’s readiness to start bringing new workloads and promoting best practices from CI/CD, accessing more resources, and setting up third-party tools to fulfill business needs.

From Customers (platform end-users):

As the platform expands to fulfill technical and user needs, each expansion means new components that make the platform composable. A composable platform will bring value to the business by providing scalability and flexibility to different personas to achieve business goals. Examples of this might be increased trust, speedy service, or cost efficiency for those funding the service. While this persona will receive an indirect benefit from platform composability, they have a sizable role in driving the priority of composable capabilities and components, as facilitating their evolving needs helps determine the necessary platform capabilities for increasing business value.

Conclusion

Composability allows the different personas in an organization to get what they need from a platform without undue investment by stakeholders in areas that are of secondary importance, allowing for flexibility and more efficient onboarding. We discussed what composable platforms are, and how they can help a business to succeed in their goal of deriving the most business value from their investment in cloud native technology.

Blog: Platform as a Product: Understanding the Personas

Thu, 16 Jan 2025 12:00:00 +0000

Platform as a Product Overview

Platform as a Product is a critical topic in the industry because it helps teams to work agile in our forever changing organizations and market. Platform as a Product is key to helping us move quickly and be ready for new technology in a standardized way. According to the CNCF Platforms White Paper, Platform as a product is defined as:

Platform as a product: A platform exists to serve the requirements of its users and it should be designed and evolved based on those requirements, similar to any other software products. Platforms should provide the necessary capabilities to support the most common use cases across product teams and prioritize those over more specific capabilities that are only used by a single team to maximize the value delivered.

Understanding the personas

When considering its users, we need to identify the different personas such as: Developers, ML Engineers, Data Scientists, Platform Engineers, and others.

The personas would focus on different aspects of the platform, and these could be represented in different organizations on one or multiple roles and teams.

Builders (focus on platform and beneficiary experience):
- Enabling Day2 Platform Operations
- Setting up the platform
- Setting up security
- Platform Capabilities for AI
- Platform capabilities for HPC workloads
Enablers (focus on end-users and application capabilities):
- Enabling cloud-native and legacy workloads with best practices such as CI/CD and security guardrails.
- Enabling user experience and self-service capabilities
- Build automation practices to simplify onboarding experiences for end users.
- Configuration integrations and third-party tools
Consumers - Platform Customers (Developers):
- Access the platform to maintain and build applications
- Build any applications from legacy systems, cloud-native, AI
- Might not interact with the platform directly.
End-Users (customers):
- Customers who are the end-users of the applications and systems running on top of the platform.
- Might not interact with the platform directly.
Service Providers:
- Organizations who provide a Platform on top of Kubernetes.
- Consulting organizations that help teams drive implementations and adoption around the platform

Personas interactions

The Platform Builders are those who are installing the platform and configuring the platform in an organization. These are the teams who are responsible for the platform itself. These personas can be a group of teams dedicated to diverse aspects of the platform, networking teams, security, and Platform Engineers, whether they are internal teams to the organization or work with service providers to provide the platform and build configurations on top of it.

The Platform Enablers could be composed of DevOps, Middleware teams, and Migration teams, who are mostly dedicated to building application capabilities and user experience with best practices on top of the platform and work directly with Developers and data Scientists. These personas will also interact with Platform Builders to ensure the platform foundation is ready for the capabilities required. They can be internal teams, external consulting teams, or a mix of both.

The Platform Consumers could be composed of Developers, Software Engineers. They are the personas who build and enrich applications from legacy systems, AI workloads, microservices or any workload that will run on top of the Platform to serve a business needs. They usually are in the Line of Business, and work might work directly with Platform Enables to ensure application capabilities are in place and access self-service and user experience. They can be internal teams, external consulting teams, or a mix of both.

The Platform End-Users, these personas are an organization customer. They are the main reason why the platform exists and what drives the organization’s business. They might interact or not directly with the platform by accessing a UI, service, AI, or any other workload. They will often work with teams closer to the platform consumers.
The Service Providers provide platforms, toolings, and services to help organizations move faster into the cloud native ecosystem. These organizations are focused on resolving common problems that the industry faces and provide diverse tools and frameworks to resolve them. They can be internal teams, external consulting teams, or a mix of both.

Blog: Platform as a Product Research - Now with a Survey!

Mon, 25 Nov 2024 12:00:00 +0000

The CNCF Platforms Working Group is continuing its research to learn how companies build their internal platforms. After sharing our Platform Engineering Maturity Model, we are now focusing on Platform As A Product. This means treating platform users like customers, making sure the platform meets their needs. Even though many talks and articles discuss this topic, we don’t know exactly how many companies use this approach.

At first, we started talking to platform builders from different industries through interviews. If you haven’t joined yet, you can still sign up by following our Invitation To Participate in Platform As A Product Research. blog post.

The qualitative insights gained from these interviews are invaluable. However, we found that interviews take a lot of time to plan and conduct. To be more inclusive for people with less time at their hands and collect more data quickly, we created a survey based on our interview questions. The survey helps us gather quantitative data to go along with the detailed stories from interviews. This combination will give us a better understanding of how teams decide what to build for their users.

And like with the interviews we want to hear from everyone involved in building platforms, not just product managers. Whether your team uses product management or another method, your answers are important to our research.

So, if you want to help us with this study, please fill out our survey here! Thank you!

We also invite you to join the CNCF Slack and find us in the #platform-engineering channel. Working group meetings are every two weeks and participation is open to all, you can find the schedule here!

Blog: Platform As A Product: Getting Into The Mindset With User Stories

Wed, 30 Oct 2024 15:00:00 +0000

When you create user stories for your platform, do they all start with “As a platform engineer …”? Switching up how you write stories can engage Product Thinking and give you much happier customers!

The Traditional Approach

Historically, systems people (including everyone from sysadmins through SREs, platform engineers, etc) have had a tough time with creating agile user stories. They frequently go through contortions to describe the work as internal (making our work feel user-oriented can be tough).

These contortions are a result of having a “behind-the-scenes” mentality which creates an us-versus-them approach to things and a disconnect with the user as a customer you are building for. Instead systems people (aka Platform Engineers) have typically written stories from their own perspective.

Here are a couple of examples:

As a platform engineer, I want to create a Terraform module for DNS to make it easier to manage DNS with local standards.

As a platform engineer, I want to deploy Loki so that users can see logs for their applications.

The problem with writing user stories this way is that it is task oriented rather than addressing user needs. As we start to build platforms for other people to use, we need to get out of our usual pattern of building tools that accelerate our own style of work and think about how others will use these tools, preferably self-service.

Let’s see what we can do about that.

Changing Perspectives

Take the first example. What would a customer need from our platform that would relate to DNS?

This could be rewritten this way:

As an application developer, I need to publish my application at a well-known address that I can share with users.

This is interesting. The way we originally wrote it, we were trying to solve an issue with making DNS easier to manage through terraform. Do developers ever ask for new terraform modules? When we change the perspective, we can see that the problem the developer is trying to solve is making their app available in a well known place. Our original story is looking more like a subtask for the real user need. There are probably other things to take care of like TLS certificates, too.

In our second example with Loki, what does the customer need from the platform? Maybe, after sitting down with the customer and observing their day to day work, you might rewrite the story as:

As an application developer, I need to be able to respond to outages in my application by being able to quickly pinpoint the root cause and then understand if the resulting fix is working correctly.

Okay, this one is more than just a story, much more. This is sounding like a whole epic! The user needs observability tooling to respond to outages and while Loki may be part of the solution, logs are just a component of an observability strategy alongside metrics or tracing. Maybe you already have a logging solution and Loki seemed like the next obvious thing to work on. But if you put yourself in the customer’s shoes, you might start to see new priorities that will directly address their needs.

Getting Started

In order to get user stories that are focused on, well, users, we need to get into a product mindset. This involves working to understand the customer’s needs and only then working on solutions that address those needs.

The easiest way to get into the mindset and gain that understanding is to talk to users and really listen.

Sure, there are many product management practices you can use to get customer insight, but talking to them and reframing how you write user stories is a great first step. By challenging ourselves to make our work relevant to users, it pushes us into good behaviors of researching user needs and listening.

It’s a lot like architecture and unit tests. Good unit testing practices naturally push engineers into good architecture patterns and reframing user stories can do the same for platform engineering.

So the next time you start a story with “As a platform engineer…”, stop and think about what you’re doing and why. Then go out and get the clarity you need to confidently build the right thing.

Originally posted at: https://www.missingmass.io/blog/platform-as-a-product-getting-into-the-mindset-with-user-stories

Disclaimer: This blog post represents the viewpoint of its author(s) and does not necessarily reflect an official position or perspective of the TAG or any subsidiary working group. See this blog and contributions guidelines for more information on how you too can contribute.

Blog: Join the Infrastructure Lifecycle Working Group 💙

Wed, 16 Oct 2024 12:00:00 +0000

Are you looking to influence the future of infrastructure management in cloud-native environments? The CNCF Technical Oversight Committee has recently approved the Infrastructure Lifecycle working group within the TAG App Delivery, and we need your expertise!

As cloud-native practices evolve, infrastructure needs are becoming more complex. IT professionals now face the challenge of managing workloads across different environments. These include cloud platforms, IoT edge devices, on-premises data centers, and hybrid setups. There were already considerable investments in both open-source and commercial solutions. However, IT professionals still look for standards that address the whole infrastructure lifecycle.

This working group aims to define technology-agnostic best practices in infrastructure lifecycle management. We will deliver a framework for managing the lifecycle of cloud-native infrastructure, with a focus on components such as:

Infrastructure as Code
Control Planes
State Management
Disaster Recovery
Automation
Testing
Observability

To accomplish our deliverable, we need your help in collaborating with relevant Technical Advisory Groups (TAGs), Working Groups (WGs), vendors, and end-users to integrate domain-specific expertise. Your participation will help us develop practical, real-world solutions that address the diverse needs of the community. Every voice matters, and we warmly welcome and encourage contributions from everyone interested in this field! 🎤

Co-chaired by Ryan Nowak (Microsoft), Bruno Schaatsbergen (HashiCorp), and Jochen Zehnder (Inventx) and various active members, the group brings together expertise from leading industry professionals.

Excited?

Are you a developer interested in shaping the future of cloud-native infrastructure lifecycle management? We’d love for you to be part of it! 💙 The charter provides more information about the working group’s goals and direction.

Bi-weekly meetings: Participate in our discussions on each month’s first and third Fridays.
CNCF Slack Channel: Connect with us on the #wg-infra-lifecycle channel on the CNCF Slack workspace.

Are you attending KubeCon + CloudNativeCon North America 2024? Don’t miss the chance to connect with the co-chairs and members of the Infrastructure Lifecycle Working Group at the TAG App Delivery booth to learn more about our initiatives!

- Ryan Nowak, Bruno Schaatsbergen, and Jochen Zehnder.

Blog: Platform Engineering in 2024, Industry Trends and Emerging Focus (An holistic proposal for Internal Developer Platforms named Platform Engineering ++)

Tue, 25 Jun 2024 15:21:00 +0000

In this blog post, I’ll explore Platform Engineering, covering its diverse interpretations and implementations across organizations.

I’m trying to answer to the following question: “Should Internal Developer Platforms be limited to self-service infrastructure provisioning and application deployment?”

Organizations approach Platform Engineering in different ways:

Some build self-service tools for infrastructure provisioning, giving developers autonomy over infrastructure management.
Others focus on enhancing the developer experience, simplifying coding and deployment.
Some adopt a marketplace-centric approach, creating a repository of reusable components like containers, data, and APIs.

While current Platform Engineering practices effectively address many aspects of simplifying people’s lives, there are other areas that require attention. Data, ML, API, Security, Privacy, and others are crucial for a better Software Product Lifecycle. It is essential to consider whether these teams can benefit from the existing Platform Engineering practices.

By expanding the focus beyond a singular aspect, we can ensure a comprehensive and enhanced experience for all users.

I believe that Platform Engineering should encompass the entire end-to-end value chain to deliver products and applications to end users effectively. Often, Platform Engineering is “solely” associated with Infrastructure and DevOps simplification, with the primary goal of providing a self-service platform for developers.

By envisioning this, we run the risk of creating new barriers between Platform Engineers and Developers, similar to the barriers that existed in the past between IT Operations and Developers. Furthermore, we now have Data, ML, API, and other engineers to consider.

In my opinion, expanding Platform Engineering and its tools to encompass the entire spectrum of Digital Applications is a worthwhile endeavor also for not typical aspects of what is usually called platform.

For example we can consider parts of a Platform also:

a Design System reusable by several teams;
a repository of libraries;
a metadata catalog;
the teams working agreements;
a set of guardrails for ensuring legal and compliance requirements;
a set of standards that application should follow.

I suggest calling this expansion Platform Engineering ++. The core elements, Infrastructure and DevOps plus Data/ML Engineering and plus Software Composability (API, Events, Micro Frontend, Libraries and all aspects of a Software Application that can be reusable and evolved with an Inner Source approach).

I believe this will lead to a more comprehensive and effective approach to application development and management. I will provide further justification for this position in the following paragraphs.

Introduction to Platforms, a metaphor

Like a library where people bring books to share and people borrow books to read, Platforms are a place where people share resources. In the digital realm, platforms serve as virtual libraries, enabling individuals to share and access a diverse range of resources.

You can represent a Library as a Platform where Assets are provided in the form of Books, Video, Papers and more. The Library offers Capabilities like Catalog Collections, Reading Rooms, Cafes and people can use those capabilities thanks to a Research Assistant, a Website, a Subscription Service. Librarians keep everything working and well organized providing a self service experience to users that can be private people, schools, and other groups.

Library users are not only readers, but also potential writers. They may draw inspiration, gather facts, and formulate thesis from other books they find in the library, which they can use to create new works.

Internal Developer Platforms

In this analogy, we can liken Librarians to Platform Engineers. Their primary objective is to make sure that users of the platform receive top-notch services and have a delightful experience while interacting with it. In the realm of metaphors, books can be likened to various resources and tools (just some examples):

A kubernetes cluster with all necessary software and configurations.
A crossplane CRD to create an infrastructure resource.
A kafka topic used for data streaming.
A data product available for consumption.
A software library designed for logging in the correct format, including the traceId.
A web component library to compose in a micro frontend.
An API for initiating payments through a digital payment provider.
A complete application accessible to end users.

The facilities of a library can be (just some examples):

A service catalog where all consumable and composable resources are listed.
A self-service UI to create a new software element (infrastructure, application or other).
A command line interface to view live logs in production.
A monitoring system to see microservice metrics.
A set of scorecards to gather information about team metrics.

The library, which I like to call the Internal Platform (or Internal Developer Platform, see next paragraph), is where all the technological magic happens. Platform Engineers manage this platform, ensuring that all Internal Platform Users have an exceptional experience. Their approach is product-oriented, focused on delivering a seamless and intuitive user experience.

Depending on the usage patterns of the platform users and the organization’s specific requirements, Platform Engineers fine-tune the platform to deliver the necessary services.

A platform is a virtual place where resources (I will call them Items as a synonym) are systematically arranged and disseminated. By optimizing time utilization, more resources become accessible.

By effectively leveraging these items, that were books in the analogy, organizations can create innovative solutions, optimize operations, and gain a competitive advantage.

Within the Internal Platform’s Platform Capabilities, capability providers offer a range of fundamental resources (Items), including Infrastructure, DevOps Toolchains, SaaS Services, and Tools, which are organized and consistently curated by Platform Engineers.

Each capability constitutes an Item of the Platform and is characterized by a:

definition: describes inputs, outputs, configurable behaviors, metrics and documentation;
life cycle: an item goes through a life cycle that includes development, testing, deployment, operation and decommissioning.
consumption and composition: items can be consumed and composed to create more complex applications and become other items to be consumed.

Product Teams can engage with and use these Items through user interfaces, command lines, and APIs. AI Agents can assist the team in simplifying usage, while the Items are organized in Catalogs with the corresponding documentation.

As a result of the collaboration, new applications are created. These applications can then be transformed into Raw Assets, which can be reused repeatedly by other teams, enhancing the organization’s DevX.

This cycle can be seen as a circular economy in software development, where resources are continuously reused and repurposed, leading to a more efficient and sustainable development process.

Platform Items Lifecycle

Platform items come in several varieties, but possessing common characteristics:

are described by a plain text file (there are several format that are emerging and I hope in the future we will have a standard);
can be put in relationships each to other;
have a life cycle for create, ship, run, operate and catalog that item in order to be consumed by Product and Application teams.

It’s important to note that users can interact with the system through a UI or CLI, or through another software that automates some tasks, or via an AI agent based on LLM.

If you had access to a comprehensive internal platform containing all the descriptions of behaviors and relationships necessary to run your application, how beneficial would it be to have an AI companion?

Imagine having a companion that simplifies tasks such as finding items, configuring them, and releasing and operating those items in production, all through simple conversations with the AI. This companion would significantly streamline your workflow and enhance your overall productivity.

I propose referring to this as “Conversational DevX”, which will be the focus of my next blog post.

Internal Developer Platform or Platform of Platforms or Internal Platform?

I don’t know if it should be better to say that an Internal Developer Platform is an ensemble of different platforms or that it’s just one of the platforms. This is an open discussion where we welcome conversation and other ideas.

Names are important but in this blog post I don’t want to focus on the name but on the content and I hope that together with the tag-app-delivery ( https://github.com/cncf/tag-app-delivery) people we will figure out a standard nomenclature.

As mentioned in the introduction of this blog post, Platform Engineering is more than Infrastructure and it isn’t an evolution of DevOps but is a way to reduce the cognitive load for people that are creating, delivering and operating software items like Applications, Data, API and ML Models.

The trend I see is that each type of software item has its platform. However, the user experiences across these platforms are becoming increasingly similar. This suggests a potential convergence of all software platforms into a single platform or a “platform of platforms.”

An Item in the Internal Platform can be of different types:

Infrastructure Resources: this encompasses the underlying hardware and software components that support your applications and services. It includes physical servers, virtual machines, containers, cloud platforms, data stores, databases, ML runtimes and more;
DevOps or Developer Platform Resources: all tools for defining and running a toolchain, plus tools to operate and observe workloads, data, APIs, event, AI models at runtime;
Data, Events, APIs Resources: data is a critical asset for many organizations. It can be structured, unstructured, or semi-structured and can come from various sources. It can be at rest or in movement. It can be updated by policies and emit events;
ML and AI Resources: machine learning models are algorithms that can learn from data and make predictions. You can define, training, deploy and improve your models;
Composable Resources: items can be orchestrated or choreographed. Depending on the type of the item you may use different patterns like Sagas or Micro Frontend composition; Composable Items are tools that allow you to compose;
DevX Resources: DevX as a Product Resources (internal marketplace and software catalog): each item of the Internal Platform can be a valuable asset for other people. Provide a marketplace for that items and a way to manage the marketplace lifecycle: create, publish, curate, consume, review is relevant for creating a software circular economy;
Team Collaboration Resources: teams work together within internal platforms, utilizing various items to facilitate communication and organize workflows. These items include issues, backlogs, documentation, and more. Understanding the Platform itself is essential for effective collaboration.

We can explode the diagram with different Platforms of the Internal Platform.

The Internal Platform obtains “raw” assets such as virtual machines (VMs), Kubernetes as a Service clusters, and Function Platform as a Service tools as runtime resources.

DevOps tools are essential for managing resources and code lifecycle, forming a fundamental pillar alongside observability, security, and identity management services. Data stores such as NoSQL and SQL databases, data streams and object storage are commonly utilized in application development.

Additionally, LLMs models are frequently employed as a service via APIs or embedded within the runtime to enhance applications created by teams. SaaS applications, such as Salesforce, Dynamics 365, and SAP, serve as central hubs for various customer and product information, playing a significant role in the development of cloud-native applications.

All capabilities are managed with an Infrastructure as Code approach and the IaC manifests are managed with the lifecycle: code, ship, run, operate and organize team collaboration.

The same approach resource lifecycle is shared among different kinds of platform items: developers, infrastructure, application orchestration (API, Events, Flows), machine learning and data. All of them are Platforms inside the Internal Platform that are interconnected and may share resources like a database cluster or a security policy.

The descriptor of that items are manifest that can be gathered in the concept of Application as Code (AaC): with the same approach of describing the desired state of the infrastructure you describe the desired state of the items of all platforms that compose the cloud native application.

Inside the Internal Marketplace are defined: the reusable blueprint for all platform items, the software catalog that you can reuse and all other items that come from each team that can be a building block for other teams. Thanks to that Marketplace you can enable a Platform Composability strategy.

The Internal Platform offers various interfaces, including a user interface (UI) and a command-line interface (CLI). Additionally, users can interact with the platform programmatically through an API. The AI serves as a companion. IaC and AaC manifests are provided as a context to the LLM and the AI simplify tasks for platform users. These tasks include resource discovery, troubleshooting microservices and resources, and creating new resources.

Platform Engineering++ is a software engineering discipline that focuses on the Internal Platforms with a holistic approach.

Conclusions

By adopting that Platform Engineering++ paradigma, I believe that the Platform Engineering initiative’s return on investment (ROI) will grow. This is because the platform becomes the foundation for business applications, eliminating obstacles between teams. As a result, all teams have a unified perspective of the end-to-end applications built on top of it.

For now I stop here and I ask for feedbacks :-). What do you think about it? Please leave a comment in the issue https://github.com/cncf/tag-app-delivery/issues/586 and join CNCF wg-platforms working group if you are interested.

The journey is just at the beginning!

References

In this blog post, when I mention “Platform,” I specifically refer to “Internal Developer Platform” or, as I prefer, “Internal Platform,” as these platforms are not exclusive to developers but also encompass Data, ML, Cloud, and other engineers. It’s important to distinguish these Internal Platforms from Business Platforms such as Uber, Airbnb, or Netflix. Internal Platforms assist organizations in building and operating their Business Platforms. For example, Spotify created Backstage as an Internal Developer Portal and has incorporated additional tools to develop its Internal Platform to operate Spotify consumer platform.

Please note that there are already papers that are defining some aspects that I’m going to discuss in this blog post:

the CNCF Platform White Paper ( https://github.com/Cloud-Native-Platform-Engineering/cnpe-community/blob/main/platforms-whitepaper/v1/index.md)
the Platform Glossary ( https://github.com/Cloud-Native-Platform-Engineering/cnpe-community/blob/main/platforms-wg/glossary/_index.md)
the Platform Engineering Maturity Model ( https://github.com/Cloud-Native-Platform-Engineering/cnpe-community/blob/main/platforms-maturity-model/v1/index.md)
the Platform of Platform initiative ( https://github.com/cncf/tag-app-delivery/issues/542).

I hope that this blog will contribute to brainstorming.

Thanks

This is an original blog post written for the CNCF TAG App Delivery community. I hope you enjoy it!

I express my sincere gratitude to the CNCF TAG App Delivery community for establishing a virtual place that fosters open-minded discussions and inclusivity.

Special thanks to who has reviewed this post! Atulpriya Sharma, Lou Bichard, Tyler Pate, Abby Bangser, Stefan Daugaard Poulsen, Chris Plank, Marsh Gardiner, Rob White and David Stenglein for their insightful reviews.

Blog: Invitation To Participate in Platform As A Product Research

Thu, 13 Jun 2024 12:00:00 +0000

Invitation to participate in Platform As A Product Research

(Don’t worry, you don’t need to be a product manager!)

The CNCF Platforms Working Group is pleased to announce a research effort to increase our understanding of how the industry is building internal platforms and we’d like you to participate.

After the publishing of the Platform Maturity Model, the working group’s next area of focus has been on Platform As A Product. This approach treats our users as customers of the platform, which is a product that should address their needs. While there is certainly a lot of coverage on this topic in conferences and public forums, it is unclear how widespread these practices are.

While the name of the research project might imply that we’re only interested in talking to product managers, that definitely isn’t the case! We want to talk to anyone building platforms to find out how teams decide what to build for their users. So regardless of your methods for building a platform, whether its using product management or not - we’d love to talk to you!

The interview process itself is pretty simple and consists of a 45 minute conversation with a couple of interviewers. Hopefully we can get multiple people from the same organization to participate between different roles like platform engineers, internal platform users and platform leaders.

All interview data is kept confidential and will only be used after being anonymized during our analysis across interviews.

Our goal is to have some initial analysis to talk about for the upcoming Platform Day at KubeCon NA in Salt Lake City.

So if you’re interested in participating in this research, please sign up here.

Blog: Enterprise IDPs must mature fast. Here’s how infrastructure optimization can help

Tue, 28 May 2024 12:00:00 +0000

Enterprises are expected to benefit from platform engineering sooner and bigger than anyone else. This has 2 main reasons:

With engineering bodies 100’s or 1,000’s large, standardization across the board is both a pressing need and a major event.
In large organizations, concerns like security, regulatory compliance, and cost efficiency can impede development.

With these two combined, the enterprise route to faster time-to-market must also go through scalable, effective guardrails along the product lifecycle. Therefore, Internal Developer Platform (IDP) capabilities that might be dismissed as day-2, effectively become day-1 imperatives. In other words, enterprise IDPs must mature fast – at least in some respects like compliance, security, and efficiency. When the MVP emerges, those must be already thoroughly worked.

3 cloud-native frameworks might prove useful in designing your platform MVP for faster time-to-(business) value. Let’s see what they mean to enterprises, identifying one takeaway from each towards that value.

CNCF IDP maturity model

The platform engineering maturity model presents five IDP aspects (investment, adoption, interfaces, operations, measurement), each described in four maturity states (provisional, operational, scalable, optimizing).

This matrix format means that, as the IDP evolves, some aspects might mature before the others. For instance, a certain IDP might be merely “operational” on its investment and adoption aspects, but demonstrate “optimized” interfaces, operations, and measurement.

This scenario becomes likely once your IDP includes cost optimization capabilities; unlike other managed SaaS, those typically repay their subscription price and beyond. So, no need to wait until a fully “scalable” stage before they’re introduced. Let’s illustrate it with a hypothetical example:

Example: soonicorn.io IDP status


Aspect	Descriptive example	Maturity
Investment	Dedicated platform team of 4: Senior platform engineer, full-stack developer, product manager, part-time UX designer.	Scalable
Adoption	Push: Teams are instructed to stop using their own deployment scripts; Cloud expenditure for Dev & QA is only authorized if resources are spun up via the IDP.	Operational
Interfaces	Integrated services: Every new project receives a space in a task runner (pipelines) and a runtime environment (Kubernetes namespace) and can opt into serverless runtime.Automation schedules pods to run on spot instances in a Dev cluster configured for bin packing.	Optimized
Operations	Managed services: VM users are only asked to declare their app as stateful/stateless and fault tolerant/intolerant. VM provisioning then spins up a spot instance configured for state reattachment and fallback or utilizes existing commitments.	Optimized
Measurement	Qualitative and quantitative: Within 1 year of MVP release, Dev vCPU hours increase by 30% while total Dev cloud costs only increase by 15%. Integrated FinOps tool analyzes each node to provide detailed cost attribution per project and team.	Optimized

This scenario is highly relevant for enterprises, which are financially conservative by nature. For those, measurement is, above all, financial. The board is all in for engineering productivity, but it must be shown expected ROI to approve any IDP investment. In other words, an enterprise platform is expected to repay itself by saving software engineering hours or otherwise money in a direct, measurable way. Therefore, the IDP mandate may depend on the ability to demonstrate how it saves the organization money:

Does it improve DORA time-centric metrics? (i.e. lead time for change, time to restore service)
Does it decrease downtime whereas the cost of downtime is known?
Does it lead to reduced cloud expenditure?

Achieving this may require the IDP to integrate both monitoring (e.g. GitLab) and guardrails for responsible cloud usage, backed by integrated tooling that applies savings techniques and visually tracks their results.

One takeaway: an IDP can earn initial credit by generating financial value as soon as its provisional investment or adoption phase, i.e. without changing the way the engineers work. For example, it can enforce cost-effective cloud resource provisioning policies in the backend. The cost reduction generated by those is easily measured and understood. Thus, this so-called day-2 capability makes more sense as part of the platform’s MVP. The value they unlock might actually win the mandate to keep maturing the platform in other aspects.

CNoE from a platform engineering perspective

The Cloud Native Operational Excellence (CNoE) initiative aims to help enterprises ease into cloud-native tooling, especially open-source. Platform engineering is its #1 use case. CNoE advocates several tenets to increase cloud-native maturity in enterprises:

Tools over practices – as hard as it is, tool adoption is still easier than culture change. Pick tools that automate the practices you want to implement – especially those your developers are not used to perform hands-on, like cloud resource provisioning.
Convergence – prefer tools that fulfill multiple desired capabilities, and can close a circle on data collection, reporting, recommendations and execution.
Managed solutions – the support gap can be closed by commercial vendors offering manages services for popular cloud-native solutions like Kubernetes, Linux and distros, or popular IDP projects Backstage and Crossplane.
Customizable standard for infrastructure – balance between the enforcement of requirements to developer usability
Powered by Kubernetes but not confined to it – prefer tooling that isn’t confined to a specific container engine, so besides different types of managed Kubernetes they may use ECS, Openshift, VMWare Tanzu and others.

These tenets are no longer mere theory: CNoE’s centerpiece is the idpBuilder tool, accompanied by a non-branded reference architecture. For enterprises looking for even more out-of-the-box open-source IDPs, BACK Stack emerges as another practical application of CNoE principles, with opinionated tooling selection which can be modified later.

Is this nirvana? I’m not sure. In my opinion, the challenges CNoE sets to solve (rapid evolution of standards, DevOps not always a good fit) aren’t the most painful ones in enterprises.

To complete the picture, here are some grave open-source concerns I heard directly from enterprise engineers:

Fragmentation – open-source tools often don’t integrate natively, so they don’t always work off the same datasets and policies – which might lead to frequent failures. When one tool only recommends (e.g. Opencost) and another only executes (e.g. Terraform), you need at least one critical pair of human eyeballs at the intersection. And at scale, there are 1,000’s of those intersections daily.
No support - Open-source tools lack professional support, which places extra load on in-house engineers.
Privacy concerns – GDPR-compliant organizations usually rule out open-source software and DBs, and also SaaS hosted on hyperscalers (AWS, Azure or GCP)The potential fine for working on data acquired unethically or exposing their own users’ data is just too big. Even if it’s free, some procurement guidelines still apply!
Inability to measure added value – in the case of cloud-native SaaS which serves as part of a stack, or does something specific along the software supply chain, its contribution to metrics improvement cannot always be isolated.

One takeaway: in the previous section we established the importance of cost optimization as an IDP’s day-1 capability. To realize this according to CNoE tenets, we want to integrate into our IDP a cost optimization solution that is managed, converged, highly automated, and suits all containerized and non-containerized settings.

Well-architected AWS, Azure, and GCP

Cost optimization is a Well-Architected pillar in AWS, Azure, and GCP alike. Each hyperscaler outlines pillar slightly different, but all touch aspects of designing & planning, monitoring & managing, and cost-reducing activities. Out of all 3, I find Azure’s to be the most straightforward:

Develop cost-management discipline (by FinOps practices)
Design with a cost-efficiency mindset
Design for usage optimization
Design for rate optimization
Monitor and optimize over time

Intuitively, what’s right for your entire public cloud estate is right for an IDP that runs in it. This leads to a two-fold conclusion: the IDP itself must run cost-efficiently on cloud resources; and it should help cultivate the same cost efficiency practices in the entire Dev organization.

From a FinOps standpoint, you want your IDP to provide visualized cost observability and accurate usage attribution of each node’s components by label, namespace etc. This means it will facilitate collaboration between the engineering body and other cost stakeholders, FinOps first.

From an activity standpoint, you want it to leverage discount compute (spots, RIs, SPs) and to continuously “squeeze the lemon” of each machine’s CPU, memory, storage, and network capacity. This is achievable by automating Kubernetes optimization techniques like bin packing, rightsizing, shutdown scheduling, and dynamic storage.

One takeaway: Your IDP’s ideal integrated cost optimization solution has dual DevOps/FinOps qualities: On one hand, it has workable IaC integrations to automate the actions needed to lower cloud spend and minimize it going forward; On the other hand, it provides cost visibility & analysis that support cloud cost attribution and accountability.

How do I make this my own?

If you work with external platform or portal tooling, e.g. Backstage, Port, Crossplane, Cortex etc. – follow the next steps:

Check integration catalog for cloud resource management solutions. These might also be classified as provisioning, autoscaling & optimization.
Consider starting with open-source reporting tools like Kubecost to learn about your optimization potential.
Research externally for tools capable of automating the heavy lifting of continuous optimization. Today, these are nearly non-existent in the integration catalogs of platform vendors – because they are considered day-2 capabilities, and most platforms are simply not there. No reason to worry though – autoscaling & co. is a saturated, competitive segment. Once you start a POC, it’s in the vendor’s best interest to develop a plugin or provider for your chosen platform tool. Such a plugin might help put them in front of many other users.

Platforming from scratch? Start off from step 2 above. When you hit step 3, look for the vendor’s GitHub repo, or Terraform module, to create the integration. If you don’t have Dev resources to dedicate, you may request it from the vendor once your POC becomes a paid subscription.

Special thanks to Artem Lajko and Abby Bangser for their insightful reviews.

Blog: Platform Engineering At KubeCon EU 2024 - Recap

Mon, 06 May 2024 12:00:00 +0000

Exactly a month ago, Kubernetes users and experts gathered in the City of Lights, Paris, for KubeCon’s Europe edition. With over 12,000 in-person attendees, this KubeCon was amongst the largest in recent times.

While there was A LOT of attention and chatter around Artificial Intelligence and Generative AI, Platform Engineering was well represented. This was the first “Platform Engineering Day,” where enthusiasts had dedicated space to discuss anything and everything related to platform engineering.

In this KubeCon 2024 recap post, I’ll shed light on the platform engineering talks and discussions that took place at KubeCon.

TAG App Delivery & WG-Platforms at KubeCon EU

The TAG App Delivery is dedicated to supporting projects and initiatives related to delivering cloud-native applications, including building, packaging, deploying, managing, and operating them. One of the initiatives at TAG is the Platforms working group, which is dedicated to supporting, improving, and advancing the platform engineering initiatives.

At KubeCon EU, talks and sessions were organized at the booth to spread awareness about the group’s work. You can read more about it in our previous blog post. If you’re new to the TAG App Delivery or platforms working groups, you can watch the following talks to know more:

Platform Engineering Day

First up, I’ll share the amazing talks from the first edition of Platform Engineering Day. I remember entering the hall a little late and seeing the place full. People were standing and listening to the talks.

Talks

TAG App Delivery Platforms Working Group Update - Colin Griffin from Krumware provides an overview of what the Platforms Working Group is and how to get involved.
Sometimes, Lipstick Is Exactly What a Pig Needs! - Abby Bangser from Syntasso & Whitney Lee from VMware highlight the importance of investing in user interfaces and adoption for internal developer platforms, even more than perfecting the underlying technology.
Beyond Platform Thinking at Ritchie Brothers - Build Things No One Expects, in a Place No One Expect - Bryan Oliver from Thoughtworks & Ranbir Chawla from Ritchie Bros discuss how Ritchie Bros has expanded the capabilities of Kubernetes beyond just a delivery platform to drive their core enterprise.
Blueprints of Innovation: Engineering Paved Paths for a User-Friendly Developer Platform- Ahmed Bebars from The New York Times - provides valuable insights into the nuances of platform engineering in the cloud, giving a blueprint for implementing strategies in their organizations.
Building a Platform Engineering API Layer with kcp - Marvin Beckers from Kubermatic GmbH discusses how kcp supercharges platform engineering with a global control plane for all internal services.
Breaking the Mold: Unveiling Anti-Architectural Patterns in Platform as a Product - Vamshi Krishna Samudrala from American Airlines - discusses the intricate landscape of designing and implementing effective platforms.
Empowering Giants: Guide Your Enterprise with CNOE in Operational Tech Choices - Engin Diri from Pulumi introduces the CNOE Framework, and explores how participation can benefit organizations in overcoming challenges.
Designing for Success: UX Principles for Internal Developer Platforms - Kirsten Schwarzer from Octopus Deploy shows practical UX principles and tools you can use to design an Internal Developer Platform (IDP) that your developers love using.
Boosting Developer Platform Teams with Product Thinking - Samantha Coffman from Spotify talks about the effectiveness of taking the product approach for building platforms.
Building an AI-Powered, Paved Road Platform with Cloud-Native OSS - Todd Ekenstam & Avni Sharma from Intuit share insights on how open-source projects, such as Open Application Model, Istio, Karpenter, Argo Rollouts, can be integrated and extended to build your AI-native application platform.
Unlocking Innovation: How NatWest Bank Uses Cloud Native Tools to Deliver Platform as a Product - Chris Plank from Natwest Group & Derik Evangelista from Syntasso discuss how they focused on a GitOps approach and incorporated a range of tools to enable platform users to have a seamless developer experience.
To K8S and Beyond – Maturing Your Platform Engineering Initiative - Nicki Watt, from OpenCredo shares insights on how the recently released CNCF platform maturity model can be used as part of a toolbox to help guide organisations think through.

Panel Discussions

Panel: Navigating the Path to Platform Engineering Excellence: A Comprehensive Guide- Cortney Nickerson from Kubeshop; William Rizzo from SUSE; Abby Bangser from Syntasso; Areti Panou from SAP SE and Aparna Subramanian from Shopify discussed actionable steps to ensure effective platform engineering and dissect critical considerations for anyone looking to invest in their own platform.
Panel: The Platform Rock-Paper-Scissors: Build, Adopt, Buy- Jorge Lainfiesta, Independent Contributor; Leena Mooneeram from Chainalysis; Victor Araujo from Wolt; Jinhong Brejnholt from Saxo Bank and Edgaras Petovradžius from LEGO share their thoughts on how to wrangle budgets, lock-ins, and licenses to make decisions as you put together the foundations of your platform

Talks and Panel Discussions at KubeCon

While Platform Engineering was one of the hottest co-located events at KubeCon 2024, talks and panel discussions took place in other co-located events, such as BackstageCon, AppDevCon, ArgoCon, MultitenancyCon, and OpenShift Commons to name a few.

Here’s a round-up of all of them:

Day 1 - March 20

Bloomberg’s Journey to a Multi-Cluster Workflow Orchestration Platform - Yao Lin & Reinhard Tartler from Bloomberg talk about how they investigated related projects and what inspiration we took from Karmada, OCM, and others to build their own orchestration platform.
Building a Large Scale Multi-Cloud Multi-Region SaaS Platform with Kubernetes Controllers- Sébastien Guilloux from Elastic describes an architecture made of hundreds of Kubernetes clusters, and talks about the challenges we have faced along the way to build a multi-cloud, multi-region platform.
Simplified Inner and Outer Cloud Native Developer Loops - Oleg Šelajev from AtomicJar & Alice Gibbons from Diagrid explore tools to simplify and improve developer productivity through a platform engineering and polyglot approach.
Building AI-Ready Platforms - Symphony for Developer and Platform Engineer - Thomas Vitale from Systematic & Lize Raes from LangChain4j share details to bridge the gap between platform engineers and developers, focusing on adapting your platform for AI while providing a smooth developer experience.
State of Platform Maturity in the Norwegian Public Sector - Hans Kristian Flaatten from the Norwegian Labor and Welfare Administration uses the newly published CNCF Platform Engineering Maturity Model to measure how mature their platforms are, and what technologies they have chosen.
Cultural Shifts: Fostering a Chaos First Mindset in Platform Engineering - Sayan Mondal from Harness & Raj Vadheraju from FIS talk about how organizations can enhance their platform engineering practices by leveraging chaos-first principles.

Day 2 - March 21

Unlocking New Platform Experiences with Open Interfaces - Thomas Vitale from Systematic & Mauricio “Salaboy” Salatino from Diagrid explore existing CNCF projects to implement an end-to-end experience to build platforms.
Keeping the Bricks Flowing: The LEGO Group’s Approach to Platform Engineering for Manufacturing - Mads Høgstedt Danquah & Jeppe Lund Andersen from The LEGO Group share their story of how the LEGO Group builds platforms and products that cope with constraints like 24/7 production, limited internet connectivity, high resilience and low latency.
Why Kubernetes Is Inappropriate for Platforms, and How to Make It Better. - Stefan Schimanski from Upbound; Mangirdas Judeikis from Cast AI; Sebastian Scheele from Kubermatic - extending Kube, adapting its architecture to be a better fit for a platform.

Day 3 - March 22

Rapid IDP Capability Development and Automated Testing at Autodesk - Jesse Sanford & Greg Haynes from Autodesk show how IDPBuilder can stand up a CNOE reference architecture in minutes, with nothing other than Docker as a pre-dependency.
Search at Shopify: Highly Available Platform for Data Resilience and Compliance - Leila Vayghan from Shopify show how Kafka is used for in-order and real-time indexing of millions of documents per minute to achieve high availability

Panel Discussions and On-Booth Talks

You see, there was so much chatter about platform engineering at KubeCon!

Apart from the talks in the main conference, there were panel discussions, and talks at other co-located events and the TAG App delivery booth as well. While we don’t have videos for all these talks, there were some interesting ones around self-service infrastructure with Backstage, and Kubernetes developer experience patterns, to name a few.

Lian, Thomas, Mauricio and Atul participated in a panel discussion - From Platform Engineering To Developer Success at the RedHat OpenCommons colocated event.

Based on my experience of attending KubeCons, this has to be one of the most prolific ones when it comes to platform engineering. The talks were engaging, the platform working group was active and the booth was abuzz.

With the CFPs for the KubeCon NA 2024 already open, share your interesting proposals on platform engineering and let the world know about the amazing things you’re doing.

Platform Coffee Meetups - Paris Edition

One of the things that I absolutely love about the platforms working group is the in-person coffee meetups that we organize during KubeCons. The first one I attended was in KubeCon Chicago and I loved the concept.

Anyone and everyone is invited to discuss anything and everything about platforms and platform engineering over a cup of coffee. It’s an amazing experience to learn about what is happening in the world of platform engineering and to meet the amazing people behind it.

The Platform Lean Coffee meetup in progress. We've some REALLY amazing topics that we're discussing today.

Kudos to the WG-Platforms group at @CNCFTAGApp for this one.

Also thanks to our sponsors @syntasso @krumware @gitpod 🙏🏻😇 pic.twitter.com/NPMHZBlgQo
— Atulmaharaj 🥑 (@TheTechMaharaj) March 21, 2024

This time, we had it for four days, and we had people turning up in great numbers each day. With Croissants, Pain au Chocolat, and Cafe Au Lait, we all discussed everything from what platforms actually mean to discussing the success criteria for platforms - these were some insightful discussions.

Key Takeaways

This was my second in-person KubeCon and fourth overall. It was also the second time I met the members of the platform working group in person. While we connect over calls monthly and work on exciting things like the Platform Whitepaper, Platform Maturity Model, and ongoing Platform as a Product paper, meeting in person is way better.

Further, a huge shoutout to the members of the TAG App delivery and WG Platforms who were helpful right from the day KubeCon was announced to being hyperactive during KubeCon. Further, the first Platform Engineering Day was a huge success, and rooms filled with people are a testament to the importance and interest in platform engineering.

Apart from being a part of a platform engineering panel, I got to listen to some interesting talks on how people navigate the platform engineering maze. I was particularly intrigued by discussions around customized measuring the success of platforms and AI-ready platforms. The one thing I absolutely enjoyed about the discussions was the focus on developer experience and its larger productivity aspects.

Join Us!

Being a part of this fun and intellectual group, I can vouch that this is one of the most interactive and helpful groups. Whether you’ve just learned about Platform Engineering or are professional building platforms for years, you’re welcome to join the platform working group and help us shape the future of platforms.

Join our WG-Platforms Slack to get started. Just drop a note to introduce yourself and let us know what you want. Someone from the team will help you get started :)

Blog: Practical Paths to Platform Maturity: Insights from KubeCon Paris

Wed, 27 Mar 2024 12:00:00 +0000

If you were lucky enough to be at KubeCon in Paris last week, you might have heard Nicki Watt from OpenCredo give a practical walkthrough of the CNCF’s Platform Maturity Model. Nicki was an early contributor to the development of the model, and she spoke about her experience using it with client organizations.

Some of the topics that Nicki covers include:

Understanding Starting Points and Goals: how to assess an organization’s current position within the CNCF Platform Maturity Model aspects, and then how to identify opportunities. This involves surveying existing capabilities and identifying the steps required to meet business needs.
Tailoring Strategies to Context: ways to adapt the maturity model to an organization’s unique context. Instead of aiming for higher levels of maturity for their own sake, Nicki advocates for work that is appropriate to the organization’s specific goals, constraints, and current state.
Center of Excellence Challenges: insights on overcoming obstacles in advancing the maturity of a platform engineering center of excellence. This includes knowing organizational goals, understanding the current landscape of people, processes, and technology, and ensuring that any tools or systems adopted are API-driven to facilitate integration and flexibility.
Practical Application of the Maturity Model: Through examples from her experience, Nicki illustrates how the maturity model can guide organizations in enhancing their platform engineering practices. This involves plotting out paths and options that align with achieving business outcomes, based on the model’s framework.

If you are looking for fresh ideas about how to improve your platform initiative, this talk is full of practical strategies and insights for using the Platform Maturity Model effectively.

Blog: Getting started with contributing in WG Platforms

Wed, 20 Dec 2023 12:00:00 +0000

Similar to the advice on the TAG App Delivery contributions page, we highly encourage new faces and new voices in existing forums, including asynchronous chats on Slack, GitHub issues, and the fortnightly working group Zoom calls.

In addition, the WG Platforms has noticed a number of exciting new ideas generated by new joiners and wants to create an avenue for those ideas to be supported and successful, even coming from the newest voices. With that in mind, we have created a path that will help these ideas get the support they need!

When you have a new idea

You are passionate in the platform engineering space and have an idea on how to share that passion with the CNCF community, that is exciting and we want to help!

Even with this excitement, we understand that contributing your own content for the first time can be confusing or intimidating. Don’t worry, we are a welcoming community and always open to new ideas and thoughts. If you’ve been wanting to be a part of the Platform WG, you’ve come to the right place.

The following process builds on the wider TAG contribution guidelines to provide a lightweight way to ensure that all of these great ideas get the support they deserve within the scope of the WG.

We have had ideas raised from new WG roles (e.g. a proposal for a community outreach role), a new white paper (e.g. the platform as a product white paper), a blog post (e.g. two sided market theory), and more. Some of these have garnered more traction than others, but the overriding criteria that we see for success is building enough momentum within the WG to get reviews for publication. This process is built to support new voices with an advocate who has the skills and experience in this process to make sure new joiner friction doesn’t cause a great idea to be silenced.

If anything about this process is stopping you from contributing, the most important thing is to raise the idea. You can reach out to the WG Platform leads at any time to bounce an idea around and learn more about how the WG can help.

With that in mind, the three steps are:

Step 0 - Idea generation

Before you can publish you will need to have an idea to share! Therefore, you may start by asking “Is my idea suitable for this working group?” While you should always feel empowered to ask, you can first evaluate if your idea relates to platforms and platforming engineering. Some examples of relevant topics including:

Technical overview or experiences working with platform tools and related technologies
Experience report or interviews about platform building or using thought leadership in regards to supporting developer experience and * productivity
Hands-on DIY posts helping readers learn a tool

Please keep in mind this is not an exhaustive list and are very open to new ideas. It may be easier to enumerate what is not fit for the working group:

Vendor or other promotional pitches
Topics not related to application delivery or cloud native technologies
Discriminatory or abusive content

Step 1 - Submission

We would encourage you to generate a GitHub issue and open a Slack thread with your idea. You can use the community contribution template to create an issue and also start a thread on the CNCF Slack in the #platform-engineering channel.

The GitHub issue will present you with a template where you can follow the prompts. First, You must write a descriptive title, then answer each question in the description field. If you find one is not relevant that is OK, write that and there is always a chance to chat more on these things after your initial submission.

For Slack, you can start a new thread in this format: [Proposal]<My_New_Idea>

This submission will act as two things:

A call for support from others in the community. You may naturally pick up a project advocate or set of collaborators who are as passionate as you are on the topic.
A home for all the work done on this piece of work. You will be updating this frequently to indicate goals, progress, and asks for help.

Step 2 - Initial acceptance

Once submitted, you can expect a WG lead to respond within a week. They will help clarify any open questions, confirm that the idea is within the scope of this WG, and guide you towards any existing work that you may be able to benefit from or where your idea may fit better if it doesn’t fit best within the WG.

They will also recommend next steps for finding a project advocate so that you have someone to work with to find WG support during the project lifecycle including publishing and publicising the work.

Step 3 - Drafting & Reviews

Now comes the fun work! You can work with your advocate and the entire WG community to refine your idea and produce the best possible content. Depending on what you are working on this could take days, weeks, or even months. Even if your idea has a scope of work that could last months, we highly suggest you find ways to release smaller content pieces first to generate more interest and also more confidence in the alignment of your work. For example, most blogs are written, reviewed, and published within 1-2 months. These are the types of things your advocate will help support you with.

Once you’ve finished drafting your blog post, you can update the GitHub issue saying it’s ready for review as well as update the Slack with the draft as a Google Doc and tag anyone who you feel can review it. (PS: It’s an open forum so anyone is free to review, but if you feel there’s someone who must have a look at it, tag them)

During the review process, we’ll do the following checks:

Basic grammar, syntax, and language check - we suggest using a tool like Grammarly before submitting for review.
Technical correctness - we’ll validate the technical accuracy of what you’ve written.
Vendor Pitches/Promotional links - we’ll carefully go through the content to ensure there’s no promotional material.

Step 4 - Final Approval and Publishing

After the review is complete, and there’s a consensus from everyone that this is good to go, the WG lead will initiate the process of raising a PR and merging it.

What next?

Congratulations, your blog post will be live by now and a handful of people will have already read it. So what next? Well, feel free to share the blog posts on social media, and tag us (TAG App delivery). Also, don’t forget to thank everyone who helped to make the blog post live. And lastly, you should start thinking about your next blog post. So shall start again from the beginning?

Blog: Clusters for all cloud tenants

Thu, 02 Jun 2022 13:04:00 +0200

A decision which faces many large organizations as they adopt cloud architecture is how to provide isolated spaces within the same environments and clusters for various teams and purposes. For example, marketing and sales applications may need to be isolated from an organization’s customer-facing applications; and development teams building any app usually require extra spaces for tests and verification.

Namespace as unit of tenancy

To address this need, many organizations have started to use namespaces as units of isolation and tenancy, a pattern previously described by Google and Kubernetes contributors. But namespace-scoped isolation is often insufficient because some concerns are managed at cluster scope. In particular, installing new resource types (CRDs) is a cluster-scoped activity; and today independent teams often want to install custom resource types and operators. Also, more developers are themselves writing software operators and custom resource types and find themselves requiring cluster-scoped access for research and tests.

Cluster as unit of tenancy

For these reasons and others, tenants often require their own isolated clusters with unconstrained access rights. In an isolated cluster, a tenant gets its own Kubernetes API server and persistence store and fully manages all namespaces and custom resource types in its cluster.

But deploying physical or even virtual machines for many clusters is inefficient and difficult to manage, so organizations have struggled to provide clusters to tenant teams. Happily :smile:, to meet these organizations’ and users’ needs, leading Kubernetes vendors have been researching and developing lighter weight mechanisms to provide isolated clusters for an organization’s tenants. In this post we’ll compare and contrast several of these emergent efforts.

Do you have other projects and ideas to enhance multitenancy for cloud architecture? Then please join CNCF’s App Delivery advisory group in discussing these here; thank you!

vcluster

vcluster is a prominent project and CLI tool maintained by loft.sh that provisions a virtual cluster as a StatefulSet within a tenant namespace. Access rights from the hosting namespace are propogated to the hosted virtual cluster such that the namespace tenant becomes the cluster’s only tenant. As cluster admins, tenant members can create cluster-scoped resources like CRDs and ClusterRoles.

The virtual cluster runs its own Kubernetes API service and persistence store independent of those of the hosting cluster. It can be published by the hosting cluster as a LoadBalancer-type service and accessed directly with kubectl and other Kubernetes API-compliant tools. This enables users of the tenant cluster to work with it directly with little or no knowledge of its host.

In vcluster and the following solutions, the virtual cluster is a “metadata-only” cluster, in that resources in it are persisted to a backing store like etcd, but no schedulers act to reify the persisted resources - ultimately as pods. Instead, a “syncer” synchronization service copies and transforms reifiable resources - podspecs - from the virtual cluster to the hosting namespace of the hosting cluster. Schedulers in the hosting cluster then detect and reify these resources in the same underlying tenant namespace where the virtual cluster’s control plane runs.

An advantage of vcluster’s approach of scheduling pods in the hosting namespace is that the hosting cluster ultimately handles all workloads and applies namespace quotas - all work happens within the namespace allocated to the tenant by the hosting cluster administrator. A disadvantage is that schedulers cannot be configured in the virtual cluster since pods aren’t actually run there.

vcluster on GitHub

Cluster API Provider Nested (CAPN)

In vcluster, bespoke support for control plane implementations is required; as of this writing, vcluster supports k3s, k0s and vanilla k8s distributions.

To support any control plane implementation, the Cluster API Provider Nested project implements an architecture similar to that of vcluster, including a metadata-only cluster and a syncer, but provisions the control plane using a Cluster API provider rather than a bespoke distribution.

CAPN promises to enable control planes implementable via Cluster API to serve virtual clusters.

HyperShift

Similar to the previous two, Red Hat’s HyperShift project provisions an OpenShift (Red Hat’s Kubernetes distro) control plane as a collection of pods in a host namespace. But rather than running workloads within the hosting cluster and namespace like vcluster, HyperShift control planes are connected to a pool of dedicated worker nodes where pods are synced and scheduled.

HyperShift’s model may be most appropriate for a hosting provider like Red Hat which desires to abstract control plane management from their customers and allow them to just manage worker nodes.

kcp

Finally, kcp is another proposal and project from Red Hat inspired by and reimagined from all of the previous ideas. Whereas the above virtual clusters run within a host cluster and turn to the host cluster to run pods, manage networks and provision volumes, kcp reverses this paradigm and makes the hosting cluster a metadata-only cluster. Child clusters - workspaces in the kcp project - are registered with the hub metadata-only cluster and work is delegated to these children based on labels on resources in the hub.

As opposed to hosted virtual clusters, child clusters in kcp could manage their own schedulers. Another advantage of kcp’s paradigm inversion is centralized awareness and management of child clusters. In particular, this enables simpler centralized policies and standards for custom resource types to be propogated to all children.

Conclusion

vcluster, CAPN, HyperShift, and kcp are emerging projects and ideas to meet cloud users’ needs for multitenancy with clusters as the unit of tenancy. Early adopters are already providing feedback on good and better parts of these approaches and new ideas emerge daily.

Want to help drive new ideas for cloud multitenancy? Want to help cloud users understand and give feedback on emerging paradigms in this domain? Then join the discussion in CNCF’s TAG App Delivery. Thank you!

Cloud Native Platform Engineering Community – Article

Blog: A Living Blueprint: Evolving the Platform Engineering Whitepaper and Maturity Model

Blog: Cloud Native Platform Engineering Japan Kickoff in Japan!

Keynote

Platform Engineering from CNCF’s Perspective

Avoiding Pitfalls When Adopting Gateway API: Secure-by-Default Design Achieved with Kuadrant and Keycloak

The Evolution of Platform Engineering at Mercari

Summary

Bonus: A Thinking Framework for Platform Engineering

Blog: Navigating Platform Engineering with the maturity model assessment

Platform orienteering

The maturity model is the map

The assessment is a compass

Route planning

Prototype assessment feedback

Helping you navigate

Blog: From Chaos to Clarity: Navigating Observability in the Platform Engineering Era (and a Dash of AI)

Introduction

The Observability Headaches: Where Do We Feel the Pinch?

Internal Developer Platforms: The Unified Solution

The Open Question: Balancing Trust and Cost with Benefits in the LLM Era

Wrapping Up

Blog: Demystifying Composability on Platforms: extending Platforms to business needs

Introduction

Characteristics of a Composable Platform according to the personas

How do the different personas view a platform?

From Builders:

From Platform Enablers:

From Developers (platform end-users):

From Customers (platform end-users):

Conclusion

Blog: Platform as a Product: Understanding the Personas

Platform as a Product Overview

Understanding the personas

Personas interactions

Blog: Platform as a Product Research - Now with a Survey!

Expanding Our Platform as a Product Research: Share Your Insights Through Our New Survey!

Blog: Platform As A Product: Getting Into The Mindset With User Stories

The Traditional Approach

Changing Perspectives

Getting Started

Blog: Join the Infrastructure Lifecycle Working Group 💙

Excited?

Blog: Platform Engineering in 2024, Industry Trends and Emerging Focus (An holistic proposal for Internal Developer Platforms named Platform Engineering ++)

Introduction to Platforms, a metaphor

Internal Developer Platforms

Platform Items Lifecycle

Internal Developer Platform or Platform of Platforms or Internal Platform?

Conclusions

References

Thanks

Blog: Invitation To Participate in Platform As A Product Research

Invitation to participate in Platform As A Product Research

(Don’t worry, you don’t need to be a product manager!)

Blog: Enterprise IDPs must mature fast. Here’s how infrastructure optimization can help

IDP-related frameworks you should know

CNCF IDP maturity model

Example: soonicorn.io IDP status

CNoE from a platform engineering perspective

Well-architected AWS, Azure, and GCP

How do I make this my own?

Blog: Platform Engineering At KubeCon EU 2024 - Recap

TAG App Delivery & WG-Platforms at KubeCon EU

Platform Engineering Day

Talks

Panel Discussions

Talks and Panel Discussions at KubeCon

Day 1 - March 20

Day 2 - March 21

Day 3 - March 22

Panel Discussions and On-Booth Talks

Platform Coffee Meetups - Paris Edition

Key Takeaways

Join Us!

Blog: Practical Paths to Platform Maturity: Insights from KubeCon Paris

Blog: Getting started with contributing in WG Platforms

When you have a new idea

Step 0 - Idea generation

Step 1 - Submission

Step 2 - Initial acceptance