The ultimate guide to platform engineering

Platform engineering is often misunderstood as highly technical and time-consuming to set up and run. But with the right design, it doesn’t have to be. So, what is platform engineering? Overall, platform engineering is a set of tools and practices around development toolchains that make it easier for developers to create great software and safeguard time, money, and competitiveness.

It establishes new (or enhances existing) internal development platforms (IDPs).  Focusing on IDP achieves efficiencies that free up developers to think innovatively. A well-structured IDP helps support teams and maintains the integrity of all aspects of the software development lifecycle, from ideation to production.

Platform engineering with cleverly structured IDPs will ultimately provide “golden paths” that developers can quickly adopt. The term IDP often has several connotations and is even used as a catch-all term. At Eficode, we refer to its classic definition, which respects its positioning, those who use it, and the technology it captures. To us, it’s an additional layer that simplifies operations and allows developers to help themselves with existing technology and tools.

With AI's continued prevalence, platform engineering can help your organization raise its competitiveness in multiple ways, whether through building your own model, storing organizational-specific context, building supporting models, or running AI workloads.

What is an internal developer platform? Learn more in our blog post on "Internal developer platforms: What they are and why you need one."

Platform engineering vs. DevOps

What is DevOps? defines DevOps as being about product management, site reliability, technical excellence, and healthy company culture. Breaking down silos between software development and operations, DevOps connects feedback from end-users in production environments to development teams such that they can react to market conditions and out-learn their competition. Providing a proven track record in enabling Business Agility and performance. 

We help hundreds of organizations refine their DevOps and believe in the practice of getting the architecture right. This is why we see platform engineering as an evolution of DevOps. Platform engineering is a focused approach and philosophy that is applied strategically and tactically to achieve many results simultaneously.

Some say, “DevOps is dead; long live platform engineering!” or search for results online such as “DevOps vs platform engineering,” but the two approaches are complementary. Platform engineering leverages DevOps practices while reducing cognitive load, aiming to enable developers' self-service capabilities.

Application development becomes increasingly complex as DevOps continues to expand. Developers need to learn new digital platform tools, manage infrastructure, and prioritize operational tasks while coding to develop new features. These demands reduce productivity, increase burnout, and lead to job weariness.

Platform engineers are crucial in simplifying standard DevOps processes

Imagine a business that creates web applications using a consistent structure with a database, a backend that provides RESTful APIs, and a web-based front end. While modern software tools and templates have been adopted, the development process still heavily relies on manual efforts. DevOps engineers are responsible for creating Docker files, writing Terraform scripts, setting up project-specific build pipelines, and managing environment updates.

Usually, a platform engineer serves as the expert who collaborates closely with developers, guiding and steering the setup and execution of the system to address their requirements while handling monitoring and alerts and helping the developers meet their Service Level Agreements (SLAs). In a context with DevOps teams, this would lead to critical bottlenecks, increasing developer lead times, and significantly stressing DevOps engineers.

Platform engineers simplify this using an internal developer platform (IDP) to automate tasks, starting with self-service. Developers don't need to set up Git repositories manually, as users can quickly request an IDP to create user groups and automatically integrate the right CI/CD template.

Some platform engineers even work with management to incorporate long-term goals and company vision into the development lifecycle. Many companies either have their own engineers or need consultants. This is where Eficode steps in.

Platform engineering doesn’t replace DevOps practices; it builds on them

Platform engineering offers teams an easy way to start projects through standardized patterns. These patterns are built into an IDP with self-service capabilities, and with them, teams start adding value immediately instead of spending weeks on project setup and problem-solving.

Self-service allows developers to be autonomous yet compliant without being overloaded. Because of this, platform engineers can focus on more significant architectural challenges, improve current features, and adjust the system to changing needs.

Check out the podcast episode "Platform engineering done right" about the capabilities of an IDP in a Portuguese bank, Millenium BCP.

Platform engineering incorporates Site Reliability Engineering as a best practice

Google pioneered Site Reliability Engineering (SRE), which focuses on operating and improving software applications at a large scale. Despite sounding alike, platform engineering and site reliability engineering are different. SRE is mostly about operations: Running a service and ensuring it’s continuously available and up-to-date. However, SRE also provides a model for service management, which can also be applied to IDPs. The approach discussed in “Site Reliability Engineering - How Google Runs Production Systems” is particularly valid.  On the other hand, platform engineering includes SRE thinking and is more holistic in its aim to understand interplays between process, people, and technology.

Reliability in platform engineering

A software application can’t offer a higher Service Level Agreement (SLA) than the lower layers in the stack. For an app to have a 99.9% availability guarantee, all its infrastructure components must offer the same deal. SLAs matter between a platform team and the development teams that use the platform. They constitute a promise of overall SLAs and thereby provide development teams with an expected level of reliability.

A Service Level Objective (SLO) targets a service level measured by a Service Level Indicator (SLI). Choosing the right SLOs is challenging but vital for the platform team's performance measurement and business success. It helps platform teams balance innovation and reliability using an error budget and the margin between SLI and SLO.

Note: SLOs and error budgets should also be published to set expectations for its stakeholders.

Platform teams and incident management

Platform teams are essential for the reliability of software applications running on their platform and infrastructure. Additionally, they must take responsibility for problems with parts belonging to the platform team during outages or issues.

Team interaction modes

Site Reliability Engineering (SRE) teams work closely with development teams, and their interaction changes as the application progresses. An SRE team may be a mix of enabling and operations teams, providing coaching on scaling and building reliable services up to a certain point. The SRE team should take full responsibility for the reliability of one (or more) digital platform services, which is different from a platform team, which is expected to deliver self-service interfaces for developer teams to consume. The platform team needs a product mindset and a close feedback loop with development teams to build the right things.

Knowing that any platform has two aspects—operational capabilities and developer experience—there are some rules of thumb that help create a long-lasting, solid foundation for any organization.

The power of collaboration

Technical teams must collaborate with organizational stakeholders to obtain the necessary support and resources for building an internal developer platform (IDP). Getting executive support is crucial, as building an IDP alone requires significant resources and isn’t just about technical challenges.

Team leaders are also charged with gaining buy-in: They must inspire cooperation with their developers. The key to success is a transparent building process that keeps focus on adoption and ease-of-use interfaces while supporting the operational side of applications.

Learn more in our blog post on "Establishing a platform engineering organization.

Choosing your tools wisely

While building a successful IDP is much more than the tools of the trade, there is no denying that choosing the right tools also matters. Should you choose one or the other, build vs. buy/adopt are some of the key questions to ask. Each organization has its own needs, processes, philosophies, and budgets; it’s all a matter of building something that fits your organization.

We can even help you evaluate how to improve with our toochain assessment or technology-specific assessment.

Whether you are choosing between virtual machines or Kubernetes, on-premise or cloud, one deployment tool or the other, the right choice is a matter of thinking about what the tool brings to the table, either as a platform builder or a platform user. You'll often have to consider whether you pass through a one-way door or can easily switch to something else when technologies and workflows change.

Making the bigger decisions can require another layer of collaboration as many stakeholders are involved, including the intersection of technical capabilities, policies, security, and legal requirements, mixed with procurement, finance, and other cross-organizational entities. Luckily, not all decisions are that complex when building an IDP. There are a range of industry standards and best practices that will guide your journey, and many open source tools have matured, and likewise, for the standards, which many commercial products have decided to follow. In many cases, this will allow you to switch many parts of your setup or postpone the slightly harder decision until later in the process, as long as you adhere to a standard.

No matter what you choose, you can always get up and running without building a complete custom solution. But be sure to adjust the tools available in the market so you can free up time to fulfill the needs of the platform users instead of fighting overly complex implementations. While we like the creative process of the implementations, there are limits to how much we can and should invest in building our things.

Stay curious: Experiment and improve

Falling into the rabbit hole of building the solid foundational layer is common. We want everything in place before the first team moves into the platform. We also want our developers to have a great experience, so we need to “have it finished” for them to run their services.

In reality, we’ll never be done with the foundational layer. We can focus on the thinnest viable platform and shift our mindset to incremental improvements. This will allow us to build smaller (and more Agile) and validate with our developers more often. On the other hand, we would spend a lot of time building what we think the developers want without getting validation.

While we want to build in small increments, we should always keep our heads up, look forward, and think about a roadmap, as there are many requirements for modern software. Sometimes, there are non-negotiable requirements that we have to include and have in the initial step, but we should be able to solve those without delaying the initial consumers.

If you’re hesitant about going into production and need to do a little more work, reach out to your colleagues who will want you to succeed and offer their help to test the platform.

To learn more about the challenges and pitfalls of platform engineering, check out Dan Grøndahl Glavind’s talk on “Platform Engineering is Hard, and We are Doing it Wrong” at DevOpsDays Denmark 2023.

Most options to assess software development focus on local productivity, quality, and consistency in the delivery process. When it comes to DevOps, there are well-established metrics thanks to the DevOps Research Assessment (DORA) program:

• Deployment frequency.
• Lead time for changes.
• Mean time to recover.
• Change failure rate.
• Reliability.

These metrics allow us to capture the impact of DevOps culture and practices in quantitative terms related to IT performance and quality. Lately, attention has shifted towards evaluating how developers feel about and value their work by taking a DevX-centric approach to measuring success.

Measuring Developer Experience

Measuring DevEx is daunting, as the concept itself is very broad. That’s why it’s helpful to start with a framework that identifies the three core dimensions of DevEx: Feedback loops, cognitive load, and flow state. These dimensions emphasize that DevEx is not only a technical matter but also primarily about developers’ perceptions. Getting insight is a good start, but it’s important not to forget the importance of informal conversations between colleagues and teams.

Developers are human, and no single quantitative metric can accurately capture DevEx, so collecting qualitative feedback from developers through surveys is crucial. This feedback describes the perceived complexity and satisfaction with deploying code and using a platform, among other aspects. For example:

• Easy-to-use tools help developers streamline development and deployment quickly, without hassle.
• Self-service capabilities support developer independence and control.
• High-quality documentation lets developers navigate complex infrastructures. 
• The platform enables the collection of diverse metrics related to DevEx.
• The fast development environment setup fosters productivity and creativity in realizing minimum viable products.

Platform engineering goes beyond improving productivity and leading to faster innovation. By improving DevEx, a great platform makes developers happier, improving retention, talent acquisition, and success in realizing business goals.

See Nicole Forsgren’s talk "Why even DevOp—making our days better" for more information.