History and evolution of platform engineering

Platform engineering is a discipline focused on building and maintaining software development platforms that provide self-service for developer teams, offering the necessary infrastructure for provisioning an application, for example. The entire flow of development, testing, documentation, deployment, rollback, etc., can happen through developer self-service. It emerged from the need to overcome the limitations of traditional software development, which was often characterized by slow cycles, isolated teams, silos, and complex infrastructures. The main goal of Platform Engineering is to improve the developer experience (DevEx) and optimize software delivery.

In essence, platform engineering seeks to:

• Provide standardized tools and services: Create a set of tools, services, and workflows that developers can easily use to build, deploy, and manage their applications. These tools can include IDEs, CLIs, project templates, code versioning, CI/CD pipelines, container orchestration tools, configuration management tools, and observability systems.
• Reduce complexity: Abstract the complexity of the underlying infrastructure and operations, allowing developers to focus on code and business logic. The growing complexity of infrastructure, the cloud computing era with different providers and vendors, multi-cloud adoption, and the proliferation of tools and artifacts in the “everything as code” era led to an increase in developers’ cognitive load. The lack of design patterns also became a problem.
• Increase productivity: Accelerate the development and deployment cycle, eliminating bottlenecks and repetitive tasks. This is achieved through the automation of tasks such as infrastructure provisioning, application deployment, and configuration management.
• Improve governance and security: Incorporate security and compliance practices into the platforms from the beginning. platform engineering aims for a more secure and compliant setup by integrating security controls and governance policies.
• Promote collaboration: Facilitate collaboration between development, operations, security, and product teams. The platform acts as an interface for different teams to work more cohesively, have visibility into solutions, and find what is necessary for the development of their work.

To understand how platform teams operate, it’s essential to view the platform as a product.

Platform as a product

A platform engineering team acts as an “internal provider,” offering a layer of abstraction that allows developers to operate more independently and efficiently. This approach is driven by the “platform as a product” mindset, where the platform is treated as a product with internal developers as its customers. This means the platform team focuses on understanding the needs of its users (primarily application developers) and delivering value to them.

Instead of each development team building and maintaining its own infrastructure and tools, platform engineering centralizes this responsibility, ensuring consistency, scalability, and optimization across the organization.

Comparison with other traditional models (ticket ops vs. self-service)

Traditionally, developers depended on operations teams to provision infrastructure and perform operational tasks. This model, often called “ticket ops,” generated significant delays, bottlenecks, and an unsatisfactory developer experience, as they were “blocked” waiting for ticket resolution; creating a mere user on a server could take a long time, delaying development. This approach resulted not only in delays but also in even more fragmented knowledge, which could become a bottleneck, especially with undocumented solutions.

Contrary to what some may think, platform engineering is a natural evolution of DevOps and transforms this scenario by focusing on creating self-service solutions. It allows developers to perform common tasks on their own, such as provisioning resources, deploying applications, and managing configurations, without depending on operations teams. This is done through:

• Self-service interfaces: APIs, automation tools, and web portals that simplify access to and interaction with the infrastructure. An internal developer portal (IDP) is a centralized environment where developers can create, discover, consume, explore, test APIs, and get support.
• Golden paths and paved roads: Defines and automates the preferred and approved approaches for software development and deployment. This reduces cognitive load, as developers do not need to learn every detail of each tool, and it ensures compliance and security. The implementation of a “check/scorecard” system in an existing IDP, for example, significantly impacted developer engagement by providing “applicable and concrete actions” for improvement.
• Platform orchestrators: Tools that act as a central link, managing the platform’s lifecycle and orchestrating the infrastructure and application configurations, potentially even replacing categories of tools like Terraform and Ansible.

The benefits of platform engineering

The adoption of platform engineering aims to solve several common challenges in software development and, consequently, offer a series of tangible benefits for organizations:

Problems it seeks to solve

• High cognitive load for developers: The need for developers to understand and manage the complexities of infrastructure, Kubernetes, CI/CD pipelines, and security. platform engineering reduces this cognitive load by abstracting infrastructure and automating tasks.
• Lack of standardization and consistency: Different teams using inconsistent tools and processes, leading to errors and inefficiencies. PE promotes standardization and consistency in environments and workflows.
• Slow development cycles and operational bottlenecks: Manual processes, dependencies on operations teams, and difficulties in continuous integration and delivery. The “ticket ops” model exemplifies this slowness.
• Fragmented security and compliance: Difficulty in ensuring that all applications and infrastructures follow security policies and regulations.
• High cost of development and maintenance: Redundancy of efforts and time spent on repetitive tasks.

Benefits:

• Increased developer productivity and efficiency (DevEx): By automating repetitive tasks and providing ready-to-use components, PE accelerates development and deployment, freeing up developers to focus on innovation and writing code. This improves the developer experience by providing a friendly, self-service environment.
• Reduced time-to-market: Accelerates product and feature release cycles, allowing companies to respond quickly to market demands.
• Improved security and compliance: Integrating security controls and governance policies from the platform’s design phase. This reduces risks and vulnerabilities and makes it easier to implement uniform security policies.
• Cost optimization: Automation and standardization of processes lead to reduced operational expenses and more efficient use of IT resources.
• Greater agility and scalability: Allows organizations to adapt more quickly to changes and expand their operations efficiently. PE provides the necessary tools to handle demand growth without compromising performance or security.
• Attraction and retention of talent: A modern and efficient work environment, with less friction and more autonomy, is more attractive to top engineering professionals.

Competitive differentiation: Enables the development of unique and innovative solutions, making the company stand out in the market.