Guest post originally published on InfraCloud’s blog by Vishal Biyani

In the decade of 2000, a lot was happening in the software industry. The Toyota lean movement had proven to be of significant value in the production of cars. A lot of industries beyond manufacturing and obviously the software industry took note of the parallels to “lean” in their world. The early innovators in software industry also were dabbling with the idea of “continuous integration” as can be seen from birth of tools such as CruiseControl (2001), and Hudson (2005) – a fork of which was named Jenkins (2011) was adopted by the OSS community & became one of the most popular tools to implement CI/CD! In year 2009, the official term DevOps was coined at O’Reilly Velocity Conference where Flickr engineers presented their famous talk: “10+ Deploys per Day: Dev and Ops Cooperation at Flickr” & DevOpsDays conference was formed.

Star Trek Meme from '10+ Deploys per Day: Dev and Ops Cooperation at Flickr' talk

(From the popular talk “10+ Deploys per Day: Dev and Ops Cooperation at Flickr”)

The basic idea of DevOps was to align the “Development” and “Operations” teams towards the goal of delivering software at higher speeds. While the alignment of technologies, processes & people took a good part of the decade of 2010!

Cloud, scale & complexity growth: decade of 2010

The decade of 2010 saw tremendous growth in cloud, SaaS & technologies like containers. Just to get a sense of the growth, AWS went from 100K customers in 2010 to a million in 2015 or Docker which was introduced in 2013 had 100K companies using/evaluating in 2014, and within a year 2015, a million companies were using containers in the form of Docker!

Chart showing Docker adoption behavior is up 40% in one year

(Source: Docker Adoption Research by Datadog)

The growth of cloud & technologies did change a few things such as:

Chart showing 40% of Docker organizations use orchestrators

Growth in orchestrators such as Mesos, Docker Swarm, Kubernetes or Rancher Cattle in early years. Source: Docker Adoption Research by Datadog 2017

As SaaS became more mainstream – things like reliability, security, cost all took the main stage. The scope of “operations” expanded way more than before!

ShiftLeft + “You Build It, You Run It” (YBYRI)

While the scope of operations expanded way more, the things on Dev side were also increasing equally. A lot of concerns around things such as security or reliability were being defined “as code”. Hence, developers had to be aware of these things, even if at a superficial level. Shifting left was also driven by the agile movement so that fixes can happen as close to the developer’s inner development as possible instead of the code going all the way to stage for the required changes and flowing through the entire pipeline again.

Some large organizations also experimented with YBYRI – You Build It, You Run It! Which meant the developer teams had to be aware of many things beyond just writing code. The developer teams could end up owning evetything from changes in code related to business to reliability, security etc.

Development+++: definitions & confusion!

So DevOps movement started with great intent and did solve many problems. However, the growth of cloud, SaaS & complexity of applications blurred the boundaries quite fast – leading to increased cognitive load for almost everyone!

Barcode image of You Built it You Run it Doesn't Scale!

(Image Source)

So should a developer apart from writing code also know how Terraform provisions everything or how to write all manifests and charts for Kubernetes? Similarly, should operations team end up writing code and if yes, where do you draw the boundaries? It is not a question of creating walls, but definitely the cognitive load of so many things on everyone is not productive. So if we go back to our original diagram of Dev aligning to Ops, it is now an alignment between many more functions such as security, reliability, cost, and of course, Dev & Ops!

Round chart puzzle of Developer Cognitive Log

Rise of Platform Engineering

All these things lead to re-thinking of the structure of teams to reduce cognitive load, increase flow state & faster feedback loops. Some organizations which were scaling fast & were leading adopters of DevOps were quick to recognize the pattern and improvise the team structures, tools & processes to build out platforms which enable developers.

So what is platform engineering? The goal of platform engineering within an organization is to enable the delivery teams to get what they need & move fast while focusing on application. With the rise in complexity & breadth of things that an application goes through, it is impossible for developers to know everything in depth. This is where the platform enables them to use sensible defaults for things they want to move fast on, while later changing it to their needs. Over time the platform will become a full blown product that enables the delivery of software but that does not have to be the goal to start with.

That’s a wrap! You can learn more about Platform Engineering in our detailed blog post on Platform Engineering 101. If you’re looking to get started with platform engineering, do learn about Backstage and the need for IDPs in the development ecosystem and how it can help organizations to grow efficiently. For a weekly dose of cloud native technologies, don’t miss to subscribe to our newsletter.