We are thrilled to announce that the first ever EnvoyCon will be held on December 10 in Seattle, WA as part of the KubeCon / CloudNative Con Community Events Day. The community growth since we first open sourced Envoy in September 2016 has been staggering, and we feel that it’s time for a dedicated conference that can bring together Envoy end users and system integrators. I hope you are as excited as we are about this!
CNCF offers diversity scholarships to developers and students to attend KubeCon + CloudNativeCon events. In this post, scholarship recipient, Yang Li, Software Engineer at TUTUCLOUD, shares his experience attending sessions and meeting the community. Anyone interested in applying for the CNCF diversity scholarship to attend KubeCon + CloudNativeCon North America 2018 in Seattle, WA December 11-13, can submit an application here. Applications are due October 5th.
This guest post was written by Yang Li, Software Engineer
When I was in college, I wrote software with Python on Ubuntu, and read Cathedral and the Bazaar by Eric S. Raymond. These were my first memories of open source.
Later on, I worked with a variety of different open source projects, and I attended many tech conferences. But none of them were like KubeCon, which gave me the opportunity to take part in the open source community in real life.
Not only was I able to enjoy great speeches and sessions at the event, but I also met and communicated with many different open source developers. I made many new friends and amazing memories during the four days in Copenhagen.
Although I haven’t been to many cities around the world, I can safely say that Copenhagen is one of my favorites.
“Diversity is essential to happiness.”
This quote by Bertrand Russell is one of my firm beliefs. Even as a male software engineer and a Han Chinese in China, I always try to speak for the minority groups which are still facing discrimination. But to be honest, I haven’t found much meaning in the word diversity for myself.
However, soon after being at KubeCon, I understood that I’m one of the minorities in the world of open source. More importantly, with people from all over the world, I learned how inclusiveness made this community a better place. Both the talk by Dirk Hohndel and the Diversity Luncheon at KubeCon were very inspirational.
I started working with Kubernetes back in early 2017, but I only made a few contributions in the past year. Not until recently did I become active in the community and joined multiple SIGs. Thanks to the conference, I have a much better understanding of the culture of the Kubernetes community. I think this is the open source culture at its best.
Distribution is better than centralization
Community over product or company
Automation over process
Inclusive is better than exclusive
Evolution is better than stagnation
It is the culture that makes this community an outstanding place which deserves our persevering.
This guest post was written by Jean de Klerk, Developer Program Engineer, Google
Much of the web today runs on HTTP/1.1. The spec for HTTP/1.1 was published in June of 1999, just shy of 20 years ago. A lot has changed since then, which makes it all the more remarkable that HTTP/1.1 has persisted and flourished for so long. But in some areas it’s beginning to show its age; for the most part, in that the designers weren’t building for the scale at which HTTP/1.1 would be used and the astonishing amount of traffic that it would come to handle.
HTTP/2, whose specification was published in May of 2015, seeks to address some of the scalability concerns of its predecessor while still providing a similar experience to users. HTTP/2 improves upon HTTP/1.1’s design in a number of ways, perhaps most significantly in providing a semantic mapping over connections. In this post we’ll explore the concept of streams and how they can be of substantial benefit to software engineers.
Semantic Mapping over Connections
There’s significant overhead to creating HTTP connections. You must establish a TCP connection, secure that connection using TLS, exchange headers and settings, and so on. HTTP/1.1 simplified this process by treating connections as long-lived, reusable objects. HTTP/1.1 connections are kept idle so that new requests to the same destination can be sent over an existing, idle connection. Though connection reuse mitigates the problem, a connection can only handle one request at a time – they are coupled 1:1. If there is one large message being sent, new requests must either wait for its completion (resulting in head-of-line blocking) or, more frequently, pay the price of spinning up another connection.
HTTP/2 takes the concept of persistent connections further by providing a semantic layer above connections: streams. Streams can be thought of as a series of semantically connected messages, called frames. A stream may be short-lived, such as a unary stream that requests the status of a user (in HTTP/1.1, this might equate to `GET /users/1234/status`). With increasing frequency it’s long-lived. To use the last example, instead of making individual requests to the /users/1234/status endpoint, a receiver might establish a long-lived stream and thereby continuously receive user status messages in real time.
Streams Provide Concurrency
The primary advantage of streams is connection concurrency, i.e. the ability to interleave messages on a single connection.
To illustrate this point, consider the case of some service A sending HTTP/1.1 requests to some service B about new users, profile updates, and product orders. Product orders tend to be large, and each product order ends up being broken up and sent as 5 TCP packets (to illustrate its size). Profile updates are very small and fit into one packet; new user requests are also small and fit into two packets.
In some snapshot in time, service A has a single idle connection to service B and wants to use it to send some data. Service A wants to send a product order (request 1), a profile update (request 2), and two “new user” requests (requests 3 and 4). Since the product order arrives first, it dominates the single idle connection. The latter three smaller requests must either wait for the large product order to be sent, or some number of new HTTP/1.1 connection must be spun up for the small requests.
Meanwhile, with HTTP/2, streaming allows messages to be sent concurrently on the same connection. Let’s imagine that service A creates a connection to service B with three streams: a “new users” stream, a “profile updates” stream, and a “product order” stream. Now, the latter requests don’t have to wait for the first-to-arrive large product order request; all requests are sent concurrently.
Concurrency does not mean parallelism, though; we can only send one packet at a time on the connection. So, the sender might round robin sending packets between streams (see below). Alternatively, senders might prioritize certain streams over others; perhaps getting new users signed up is more important to the service!
Concurrent streams, however, harbor some subtle gotchas. Consider the following situation: two streams A and B on the same connection. Stream A receives a massive amount of data, far more than it can process in a short amount of time. Eventually the receiver’s buffer fills up and the TCP receive window limits the sender. This is all fairly standard behavior for TCP, but this situation is bad for streams as neither streams would receive any more data. Ideally stream B should be unaffected by stream A’s slow processing.
HTTP/2 solves this problem by providing a flow control mechanism as part of the stream specification. Flow control is used to limit the amount of outstanding data on a per-stream (and per-connection) basis. It operates as a credit system in which the receiver allocates a certain “budget” and the sender “spends” that budget. More specifically, the receiver allocates some buffer size (the “budget”) and the sender fills (“spends”) the buffer by sending data.The receiver advertises to the sender additional buffer as it is made available, using special-purpose WINDOW_UPDATE frames. When the receiver stops advertising additional buffer, the sender must stop sending messages when the buffer (its “budget”) is exhausted.
Using flow control, concurrent streams are guaranteed independent buffer allocation. Coupled with round robin request sending, streams of all sizes, processing speeds, and duration may be multiplexed on a single connection without having to care about cross-stream problems.
The concurrency properties of HTTP/2 allow proxies to be more performant. As an example, consider an HTTP/1.1 load balancer that accepts and forwards spiky traffic: when a spike occurs, the proxy spins up more connections to handle the load or queues the requests. The former – new connections – are typically preferred (to a point); the downside to these new connections is paid not just in time waiting for syscalls and sockets, but also in time spent underutilizing the connection whilst TCP slow-start occurs.
In contrast, consider an HTTP/2 proxy that is configured to multiplex 100 streams per connection. A spike of some amount of requests will still cause new connections to be spun up, but only 1/100 connections as compared to its HTTP/1.1 counterpart. More generally speaking: If n HTTP/1.1 requests are sent to a proxy, n HTTP/1.1 requests must go out; each request is a single, meaningful request/payload of data, and requests are 1:1 with connections. In contrast, with HTTP/2 n requests sent to a proxy require nstreams, but there is no requirement of n connections!
The proxy has room to make a wide variety of smart interventions. It may, for example:
Measure the bandwidth delay product (BDP) between itself and the service and then transparently create the minimum number of connections necessary to support the incoming streams.
Kill idle streams without affecting the underlying connection.
Load balance streams across connections to evenly spread traffic across those connections, ensuring maximum connection utilization.
Measure processing speed based on WINDOW_UPDATE frames and use weighted load balancing to prioritize sending messages from streams on which messages are processed faster.
HTTP/2 Is Smarter At Scale
HTTP/2 has many advantages over HTTP/1.1 that dramatically reduce the network cost of large-scale, real-time systems. Streams present one of the biggest flexibility and performance improvements that users will see, but HTTP/2 also provides semantics around graceful close (see: GOAWAY), header compression, server push, pinging, stream priority, and more. Check out the HTTP/2 spec if you’re interested in digging in more – it is long but rather easy reading.
To get going with HTTP/2 right away, check out gRPC, a high-performance, open-source universal RPC framework that uses HTTP/2. In a future post we’ll dive into gRPC and explore how it makes use of the mechanics provided by HTTP/2 to provide incredibly performant communication at scale.
CNCF offers diversity scholarships to developers and students to attend KubeCon + CloudNativeCon events. In this post, scholarship recipient, Mohammed Zeeshan Ahmed, Associate Software Engineer at Red Hat, shares his experience attending sessions and meeting the community. Anyone interested in applying for the CNCF diversity scholarship to attend KubeCon + CloudNativeCon North America 2018 in Seattle, WA December 11-13, can submit an application here. Applications are due October 5th.
By Mohammed Zeeshan Ahmed, Associate Software Engineer at Red Hat
Hailing from Bangalore, India, I got introduced to computers when I was about 13 years old. I still remember the old Unix based operating system, on which I wrote my first C program and compiled it with gcc, on a friends computer. That feeling you get when you get a box to do something you want is amazing.
Ever since then I was hooked, growing up with various Linux distributions and technology. I am also a gamer, and that only served to increase my hunger for technology.
My introduction to the world of containers happened two years ago, when I stumbled upon one of my now great friends, Neependra Khare, at my first technical conference, in India.
I am very grateful for the introduction to this world that happened on that fateful day. The day I found Kubernetes (amongst many other things).
I also want to give a shout out to Kelsey Hightower, whose “Kubernetes the Hard Way” talk helped me gain a good understanding regarding this complex container orchestrator.
The Diversity Scholarship:
Receiving the scholarship came as a pleasant surprise. I have been working with Kubernetes and OpenShift as a user as well as a developer, building tools around it for some time now. I really wanted to dig deeper into this world. This scholarship actually gave me an opportunity to interact with the amazing communities working on cloud technology and to learn a lot from them. It also helped me get started on one of my biggest goals of contributing back to these amazing communities.
I identify myself as a person of color, from India. It was therefore really amazing to see steps being taken by the communities, to ensure inclusion of women and people of color.
Kubecon EU 2018, in the Bella Center, Copenhagen, Denmark, turned out to be one of the most amazing conferences I have attended. To see such a well-organized event, at such scale is pretty damn amazing. The venue was huge and really well decked out. There were barely any problems for the attendees.
I had signed up for the Kubernetes Contributor Summit that happened on May 1st and it was awe inspiring to be in the room with the people who build this amazing software. It was the largest turnout so far in a Kubernetes Contributor Summit.
I was present for the new contributor workshop which helped me learn a lot about how to contribute to Kubernetes. It has now lead me to join the Kubernetes sig-release, the first step for me in this direction.
I met some really amazing people there. I even got some really cool swag such as the Kubernetes Contributor patch, which has encouraged me to step up my game.
The next three days worth of keynotes were really amazing, especially when Kelsey Hightower showed off his no code repository to everyone. The list of projects under CNCF is growing, and it was very amazing to see the various stages in which the projects were. Congratulations to Kubernetes by the way, for graduating.
The sponsors came on stage and talked about their journeys and shared their wisdom. The ones that i enjoyed the most personally was CERN, who showed a lot of information about how they were using a large number of Kubernetes clusters, in their day-to-day research.
Another keynote speech that I enjoyed was by Dan Kohn about the status of code in the projects. Makes you think about the sheer size and scale of these projects and the amount of testing, like three million lines of testing code and man hours that go into maintaining all of it.
Another really commendable part of the conference that I enjoyed were the face-to-face interactions with the maintainers. It was really nice to hear about the projects from the people who maintain them. I am looking to start contributing to some of them as well, apart from Kubernetes.
We also had amazing diversity lunch, courtesy of our sponsors. Great people, amazing discussions about inclusion (and other stuff of course) and good food. What more could you want?
Also, what an amazing dinner that was at the Tivoli Gardens!! To be in a magical place, surrounded by amazing folks, all you can eat food, and all you can ride rides. An experience I will never forget.
One of the best parts though was hanging out with people from all over the world, and my fellow scholarship recipients. The stuff we did, I will never forget.
“Unity in Diversity.” It reverberated in every fabric of KubeCon + CloudNativeCon EU 2018. At the times when the world is tearing itself apart, it is nice to see so much love, cooperation, and sharing going around.
This was my first KubeCon, heck first international conference. I thoroughly enjoyed my experience, and am looking forward to many more.
I have also started on the path to much better contributions to amazing upstream projects, and am looking forward to stepping up and doing a whole lot more.
Thanks again, to the Linux Foundation, to Cloud Native Foundation, for giving me this opportunity to expand my horizons. To meet great people, to make cool, amazing friends. To learn and to grow …
CNCF has held two flagship conferences in the last six months that demonstrate the significant momentum of Kubernetes and the adoption of cloud native – the first KubeCon + CloudNativeCon North America 2017 in Austin, Texas and more recently KubeCon + CloudNativeCon Europe 2018 in Copenhagen, Denmark. These events bring together thousands of adopters and technologists from leading open source and cloud native communities across the world. Rounding out the impressive audience are leading industry analysts representing a wide variety of firms and they have a lot to say. Highlights include:
“Last week’s KubeCon + CloudNativeCon Europe conference in Denmark demonstrated the extent to which Kubernetes has become the industry standard for orchestrating and managing cloud-native applications.”
“The conference saw Kubernetes announcements from Cisco, Red Hat, and Oracle, illustrating the growing commitment of data center infrastructure vendors to open source and application performance management (APM) technologies.“
“CNCF has three classes of projects: sandbox, incubated and graduated. To move from sandbox to incubated status means a project is in production use at three or more users. Graduating means going mass market, and so far, only Kubernetes has made this leap. CNCF
is clearly on this path as well and the event was a clear demonstration of how it is moving beyond the sandbox.”
“Cloud native is a bit of a Lego market right now, with all kinds of building blocks being created by all kinds of interested groups and individual entities. This report looks at some of the ‘cloud native’ building blocks that we saw during the recent KubeCon + CloudNativeCon Europe in Copenhagen.”
“Standing in the main expo hall of KubeCon + CloudNativeCon Europe 2018in Copenhagen, the richness of the Kubernetes ecosystem is readily apparent. There are booths everywhere, addressing all the infrastructure needs for an enterprise cluster. There are meetings everywhere for the open source projects that make up the Kubernetes and Cloud Native base of technology. The keynotes are full. What was a 500-person conference in 2012 is now, 6 years later, a 4300-person conference even though it’s not in one of the hotbeds of American technology such as San Francisco or New York City.”
“At the recent KubeCon + CloudNativeCon [December 2017], the Cloud Native Computing Foundation (CNCF) and others in the community made several significant product announcements. The CNCF discussed both an expansion of its membership and new releases for projects under the CNCF. For Kubernetes, the releases of Kubernetes 1.9 showed the maturity and momentum that surrounds the project. The CNCF has established itself as the industry group for working with and promoting cloud native technologies.”
Click here to see full ESG Brief:Kubernetes and the Cloud Native Computing Foundation Drive Continued Momentum with New Community Announcements (subscription access required to see the full brief)
SAVE THE DATE: Next stop is Shanghai and then Seattle in 2018!
With KubeCon now behind us, here’s a snapshot of what was accomplished at our most jam-packed show to date. KubeCon + CloudNativeCon Europe 2018 had the largest attendance of any past CNCF event with more than 4,300 contributors, end users, vendors and developers from around the world gathering for over three days in Copenhagen, Denmark to further the education and adoption of cloud native computing, and share insights around this fast growing ecosystem.
The annual European event grew by three times last year’s event in Berlin. It was incredible to see the growth of contributors and projects vying to be part of the community. But most impressively was the growth in the End-User Community, which is now at 52, and spans across a number of industries including finance, healthcare, automotive, technology, government, insurance and more.
However, even with the massive growth in the End-User Community, it was apparent walking through the expo hall and overhearing conversations and the excitement about new projects that weren’t around just a little while ago, a sea of laptops covered in stickers and not a suit in site, that this is still very much a developer conference.
Once again, the conference co-chairs, Liz Rice and Kelsey Hightower, rocked the house in every way possible, from rockstar-level keynotes to curating an amazing list of speakers from companies like Adidas, Booking.com, Checkfront, eBay, Lyft, New York Times, Norwegian Tax Administration, Spotify and The Financial Times, covering emerging trends around serverless, hardware hacking, service mesh, machine learning and APIs of the future. They were essential in keeping this a true developer conference.
While education and collaboration are vital to the future of the cloud native ecosystem, it is imperative to CNCF’s success that everyone in the community that wants to participate feels welcome to do so regardless of gender, sexual orientation, disability, race, ethnicity, age, religion or economic status.
To increase and encourage multiple perspectives, CNCF — aided with generous donations made by AWS, Google, Helm, Heptio and VMWare — offered 62 diversity and need-based registration scholarships.
We also partnered with Google and Heptio to host a diversity luncheon & program with engaging discussions focused on how to recruit more women + underrepresented groups into tech, the role of mentors and perceived gender biases around soft and hard skills in the workplace.
The May 1st EmpowHer Evening Event, sponsored by Google Cloud, gathered attendees to discuss ways to increase inclusivity in our fast-growing ecosystem, how to get involved with different cloud native projects, and network with others in the tech industry from around the globe.
As mentioned earlier, the growth of the End-User Community has been tremendous and we can proudly say that CNCF has one of the largest end-user community membership of any open source foundation.
Members of the CNCF End-User Community have an official voice and the ability to help shape the future of the ecosystem. To recognize this rapidly growing community, we announced the first ever Top End-User Award. The recipient, Bloomberg, was chosen by its peers in recognition of major contributions to the cloud native ecosystem
KubeCon was the center of a lot of exciting news being announced from the end-user, vendor and project communities, but we’d be remiss not to highlight some of the major technical announcements from the community, which help advance all projects and companies in the cloud native ecosystem.
Red Hat Introduced the Operator Framework, an open source toolkit designed to manage Kubernetes native applications in a more effective, automated and scalable way.
To much excitement, the CNCF Serverless Working Group released version 0.1 of CloudEvents, which is an industry-wide project designed to ease serverless event and tooling interoperability. It’s expected to be proposed as a CNCF sandbox project in May.
Also, Google Cloud announced Stackdriver for Kubernetes monitoring, which will be available in production sometime in version Kubernetes 1.10. Google Cloud also announced it will be open sourcing gVisor, a sanboxed container runtime.
We’re also proud to say that our events team out did themselves with one of the all-time best All Attendee Party at the beautiful Tivoli Gardens. We ate. We drank. We continued conversations about Cloud Native, and some of us even went on rides!
“Last week at KubeCon and CloudNativeCon in Copenhagen, we saw an open source community coming together, full of vim and vigor and radiating positive energy as it recognized its growing clout in the enterprise world.
The hotel and conference center were buzzing with conversation. Every corner and hallway, every bar stool in the hotel’s open lobby bar, at breakfast in the large breakfast room, by the many coffee machines scattered throughout the venue, and even throughout the city, people chatted, debated and discussed Kubernetes and the energy was palpable.” Ron Miller, TechCrunch
“That optimism was on display this week at the latest edition of a conference called KubeCon + CloudNativeCon Europe 2018 in Copenhagen, Denmark. The event drew… 4,300 other attendees from around the world.
CNCF says it now has 20 projects — including Kubernetes — in some stage of development. Just as critically, CNCF now counts every major cloud service provider as a member. Over the past 12 months, Dan Kohn, executive director of CNCF, said he has been surprised by how quickly industry partners, many of them cloud platform rivals, have bought into the movement and joined the foundation.” Chris O’Brien, VentureBeat
“Today, the furor over Kubernetes (and containers, generally) is loud, and rightly so: Containers mark a demonstrably better way to build applications, with Kubernetes the runaway leader for making it easy to manage those containers at scale.” Matt Assay, TechRepublic
“”I do think Kubernetes in particular has driven a level of consistency across the industry that we’ve never seen,” [Jason] McGee [CTO, IBM Cloud Platform]. “You literally can go to any public cloud now and all the on-premises environments and have a pretty high success rate at taking a workload and running it in another place.
While the workload movement that Kubernetes enables is not a live migration, that’s not necessarily what every organization needs or wants. What Kubernetes and the container ecosystem provide is a standard application packaging approach, though other things are still needed to fully enable portability, according to McGee.” Sean Kerner, eWeek
completely humbled with how much the @CloudNativeFdn has grown, hope you enjoyed #kubecon#cloudnativecon and please feel free to personally send me feedback on how we can make things better for you in the future
CNCF offers diversity scholarships to developers and students to attend KubeCon + CloudNativeCon events. In this post, scholarship recipient Kathryn Hamon, Principal Business Management at AT&T, shares her experience attending sessions and meeting the community. Anyone interested in applying for the CNCF diversity scholarship to attend KubeCon + CloudNativeCon North America 2018 in Seattle, WA December 11-13, can submit an application here. Applications are due October 5th.
By Kathryn Hamon, Principal Business Management at AT&T
To start off with, I need to say all statements and opinions are my own, I do not represent any individual or company except for myself. That said, I work for a large communications and software company, possibly the largest in the world and I have cancer. I am the mother of a six year old future rocket scientist (watch out, NASA!) and my husband makes bagpipes, and I am super proud of both of them. I am exhausted and so sick of effing cancer effing up everything (can I say that? Who cares, I’m gonna). Even with social media, and a decent social life tbh, it’s hard to connect to other people who can completely identify. A lot of times I feel like the only one of my kind. Despite this, I know you are out there my fellow exhausted moms and driven, career-minded cancer patients who are trying to move with the industry. I dedicate this to you.
What I hope to communicate to you through this blog is that despite the monolithic need for pretty much every human being in every industry and career to be constantly building our technical expertise and knowledge lexicon, there are gaps that seem insurmountable. This is partly because we don’t even know what they are. I’m going to learn how to fill those gaps for myself, and I’m going to share how to do it.
Prior to KubeCon, here are the thoughts that were going through my head:
For more than 15 years, I’ve studied and lived network mapping and building, both physical and virtual. I can personally describe, and even build with my hands in some cases, how internet signal gets to and from the central offices, or remote terminals, or satellite, or cell site, to your homes and businesses. Once there, I know how the heck to use it, I know how to network the devices together, I know how to troubleshoot when it doesn’t work. I can get on those devices, I can build files and connect to data and analyze that data in a multitude of ways using at least four completely different softwares and at least three programming languages. I can build interactive tools that will save you hours of time. I feel like I know a lot; I describe myself as “techie.” Paradoxically, I have known about Kubernetes for about 1.5 years, and have yet to link it to anything tangible in my technical lexicon.
What am I missing to make the leap to comprehending DevOps, container strategy, cloud native, ONAP? After all, this leap is what my workplace has been communicating to its employees as the best chance to remain competitively employed.
So I went to KubeCon (thank you, Diversity Committee and sponsors!) I left with eleven and a half pages of notes which will provide me with at least a year of things to study and learn. Eben Freeman’s Queueing Theory PDF and Aaron Schlesinger’s HPA (Horizontal Pod Autoscale, I had to look it up) recommendation will likely be six months of study alone, minimum. I also left with a sense of newness and freshness and community, lots of Twitter follows and hopefully a few friends.
One of the biggest challenges I had at the 2017 K8sCon was that everything seems to be so far reaching, all encompassing. All of the seminars seemed to be offering solutions for steps A-Z. Like “The Cloud” is here, there, and everywhere but (virtually) nowhere all at once. How is a lay person supposed to understand where to start? Finally, listening to the Google team speak, I got that at least one thing we are talking about is the masses of data the come across giant websites, accessing, purchasing, linking, researching, bots, people, governments. I asked a fellow conferee and she confirmed that is one aspect of it. My reaction: “You mean there’s more?” But at least I had a place to start. And it’s also astounding how just a search engine and, more so, those who run it, really reflect so much about our world in a way that was never possible before.
This meta is growing exponentially in a way that many have predicted but only an elite and seriously impressive few are actually putting in motion. I admire them greatly. My goals with learning more about ONAP and K8s are not to become a better software engineer, developer, or open networking community contributor. My goal is understanding, and applying that understanding towards strategy. I hope one day to lead a company with a team of specialists, doing work that matters, innovating and impacting industry.
Bloomberg is a big believer in open source. Not only does Bloomberg use open source software widely, but its engineers contribute widely. In the case of Kubernetes, says Drew Rapenchuk, team lead for Bloomberg’s Kubernetes-as-a-Service offering (which is meant to provide a unified method for provisioning Kubernetes infrastructure internally), it was a particularly easy sell. “As engineers, this is something that we want to be involved in,” he says. “The community around Kubernetes is so awesome. There are all these people from different backgrounds, and interacting with each other makes us better engineers.”
But the reasons for contributing to open source projects go well beyond personal and professional development, says Steven Bower, Bloomberg’s Data & Analytics Infrastructure Engineering lead. “If you’re going to use a piece of open source software, you have to be willing to contribute back to the project,” says Bower. “Because when it falls over in the middle of the night, there is no one to call. We have to be able to solve that problem.”
Bower says Bloomberg started investigating Kubernetes about two-and-a-half years ago, after building a tool to manage the deployment of about 6,000 instances of Apache Solr, an open source enterprise search platform, across roughly 1,000 servers. Bloomberg also started writing containerization software and orchestration software to support it. “Then we started looking at, why do we have to build all this code ourselves? Why are we spending all this time?” says Bower. And, importantly: “Is there something better?”
At the same time, Bloomberg engineers began poking around Kubernetes, trying to understand what the gaps were, how stable it was, how it could be monitored, and how it would fit into the existing Bloomberg tech stack. About a year ago, Bloomberg starting running its first production software on top of Kubernetes.
Bloomberg recently completed building a Data Science Platform that uses Kubernetes. “The Data Science Platform has given us a huge amount of knowledge about how we can monitor and run Kubernetes,” says Bower. “Now we can begin running more complex services and sharing that experience and insight with other teams of engineers.”
“Kubernetes, as a project, has grown really rapidly,” says Bower. Crucial to Bloomberg’s continued involvement is that Kubernetes has an unusually strong governance model. There is a detailed schedule of releases, and a list of features that are to be included in each release. “The releases and the features get delivered, and if there are changes, there’s clear communication around it,” says Bower. Late last year, Bloomberg officially joined the CNCF End User Community.
As different teams at Bloomberg got deeper into Kubernetes, engineers across the organization started contributing bug fixes and patches to the core project and community tools like kubeadm, kube-ops-view and Kubo. They also contributed towards a way to manually create a Job instance from a CronJob that will run right away, rather than periodically.
And they are regular presenters of use cases, implementations, best practices and lessons learned at technical conferences and meetups, including KubeCon + CloudNativeCon, Strata and the San Francisco Kubernetes meetup.
“A good chunk of the work we’re doing can be contributed to the open source community around Kubernetes,” says Rapenchuk. “We’re addressing our own challenges, but making sure we’re solving them in a way that we can contribute so that everyone else in the community wins too.”
Every day, China’s largest retailer and leading global technology company, JD.com, is required to process a staggering amount of data to ensure that its nearly 300 million customers have the best, most efficient and convenient experience possible. In order to achieve this, JD leverages Kubernetes®, one of the Cloud Native Computing Foundation’s (CNCF) flagship projects. Now the company aims to not only further develop these technologies, but also to work hand-in-hand with the CNCF to chart the direction of the overall industry. This is why JD has joined the CNCF.
JD runs one of the largest Kubernetes clusters in production in the world. A few years ago, the company rolled out containerized infrastructure. When clusters grew from 5,000 to 150,000 containers, JD shifted from OpenStack to Kubernetes. The move, named JDOS 2.0, involved separating JD’s application and infrastructure layers by deploying a DevOps stack on Kubernetes that includes GitLab, Jenkins, Logstash, Harbor, Elasticsearch and Prometheus. Apart from leveraging Kubernetes, JD is currently working on its own open source projects and plans to contribute its immense expertise and resources to empower others in the community.
As part of JD’s commitment to the CNCF, Haifeng Liu, Chief Architect and Vice President at JD.com, has joined the CNCF’s Governing Board alongside the likes of companies including Amazon Web Services, SAP, Microsoft Azure and Samsung SDS, among others. The group is responsible for providing recommendations about the future direction of the CNCF. Liu is also a member of the CNCF’s End User Community, which meets monthly to discuss key challenges, emerging use cases and new opportunities for cloud native technologies.
Liu said, “We’re in this to provide our customers with an unrivaled experience. In order to do that, we rely heavily on innovative technology in areas like container-based infrastructure, AI and big data. Joining the CNCF gives us a huge leg up in charting the future of cloud native projects that are critical to our business and can also enable us to empower our partners and realize our Retail as a Service vision.”
“It’s great to see JD.com invest and engage in collaborative development by joining CNCF at the highest level possible as our first end user Platinum member,” said Dan Kohn, Executive Director of the Cloud Native Computing Foundation. “Chinese companies like JD.com are at the forefront of retail innovation in part because of their success with cloud native computing. Containers and technologies like Kubernetes allow JD to optimize purchase data, develop an agile supply chain, deliver new products quickly and scale when necessary.”
you tell us about yourself and what Scalefastr does?
My name is Kevin Burton and I’m the CEO of Scalefastr. My background is in distributed systems and I’ve previously ran Datastreamer, a company that built a petabyte scale distributed social media crawler and search engine.
At Datastreamer we ran into scalability issues regarding our infrastructure and built out a high performance cluster based on Debian, Elasticsearch, Cassandra, and Kubernetes.
We found that many of our customers were also struggling with their infrastructure and I was amazed at how much they were paying for hosting large amounts of content on AWS and Google Cloud.
We continually evaluated what it costs to run in the cloud and for us our hosting costs would have been about 5-10x what we currently pay.
We made the decision to launch a new cloud platform based on Open Source and cloud native technologies like Kubernetes, Prometheus, Elasticsearch, Cassandra, Grafana, Etcd, etc.
We’re currently hosting a few customers in the petabyte scale and are soft launching our new platform this month.
What was your pre-Prometheus monitoring experience?
At Datastreamer we found that metrics were key to our ability to iterate quickly. The observability into our platform became something we embraced and we integrated tools like Dropwizard Metrics to make it easy to develop analytics for our platform.
We built a platform based on KairosDB, Grafana, and our own (simple) visualization engine which worked out really well for quite a long time.
They key problem we saw with KairosDB was the rate of adoption and customer demand for Prometheus.
Additionally, what’s nice about Prometheus is the support for exporters implemented by either the projects themselves or the community.
With KairosDB we would often struggle to build out our own exporters. The chance that an exporter for KairosDB already existing was rather low compared to Prometheus.
For example, there is CollectD support for KairosDB but it’s not supported very well in Debian and there are practical bugs with CollectD that prevent it from working reliability in production.
With Prometheus you can get up and running pretty quickly (the system is rather easy to install), and the chance that you have an exporter ready for your platform is pretty high.
Additionally, we’re expecting customer applications to start standardizing on Prometheus metrics once there are hosted platforms like Scalefastr which integrate it as a standardized and supported product.
Having visibility into your application performance is critical and the high scalability of Prometheus is necessary to make that happen.
Why did you decide to look at Prometheus?
We were initially curious how other people were monitoring their Kubernetes and container applications.
One of the main challenges of containers is the fact that they can come and go quickly leaving behind both log and metric data that needs to be analyzed.
It became clear that we should investigate Prometheus as our analytics backend once we saw that people were successfully using Prometheus in production along with a container-first architecture – as well as the support for exporters and dashboards.
How did you transition?
The transition was somewhat painless for us since Scalefastr is a greenfield environment.
The architecture for the most part is new with very few limiting factors.
Our main goal is to deploy on bare metal but build cloud features on top of existing and standardized hardware.
The idea is to have all analytics in our cluster backed by Prometheus.
We provide customers with their own “management” infrastructure which includes Prometheus, Grafana, Elasticsearch, and Kibana as well as a Kubernetes control plane. We orchestrate this system with Ansible which handles initial machine setup (ssh, core Debian packages, etc.) and baseline configuration.
We then deploy Prometheus, all the required exporters for the customer configuration, and additionally dashboards for Grafana.
One thing we found to be somewhat problematic is that a few dashboards on Grafana.com were written for Prometheus 1.x and did not port cleanly to 2.x. It turns out that there are only a few functions not present in the 2.x series and many of them just need a small tweak here and there. Additionally, some of the dashboards were written for an earlier version of Grafana.
We’re hoping this makes it easy for other people to migrate to Prometheus.
One thing we want to improve is to automatically sync it with our Grafana backend but also to upload these dashboards to Grafana.com.
We also published our Prometheus configuration so that the labels work correctly with our Grafana templates. This allows you to have a pull down menu to select more specific metrics like cluster name, instance name, etc.
What improvements have you seen since switching?
The ease of deployment, high performance, and standardized exporters made it easy for us to switch. Additionally, the fact that the backend is fairly easy to configure (basically, just the daemon itself) and there aren’t many moving parts made it an easy decision.
What do you think the future holds for Scalefastr and Prometheus?
Right now we’re deploying Elasticsearch and Cassandra directly on bare metal. We’re working to run these in containers directly on top of Kubernetes and working toward using the Container Storage Interface (CSI) to make this possible.
Before we can do this we need to get Prometheus service discovery working and this is something we haven’t played with yet. Currently we deploy and configure Prometheus via Ansible but clearly this won’t scale (or even work) with Kubernetes since containers can come and go as our workload changes.
We’re also working on improving the standard dashboards and alerting. One of the features we would like to add (maybe as a container) is support for alerting based on holts winters forecasting.
This would essentially allow us to predict severe performance issues before they happen. Rather than waiting for something to fail (like running out of disk space) until we take action to correct it.
To a certain extent Kubernetes helps with this issue since we can just add nodes to the cluster based on a watermark. Once resource utilization is too high we can just auto-scale.
We’re very excited about the future of Prometheus especially now that we’re moving forward on the 2.x series and the fact that CNCF collaboration seems to be moving forward nicely.