All Posts By

cncf

My KubeCon & CloudNativeCon Europe Experience 2019

By | Blog

Guest post by Ines Cheikhrouhou, DevOps and Cloud consultant, Agyla originally published on Medium

Hi, My name is Ines and I’m one of the lucky people who was sent an invite as a diversity scholar to Kubecon + CloudNativeCon in Barcelona.

First of all, I want to thank CNCF for this great and life-changing opportunity for me.

I was very happy to receive the email and I was very excited to meet the kind of people who love staying in front of computers. It motivates me a lot to know more technical aspects of Kubernetes, mainly the core project. However, little did I know that this event was much more than that. It gave me all of the motivation and also all of the technical information and knowledge that I wanted to have.

My first day at KubeCon was the AWS container day co-located event which was a perfect choice for me since I work with AWS and I wanted to dig deeper into it and other cloud-native projects.

Throughout the day, I learned a lot about what’s new in AWS with relation to Kubernetes or simply other cloud-native tools.

One of the best discoveries for me was the APP mesh which is based on Envoy Proxy, as well as some advanced services for observability such as cloudwatch container insights.

In addition to that, I learned about the huge benefit of doing machine learning development workflow on Kubernetes, as well as the advantages of kubeflow.
And mostly, the famous eksctl CLI that helps provision your cluster in an easy way.

And here are my favorite pictures for the first day.

 

 

The end of the first day was very successful and it made me feel like I belonged with these smart and motivating people. I also got the chance to tour the beautiful Barcelona city (Thanks CNCF for the great choice).

The second day was in a different place which is the main event place at the Fira Gran Via, which was a HUGE building. It was very organized and you would find the planning of the day in each corner.

The day started with a perfect keynote where we learned about almost all of the CNCF projects with details from people who work daily on these projects and I got to listen to some of the best speakers such as Dan Kohn, Bryan Liles and especially Cheryl Hung, a woman who inspired me a lot. Seeing all of these women who participated as speakers made me want to work hard and be there on stage one day.

And last but not least, the famous presentation from Lucas and Nikhita that marked a starting point in my life which is a contribution.

I always thought it was just some smart people coding in a shared GitHub account on a project that I would probably not understand. But it’s more than that, it’s not even about coding, it’s the family that it creates, a family that is composed of people who encourage you, help you and inspire you to show your best. It’s about sharing and improving.

And here is the famous picture that presented the CNCF projects.

After the keynote, there were so many sessions that I wouldn’t want to miss and there was a huge place for sponsor showcase which was the perfect place for you if you had a question in mind or request of a demo or a sticker and a t-shirt. I got to meet so many people and I got to see a lot of demos and know about new CNCF projects that I didn’t hear about before.

And It was also such a pleasure meeting talented people such as Joe Beda, Ali Saad, Arun Gupta, Janet Kuo and all of the speakers that taught us so many things in a short period of time.

I also participated in the Networking and Mentoring session the next day which was the best part of the whole event for me. I was lucky to share a table with 3 of the greatest people I met in the event, Nikhita, Hippie Hacker and carolynvs. They introduced me to the world of contribution, the steps to follow, they taught me where to start and I even made my first PR that day.

These kinds of people and all of the community’s love of Open Source is what makes it fun and interesting to be a part of. And I hope everyone like me whom at a certain point were scared or doubtful, knows that it’s really a safe place to be around.

In the end, the event was a SUCCESS for me and I enjoyed the 5th-anniversary party of Kubernetes and the big Poble Espanyol party with free food and drinks.

Overall, Kubecon was a dream experience for me. Before this conference, I wouldn’t have been able to talk about Kubernetes or other Cloud Native Projects with confidence. But after this event, I gained a lot of knowledge and met so many people who offered me help. The whole experience offered me great opportunities to improve my personal and professional development. I’m excited to share this experience with my friends and I’m inspired to start being an active member of the community.

With this, I look forward to KubeCon + CloudNativeCon Europe 2020. Thank you KubeCon + CloudNativeCon, Europe 2019 and thank you CNCF for the amazing opportunity.

Here is a quick summary about what you’ve probably missed during this event, and let’s start with OpenTelemetry that is the next major version of the openTracing and openCensus projects, CNAB that allows one to package up multiple formats and their toolchains into a single artifact, different Kuberentes operators, how bezel helps with streamlining Kubernetes application CI/CD, containerd, crio-o, Autoscaling multi-cluster observability using Prometheus, and linkerd, building docker on Kubernetes using build kit and the great cloud-native storage orchestrator Rook.

In addition to Jaeger, its agents and its scaling, enhancing security made by Envoy SDS, the new fluentbit after fluentd for extending your logging pipeline with Go, the amazing Grafana Loki for logs and its integration with existing observability tools, multiple types of load balancing such as gRPC load balancing and its benefits with service mesh and Kubert VMs that provides networking functions to Kubernetes objects.

And lastly, the superstar Helm, Prometheus and custom metrics of k8s, Calico and SPIRE and their integration with Envoy, the new trending GitOps strategies and the serverless future of cloud computing.

Apple Joins Cloud Native Computing Foundation as Platinum End User Member

By | Blog

The Cloud Native Computing Foundation (CNCF), which sustains and integrates open source technologies like Kubernetes® and Prometheus™, today announced that Apple has joined the CNCF as a Platinum End User Member.

Apple has completely revolutionized personal and enterprise technology, and has long been a pioneer in cloud native computing and one of the earlier adopters of container technology. Apple has also contributed to several CNCF projects, including Kubernetes, gRPC, Prometheus, Envoy Proxy, Vitess and hosted the FoundationDB Summit at KubeCon + CloudNativeCon last year.

“Having a company with the experience and scale of Apple as an end user member is a huge testament to the vitality of cloud native computing for the future of infrastructure and application development,” said Chris Aniszczyk, CTO of the Cloud Native Computing Foundation. “We’re thrilled to have the support of Apple, and look forward to the future contributions to the broader cloud native project community.”

As part of Apple’s Platinum membership, Tom Doron, Senior Engineering Manager at Apple has joined CNCF’s Governing Board.

Apple will join 87 other end user companies including Adidas, Akatsuki, Amadeus, Atlassian, AuditBoard, Bloomberg, Box, Cambia Health Solutions, Capital One, Concur, Cookpad, Cruise,  Curve, DENSO Corporation, DiDi, Die Mobiliar, DoorDash, eBay, Form3, GE Transportation, GitHub, Globo, Goldman Sachs, Granular, i3 Systems, Indeed,Intuit,JD.com, JP Morgan, Kuelap, Mastercard, Mathworks, Mattermost, Morgan Stanley, MUFG Union Bank, NAIC, Nasdaq, NCSOFT, New York Times, Nielsen, NIPR, Pinterest, PostFinance, Pusher, Reddit, Ricardo.ch, Salesforce, Shopify, Showmax, SimpleNexus, Spotify, Spredfast, Squarespace, State Farm, State Street, Steelhouse, Stix Utvikling AS, Testfire Labs, Textkernel, thredUP, TicketMaster, Tradeshift, Twitter, Two Sigma,University of Michigan – ARC, Upsider, Walmart, Werkspot, WeWork, WikiMedia, WooRank, Workday, WPEngine, Yahoo Japan Corporation, Zalando SE, and Zendesk in CNCF’s End User Community. This group meets monthly and advises the CNCF Governing Board and Technical Oversight Committee on key challenges, emerging use cases and areas of opportunity and new growth for cloud native technologies.

Additional Resources

Square: How Vitess Enables ‘Near Unlimited Scale’ for Cash App

By | Blog

Four years ago, Square branched out into peer-to-peer transactions via its Cash App. After doing so, users started increasing by the minute and they needed to come up with a long term solution for scalability.  Vitess was the answer to the scalability issue. With Vitess, Cash App didn’t have to completely change how developers built applications and were able to change only 5% of their system vs. 95% to respond to increased customer demand.  Additionally, Cash App developers can do multiple shard splits per week with less than a second of downtime. Read the full case study here.

 

Reflections on the Fifth Anniversary of Kubernetes

By | Blog

Guest post from the Kubernetes Project

Five years ago, Kubernetes was released into the world. Like all newborns, it was small, limited in functionality, and had only a few people involved in its creation. Unlike most newborns, it also involved a great deal of code written in Bash. Today, at the five year mark, Kubernetes is full grown, and while a human would be just entering kindergarten, Kubernetes is at the core of production workloads from startups to global financial institutions.

They say that success has a thousand parents and failure is an orphan, but in the case of Kubernetes the truth is that its success is due to its thousands (and thousands) of parents. Kubernetes came from humble beginnings, with just a handful of developers and in record time grew into its current state with literally thousands of contributors – and even more people involved in meetups, docs, education, release management, and supporting the broader community. At many points in the project, when it seemed that it might be moving too fast or becoming too big, the community has responded and stepped up with new ways of organizing and new ways of supporting the project so that it could have continued success. It is an amazing achievement to see a project reach this scale and continue to operate successfully, and it is a tribute to each and every member of our amazing community that we’ve been able to do this while maintaining an open, neutral and respectful community.

Five years in, it’s worth reflecting on the things that Kubernetes has achieved. It is one of the largest, if not the single largest open source project on the planet. It has managed to sustain a fast pace of development across a team of thousands of distributed engineers working in a myriad of different companies. It has merged tens of thousands of commits while sustaining a regular release cadence of high-quality software that has become mission-critical for countless organizations and companies. This would be no small achievement within a single company, but to do this while being driven by dozens of different companies and thousands of individuals (many of whom have other jobs or even school!) is truly amazing. It is a credit to the selflessness of all of the folks in the community who chop wood and carry water every single day to ensure that our tests are green (ish), our releases get patched, our security is maintained, and our community conducts itself within the bounds of our code of conduct. To all of the people who do this often tedious, and sometimes emotionally draining work, you deserve our deepest thanks. We could never have gotten here without you.

Of course, the story of Kubernetes isn’t just a story of community, it is also a story of technology. It is breath-taking to see the speed with which the ideas of cloud-native development have shaped the narrative of how reliable and scalable applications are built. Kubernetes has become a catalyst for the digital transformation of organizations toward cloud-native technologies and techniques. It has become the rallying point and supporting platform for the development of an entire ecosystem of projects and products that add powerful cloud-native capabilities for developers and operators. By providing a ubiquitous and extensible control-plane for application development, Kubernetes has successfully uplifted a whole class of higher-level abstractions and tools.

One of the most important facets of the Kubernetes project was knowing where it should stop. This has been a core tenet of the project since the beginning and though the surface area of Kubernetes continues to grow, it has an asymptotic limit. Because of this, there is a flourishing ecosystem on top of and alongside the core APIs. From package managers to automated operators, from workflow systems to AI and deep learning, the Kubernetes API has become the substrate on which a vibrant cloud-native biome is growing.

As Kubernetes turns five, we naturally look to the future and contemplate how we can ensure that it continues to grow and flourish. In the celebration of everything that has been achieved, it must also be noted that there is always room for improvement. Though our community is broad and amazing, ensuring a diverse and inclusive community is a journey, not a destination, and requires constant attention and energy. Likewise, despite the promise of cloud-native technologies, it is still too hard to build reliable, scalable services. As Kubernetes looks to its future, these are core areas where investment must occur to ensure continued success. It’s been an amazing five years, and with your help the next five will be even more amazing. Thank you!

A Look Back At KubeCon + CloudNativeCon Barcelona 2019

By | Blog

 

Hot off an amazing three days in Barcelona, here is a snapshot into some of the key highlights and news from KubeCon + CloudNativeCon Europe 2019! This year we welcomed more than 7,700 attendees from around the world to hear compelling talks from CNCF project maintainers, end users and community members.

The annual European event grew by more than 3,000 attendees than last year in Copenhagen. At the conference, CNCF announced its ever-growing ecosystem has hit over 400 member companies, of which there are now more than 88 end user members. We also learned that Kubernetes has more than 2.66 million posts from 26,214 contributors.

This year we welcomed Bryan Liles as a KubeCon + CloudNativeCon co-chair! He took the stage to announce all the great project news that has come out in the last couple months.

During the opening keynotes we also heard from CNCF Executive Director, Dan Kohn who spoke about the key factors that contributed to the massive growth of the Kubernetes ecosystem, and CNCF Director of Ecosystem, Cheryl Hung who shared CNCF’s growth and plans to continue growing a positive community. Lucas Käldström, CNCF Ambassador, Independent & Nikhita Raghunath, Software Engineer, Loodse shared insights on the what, why and how of contributing to Kubernetes.

Kubernetes Boothday Party!

While we celebrated the cloud native community, we also got to celebrate Kubernetes’ fifth birthday with a “Boothday Party” and donut wall!  

Continuing to Embrace Diversity in the Ecosystem

At KubeCon + CloudNativeCon EU, CNCF’s diversity program offered scholarships to 56 recipients, from traditionally underrepresented and/or marginalized groups, to attend the conference! The $100K investment for Barcelona was donated by CNCF, Aspen Mesh, Google Cloud, Red Hat, Twistlock and VMware.

CNCF has offered more than 300 diversity scholarships to attend KubeCons since November 2016.

We also had a wonderful time at the Diversity lunch and EmpowerUs events!

Take Good Care: Open Sourcing Mental Illness

This year at KubeCon + CloudNativeCon EU, we made sure that self care and mental wellness was top of mind for everyone. As a result, we got to hear inspiring talks throughout the conference on these topics, plus, we had a booth 100% dedicated to relaxing and mental health. We felt so much community support!

All Attendee Party at Poble Espanyol!

Our events team organized a fantastic party at Poble Espanyol, celebrating the many achievements of the cloud native ecosystem in the beautiful Spanish courtyard!

Keynote and Session Highlights

All presentations and videos are available to watch. Here is how to find all the great content from the show:

  • Keynotes, sessions and lightning talks can be found on the CNCF YouTube
  • Photos can be found on the CNCF Flickr
  • Presentations can be found on the Conference Event Schedule, click on the session and scroll to the bottom of the page to see the PDF of the presentation for download

“From the people Computer Weekly spoke to at Kubecon-CloudNativeCon, there is a sense that Kubernetes is breaking out of the open source developer space into the enterprise.” Cliff Saran, ComputerWeekly

“Five years after Google released the toolkit for managing workloads to the open source community, Kubernetes became the celebrated boy. Nothing seems to stop its advance, especially because the developers have embraced this system.” Alfred Montie, Computable.nl

“The Kubernetes container-orchestration system is one platform that is both surviving and thriving.” Nick Marinoff, SiliconANGLE

“It’s apparent that whether an application lives on the Google Cloud Platform or in an on-premises data center, it can be done with containers.” Kristen Nicole, SiliconANGLE

“As stated by Dan Kohn, Kubernetes has emerged on the shoulders of giants: on Linux, the Internet and various cluster manager implementations of cloud-native businesses from Spotify to Facebook to Google. The exciting question for the conference is which innovations will come on the shoulders of the giant Kubernetes in the next few days.” Josef Adersberger, Alex Krause, Heise Online

“The obvious conclusion: If you’re interested in enterprise IT infrastructure, Kubernetes should be your technology of choice, and KubeCon is the place to be.” Jason Bloomberg, SiliconANGLE

It’s a Wrap!

Save the Dates!

Register now for KubeCon + CloudNativeCon + Open Source Summit China 2019, scheduled for June 24-26, 2019 at the Shanghai Expo Centre, Shanghai, China

Register now for KubeCon + CloudNativeCon North America 2019, scheduled for November 18-21, 2019 at the San Diego Convention Center, San Diego, California. The CFP closes July 12.

Save the date for KubeCon + CloudNativeCon Europe 2020, scheduled for March 30-April 2, 2020 in Amsterdam, The Netherlands

Observability should not slow you down

By | Blog

Originally published on Medium by Travis Jeppson, Sr. Director of Engineering, Nav Inc

In any application, the lack of observability is the same as riding a bike with a blindfold over your eyes. The only inevitable outcome is crashing, and crashing always comes with a cost. This cost tends to be the only focus we have when we look at observability, but this isn’t the only cost. The other cost of observability isn’t usually addressed until it becomes more painful than the cost of crashing —the cost of maintenance and adaptability.

I’ve listened to, and watched, many conference talks about this subject; and had my fair share of conversations with vendors as well. Maintenance and adaptability aren’t generally mentioned. I’ve only had these topics come up when I’m talking to other companies about their adopted platform, how they were actually able to integrate observability into real-life situations, and from my own experiences doing the same. The reason these topics come up after some practical application is that we’ve all hit the proverbial wall.

We’ve all run into problems, or incompatibilities, or vendor lock-in that feels almost impossible to get rid of. Our observability begins to dwindle, the blindfold starts falling down over our eyes, and we’re again heading to an inevitable fate. What can be done? Revisit the entire scenario? Wait for a major crash and create an ROI statement to show we have to re-invest in major parts of our applications? This can’t possibly be the only way to deal with this problem. This is an anti-pattern to the way we build software. Observability is supposed to empower speed and agility, not hold it back.

There is another way, and it starts by determining the key elements on which you won’t make concessions. During the last iteration of trying to get this right at Nav, we had a lot of discussions around our previous attempts. The first attempt was a solution that we thought initially had unlimited integrations; it turns out it didn’t have the one we needed, Kubernetes. We also couldn’t produce custom metrics from our applications, so that solution had to go. We weren’t about to wait for them to tell us an integration was ready, we were ready to move. We decided to go with a solution that was end-to-end customizable, we could spend time developing our telemetry data, and how to interpret it. This, unfortunately, forced us into a maintenance nightmare. On the third iteration, however, we decided to settle somewhere in the middle. We sat down and defined our “no compromise” priorities, and started finding solutions that fit. Here’s how we saw the priorities for Nav.

1. Customization! We needed adaptability, no waiting for integrations

First and foremost the solution needed to allow for custom metrics, and handle them like a first-class citizen. This needed to be true for our infrastructure metrics as well as anything coming from our applications. Adaptability was key in our decision: If the solution we chose was adaptable, then we should be free to adjust any component of our infrastructure without having to check if our observability would be affected.

2. No vendor-specific code in our applications, not even libraries

This may seem a little harsh at first, but the fact of the matter is that we didn’t want to have a dependency on a vendor. We use a wide variety of languages at Nav —Ruby, Elixir, Go, Python, Javascript, Java, the list goes on. It was almost impossible to find a vendor solution that would work with all of those languages. We decided the language needed to be agnostic, which means we couldn’t have any vendor code or libraries in our applications. The other side of this is that we didn’t want to be locked to the solution, since we had previously run into issues with that problem.

3. HELP! The maintenance cannot be overwhelming

This meant that at some point we would probably need a vendor to help us out. We didn’t want a ridiculous uptime for our observability platform to be our concern, we wanted to worry about the uptime of our application instead. We also didn’t want to worry about the infrastructure of the observability platform, we wanted to worry about our own. Catch my drift? We also wanted some guidance about what to pay attention to. We wanted a simple way to build dashboards, and the ability to allow pretty much every engineer to build their own dashboards around their own metrics.

Now the Rest: Our Second Tier of Priorities

Now we get into the “like to have” priorities. The following were more of a wish list, the top three were dealbreakers for the solution we came up with. Fortunately, as will be illustrated later, we didn’t need to compromise on any of our priorities.

4. Alerting needed to be easy to do, and integrate with our on-call solution

With our end-to-end customized solution (attempt #2 in observability) alerting was ridiculously tedious. It was a JSON document that had so many defining parts that we never really had any good alerts setup. We also caused a lot of on-call burnout due to large amounts of false positives. We didn’t want to repeat this.

5. We didn’t want to pay the same price for our non-production environments as we do for production

It is a giant pet-peeve of mine that it is required of anyone to pay the same price for observability, just because the size of the environments is the same. Why must this be? I don’t actually care nearly as much if a development environment goes down for 5 minutes; but I definitely care if production is down for 5 minutes.

The Final Decision: Nav’s Tri-Product Solution

With these priorities in hand, we set out to create a solution that worked. To cut a long story short, there didn’t end up being the perfect solution, there wasn’t a solution that could give us the top 3 priorities … on their own. It turns out we needed multiple pieces to work seamlessly together.

Prometheus

Prometheus is an open source metric aggregation service. The fantastic thing about Prometheus is that it is built around a standard, which they also created. This standard is called the exposition format. You can provide a text-based endpoint and Prometheus will come by and “scrape” the data off of this endpoint and feed it into a time series database. This … is … simply … amazing! Our developers were able to write this endpoint in their own code bases, and publish any kind of custom metric they desired.

StatsD

StatsD was a solution originally written by Etsy, StatsD provided a way for us to push metrics on our software that wasn’t associated with a web server, such as short-lived jobs, or event-driven computations.

Between StatsD and Prometheus, we were able to publish custom metrics from virtually anywhere. The other great thing, is with both of these solutions being open source, there was already a thriving community building out assistive components to these two libraries.

The final piece of the puzzle for us was where the vendor came into play. With our priorities set, we found a vendor that did seamlessly integrate with Prometheus metrics, they would even scrape the metrics for us, so we didn’t even need to run Prometheus, just use their standards. They also ingested our StatsD metrics without a hitch.

SignalFx

SignalFx was the vendor we ended up selecting, this is what ended up working for us, and our priorities. The key component with the vendor selection is that the solution fulfills your needs from a managed, and ease-of-use view point. That being said, I’ll illustrate how SignalFx fulfilled this for us.

The tailing part of our third priority is we wanted some guidance on what to pay attention to, SignalFx had some very useful dashboards out of the gate that used our Prometheus metrics to pinpoint some of our key infrastructure components, like Kubernetes and AWS.

They also have a very robust alerting system which was as simple as identifying the “signal” we wanted to pay attention to, and adding a particular constraint to it. These constraints could be anything between a static threshold, to outliers, to historical anomalies. This was significantly simpler than our second attempt, and this was built around custom metrics! Win, Win!

Finally, SignalFx charges per metric you send them, the great thing about this is that our non-prod environments are pretty quiet, we dialed down their resolution to a minute or two, so the metrics that are constantly being generated, like CPU, or memory, didn’t cost an arm and a leg. This fulfilled our final priority, and allowed us to save a significant amount of money over other vendor solutions.

The takeaway from all of this is that the observability platform we use, if built around standardized systems, doesn’t have to be painful. In fact it can be just the opposite. We have been able to accelerate our development and we have never had surprises due to the maintainability and adaptability of our observability platform.

For more on Nav’s cloud native journey, check out the case study and video