Prometheus Graduates Within CNCF

By | Blog

Originally posted August 9, 2018 by Richard Hartmann on

We are happy to announce that as of today, Prometheus graduates within the CNCF.

Prometheus is the second project ever to make it to this tier. By graduating Prometheus, CNCF shows that it’s confident in our code and feature velocity, our maturity and stability, and our governance and community processes. This also acts as an external verification of quality for anyone in internal discussions around choice of monitoring tool.

Since reaching incubation level, a lot of things happened; some of which stand out:

  • We completely rewrote our storage back-end to support high churn in services
  • We had a large push towards stability, especially with 2.3.2
  • We started a documentation push with a special focus on making Prometheus adoption and joining the community easier

Especially the last point is important as we currently enter our fourth phase of adoption. These phases were adoption by

  1. Monitoring-centric users actively looking for the very best in monitoring
  2. Hyperscale users facing a monitoring landscape which couldn’t keep up with their scale
  3. Companies from small to Fortune 50 redoing their monitoring infrastructure
  4. Users lacking funding and/or resources to focus on monitoring, but hearing about the benefits of Prometheus from various places

Looking into the future, we anticipate even wider adoption and remain committed to handling tomorrow’s scale, today.

Pinning its Past, Present, and Future on Cloud Native

By | Blog

After eight years in existence, Pinterest has grown into 1000 microservices, multiple layers of infrastructure, and a diverse set-up of tools and platforms. In order to manage all of this they needed a compute platform that enabled the fastest path from idea to production with simplicity for their engineers.

Pinterest turned to Kubernetes on Docker containers. In this case study, they describe their journey, using Jenkins clusters, and how the team was able to build on-demand scaling and new failover policies in addition to simplifying the overall deployment and management.

Running on AWS since 2010, Pinterest sees themselves as a ‘cloud native pioneer’ and is eager to share their ongoing cloud native journey and contribute their learnings back to the community.

See a summary of Pinterest, with metrics, here.

Watch the video of Pinterest’s journey from VM’s to Containers presented at KubeCon + CloudNativeCon North America.

CNCF to Host OpenMetrics in the Sandbox

By | Blog

Today, the Cloud Native Computing Foundation (CNCF) accepted OpenMetrics, an open source specification for metrics exposition, into the CNCF Sandbox, a home for early stage and evolving cloud native projects.

OpenMetrics brings together the maturity and adoption of Prometheus, Google’s background in working with stats at extreme scale, along with the experience and needs of a variety of projects, vendors, and end-users – aiming to move away from the hierarchical way of monitoring to enable users to transmit metrics at scale.

The open source initiative, focused on creating a neutral metrics exposition format, provides a sound data model for current and future needs of users, and embeds this into a standard that is an evolution of the widely-adopted Prometheus exposition format. While there are numerous monitoring solutions available today, many do not focus on metrics and are based on old technologies with proprietary, hard-to-implement and hierarchical data models.

“The key benefit of OpenMetrics is that it opens up the de facto model for cloud native metric monitoring to numerous industry leading implementations and new adopters. Prometheus has changed the way the world does monitoring and OpenMetrics aims to take this organically grown ecosystem and transform it into a basis for a deliberate, industry-wide consensus, thus bridging the gap to other monitoring solutions like InfluxData, Sysdig, Weave Cortex, and OpenCensus. It goes without saying that Prometheus will be at the forefront of implementing OpenMetrics in its server and all client libraries,” said Richard Hartmann, technical architect at SpaceNet, Prometheus team member, and founder of OpenMetrics. “CNCF has been instrumental in bringing together cloud native communities. We look forward to working with this community to further cloud native monitoring and continue building our community of users and upstream contributors.”

OpenMetrics contributors include AppOptics, Cortex, Datadog, Google, InfluxData, OpenCensus, Prometheus, Sysdig and Uber, among others.

“Metrics, when combined with traces and logging, are key to measuring the success of any technology initiative. By offering support for open standards such as OpenMetrics, we’ve enabled our customers to quickly gain visibility into the growing ecosystem technologies that are key to their cloud native transitions,” said Ilan Rabinovitch, VP of Product & Community at Datadog. “We look forward to our continued collaboration with the CNCF and observability community on developing standards and best practices for monitoring in cloud native architectures.”

“Google has a history of innovation in the metric monitoring space, from its early success with Borgmon, which has been continued in Monarch and Stackdriver. OpenMetrics embodies our understanding of what users need for simple, reliable and scalable monitoring, and shows our commitment to offering standards-based solutions. In addition to our contributions to the spec, we’ll be enabling OpenMetrics support in OpenCensus” said Sumeer Bhola, Lead Engineer on Monarch and Stackdriver at Google.

“At InfluxData, we’re very excited about the work to create a standard metrics format. Even though metrics is a subset of our larger focus on time series data, we see real value in creating a standard that can work across vendors and open source projects,” said Paul Dix, founder and CTO of InfluxData. “We’re excited to collaborate with members from Prometheus, OpenCensus and others to push the OpenMetrics standard forward. It will be a first class citizen in the InfluxData set of open source tools, including InfluxDB, our open source time series database and Telegraf, our open source data collector.”

OpenMetrics is a huge improvement over the current mixed bag of formats out there today, and will make systems more interoperable with each other. Uber has a lot of complexity with the challenge of ingesting all of the metrics different applications and services expose due to this problem. Complex software and data centers will become a little easier to observe, monitor and run with this standard, which makes tools work together more easily, and small to medium organizations will need to invest less time for first class monitoring,” said Rob Skillington, lead of metrics and systems monitoring at Uber. “We are excited to be a part of OpenMetrics and will natively support the standard in our open source distributed time series database M3DB, which we use to store petabytes of metrics with, alongside supporting long-term storage of metrics for Prometheus.”

TOC sponsors of the project include Alexis Richardson and Bryan Cantrill.

The CNCF Sandbox is a home for early stage projects, for further clarification around project maturity levels in CNCF, please visit our outlined Graduation Criteria.

Helm, the Package Manager for Kubernetes

By | Blog

A few weeks ago, the CNCF family was extended with a new projectHelm, the package manager for Kubernetes.

Kubernetes was developed as a solution to manage and orchestrate containerized workloads. At the same time, managing the pure containers is not always enough. At the final end, Kubernetes is being used to run applications, and having a solution that will simplify the ability to run and deploy applications with Kubernetes, was a high demand. Helm was this solution.

Originally developed by Deis, Helm shortly became a de-facto open source standard for running and managing applications with Kubernetes.

Helm history – a slide from the Helm project presentation to CNCF TOC, May 2018

Imagine Kubernetes as an Operating System (OS), Helm is the apt or yum for it. Any operating system is a great foundation, but the real value is in the applications. Package managers like apt, yum, or similar, simplify the operations so instead of building the application from the source files, you can easily install it with the package manager in a few clicks.

The same approach is with Kubernetes and Helm. It’s not difficult to write a simple YAML file describing your desired deployment for Kubernetes, but it is much easier to use the application definitions, which can be easily installed in a few clicks (especially if we are speaking about the complex applications). Also, Helm makes it easier to customise an application (e.g. setting the WordPress username) compared to the raw Kubernetes manifests where you’d either have to edit the YAML or use a sed command or similar.

These predefined application definitions in the Helm world are called Charts. The Charts definition from the Helm official documentation: “A chart is a collection of files that describe a related set of Kubernetes resources. A single chart might be used to deploy something simple, like a memcached pod, or something complex, like a full web app stack with HTTP servers, databases, caches, and so on.” In other words, Charts are similar to DEB or RPM packages for the Linux operating system distributions.

Helm consists of two main components: Helm Client and Tiller Server. Helm Client is responsible for the local chart development, repositories management, and interaction with the Tiller Server, which, in its turn, handles the interaction with the Kubernetes API (installs, upgrades and uninstalls charts).

The major prerequisite for installing Helm is a Kubernetes cluster. If you don’t have it, minikube is the easiest way to get one. Helm installation is as easy as installing the binary package (there is also an option to build it from source, though) – all the possible Helm installation options are covered in the Helm documentation.

The simplest way of Helm client installation is running a script that will fetch and install the latest version of it. (This is not recommended for the production environments, but the fastest solution to have Helm installed in a few minutes for development or demo purposes):

$ curl | bash
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 6740 100 6740 0 0 164k 0 --:--:-- --:--:-- --:--:-- 164k
Preparing to install into /usr/local/bin
helm installed into /usr/local/bin/helm
Run 'helm init' to configure helm.

After the Helm client is installed, install Tiller to have an ability to interact with the Kubernetes cluster:

helm init

Now we are ready to install applications. The easiest way to list the available applications from the stable repo is to run the following command:

$ helm search

Here we’ll install WordPress, a well-known open source content management system as an application sample:
$ helm repo update #optionally, to fetch the latest data from the helm repo
$ helm install stable/wordpress


Check if the application was installed correctly using the Kubernetes command-line tools:

$ kubectl get deployments
washed-tortoise-wordpress 1 1 1 0 1m

Here is a list of Kubernetes deployments, with the WordPress installation.

Also, listing the pods in the Kubernetes cluster shows the WordPress chart deployed two pods – wordpress itself, and mariadb as a database for the WordPress application.

idv@ihor:~$ kubectl get pods
exciting-grasshopper-mariadb-0 1/1 Running 0 8m
exciting-grasshopper-wordpress-5f666d765c-xgk85 1/1 Running 1 8m

And of course, we may check if the chart has been installed via helm command:

$ helm list
washed-tortoise 1 Mon Jul 23 19:22:33 2018 DEPLOYED wordpress-2.1.3 default

This was the easiest sample of what can be achieved with Helm, but it is definitely able to handle much more complex deployments. The best place to get started with Helm is the project website, where you may find all the necessary information about this amazing technology.

Q&A with Kubernetes, Cloud Native, and CNCF Projects Driving Big Data and AI

By | Blog

Liu Haifeng, Chief Architect at, sat down with the Cloud Native Computing Foundation (CNCF), to talk about Cloud Native,’s Kubernetes implementation, and tips for other companies looking to get started with open source. Below is their interview.


CNCF: How do you see your CNCF membership and cloud native technologies helping realize its “Retail as a Service” vision?

Haifeng: The goal of our Retail as a Service (RaaS) strategy is to open up our capabilities and resources to empower our partners, suppliers, and other industries. This is very much in line with our commitment to open source technologies. We’ve already benefited tremendously from the CNCF projects we have been a part of and our new commitment to CNCF enables us to build even stronger collaborative relationships with the industry’s top developers, end users, and vendors and ultimately enables us to contribute more to the open source community. Joining CNCF is an important step for us as we develop new container-native technologies towards an open platform to realize our RaaS vision.

CNCF: What impact has Kubernetes had on your company and/or development team?

Haifeng: is one of the earliest adopters of Kubernetes. The company currently manages the world’s largest Kubernetes clusters in production with more than 20,000 bare metal services in several clusters across data centers in multiple regions.

CNCF: How has Kubernetes helped JD to conduct AI or big-data analytics to revolutionize e-commerce?

Haifeng: JDOS, our customized and optimized Kubernetes supports a wide range of workloads and applications, including big data and AI. JDOS provides a unified platform for managing both physical servers and virtual machines, including containerized GPUs and delivering big data and deep learning frameworks such as Flink, Spark, Storm, and Tensor Flow as services. By co-scheduling online services and big data and AI computing tasks, we significantly improve resource utilization and reduce IT costs.

CNCF: How big is the Kubernetes cluster JD runs? Please describe it, your team using Kubernetes.

Haifeng: JD currently manages the world’s largest Kubernetes clusters in production with more than 20,000 bare metal services in several clusters across data centers in multiple regions.

CNCF: How is Kubernetes and cloud native empowering JD developers? What can they do now that they couldn’t do before?

Haifeng: The old deployment tools required different processes for different environments, from application packaging, container application, deployment, configuration, and scaling. The overall process was complicated and time-consuming. The introduction of Kubernetes dramatically simplifies the process. Applications are now automatically packaged into images and deployed onto containers in near-real time. Scaling is now a simple one-click operation that can occur within a few seconds.

CNCF: JD runs one of the largest Kubernetes clusters in production in the world, how has the company overcome hurdles to make this possible?

Haifeng: We are constantly monitoring the performance of our systems. To address performance issues in the past, we collected and analyzed several key performance indicators and generated a detailed bottleneck analysis report. We then customized Kubernetes by removing unnecessary functions and optimizing the default scheduler. We also enhanced multiple controllers to avoid cascading failures. In addition, we developed an operational toolkit for inspection, monitoring, alarming, and fault handling, which helps operators troubleshoot and quickly resolve any issues which may come up.

CNCF: JD just celebrated its infamous June 18 anniversary sale (“618”) clocking transaction volume of over $24.7 billion during the 18-day period. That is a lot of orders. Can you talk about how your system is able to handle this much volume?

Haifeng: JDOS uses prediction-based algorithm to proactively allocate resources to meet forecasted demand and improve resource utilization. It also provides millisecond-level elastic scaling to handle the extreme workloads. Our June 18 anniversary sales period event, which we hold annually, generated $24.7 billion in transaction volume this year. With over 300 million customers on our platform, we experience a significant peak in traffic during this period. We scheduled approximately 460,000 containers (Pod) and 3,000,000 CPU cores to support the massive volume of orders.

CNCF: Tell us about your use of Vitess. What impact has this had?

Haifeng: Our elastic database is one of the largest and most complex Vitess deployments in the world. We have successfully scaled Vitess to manage large volumes of complex transitional data on JD’s Kubernetes platform. The salient features include the support of RocksDB and TokuDB as new storage engines, automatic re-sharding, automatic load balancing, and migration. Our system currently manages 2,600 MySQL clusters, 9,000 MySQL instances, 350,000 tables, 160 billion records, and 65T data in support of various business applications and services at JD. The use of Vitess enables us to manage resources much more flexibly and efficiently, which significantly reduces operations and maintenance costs. We are actively collaborating with the CNCF community to add new features such as subquery support and global transactions to Vitess.

CNCF: What’s next for Kubernetes and other cloud native technologies (GitLab, Jenkins, Logstash, Harbor, Elasticsearch, and Prometheus) in your company?

Haifeng: Our containerized platform separates the application and infrastructure layers by deploying a DevOps stack on Kubernetes that includes Vitess, Prometheus, GitLab, Jenkins, Logstash, Harbor, Elasticsearch, etc. We have contributed code to some of these projects. We would like to make more contributions in the future. One example of where we think we can really add value is Vitess, the CNCF project for scalable MySQL cluster management. We are not only the largest end-user of Vitess, but also a very active and significant contributor. We look forward to working together with others in the CNCF community to add new features to Vitess, including sub-query support, global transactions, etc. Separately, we are extending Prometheus to create a real time and high performance monitoring system. We’d like to improve Kubernetes to support multiple, diverse workloads and hopefully contribute code to Kubernetes as well.

We plan to release our internal and homegrown projects too. There are a bunch of them on already. We also plan to propose new CNCF projects. One such project is ContainerFS –a large scale, container-native cluster file systems that has been seamlessly integrated with Kubernetes.  

CNCF: What other technologies or practices (DevOps, CI/CD) are you currently evaluating?

Haifeng: We are actively working on our own open source projects centered around cloud native or container-native software and technology, from computing, storage and middleware to applications. One focus is container platforms for diverse workloads, including online services, data analytics, edge computing, and IoT. Another focus is scalable and high-performance data storage for container platforms.  

CNCF: For other Chinese companies just getting started with cloud native, what are the most important things to consider?

Haifeng: With Docker, Kubernetes, and microservices, you can get a lot of value out of cloud native without having to endure high costs. A cloud native solution doesn’t only function in the cloud. It is flexible enough to be deployed across on-premise, private cloud, public cloud, and hybrid environments. It is important to keep a close eye on new technology and on industry trends, leverage open source technologies and actively engage with open source communities.

CNCF: What advice do you have for other companies looking to deploy a cloud native infrastructure?

Haifeng: Think about how to meet your business needs from an ecosystem perspective, including containerized infrastructure, data storage, microservice platform, messaging, monitoring systems, etc. In terms of container orchestration and management, Kubernetes is the de facto standard and a sure thing to bet on. You should also take advantage of emerging serverless architectures to simplify the process of application development, packaging, deployment, and management.

CNCF: Why is cloud native such a business imperative today for JD?

Haifeng: With over 300 million customers, in addition to our merchants, it is imperative that our infrastructure is scalable and extremely efficient. To put it in perspective, five years ago, there were about two billion images in our product images system. Today, there are more than one trillion, and that figure increases by 100 million every day. Furthermore, as not only China’s largest retailer, online or offline, but also the operator of China’s largest e-commerce logistics infrastructure – developed fully in-house – our business is complex and changing rapidly by the day. Accordingly, our infrastructure has to be extremely agile and support a wide range of workloads and application scenarios in a host of areas such as online services, data analytics, AI, supply chain, finance, IoT, or edge computing. Cloud native technologies are well suited to handle our ever-changing requirements.

CNCF: Is this the first open source foundation JD has joined?

Haifeng: Yes.  We are a firm believer in open source and it closely aligns with our own strategy. Through CNCF, we aim to have more and stronger engagement with the open source community and fully see the potential mutual benefit of contributing to the open source community. As the third largest internet company in the world by revenue, JD has already developed many leading technology innovations and we recognize our responsibility to take a leadership role in the open source community.

CNCF: How do you plan to work hand-in-hand with the CNCF?

Haifeng: The areas where we can collaborate are unlimited. Joining CNCF and working with the other members will be extremely helpful as we take some of our new projects forward. In addition, CNCF provides us with a platform with which to raise awareness about some of our projects and recruit leading developers to collaborate and contribute to our efforts.  

CNCF: What are you excited to learn or see at KubeCon China?

Haifeng: We look forward to meeting with the industry’s top developers, end users, and vendors and continuing to learn about the newest technological developments. We also plan to showcase our own work and identify potential collaboration opportunities with companies, end users, and independent developers.


Want to learn more about how technology leaders in China are leveraging Cloud Native technologies? Join us for our inaugural KubeCon + CloudNativeCon China in Shanghai from Nov. 14-15. Hope to see you there!

Demystifying RBAC in Kubernetes

By | Blog

Today’s post is written by Javier Salmeron, Engineer at Bitnami

Many experienced Kubernetes users may remember the Kubernetes 1.6 release, where the Role-Based Access Control (RBAC) authorizer was promoted to beta. This provided an alternative authentication mechanism to the already existing, but difficult to manage and understand, Attribute-Based Access Control (ABAC) authorizer. While everyone welcomed this feature with excitement, it also created innumerable frustrated users. StackOverflow and Github were rife with issues involving RBAC restrictions because most of the docs or examples did not take RBAC into account (although now they do). One paradigmatic case is that of Helm: now simply executing “helm init + helm install” did not work. Suddenly, we needed to add “strange” elements like ServiceAccounts or RoleBindings prior to deploying a WordPress or Redis chart (more details in this guide).

Leaving these “unsatisfactory first contacts” aside, no one can deny the enormous step that RBAC meant for seeing Kubernetes as a production-ready platform. Since most of us have played with Kubernetes with full administrator privileges, we understand that in a real environment we need to:

  • Have multiple users with different properties, establishing a proper authentication mechanism.
  • Have full control over which operations each user or group of users can execute.
  • Have full control over which operations each process inside a pod can execute.
  • Limit the visibility of certain resources of namespaces.

In this sense, RBAC is a key element for providing all these essential features. In this post, we will quickly go through the basics (for more details, check the video below) and dive a bit deeper into some of the most confusing topics.

The key to understanding RBAC in Kubernetes

In order to fully grasp the idea of RBAC, we must understand that three elements are involved:

  • Subjects: The set of users and processes that want to access the Kubernetes API.
  • Resources: The set of Kubernetes API Objects available in the cluster. Examples include Pods, Deployments, Services, Nodes, and PersistentVolumes, among others.
  • Verbs: The set of operations that can be executed to the resources above. Different verbs are available (examples: get, watch, create, delete, etc.), but ultimately all of them are Create, Read, Update or Delete (CRUD) operations.

With these three elements in mind, the key idea of RBAC is the following:

We want to connect subjects, API resources, and operations. In other words, we want to specify, given a user, which operations can be executed over a set of resources.

Understanding RBAC API Objects

So, if we think about connecting these three types of entities, we can understand the different RBAC API Objects available in Kubernetes.

  • Roles: Will connect API Resources and Verbs. These can be reused for different subjects. These are binded to one namespace (we cannot use wildcards to represent more than one, but we can deploy the same role object in different namespaces). If we want the role to be applied cluster-wide, the equivalent object is called ClusterRoles.
  • RoleBinding: Will connect the remaining entity-subjects. Given a role, which already binds API Objects and verbs, we will establish which subjects can use it. For the cluster-level, non-namespaced equivalent, there are ClusterRoleBindings.

| TIP: Watch the video for a more detailed explanation.

In the example below, we are granting the user jsalmeron the ability to read, list and create pods inside the namespace test. This means that jsalmeron will be able to execute these commands:

But not these:

Example yaml files:

Another interesting point would be the following: now that the user can create pods, can we limit how many? In order to do so, other objects, not directly related to the RBAC specification, allow configuring the amount of resources: ResourceQuota and LimitRanges. It is worth checking them out for configuring such a vital aspect of the cluster.

Subjects: Users and… ServiceAccounts?

One topic that many Kubernetes users struggle with is the concept of subjects, but more specifically the difference between regular users and ServiceAccounts. In theory it looks simple:

  • Users: These are global, and meant for humans or processes living outside the cluster.
  • ServiceAccounts: These are namespaced and meant for intra-cluster processes running inside pods.

Both have in common that they want to authenticate against the API in order to perform a set of operations over a set of resources (remember the previous section), and their domains seem to be clearly defined. They can also belong to what is known as groups, so a RoleBinding can bind more than one subject (but ServiceAccounts can only belong to the “system:serviceaccounts” group). However, the key difference is a cause of several headaches: users do not have an associated Kubernetes API Object. That means that while this operation exists:

this one doesn’t:

This has a vital consequence: if the cluster will not store any information about users, then, the administrator will need to manage identities outside the cluster. There are different ways to do so: TLS certificates, tokens, and OAuth2, among others.

In addition, we would need to create kubectl contexts so we could access the cluster with these new credentials. In order to create the credential files, we could use the kubectl config commands (which do not require any access to the Kubernetes API, so they could be executed by any user). Watch the video above to see a complete example of user creation with TLS certificates.

RBAC in Deployments: A use case

We have seen an example where we establish what a given user can do inside the cluster. However, what about deployments that require access to the Kubernetes API? We’ll see a use case to better understand this.

Let’s go for a common infrastructure application: RabbitMQ. We will use the Bitnami RabbitMQ Helm chart (in the official helm/charts repository), which uses the bitnami/rabbitmq container. This container bundles a Kubernetes plugin responsible for detecting other members of the RabbitMQ cluster. As a consequence, the process inside the container requires accessing the Kubernetes API, and so we require to configure a ServiceAccount with the proper RBAC privileges.

When it comes to ServiceAccounts, follow this essential good practice:

Have ServiceAccounts per deployment with the minimum set of privileges to work.

For the case of applications that require access to the Kubernetes API, you may be tempted to have a type of “privileged ServiceAccount” that could do almost anything in the cluster. While this may seem easier, this could pose a security threat down the line, as unwanted operations could occur. Watch video above to see the example of Tiller, and the consequences of having ServiceAccounts with too many privileges.

In addition, different deployments will have different needs in terms of API access, so it makes sense to have different ServiceAccounts for each deployment.

With that in mind, let’s check what the proper RBAC configuration for our RabbitMQ deployment should be.

From the plugin documentation page and its source code, we can see that it queries the Kubernetes API for the list of Endpoints. This is used for discovering the rest of the peer of the RabbitMQ cluster. Therefore, what the Bitnami RabbitMQ chart creates is:

A ServiceAccount for the RabbitMQ pods.A Role (we assume that the whole RabbitMQ cluster will be deployed in a single namespace) that allows the “get” verb for the resource Endpoint.

A RoleBinding that connects the ServiceAccount and the role.

The diagram shows how we enabled the processes running in the RabbitMQ pods to perform “get” operations over Endpoint objects. This is the minimum set of operations it requires to work. So, at the same time, we are ensuring that the deployed chart is secure and will not perform unwanted actions inside the Kubernetes cluster.

Final thoughts

In order to work with Kubernetes in production, RBAC policies are not optional. These can’t be seen as only a set of Kubernetes API Objects that administrators must know. Indeed, application developers will need them to deploy secure applications and to fully exploit the potential that the Kubernetes API offers to their cloud-native applications. For more information on RBAC, check the following links:

CNCF to Host Harbor in the Sandbox

By | Blog

Today, the Cloud Native Computing Foundation (CNCF) accepted Harbor, a cloud native registry, into the CNCF Sandbox, a home for early stage and evolving cloud native projects.

Project Harbor is an open source cloud native registry that stores, signs, and scans content. Created at VMware, Harbor extends the open source Docker Distribution by adding the functionalities usually required by users – such as security, identity, and management – and supports replication of images between registries. With more than 4,600 stars on GitHub, the project also offers advanced security features such as vulnerability analysis, role-based access control, activity auditing, and more.

“Container registries are essential to healthy cloud native environments, and enterprises require the security, features, flexibility, and portability that a trusted registry provides,” said Haining Henry Zhang, Technical Director, Innovation and Evangelism, at VMware, and Harbor project founder. “We’re thrilled to have Harbor in an neutral home that fosters open collaboration, which is incredibly important for creating new critical features. The project will benefit greatly from the contributions of CNCF’s thriving community.”

Harbor users include CNCF members Caicloud, China Mobile,, Pivotal, Rancher, Tencent Cloud, Tenxcloud and Wise2c, along with OnStar Shanghai, Talking Data and TrendMicro, among others.

TOC sponsors of the project include Quinton Hoole and Ken Owens.

The CNCF Sandbox is a home for early stage projects, for further clarification around project maturity levels in CNCF, please visit our outlined Graduation Criteria.



Harbor项目是一个具有存储、签署和扫描内容功能的开源云原生registry。Harbor 由VMware创建,通过添加用户所需功能(如安全性,身份认证和管理)来扩展开源Docker Distribution,并支持在registry之间复制镜像。Harbor还提供高级安全功能,比如漏洞分析,基于角色的访问控制,活动审计等等。该项目在GitHub上已获得超过4600颗星。


Harbor用户包括CNCF成员才云、中国移动、京东、Pivotal、Rancher、腾讯云、时速云、睿云智合,以及 上海安吉星,Talking Data和TrendMicro等。

该项目的TOC发起人包括Quinton HooleKen Owens


Meet the Ambassador: Cheryl Hung

By | Blog

Cheryl Hung, StorageOS, sat down with Kaitlyn Barnard, Cloud Native Computing Foundation (CNCF), to talk about cloud native, the cloud native Meetup group in London, and being an Ambassador. Below is their interview. You can also view the video.


Kaitlyn: Thank you so much for joining me today to talk about your community involvement and our Ambassador Program. Can you tell us a little bit about yourself?

Cheryl: I’ve been involved with Cloud Native for a couple of years, I used to work at Google as a software engineer. Now I’m the Product and DevOps Manager at a London start-up called StorageOS, building persistent storage for containers, and I also run the Cloud Native London Meetup group.

Kaitlyn: You’ve been talking about storage recently. Can you talk a little bit about some of the trends and challenges you’re seeing the in cloud native storage space right now?

Cheryl: It’s definitely an evolving space, because containers were designed to be stateless and immutable so storage is not really a concern. Except that at the end of the day, pretty much everybody has data that they need to store somewhere! It’s clearly an unsolved problem about what’s the best way to do databases and other stateful applications with Kubernetes, which is why I work at StorageOS, because it provides that storage abstraction layer and that means you can run databases within Kubernetes and have replication and failover, etc.

In terms of what I see coming next, the Container Storage Interface is one of the big things to be aware of in the cloud native space. Over the course of the next six months to a year, all of the vendors, cloud providers, and cloud orchestrators are going to get behind this interface. So hopefully, by this time next year, storage will be a solved problem as far as end users are concerned.

Kaitlyn: You run one of the largest CNCF meetups that we have. Why and how did you start the Cloud Native London Meetup?

Cheryl: I started it in about June of 2017. At the time there was Cloud Native Paris, Cloud Native Berlin, Cloud Native Barcelona, and there was a Cloud Native London, but it had been quiet and dormant for a couple of years. I thought this was a really good time to revive that Meetup and to bring in all the new knowledge, community, and interest around Kubernetes and Docker and also Prometheus, Linkerd, and all the other projects.  

It was about bringing in people from all different interests, but all in the same infrastructure and DevOps mindset and having a space where people can teach others as well. So I really encourage new speakers to join and share their stories, because there’s always a mindset around, “Oh, am I good enough to do public speaking?”. I see it as part of my role as the organizer to tell people that, “Yes, your stories are interesting and people do want to hear from you.”

Kaitlyn: Why did you want to be a CNCF ambassador?

Cheryl: I’m probably one of the truest cloud natives in that when I joined Google, I was 21. I was using Borg, which was Google’s internal predecessor to Kubernetes. Because I was so young, I really didn’t have a memory of how software was done before. So to me, it’s always been natural to containerize your software into running with an orchestrator, like Kubernetes, and to package it as microservices. It seems like the whole industry is moving that way. So becoming a CNCF ambassador has been about taking what I know and bringing that attitude, culture, tools, and infrastructure out to the entire industry.

Kaitlyn: You travel a lot, I see you at a lot of conferences. What has been your favorite place that you visited so far?

Cheryl: I travel mostly in the Bay Area and then around Europe. Last year we met at Los Angeles, Open Source Summit, that was really cool because I had not been to Los Angeles before and that was a really interesting place. Some really great stuff and some stuff that was a bit scary, but fun. That was really cool to see.

One thing about being at all these conferences, when I was here last year at the Berlin KubeCon I really didn’t know anybody, I was completely on my own. This time, as I’ve been walking around, it’s been, “Hey, Cheryl” and  “Oh, I know you,” “Oh, you run the meetup, right?”. That’s been awesome, that’s been fantastic to have so many people get involved with the community, know about me, and come up and say hi.

Kaitlyn: It’s fun. Even though it’s 4,300 people now, which is crazy growth in the first place, it’s still a small community and everyone still kind of knows each other.

Cheryl: Exactly. And it’s still a really friendly and open community, which I love.

Kaitlyn: What do you do in your free time?

Cheryl: I’m getting married at the end of August! I’m incredibly excited about it, but it means that any time I’m not planning my work, my engineering team, and what they’re working on, I’m planning my wedding, which is a challenge in its own right! It’s a two day thing, a western wedding and a Chinese wedding, so I spend a lot of time negotiating with people about, “What kind of stationary do I like? What kind of flowers do I like?” It’s crazy!

Kaitlyn: Thank you so much for taking the time to speak with me today.

Cheryl: You are very welcome. Thank you for inviting me.

Announcing EnvoyCon! CFP due August 24

By | Blog

Originally published by Richard Li on the Envoy blog, here.

We are thrilled to announce that the first ever EnvoyCon will be held on December 10 in Seattle, WA as part of the KubeCon / CloudNative Con Community Events Day. The community growth since we first open sourced Envoy in September 2016 has been staggering, and we feel that it’s time for a dedicated conference that can bring together Envoy end users and system integrators. I hope you are as excited as we are about this!

The Call For Papers is now open, with submissions due by Friday, August 24.

Talk proposals can be either 30 minute talks or 10 minute lightning talks, with experience levels from beginner to expert. The following categories will be considered:

  • Using and Integrating with Envoy (both within modern “cloud native” stacks such as Kubernetes but also in “legacy” on-prem and IaaS infrastructures)
  • Envoy in production case studies
  • Envoy fundamentals and philosophy
  • Envoy internals and core development
  • Security (deployment best practices, extensions, integration with authn/authz)
  • Performance (measurement, optimization, scalability)
  • APIs, discovery services and configuration pipelines
  • Monitoring in practice (logging, tracing, stats)
  • Novel Envoy extensions
  • Porting Envoy to new platforms
  • Migrating to Envoy from other proxy & LB technologies
  • Using machine learning on Envoy observability output to aid in outlier detection and remediation
  • Load balancing (both distributed and global) and health checking strategies

Reminder: This is a community conference — proposals should emphasize real world usage and technology rather than blatant product pitches.

If you’re using or working on Envoy, please submit a talk! If you’re interested in attending, you can register as part of the KubeCon registration process.

Reinventing the World’s Largest Education Company With Kubernetes

By | Blog

Pearson, a global education company serving 75 million learners, set a goal to more than double that number to 200 million by 2025. To serve the digital learning experiences of their users, they needed to scale and adapt to this growing online audience.

In this case study, Pearson describes their process to build a platform that would enable their developers to build, manage, and deploy applications, thus improving their engineer’s productivity.

Read about the benefits they saw from their Kubernetes implementation not only for their team, but in customer experience. As Pearson states: “We’re not worried about 9s. There aren’t any. It’s 100% (uptime)”.