Prometheus User Profile: Helping Life360 Keep Families Safe

By | Blog

An open source monitoring system with a dimensional data model, flexible query language, efficient time series database and modern alerting approach, Prometheus has an active and growing user base. Counting Life360 as one of its many users, Prometheus helps the company monitor its massive data sets.

Screen Shot 2016-07-06 at 11.45.14 AM.png

Figure 1: Overall architecture of Prometheus and some of its ecosystem components

Life360 is a free smartphone app that helps families and close friends stay in sync throughout their busy day. Using the latest GPS tracking technology, Life360 allows you to see where your family and friends are on a private map, stay in touch with group and one-on-one messaging, get help in an emergency, and track lost or stolen phones. Started in 2008, Life360 has grown to 75 million members worldwide.

In this blog, originally published on the Prometheus blog, the company shares its experiences evaluating and using Prometheus.

Can you tell us about yourself and what Life360 does?

I’m Daniel Ben Yosef, a.k.a, dby, and I’m an Infrastructure Engineer for Life360, and before that, I’ve held systems engineering roles for the past 9 years.

Life360 creates technology that helps families stay connected, we’re the Family Network app for families. We’re quite busy handling these families – at peak we serve 700k requests per minute for 70 million registered families.

We manage around 20 services in production, mostly handling location requests from mobile clients (Android, iOS, and Windows Phone), spanning over 150+ instances at peak. Redundancy and high-availability are our goals and we strive to maintain 100% uptime whenever possible because families trust us to be available.

We hold user data in both our MySQL multi-master cluster and in our 12-node Cassandra ring which holds around 4TB of data at any given time. We have services written in Go, Python, PHP, as well as plans to introduce Java to our stack. We use Consul for service discovery, and of course our Prometheus setup is integrated with it.

What was your pre-Prometheus monitoring experience?

Our monitoring setup, before we switched to Prometheus, included many components such as:

  • Copperegg (now Idera)

  • Graphite + Statsd + Grafana

  • Sensu

  • AWS Cloudwatch

We primarily use MySQL, NSQ and HAProxy and we found that all of the monitoring solutions mentioned above were very partial, and required a lot of customization to actually get all working together.

Why did you decide to look at Prometheus?

We had a few reasons for switching to Prometheus, one of which is that we simply needed better monitoring.

Prometheus has been known to us for a while, and we have been tracking it and reading about the active development, and at a point (a few months back) we decided to start evaluating it for production use.

The PoC results were incredible. The monitoring coverage of MySQL was amazing, and we also loved the JMX monitoring for Cassandra, which had been sorely lacking in the past.

Cassandra Client Dashboard

How did you transition?

We started with a relatively small box (4GB of memory) as an initial point. It was effective for a small number of services, but not for our full monitoring needs.

We also initially deployed with Docker, but slowly transitioned to its own box on an r3.2xl instance (60GB ram), and that holds all of our service monitoring needs with 30 days of in-memory data.

We slowly started introducing all of our hosts with the Node Exporter and built Grafana graphs, up to the point where we had total service coverage.

We were also currently looking at InfluxDB for long term storage, but due to recent developments, this may no longer be a viable option.

We then added exporters for MySQL, Node, Cloudwatch, HAProxy, JMX, NSQ (with a bit of our own code), Redis and Blackbox (with our own contribution to add authentication headers).

NSQ Overview Dashboard

What improvements have you seen since switching?

The visibility and instrumentation gain was the first thing we saw. Right before switching, we started experiencing Graphite’s scalability issues, and having an in-place replacement for Graphite so stakeholders can continue to use Grafana as a monitoring tool was extremely valuable to us. Nowadays, we are focusing on taking all that data and use it to detect anomalies, which will eventually become alerts in the Alert Manager.

What do you think the future holds for Life360 and Prometheus?

We currently have one of our projects instrumented directly with a Prometheus client, a Python-based service. As we build out new services, Prometheus is becoming our go-to for instrumentation, and will help us gain extremely meaningful alerts and stats about our infrastructure.

We look forward to growing with the project and keep contributing.

Thank you Daniel! The source for Life360’s dashboards is shared on Github.

KubeCon 2016

By | Blog


November 8-9, 2016
Seattle, WA
The Cloud Native Computing Foundation is hosting KubeCon to gather leading Kubernetes technologists from multiple open source cloud native communities to further the education and advancement of Docker, Kubernetes, and Cloud Native architectures. By co-locating with CloudNativeCon, KubeCon adds to the ecosystem of technologies that support the cloud native ecosystem and leading expert discussion on cloud native projects. Learn More

AT&T, CoreOS, Weaveworks, and Fujitsu Share Thoughts on Cloud Native Market

By | Blog

Some of our member companies recently shared the market problems and misconceptions around cloud native, as well as the objectives and benefits of CNCF.

Here is what AT&T’s Lead Principal Technical Architect, Domain 2.0 Architecture and Engineering, Cloud Architecture, Doug Nassaur; CoreOS CEO, Alex Polvi; Fujitsu Director of Development Department, Linux Development Division, Kenji Kaneshige; Weaveworks CEO, Alexis Richardson, had to say.

1. What are the cloud native market problems?

Polvi, CoreOS: We’re at the beginning of the adoption curve in cloud native computing. Naturally, new markets have a growth period before stabilizing. Any emerging market and ecosystem needs to have hobbyists, commercial vendors, and vendor-neutral industry groups to grow. The CNCF is the latter for this ecosystem.

Kaneshige, Fujitsu: Cloud native is a set of technologies, but to get the benefit from “high velocity”, users need to understand what culture and rules. Organization is required for delivering apps and change, so there will be some barriers. Providing case studies that show the benefits to the market will be helpful.

Richardson, Weaveworks: Customers want to build cloud native applications and microservices.  But many are not sure where to start, and what tools to use.  We need to help by identifying the best tools, patterns and practices, and then explaining these clearly.  There’s multiple issues like data, networks, monitoring, security, governance, and orchestration. On top of all that, companies want to deliver apps with high velocity.  The CNCF can show the way to do this.

2. What are the biggest misconception about cloud native in the market today?

Nassaur, AT&T: We have an impedance mismatch between expectations and effort. Too many people believe containers to be the next silver bullet – put your software in a container and it magically removes all barriers and obstacles to performance, availability and economics. You can’t send your application to the gym each morning, have it run two miles every night, put it on a gluten-free diet and call it a micro-service. It doesn’t work that way. Cloud native applications are container-packaged, dynamically scheduled and micro-services oriented. This means they are supported and enabled by a software ecosystem which must be in place and based on open standards – that’s why we need the CNCF.

Polvi, CoreOS: There are some misconceptions to address head on and we are focused on addressing this in the CNCF. One misconception is that containers will replace virtual machines. This is not true. For example, many container deployments are on AWS, AWS uses virtualization for every server, and I do not see that changing for the foreseeable future.

Another myth is about the security of this style of infrastructure that may have come from faulty early implementations. This style of infrastructure is actually more secure. With the ability to isolate individual applications and automatically update the software stack, containers are a great way to increase security.

A misconception is that this style of infrastructure is only for new applications and that old enterprise apps can’t run on cloud native infrastructure. This is in fact how the biggest companies like Google run all of their infrastructure, and have been for years given the balance of operational efficiency and flexibility. These benefits in addition to the ability to have consistent environments will drive the adoption of cloud-native infrastructure over the next few years.

Richardson, Weaveworks: The biggest misconception is that you need to understand infrastructure technology in order to deliver cloud native applications. Fundamentally, cloud native is about enabling application developers to create business value, without thinking about infrastructure.  And the second big misconception is that PaaS delivers all things cloud native.   That is certainly not true – PaaS describes an opinionated model for application delivery, and is just one of several cloud native patterns.

3. How is CNCF uniquely positioned to address these market problems and clear up these misconceptions?

Nassaur, AT&T: Integration, interoperability with freedom of choice to adopt and consume innovation as each building block matures without having to turn over or upgrade the whole farm. Encouraging and supporting new projects is vital for innovation. Education, awareness and being an impartial lighthouse to guide newcomers will expedite understanding, and as a result promote adoption.

Polvi, CoreOS: As a vendor neutral home for cloud native computing, CNCF is positioned to help unify and clarify what is happening in this market.

Kaneshige, Fujitsu: We’ve got a lot of amazing companies involved. If you look at the governing board and technical oversight committee, it really is a who’s who in the industry right now. Lots of technology industry leadership collaborating.

Richardson, Weaveworks: The CNCF is pragmatic — we understand that customers want to adopt cloud native incrementally, and may not want to replatform or make wholesale architectural changes.  We’re not locked into a particular architecture. We’re trying to align with users’ needs as they emerge. We have a model that is designed to grow with their needs. We’re going to listen to users, which is why we created an end user board.

4. What are the benefits CNCF aims to provider developers, end users, and CIOs?

Polvi, CoreOS: For developers, the question should be how can CNCF help open source projects make cloud native possible? For instance, with Kubernetes, being vendor neutral from a copyright perspective helps a diverse set of companies get involved with the open source collaboration side of things. The CNCF neutralizes party alignment from the different vendors.

Kaneshige, Fujitsu: For end users, it is very important to provide a stable and open cloud native technology and accessible APIs. For this purpose, the cloud native technology should be harmonized by the industry leaders. CNCF has a very good opportunity to do that.

Richardson, Weaveworks: CIOs want IT to go faster, software to go faster. CNCF will provide them with the ability to go faster. They will be able to adopt a set of technologies and practices that will enable them to create software in-house for business people working with IT people to keep up with companies like Airbnb, while delivering better services to their customers. Therefore, the obvious benefit to end users is better services.

5. What are some of the key objectives for CNCF?

Nassaur, AT&T: To provide education and awareness. Cloud native is not a single architecture. Customers have multiple use cases that have quite different approaches, so it is important to support multiple, different projects. We need to educate people on the need to invest in a cloud native fabric by showing them the benefits of all kinds of open source componentry as it sits today. As it continues to evolve, they can get the benefit of the excellent innovation from the open source community, while still having a consistent, predictable, repeatable infrastructure that they can count on to run their organizations.

Polvi, CoreOS: The main objectives we see are:

  • Lowering the confusion of what’s going on in this space.

  • Providing enterprises an adoption path.

  • Fostering, supporting, and generally helping the open source projects that make cloud native computing possible in the first place.

Kaneshige, Fujitsu: To make cloud native architecture available, we would like to grow the ecosystem with more projects and to establish a set of standard APIs and software of cloud native technologies for collaborative computing.

Richardson, Weaveworks: Reduce confusion in the market and make it faster and easier to adopt cloud native applications to help customers make that transformation faster.

Cloud Native Computing Foundation Sponsoring PromCon and Software Circus

By | Blog

We are excited to sponsor several upcoming developer shows taking place in EMEA this year, including PromCon 2016, occurring August 25-26 at Google Berlin, which will be the first conference around the Prometheus monitoring system. The second is Software Circus, occurring August 31-September 2 in Amsterdam and will be two days focused on Cloud Native Computing organized by Implicit-Explicit & Container Solutions.

Check out our preview of the shows:

PromCon 2016

This exciting event will connect Prometheus users and developers from around the world in order to exchange knowledge, best practices, and experience gained around using Prometheus. PromCon will also enable collaboration for building its community and growing professional connections around systems and service monitoring.

Organized by the Prometheus developer community, they’re expecting an intimate crowd of experienced and influential infrastructure engineers.

CNCF is proud to be a Platinum sponsor, alongside fellow sponsors Robust Perception, Weaveworks, CoreOS, Google Cloud Platform, and more. The full speaker lineup can be found here and features talks from DigitalOcean, Weaveworks, Improbable, CoreOS, SoundCloud, Rancher Labs, Joyent, Raintank, and more.

This event is fully sold out, but there is a waiting list and tickets will be issued over the coming weeks. Additionally, all talks will be recorded and will be published after the event. Plus, you can still participate in the Prometheus conversations at PrometheusDay, co-located with CloudNativeCon, in Seattle November 8-9. Register to attend or submit to speak! CFPs close August 5th.

Software Circus

This show will take attendees “To the Cloud and Beyond” with two whole days centered around cloud native computing! Last year’s inaugural event brought together prominent influencers like Kelsey Hightower, Ken Owens, Luke Marsden and Adrian Cockcroft, and companies like Cisco, CloudSoft, Mesosphere and Weave.

Software Circus, an initiative of Implicit-Explicit and Container Solutions, started as a meetup group with a mission to tackle the social aspects of technology, the bleeding edges and people’s experience.

CNCF is proud to sponsor the infamously fun conference — with a focus on learning, inspiring and entertaining — which has grown to include workshops and two days of sessions, equipped with dancing, robots and drones.

Staying true to its roots, the Software Circus community will host pre-event meetups before the big show, including:

While the call for papers closes on July 31st, if the show is anything like last year’s event we can expect lively sessions discussing the basics of working with Docker, the fundamental concepts and ideas behind containers, workshops on continuous delivery with Docker on Mesos, and tutorials on using Prometheus for metrics collection, monitoring and alerting in a microservice environment.

Following this exciting event, the cloud native computing conversation will continue at CloudNativeCon in Seattle from November 8-9! Register to attend or submit to speak! CFPs close August 5th.

Register now for Software Circus — we hope to see you there!

Why ChaoSuan, Crunchy Data, Qbox, StorageOS, and Treasure Data Recently Joined The Cloud Native Computing Foundation

By | Blog

We recently announced ChaoSuan, Crunchy Data, Qbox, StorageOS, and Treasure Data have joined our efforts to accelerate the adoption of cloud native technologies and advance the open source ecosystem.

ChaoSuan CEO, Alex Yang; Crunchy Data CEO Bob Laurence; Qbox Co-Founder and CEO, Mark Brandon; StorageOS Co-Founder and CEO Chris Brandon; and Treasure Data CEO, Hiro Yoshikawa, shared with us why they joined the Foundation and their work to advance cloud native computing. You’ll find many of them at LinuxCon, CloudNativeDay, CloudNativeCon, KubeCon, PromCon and ContainerCon in the coming months, excited to discuss their reasons for joining CNCF and their experiences with the project.

1. Why did you become a member of CNCF?

Yang, ChaoSuan: We believe cloud native innovation is the future of cloud computing, and we look forward to working together closely with the CNCF community to continue advancing the adoption and development of cloud native technologies. Joining CNCF will ensure that ChaoSuan stays apprised of state-of-the-art cloud application developments throughout the industry, so that we always provide the most advanced, cutting-edge solutions to our clients.

Laurence, Crunchy Data: Crunchy Data, a leading provider of trusted 100 percent open source PostgreSQL and enterprise PostgreSQL support, technology and training, is enabling PostgreSQL for cloud native architectures. Combining the agility provided by cloud native with the power of the most advanced open source database will allow organizations to build new innovation as never before. We joined CNCF to help advance the adoption and maturity of cloud native architectures based on containers and microservices. Crunchy Data believes that these technologies have real potential to accelerate innovation for any organization.

Brandon, Qbox: DevOps is our core competency, Qbox has been running a hosted Elasticsearch business since 2013. As the creators of Supergiant – an open source datacenter management system based on Kubernetes – we felt that joining CNCF would be an opportunity to both support the platform so crucial to our development and gain insight into its roadmap.

Brandon, StorageOS: Our vision is to give developers and enterprises total control of their own storage environment with agile, on-demand storage anywhere – easily and securely. Standardizing container storage and offering agile storage gives developers freedom to innovate. StorageOS joined the CNCF because of the leadership it provides to the cloud technology ecosystem and operations market. Its mission to harmonize emerging technologies and foster innovation in the container market aligns perfectly with our vision at StorageOS. Through joining CNCF, our aim is to help the industry understand the role storage plays in these new technologies and deliver solutions that are well integrated and easy to use.

Yoshikawa, Treasure Data: Treasure Data is the original creator and primary sponsor of Fluentd, the open source log collector. For us, joining CNCF is the natural logical step in demonstrating our commitment and continued partnership with cloud native software like Kubernetes, Prometheus and Docker.

Prometheus has been one of the popular destinations for Fluentd for some time. In fact, one of the active co-users of Prometheus and Fluentd wrote a Fluentd output plugin (link) for Prometheus enabling Prometheus users to collect data from dozens of different data sources and data formats via Fluentd.

Fluentd and Kubernetes have began working together for almost two years. Fluentd, is the default logging agent for Kubernetes (link). Our collaboration with Kubernetes is one of the reasons why Fluentd was one of the first to be supported on Docker’s Logging Driver API.

2. Why is cloud native important to your company?

Yang, ChaoSuan: ChaoSuan aims to provide the most efficient, high-quality enterprise solutions. As cloud innovation is advancing rapidly and has become the de facto DevOps model, adoption of cloud native technology will significantly improve the quality of our services while reducing operational costs for customers.

Laurence, Crunchy Data: Cloud native is important to Crunchy Data because we recognize that organizations leveraging cloud native architectures will be able to realize the inherent agility and flexibility enabled through the use of containers and microservices and thereby accelerate innovation and meet ever-increasing customer demands. Crunchy Data is leading the way in the effort to bring the PostgreSQL database into the cloud native world. Crunchy Data is investing heavily in establishing PostgreSQL as a first-class citizen in a containerized environment. Leveraging containers and microservices to run and manage PostgreSQL instances will allow organizations to leverage the most advanced open source database in their cloud native application architectures.

Brandon, Qbox: In 2015, Qbox switched from using virtual machines (VMs) to cloud native technology and the result was powerful. We saw a 50 percent decrease in our AWS costs, along with improved performance of our hosted Elasticsearch service and fewer support tickets. Our discovery was so eye opening that we packaged our hand-rolled datacenter management system, attached an Apache 2 license and put it out on Github for open source use.

Brandon, StorageOS: Cloud native applications need to function in a world where storage management is simple and access is ubiquitous. To achieve this, these applications require access to data in any location in which it resides and the capability to move this data securely. At StorageOS, our mission is to deliver a persistent, cloud native storage platform that helps make applications portable and secure without hardware or cloud provider lock-in.

Yoshikawa, Treasure Data: Treasure Data believes that data collection across disparate systems is the single biggest problem that big data and analytics face today. Traditionally, data collection has been a major bottleneck for businesses to use because legacy systems and application deployment models weren’t built to make it easy to collect data. The way to solve this problem is to modernize how applications are deployed, but also to ensure that the new generation of applications collect data in a uniform way; significantly simplifying the process of data collection, instrumentation, and management.

Logging and data collection are a first-class citizen in the cloud native world, and open source tools like Fluentd, Embulk and MessagePack make it easier to collect and transport data uniformly at scale. If more companies embrace the cloud native approach, it will mean more applications will be deployed in a data collection friendly way.

3. As part of your membership, how does your company plan to get involved with the Foundation?

Yang, ChaoSuan: Aside from regular participation in CNCF meetings and discussions, ChaoSuan plans to allocate additional resources to further the development of future incubated projects.

Laurence, Crunchy Data: We plan to become actively involved in CNCF efforts around supporting databases and storage within cloud native architectures. Crunchy Data’s mission is to help organizations accelerate innovation and success by leveraging the most advanced and cost-effective open source object-relational database platform as the foundation for their application infrastructures. Crunchy Data is investing in technologies and best practices that will allow PostgreSQL to be used in modern application architectures. Leveraging containers and microservices to run and manage PostgreSQL instances will allow organizations to leverage the most advanced open source database in their cloud native application architectures.

Brandon, Qbox: Qbox looks forward to contributing to the development of Kubernetes, especially as it relates to administering stateful, distributed apps. We also look forward to participating in CloudNativeDay, CloudNativeCon, and community events as an official CNCF member to help spread awareness of the project’s mission.

Brandon, StorageOS: StorageOS is creating white papers describing best practices for deploying and managing simple-to-use, cost-effective storage that is highly portable across physical, virtual and cloud devices. Lastly, following our beta launch this month, we will donate a StorageOS Kubernetes plugin to the open source community as the Kubernetes community is essential to our strategy.

Yoshikawa, Treasure Data: In the future, data will be used more effectively to run and grow business and data management in cloud native environments will play a key part as the amount of data grows. As part of our CNCF membership, we look forward to partnering with other members via technology integration and partnership to build an open ecosystem where tools can be plugged into each other and encourage community participation. We look forward to additional contributions to Fluentd from the open source community.  Fluentd has over 300 plugins, with less than 20 written by Treasure Data. Additionally, we plan to attend and sponsor Docker and Kubernetes related events such as DockerCon, KubeCon, CloudNativeDay, and CloudNativeCon to name a few.

CNCF 2016 Events Schedule; Mark Your Calendars!

By | Blog

Mark your calendars. CNCF will be attending, speaking at and hosting a number of events for 2016 – including CloudNativeDay, CloudNativeCon, KubeCon, and PrometheusDay.

CNCF Events

CNCF will host two new events in 2016 to help further the adoption of cloud native computing; with particular focus on central orchestration processing, cloud native applications, container packaging, and microservices. Hope to see you there!

  • CloudNativeDay, a CNCF event

Date: Thursday, August 25, 2016

Location: Toronto, Canada


Register to Attend

Sponsor the Event

Agenda coming soon!

Description: CloudNativeDay, a CNCF event, gathers leading technologists from multiple open source cloud native communities to further the education and advancement of cloud native computing. CloudNativeDay is being co-located with ContainerCon North America 2016 and will feature a single-track, one-day event dedicated to cloud native education.

  • CloudNativeCon, a CNCF event

Date: November 8-9, 2016

Location: The Sheraton Seattle

Submit a CFP – Due August 5th


Register to Attend

Sponsor the Event

Description: Hosted by the CNCF community and co-located with KubeCon and PrometheusDay, this event will feature a multi-track, multi-day event dedicated to cloud native education.

  • KubeCon, co-located with CloudNativeCon

Date: November 8-9, 2016

Location: The Sheraton Seattle

Submit a CFP – Due August 5th


Register to Attend

Sponsor the Event

Description: KubeCon (a CNCF event) gathers leading Kubernetes technologists from multiple open source cloud native communities to further the education and advancement of Docker, Kubernetes, and Cloud Native architectures. This event is co-located with CloudNativeCon.

  • PrometheusDay, co-located with CloudNativeCon

Date: November 8-9, 2016

Location: The Sheraton Seattle

Submit a CFP – Due August 5th (shared with CloudNativeCon)


Registration: Complimentary with CloudNativeCon registration

Description: PrometheusDay, a CNCF event, co-located with CloudNativeCon, is a single-day event taking place this November in Seattle. The event will feature highly technical talks covering major Prometheus adopters, leading expert contributor insights, and a full range of technologies that support open source monitoring technology in the cloud native ecosystem.


CNCF Event Participation

CNCF will have a presence at a number of cloud native, container and open source events.

  • ContainerCon Japan 2016

Date: July 13-15, 2016

Location: Tokyo, Japan


Description: COO of Cloud Native Computing Foundation, Chris Aniszczyk, will speak on Cloud Native and Container Technology. Additionally, CNCF member Fujitsu will lead several sessions during the conference including: Container Standardization Introduction, Learning From Real Practice of Providing Highly Available Hybrid Cloud Service with OpenStack Neutron, Btrfs in-Band De-Duplication, and Address Range Memory Mirroring.

  • ContainerCon

Date: August 22-24, 2016

Location: Toronto, Canada


Description: Chris Aniszczyk, and CEO of Weaveworks and CNCF TOC Chair Alexis Richardson will participate in a panel discussion “So CFF, CNCF and OCI Walk Into A Room.”

  • PromCon

Date: August 25-26, 2016

Location: Google Berlin


Description: CNCF is a sponsor of PromCon 2016, the first conference around the Prometheus monitoring system.

  • VMworld

Date: August 28 – September 1, 2016

Location: Las Vegas


Description: COO of Cloud Native Computing Foundation, Chris Aniszczyk, presents “Building Cloud Native Architectures”

  • Huawei Connect 2016

Date: August 31 – September 2, 2016

Location: Shanghai, China


Description: Executive Director of Cloud Native Computing Foundation, Dan Kohn, to present at event.

  • Software Circus

Date: August 31 – September 2, 2016

Location: Amsterdam


Description: CNCF is a sponsor of Software Circus, a three-day event dedicated to cloud native computing.

Member Event Activity

CNCF member companies will participate in the following events in 2016. Come by and learn more about their involvement with the Foundation!

Platinum Member Spotlight: Fujitsu

By | Blog

Recently, Fujitsu Limited became a Platinum member of CNCF, joining the growing industry effort to advance and accelerate cloud native computing.

We had a chance to sit down with Kenji Kaneshige, director of Linux software development department, platform software division at Fujitsu, to discuss why cloud native is important to Fujitsu, how they will contribute to CNCF and what impact this work will have on Fujitsu’s digital business platform.

Q. What role will cloud native play in building the next-generation digital business platforms?

Kaneshige, Fujitsu: In the future, services will unleash massive transformations at scale. Cloud native computing is a key concept for next-generation digital business platforms to build flexible, scalable and application centric computing environment. These environments will easily connect people, devices and services to enable customers to build information and communications technology (ICT) services for their business quickly.

Q. How will these business platforms benefit developers, CIOs and consumers?

Kaneshige, Fujitsu: “Connecting people” means ICT services finally get an accurate feedback loop between providers and consumers. With consumer data, ideas, and behavior being collected, CIOs will be able to see real value and qualities of services, while developers will be able to hack services in quick ways  based on their deep insights.

Q. Why is cloud native important to Fujitsu?

Kaneshige, Fujitsu: Cloud native computing is a key concept for Fujitsu’s digital business platform, MetaArc, which provides connecting services and infrastructures in open standards such as, Linux, KVM, OpenStack and OCI. In some cases, applications are tightly coupled with the platform and customers need to make additional investment just to keep them running, not to mention the investment needed to evolve the applications in the future. Running applications on an open standard layer has been an important idea for decades, and another reason cloud native computing is important, as it allows businesses to decouple platforms and applications to protect their legacy investments.

Q. Open standards and open source are important parts of Fujitsu’s business. What other projects have you contributed to?

Kaneshige, Fujitsu: We are a long-time member of The Linux Foundation in addition to founders of the Open Container Initiative (OCI) and the Cloud Native Computing Foundation (CNCF). We have been committed to open source software and contribute to Linux, KVM, OpenStack, Kubernetes and others. By joining open source and fairly governed open projects, Fujitsu will continue to create trusted open platforms for customers to rely on as long-term architectures.

Q. How did Fujitsu contribute to Kubernetes provisioning and OpenStack integration?

Kaneshige, Fujitsu: We have been a contributor to Kubernetes, specifically in the areas of the Dashboard, Kubernetes provisioning and OpenStack integration. In March, Fujitsu released its Kubernetes-based offering FUJITSU Software ServerView Cloud Load Control (CLC) to the public, which helps platform operators to efficiently manage Kubernetes as part of the overall infrastructure.

Q. As part of its Platinum membership, how will Fujitsu continue to contribute to CNCF?

Kaneshige, Fujitsu: Fujitsu developers will provide insight and experience in supporting enterprise systems and cloud services to the different open source projects hosted by CNCF. And we’d like to make proposals to APIs to enlarge the CNCF ecosystem. Additionally, I will be joining the CNCF’s Governing Board.

Q. Why is CNCF important to the APAC region?

Kaneshige, Fujitsu: CNCF will provide the necessary infrastructure for Internet companies and enterprises to scale their businesses. By nurturing a set of emerging cloud native technologies, CNCF is paving the way for faster code reuse, improved machine efficiency, reduced costs, and increased overall agility and maintainability of applications.

Prometheus User Profile: Metrics Made Easy for ShowMax

By | Blog

Since its inception in 2012, many companies and organizations have adopted Prometheus, the open source monitoring system and time series database. The project has a very active and growing community that counts ShowMax as one of its many users.

Here are some recent stats on the growing Prometheus ecosystem:

  • 37 repositories in its GitHub organization

  • 6502 stars for all repos

  • 5033 stars for the main prometheus/prometheus repo

  • 700+ contributors

ShowMax offers a subscription video on demand service. Launched in South Africa in 2015, it has expanded its footprint to 36 additional countries across sub-Saharan Africa. In total, ShowMax now supplies subscription video on demand services to 65 countries worldwide.

In this blog, originally published on the Prometheus blog, the company shares its experiences evaluating and using Prometheus.

Can you tell us about yourself and what ShowMax does?

I’m Antonin Kral, and I’m leading research and architecture for ShowMax. Before that, I’ve held architectural and CTO roles for the past 12 years.

ShowMax is a subscription video on demand service that launched in South Africa in 2015. We’ve got an extensive content catalogue with more than 20,000 episodes of TV shows and movies. Our service is currently available in 65 countries worldwide. While better known rivals are skirmishing in America and Europe, ShowMax is battling a more difficult problem: how do you binge-watch in a barely connected village in sub-Saharan Africa? Already 35% of video around the world is streamed, but there are still so many places the revolution has left untouched.

ShowMax logo

We are managing about 50 services running mostly on private clusters built around CoreOS. They are primarily handling API requests from our clients (Android, iOS, AppleTV, JavaScript, Samsung TV, LG TV etc), while some of them are used internally. One of the biggest internal pipelines is video encoding which can occupy 400+ physical servers when handling large ingestion batches.

The majority of our back-end services are written in Ruby, Go or Python. We use EventMachine when writing apps in Ruby (Goliath on MRI, Puma on JRuby). Go is typically used in apps that require large throughput and don’t have so much business logic. We’re very happy with Falcon for services written in Python. Data is stored in PostgreSQL and ElasticSearch clusters. We use etcd and custom tooling for configuring Varnishes for routing requests.

What was your pre-Prometheus monitoring experience?

The primary use-cases for monitoring systems are:

  • Active monitoring and probing (via Icinga)

  • Metrics acquisition and creation of alerts based on these metrics (now Prometheus)

  • Log acquisition from backend services

  • Event and log acquisition from apps

The last two use-cases are handled via our logging infrastructure. It consists of a collector running in the service container, which is listening on local Unix socket. The socket is used by apps to send messages to the outside world. Messages are transferred via RabbitMQ servers to consumers. Consumers are custom written or hekad based. One of the main message flows is going towards the service ElasticSearch cluster, which makes logs accessible for Kibana and ad-hoc searches. We also save all processed events to GlusterFS for archival purposes and/or further processing.

We used to run two metric acquisition pipelines in parallel. The first is based on Collectd + StatsD + Graphite + Grafana and the other using Collectd + OpenTSDB. We have struggled considerably with both pipelines. We had to deal with either the I/O hungriness of Graphite, or the complexity and inadequate tooling around OpenTSDB.

Why did you decide to look at Prometheus?

After learning from our problems with the previous monitoring system, we looked for a replacement. Only a few solutions made it to our shortlist. Prometheus was one of the first, as Jiri Brunclik, our head of Operations at the time, had received a personal recommendation about the system from former colleagues at Google.

The proof of concept went great. We got a working system very quickly. We also evaluated InfluxDB as a main system as well as a long-term storage for Prometheus. But due to recent developments, this may no longer be a viable option for us.

How did you transition?

We initially started with LXC containers on one of our service servers, but quickly moved towards a dedicated server from Hetzner, where we host the majority of our services. We’re using PX70-SSD, which is Intel® Xeon® E3-1270 v3 Quad-Core Haswell with 32GB RAM, so we have plenty of power to run Prometheus. SSDs allow us to have retention set to 120 days. Our logging infrastructure is built around getting logs locally (receiving them on Unix socket) and then pushing them towards the various workers.

Diagram of ShowMax logging infrastructure. Shows flow of log messages from the source via processors to various consumers.

Having this infrastructure available made pushing metrics a logical choice (especially in pre-Prometheus times). On the other side, Prometheus is primarily designed around the paradigm of scraping metrics. We wanted to stay consistent and push all metrics towards Prometheus initially. We have created a Go daemon called prometheus-pusher. It’s responsible for scraping metrics from local exporters and pushing them towards the Pushgateway. Pushing metrics has some positive aspects (e.g. simplified service discovery) but also quite a few drawbacks (e.g. making it hard to distinguish between a network partition vs. a crashed service). We made Prometheus-pusher available on GitHub, so you can try it yourself.

Grafana dashboard showing April 5th 2016 log processors traffic.

The next step was for us to figure out what to use for managing dashboards and graphs. We liked the Grafana integration, but didn’t really like how Grafana manages dashboard configurations. We are running Grafana in a Docker container, so any changes should be kept out of the container. Another problem was the lack of change tracking in Grafana.

We have thus decided to write a generator which takes YAML maintained within git and generates JSON configs for Grafana dashboards. It is furthemore able to deploy dashboards to Grafana started in a fresh container without the need for persisting changes made into the container. This provides you with automation, repeatability, and auditing.

We are pleased to announce that this tool is also now available under an Apache 2.0 license on GitHub.

What improvements have you seen since switching?

An improvement which we saw immediately was the stability of Prometheus. We were fighting with stability and scalability of Graphite prior to this, so getting that sorted was a great win for us. Furthemore the speed and stability of Prometheus made access to metrics very easy for developers. Prometheus is really helping us to embrace the DevOps culture.

Tomas Cerevka, one of our backend developers, was testing a new version of the service using JRuby. He needed a quick peek into the heap consumption of that particular service. He was able to get that information in a snap. For us, this speed is essential.

Heap size consumed by JRuby worker during troubleshooting memory issues on JVM.

What do you think the future holds for ShowMax and Prometheus?

Prometheus has become an integral part of monitoring in ShowMax and it is going to be with us for the foreseeable future. We have replaced our whole metric storage with Prometheus, but the ingestion chain remains push based. We are thus thinking about following Prometheus best practices and switching to a pull model.

We’ve also already played with alerts. We want to spend more time on this topic and come up with increasingly sophisticated alert rules.

Why AppFormix, Cisco, Twistlock, and Weaveworks Joined The Cloud Native Computing Foundation

By | Blog

Here’s insights from more of our members AppFormix CEO, Sumeet Singh; Cisco Intercloud Services CTO, Ken Owens; Twistlock Chief Strategy Officer,Chenxi Wang; and Weaveworks COO, Mathew Lodge, on why they joined CNCF and their own work to advance cloud native computing. You’ll find many of them on the road and happy to talk about the future of CNCF.

1. How do you see the CNCF furthering adoption of cloud native?

Singh, AppFormix:  The critical question is how do we make it easier for cloud operators to deliver the level of performance that application developers and end users expect. There are lots of challenges to delivering that in a microservices world. CNCF can be a rallying point for companies and individuals with great ideas on how to give cloud operators the tools they need to succeed in a cloud native world.

Owens, Cisco: CNCF will help to define cloud native (architecture, reference design, taxonomy) and how software patterns are evolving toward cloud. Additionally, CNCF will include a common set of tools to further cloud native adoption.

Wang, Twistlock: CNCF, as an open source foundation, should represent different points of view. We think CNCF will provide guidance to the industry and also help to smooth out disparate views and approaches going forward.

Lodge, Weaveworks: There’s a lot of confusion among users about how to move to cloud native architectures and container-based deployments. The vendor space is very noisy, and it’s hard to figure out how all the pieces fit together. The CNCF has the opportunity to simplify that and accelerate cloud-native adoption.

2. Why did you join CNCF?

Singh, AppFormix: It’s a VM and container world, at least for the foreseeable future. In that vein, our customers need answers to both of those compute philosophies, and they don’t have the resources or patience to manage multiple control interfaces to get there. Achieving this means working with CNCF, OpenStack, Docker, Kubernetes, and Linux communities.

Owens, Cisco: We strongly believe that cloud native is the next data center and service provider operating model. We want to ensure that the current state of the application developer and the problems they face in moving to cloud native are represented. Having developed and deployed cloud native transformations, we bring this working experience to the community.

Wang, Twistlock: We like the mission and we believe it’s important that different companies come together to work on this industry-significant initiative. We also will bring a security voice in the CNCF.

Lodge, Weaveworks: We believe strongly that open source software is a great way to enable customers to go faster while retaining control of their own destiny. Given the large number of vendors in the cloud-native space, the CNCF can be more of a neutral ground for both guidance and running code in the open source projects that are housed within the CNCF. That benefits everyone, and it provides a way for big and small vendors to contribute.

3. What advice would you give other companies looking to join CNCF?

Singh, AppFormix: The reality is that companies join open source projects either because the code and the relationships help them build commercial solutions, or because they consume the code from those projects in their in-house solutions. Either way, they are users who want to engage upstream. Joining CNCF means that you’ve made a strategic decision that the success of your organization relies to one degree or another on having a voice in how microservices architectures are built and deployed at scale.

Owens, Cisco: Companies should consider joining CNCF if it is important to encourage the community to solve the real-world issues they are trying to solve. Additionally, if the access to the community, tools, and lab environment are critical to their business success, they should get involved.

Wang, Twistlock: To be effective, you have to participate and put in the time. If you become part of the CNCF, the biggest thing is be ready to invest your time and be part of the conversation. If you are passionate about something, drive that initiative within CNCF.

Lodge, Weaveworks: For end users, it gives you the opportunity to get in at the ground floor and help shape the future of cloud-native computing and container technologies. For vendors, it’s a way to collaborate and work productively with other vendors to serve bigger end user requirements. That helps grow the overall “pie” of commercial opportunity in a way that also serves the end users.

4. What can people expect to see from your company at upcoming conferences in 2016?

 OpenStack running in Containers on Kubernetes at Openstack Summit

Figure 1: OpenStack running in Containers on Kubernetes at Openstack Summit

Singh, AppFormix: We recently announced at Openstack Summit, how AppFormix and Rackspace are working together to substantially improve the performance of microservice based architectures. Read more about our announcement here.

Owens, Cisco: We will be attending multiple shows over the next several months; including recently spoke at Openstack Summit on containers with OpenStack and CNCF ability to help Openstack, focusing on CNCF and DC/OS early developments at MesosCon, talking about cloud native journey and how CNCF helps at OSCON, and presenting on CNCF scope and charter, as well as provided an overview of cloud native integrations between Cisco and Joyent at Container Summit.

Additionally, we will focus on the cloud native journey and how CFF and CNCF are working together at Cloud Foundry Summit, talking about enterprise cloud native use cases and how OCI and CNCF are working together at DockerCon, discussing K8S and CNCF efforts esp around tools at DevNation, and promoting CNCF awareness and cloud native transformation use cases related to CNCF at ContainerCon.

Cisco Live US is coming up in July and sessions will include cloud native, enterprise, service providers (NFV), and IoT use cases to name just a few. We plan to have a CNCF focus with customers that have joined the foundation.

Wang, Twistlock: We’ll have a number of releases and announcements this year. DockerCon, AWS Re:Invent, Container Summit, and DevOps Enterprise Summit are the main conferences we plan to participate in and will try to align with CNCF opportunities at the shows.

Lodge, Weaveworks: Weaveworks’ Weave Net has seen incredible growth and continues to solve difficult container networking problems in simple ways, and we’re also seeing strong interest in Weave Scope for its intuitive visual approach to developing and running microservices in cloud-native environments. We’ll be showing how customers like ISE and are putting the two together to form a flexible, easy to understand “invisible infrastructure” for cloud-native microservices and applications.

Welcome Prometheus

By | Blog

Hi – my name is Alexis Richardson and I’m the chairman of the Cloud Native Computing Foundation TOC – Technical Oversight Committee.  The TOC is an elected board of nine people.  Representing the interests of CNCF’s members, we define and execute the CNCF’s technology strategy.  I’m also the CEO and co-founder of Weaveworks, a CNCF member company.

Prometheus and the CNCF

Prometheus is high-quality software for monitoring and analysis of cloud native architectures and time series data.  The integration of these features is important for Cloud Native apps, due to the high frequency and volume of instrumentation data in modern architectures.

With today’s announcement that Prometheus is the second project to join the CNCF, I want to talk about what the TOC is doing.  Our doors are open to other projects to apply, and we are actively pursuing projects in specific areas.

The CNCF’s goal is to accelerate customer success with Cloud Native applications.  When it comes to their software decisions, we believe that customers are looking for guidance, clarity and quality.  To that end we aim to:

  1. Increase customer trust in Cloud Native software

  2. Decrease confusion about how to assemble software into real applications

The mechanism for achieving this is to unite high quality and relevant projects into a foundation.  Customers can trust the CNCF to identify and support the very best projects, and, over time, use them to implement a wide range of use cases using Cloud Native architectures.  For a summary of why this mattersplease have a look at this article I wrote on TNS last year.

What projects is CNCF looking for?

In broad terms: right now we want excellent open source projects, that are already up and running, and proven to solve a problem for Cloud Native applications.  Not all projects are suited to the CNCF.  We have brand values and selection criteria, which I spoke about at a recent Linux conference (slides).

Our criteria are: First a project must demonstrate high quality and high velocity; Second, a project must be Cloud Native; and Third, the project must have an affinity with the foundation – i.e. the community of developers and users have to want it.

The Prometheus vote was unanimous, because it exemplifies the type of project that the CNCF TOC is looking for.  To learn more, please read the project proposal and intro slides.  And the community are happy.  The TOC’s unanimous vote shows how the CNCF members can unite and act together decisively for the benefit of the wider Cloud Native community.

What’s next for Prometheus?

Originated at SoundCloud, Prometheus gained adoption in a short time with an impressive spread of end users.  For example at Weaveworks we are very happy Prometheus users: along with Docker, Kubernetes and Terraform it underpins our cloud service.

In addition to customer adoption, Prometheus’ product development continues to move at high speed.  With help from the CNCF, these will now accelerate even further.  Some CNCF members will work on the project to broaden its use cases.  And we intend to raise the profile of the project through demonstrating examples, interoperability and performance.

I’ll leave the last word with the community on

“By joining the CNCF, we hope to establish a clear and sustainable project governance model, as well as benefit from the resources, infrastructure, and advice that the independent foundation provides to its members”

There is much more to come.  Watch this space.

— Alexis