The CNCF Technical Oversight Committee (TOC) has voted to accept CubeFS as a CNCF incubating project.
CubeFS is a cloud native distributed storage platform applied as the storage infrastructure for online applications orchestrated by Kubernetes, databases, and machine learning platforms. It adopts the storage-computing separation architecture, provides elastic and scalable metadata services, supports two storage engines, multi-copy and erasure code, and is compatible with three access protocols: S3, POSIX, and HDFS.
CubeFS was open sourced and released as version 1.0.0 in March 2019. At the end of 2019, CubeFS entered the CNCF sandbox. CubeFS community has maintainers from several companies and over 90 developers, including cloud service providers and end users, contributing to the project. It is used in hybrid cloud, cloud native data lake, machine learning, online services, and other environments.
The project is used in production by more than 10 companies, including OPPO, JD.com, NetEase, and Shell, as well as some high-tech start-ups.
“At NetEase, CubeFS has solved the problem of poor scalability and handling capacity caused by the separation of computing and storage in ElasticSeach and usage of local storage in the machine learning business,” said Zcola Zhang, senior director of operations at NetEase. “The project continues to attract us for its other features, including EC and cache acceleration.”
“Using CubeFS as the underlying storage effectively improves data management efficiency and reduces our operating costs,” said Cartman, general manager of Intelligent Cloud at OPPO.
Main Components
- Resource management subsystem: manages the survival status of data and metadata nodes and creates and maintains volume and partition information.
- Metadata subsystem: composed of one or more metadata shards around memory. It enables high data availability and consistency by using MultiRaft and supports horizontal expansion by splitting.
- Data storage subsystem: provides multi-copy and erasure code storage compatible with performance and cost requirements of different business scenarios.
- Multi-protocol clients: provide access to different file system instances and external read, write, and delete functions.
Notable Milestones
- 2,600+ GitHub Stars
- 1,000+ Pull requests
- 400+ issues
- 94 contributors from 10 companies
- 28 Releases
CubeFS has a clear roadmap for adding new features and functions. The team will continue to improve the QoS function to enhance the multi-tenant isolation capability and the easy-to-use operation and maintenance tools box. It is also promoting the research and development of advanced features such as hybrid cloud acceleration, multi-version and snapshot, and lifecycle management.
As a CNCF-hosted project, CubeFS joins 35 other incubating technologies as part of a neutral foundation aligned with its technical interests and the Linux Foundation, which provides governance, marketing support, and community outreach. For more information on maturity requirements for each level, please visit the CNCF Graduation Criteria.