Guest post by Miao Zhang, Baidu
As CMB’s adoption of cloud native technologies, the Kubernetes container platform is now supporting 100k+ applications in CMB. With such huge number of applications, CMB encountered multiple issues with some popular solutions:
- The configuration reload performance will declined dramatically, and it will cost several minutes for new configuration to take effective.
- Without multi-tenancy support, hundreds of load balancer instances are required to support all the applications on CMB’s container cloud. This increased the total cost of ownership(TCO), including server cost and operation cost.
Also, with applications deployed across multiple Kubernetes clusters and available zones(AZs), CMB also need a solution to handle load balancing and failure resilience across K8s clusters and AZs. With the most popular solution of DNS, it usually takes 8-10 minutes to take effective, which is too long for a major banking industry customer. Also DNS solution cannot support load dispatching at request grain, resulting in workload unbalancing.
“We choose BFE to build the load balancer and overall traffic entrance for our private cloud. BFE has already proved its value after 2 years’ practice at CMB.” ——AiGuo Xiong, Distinguished Talent and Leader of Cloud Computing Project of China Merchants Bank.
After thoroughly evaluation and test of different solutions, in 2020, CMB decided to use BFE to build a unified application load balancer platform and traffic entrance for their private cloud and Kubernetes platform. Up to now, 100+ BFE instances have been deployed for CMB’s load balancer platform.
CMB leverages BFE’s global and distributed zone level load balancing capability to do traffic balancing and failure resilience across AZs and K8s clusters, as illustrated in below picture. BFE’s balancing weight adjustment and failover/failback can take effective within 20 seconds (compared with 8-10 minutes with DNS solutions). Applications’ availability and resource utilization rate are improved.
BFE is also integrated with CMB’s cloud management platform(CMP), thanks to BFE’s multi-tenancy model and fully RESTful API support. Now each application team can publish and configure their application load balancing through CMP. The configurations will take effect on BFE within seconds, with fully automation. Thus, the application can go into production within minutes.
“In the last 2 years, BFE runs stably in an environment of high concurrent, heavy traffic and massive number of applications. It provides a reliable traffic entrance for our cloud native services, together with other advanced capabilities such as precise traffic dispatching, zone level failure resilience.” Says ShengQiang Xie, network SME, China Merchants Bank.
And as it’s a unified load balancing platform with multi tenancy support, CMB’s applications now are sharing the same load balancer infrastructure resources. TCO for load balancer is reduced, major due to less server resource consumption and less operation/maintenance works required.
The journey of BFE in China Merchants Bank is not yet finished. “We are now expanding usage of BFE in CMB. For example, we are implementing BFE for load balancing over our virtual machine environments , and for TLS offloading”, Xie says. With CMB’s go-cloud strategy, we believe we will hear more inspiring stories about BFE and other cloud native technologies.
BFE Project website: https://www.bfe-networks.net/en_us/
Challenges
As China’s leading commercial bank, China Merchants Bank (CMB) has kept excessive investment on information technologies. With a huge number of online applications and services, CMB needs a powerful application load balance platform to support multi-tenancy and self-service, handle massive scale application route rules efficiently and safely, and support traffic balancing and failure resilience across multiple private clouds and Kubernetes clusters.
Solution
After considering several different solutions, CMB decided to build its new generation load balancer platform with BFE. Leveraging BFE’s capability of global and distributed load balancing, CMB achieves traffic balancing and failure resilience between multiple private clouds and Kubernetes clusters. Also, with BFE’s multi-tenancy model and RESTful APIs, it is easy to integrate BFE into cloud management platform(CMP) of CMB and achieve self-serviced, automated application publish.
Impact
As an important building block of CMB’s unified application load balancer platform for all applications in private cloud, BFE has been in production for over two years. In scenarios such as zone level application failures or overloading, BFE improves application availability by reducing the recovery time to less than 20 seconds (comparing to about 8-10 minutes without BFE). Also, operation efficiency is improved as each application team now can publish and configure their own application routing rule within minutes, with fully automation.
By Numbers
Recovery time
<20 seconds for zone level failure, reduced from 8-10 minutes
Scale
100+ BFE instances, supporting 100k+ applications
正文
China Merchants Bank(CMB) is China’s leading commercial bank and listed 24th in 2022 Forbes Global 2000 Companies, with assets over $1,451 billion. CMB is pushing forward the innovation-driven development strategy, accelerating the construction of “Digital Bank”, and developing business agility driven by technology agility. This strategy is also reflected on it’s IT development and evolvement.
“With our steps forward, we have encountered issues and concerns that many solutions cannot address, including disruptions triggered by configuration reload, poor reload performance with 10k+ applications, complicate application route configuration programing, requirement for multi-tenancy support and sufficient proven use cases under heavy traffic”
——AiGuo Xiong, Distinguished Talent and Leader of Cloud Computing Project of China Merchants Bank.