KubeCon + CloudNativeCon Amsterdam | March 30 – April 2 | Best Pricing Ends February 2 | Learn more

« All Webinars


CNCF Member Webinar: Improving Data Locality for Analytics Jobs on Kubernetes Using Alluxio

@ 10:00 - 11:00 AM PT (UTC-8)
Presented by Alluxio
Register Now

In the on-prem days, one key performance optimization for Apache Hadoop or Apache Spark workloads is to run tasks on nodes with local HDFS data. However, while adoption of the Cloud & Kubernetes makes scaling compute workloads exceptionally easy, HDFS is often not an option. Effectively accessing data from cloud-native storage services like AWS S3 or even on-premises HDFS becomes harder as data locality is lost.

Originated from UC Berkeley AMPLab, the open source project Alluxio approaches this problem in a new way by helping to move data closer to compute workloads efficiently and on-demand, and unify data across multiple or remote clouds, and many more. This webinar will describe the concept and internal mechanism using the stack of Spark+Alluxio in Kubernetes to enhance data locality even when the storage service is outside or remote.

Particularly, we will go over:

  • Why Spark is able to make a locality-aware schedule when working with Alluxio in K8s environment using the host network
  • Why a pod running Alluxio can share data efficiently with a pod running Spark on the same host using domain socket and host path volume
  • The roadmap of Alluxio to further improve running analytics jobs like Spark and Presto, including the on-going closer integration with Presto


Gene Pang, PMC Maintainer @Alluxio

Event Details

Date: January 21, 2020

Time: 10:00 - 11:00 AM PT (UTC-8)

Website: https://zoom.us/webinar/register/3315785951684/WN_Vr...

Get Involved

CNCF webinars are a great way to educate new and existing community members about trends and new technologies. We’re looking for project maintainers, CNCF members, and community experts to share their knowledge. Webinars are non-promotional and focus on education and thought leadership within the cloud native space.

Interested in hosting a CNCF webinar?

Let Us Know