KubeCon + CloudNativeCon + Open Source Summit + AI_dev China 2024

The rapid advancement of artificial intelligence technologies, especially the development of large language models (LLMs), has led to a sharp increase in the amount of data that enterprises need to process. Managing large-scale data clusters in Kubernetes environments presents several challenges, including storage performance, complex access control management and system stability. JuiceFS is a distributed POSIX file system designed for cloud. It was open-sourced in 2021( 9.8k stars) To deliver an optimal experience in Kubernetes, JuiceFS developed JuiceFS CSI Driver. In addition, JuiceFS CSI introduced several new designs to support large-scale, complex AI training tasks such as the mount pod mode and the sidecar mode for serverless environments. Outline: - LLM Storage challenges - JuiceFS CSI Driver Architectural - Mount pod mode\Sidecar mode - Practical experience - Future

人工智能技术的快速发展，特别是大型语言模型（LLMs）的发展，导致企业需要处理的数据量急剧增加。在Kubernetes环境中管理大规模数据集群面临着多个挑战，包括存储性能、复杂的访问控制管理和系统稳定性。 JuiceFS是一种为云设计的分布式POSIX文件系统。它于2021年开源（拥有9.8k星）。为了在Kubernetes中提供最佳体验，JuiceFS开发了JuiceFS CSI驱动程序。此外，JuiceFS CSI引入了几项新设计，以支持大规模、复杂的人工智能训练任务，如挂载Pod模式和用于无服务器环境的Sidecar模式。 大纲： - LLM存储挑战 - JuiceFS CSI驱动程序架构 - 挂载Pod模式\Sidecar模式 - 实践经验 - 未来