KubeCon + CloudNativeCon + Open Source Summit + AI_dev China 2024

Dragonfly provides efficient, stable and secure file distribution and image acceleration based on P2P technology to be the best practice and standard solution in cloud native architectures. In this talk, there is an introduction to dragonfly and the features of the latest version, and AI model distribution practice in AI inference. Additionally, Ray utilizes Dragonfly as its file distribution solution for the large-scale cluster. Subsequently, we will introduce practical problems of model distribution in LLM and multi-media service, and how Ray solves them in Ant Group's production environment.

Dragonfly提供了基于P2P技术的高效、稳定和安全的文件分发和图像加速，成为云原生架构中的最佳实践和标准解决方案。在本次讨论中，将介绍Dragonfly及最新版本的特性，以及在AI推理中的AI模型分发实践。此外，Ray将Dragonfly作为其大规模集群的文件分发解决方案。随后，我们将介绍LLM和多媒体服务中模型分发的实际问题，以及Ray在蚂蚁集团生产环境中如何解决这些问题。