KubeCon + CloudNativeCon + Open Source Summit + AI_dev China 2024

Large AI models are driving significant investment in GPU clusters. Yet, managing these clusters is hard: Slurm-based HPC setups lack of management granularity and stability, while Kubernetes poses usability challenges for AI users. This talk introduces TACC, an AI infra management solution that bridges the advantages of both K8S and Slurm setups. This is a joint-work from computer system researchers at HKUST and leading CNCF contributors at DaoCloud. TACC manages a large-scale cluster at HKUST that supports over 500 active researchers since 2020. In this talk, we share our five-year journey with TACC, covering: * [User Experience] A seamless UI for job submissions and management, supporting both container and Slurm format, all on the same backbone * [Resource Management] Multi-tenant allocation with configurable strategies, using CNCF HAMi and Kueue * [Performance and Scalability] A robust distributed infrastructure with networked storage and RDMA, via CNCF SpiderPool,Fluid...

大型AI模型正在推动GPU集群的重大投资。然而，管理这些集群很困难：基于Slurm的HPC设置缺乏管理粒度和稳定性，而Kubernetes对AI用户存在可用性挑战。 本次演讲介绍了TACC，这是一种AI基础设施管理解决方案，可以结合K8S和Slurm设置的优势。这是香港科技大学的计算机系统研究人员与DaoCloud领先的CNCF贡献者共同合作的成果。 TACC自2020年以来管理着香港科技大学支持超过500名活跃研究人员的大规模集群。在本次演讲中，我们分享了与TACC一起的五年历程，涵盖以下内容： * [用户体验] 无缝的UI界面用于作业提交和管理，支持容器和Slurm格式，均在同一基础上 * [资源管理] 多租户分配与可配置策略，使用CNCF HAMi和Kueue * [性能和可扩展性] 强大的分布式基础设施，具有网络存储和RDMA，通过CNCF SpiderPool，Fluid...