KubeCon + CloudNativeCon + Open Source Summit + AI_dev China 2024

As an Open Source veteran who've been working on secure container technology (Kata Containers), the speaker has been crafting Open Source governance and strategies for projects for years. The team joined Ant Group 5 years ago and was continuously focusing on Cloud Native and Trust technologies. In 2023, the speaker was appointed to assume the role of Vice President of Open Source Technical Oversight Committee for Ant Group.The TOC job requires setting up open source strategy and growth tactics, but now for a company with 25K employees and 13K engineers. It turned out that the experience leading a top level project was immensely valuable for the new position. In this session, we'll share first hand experiences for a tech leader to wear multiple hats of tech director, open source leader, and the go-to person for OSS strategies for a large corporation, and the learnings / reflections coming from the new challenges.

作为一位开源资深人士，演讲者一直致力于安全容器技术（Kata Containers），并多年来一直在为项目制定开源治理和战略。团队于5年前加入蚂蚁集团，一直专注于云原生和信任技术。在2023年，演讲者被任命为蚂蚁集团开源技术监督委员会副主席。TOC的工作需要制定开源战略和增长策略，但现在是为一个拥有25,000名员工和13,000名工程师的公司。事实证明，领导一个顶级项目的经验对新职位非常有价值。在这场演讲上，我们将分享一个技术领导者如何在大公司中扮演技术总监、开源领导者和开源战略的权威人士等多重角色的第一手经验，以及从新挑战中获得的经验和反思。

Are you struggling with slow deployment times, high operational costs, or scalability issues when serving your ML models? Now, imagine the added complexity when typical AI apps require not just one, but an interconnected suite of models. In this session, discover how the integration of BentoML with Dragonfly effectively addresses these challenges, transforming the landscape of multi-model composition and inference within serverless Kubernetes envs. Join the co-presentation by the BentoML and Dragonfly communities to explore a compelling case study: a RAG app that combines 3 models—LLM, embedding, and OCR. Learn how our framework not only packages these diverse models efficiently but also utilizes Dragonfly's innovative P2P network for swift distribution. We'll further delve into how other open-source technologies like JuiceFS and VLLM have enabled us to achieve remarkable deployment times of just 40 seconds and establish a scalable blueprint for multi-model composition deployments.

您是否在为机器学习模型的部署时间慢、运营成本高或可扩展性问题而苦恼？现在，想象一下当典型的人工智能应用程序不仅需要一个模型，而是一个相互连接的模型套件时所增加的复杂性。在本场演讲中，了解BentoML与Dragonfly的集成如何有效解决这些挑战，改变了无服务器Kubernetes环境中多模型组合和推理的格局。 加入BentoML和Dragonfly社区的联合演示，探索一个引人注目的案例研究：一个结合了LLM、嵌入和OCR三个模型的RAG应用程序。了解我们的框架不仅高效打包这些多样化的模型，还利用Dragonfly创新的P2P网络进行快速分发。我们还将深入探讨其他开源技术，如JuiceFS和VLLM，如何帮助我们实现仅需40秒的部署时间，并为多模型组合部署建立可扩展的蓝图。

Cluster API, designed with extensibility at its core, has revolutionized Kubernetes cluster management. Its open and pluggable architecture empowers providers to implement custom solutions tailored to their unique requirements. In this session, we will explore how Cluster API's extension-by-design philosophy has opened new horizons for organizations seeking to create bespoke Kubernetes clusters. Managing Kubernetes clusters at scale presents unique operational challenges that cannot be tamed with manual operations. Through real-world examples and lessons learned, we will demonstrate how Cluster API's flexibility allows for the integration of diverse infrastructure providers and the implementation of organization-specific customizations. Attendees will gain insights into best practices for extending Cluster API, including developing custom controllers, integrating third-party tools, and creating bespoke workflows.

Cluster API是以可扩展性为核心设计的，已经彻底改变了Kubernetes集群管理。其开放和可插拔的架构赋予提供者实施定制解决方案的能力，以满足其独特需求。在本场演讲中，我们将探讨Cluster API的“通过设计进行扩展”的理念如何为寻求创建定制化Kubernetes集群的组织开辟了新的视野。 在规模化管理Kubernetes集群时，会面临无法通过手动操作解决的独特运营挑战。 通过现实世界的例子和经验教训，我们将演示Cluster API的灵活性如何允许集成各种基础设施提供者，并实施组织特定的定制化。与会者将获得有关扩展Cluster API的最佳实践的见解，包括开发自定义控制器、集成第三方工具和创建定制工作流程。

Explore China Merchants Bank's (CMB), one of China's largest retail banks, transformative journey through cloud migration, cloud-native transformation, and platform engineering over the past three years. Despite challenges such as increased complexity in cloud technology and management, and potential risks to developer productivity and continuous assurance of financial services, CMB successfully leveraged KubeVela, OpenFeature, Envoy, Clilum, and OpenTelemetry to build the Next-Gen FinTech IDP. This led to the management of 70% of applications within a year and improved developer experience, covering thousands of R&D engineers. We'll discuss the strategic thinking, 'Golden Path' implementation, struggles, trade-offs, and key success metrics with platform engineering maturity model. This session provides a blueprint and reference architecture for financial organizations undergoing similar transformations.

在KubeCon的会议描述中，探索中国招商银行（CMB）作为中国最大的零售银行之一，在过去三年中通过云迁移、云原生转型和平台工程的变革之旅。尽管面临诸如云技术和管理复杂性增加、开发人员生产力和金融服务持续保障的潜在风险等挑战，CMB成功利用KubeVela、OpenFeature、Envoy、Clilum和OpenTelemetry构建了下一代金融科技IDP。这导致了一年内管理了70%的应用程序，并改善了开发人员体验，涵盖了数千名研发工程师。我们将讨论战略思维、“黄金路径”实施、挣扎、权衡和关键成功指标，以及平台工程成熟度模型。本场演讲提供了金融机构进行类似转型的蓝图和参考架构。

With the rise of generative AI technology, more and more applications are starting to integrate with the capabilities of generative AI. However, the high costs of training and inference can be daunting for developers. In this talk, we will discuss the issues and solutions that need additional consideration when using elastic scaling in generative AI scenarios, including: ● How to enhance the elastic startup efficiency of generative AI ● How to address the efficiency of inference when separating compute and storage in generative AI ● How to reduce the costs of training and inference ● How to solve the interruption problem in AI training scenarios using Spot instances ● How to address the issue of capacity elasticity in LLM scenarios Finally, we will introduce the practical experience of the world's leading generative AI service provider: HaiYi (seaart.ai), allowing more developers to understand the architectural methods of elastic cloud AI infrastructure.

随着生成式人工智能技术的兴起，越来越多的应用程序开始与生成式人工智能的能力集成。然而，训练和推理的高成本可能会让开发人员望而却步。在这次演讲中，我们将讨论在生成式人工智能场景中使用弹性扩展时需要额外考虑的问题和解决方案，包括： ● 如何提高生成式人工智能的弹性启动效率 ● 如何在生成式人工智能中分离计算和存储时解决推理效率的问题 ● 如何降低训练和推理的成本 ● 如何使用Spot实例解决AI训练场景中的中断问题 ● 如何解决LLM场景中的容量弹性问题 最后，我们将介绍世界领先的生成式人工智能服务提供商海艺（seaart.ai）的实际经验，让更多开发人员了解弹性云AI基础设施的架构方法。