A high-throughput and memory-efficient inference and serving engine for LLMs

An open platform for training, serving, and evaluating large language models. Release repo for Vicuna and Chatbot Arena.

DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.

The official Meta Llama 3 GitHub site

The official Python library for the OpenAI API

MLX: An array framework for Apple silicon

🤗 The largest hub of ready-to-use datasets for ML models with fast, easy-to-use and efficient data manipulation tools

The official repo of Qwen (通义千问) chat & pretrained large language model proposed by Alibaba Cloud.

Fast and memory-efficient exact attention

A scalable generative AI framework built for researchers and developers working on Large Language Models, Multimodal, and Speech AI (Automatic Speech Recognition and Text-to-Speech)

SGLang is a fast serving framework for large language models and vision language models.

Ongoing research training transformer models at scale

SkyPilot: Run AI and batch jobs on any infra (Kubernetes or 15+ clouds). Get unified execution, cost savings, and high GPU availability via a simple interface.