SGLang is a high-performance serving framework for large language models and multimodal models.

A high-throughput and memory-efficient inference and serving engine for LLMs

Autonomous coding agent as an SDK, IDE extension, or CLI assistant.

Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.

DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

An open platform for training, serving, and evaluating large language models. Release repo for Vicuna and Chatbot Arena.

MLX: An array framework for Apple silicon

The AI Toolkit for TypeScript. From the creators of Next.js, the AI SDK is a free open-source library for building AI-powered applications and agents

Fast and memory-efficient exact attention

An open-source, code-first Python toolkit for building, evaluating, and deploying sophisticated AI agents with flexibility and control.

Ongoing research training transformer models at scale

NVIDIA® TensorRT™ is an SDK for high-performance deep learning inference on NVIDIA GPUs. This repository contains the open source components of TensorRT.