FastChat (39,515 Stars) 2026: Train, Serve and Evaluate LLMs with the Open Platform

FastChat (39,515 stars) is the open platform for training, serving and evaluating LLMs - including the Vicuna models and a high-performance serving engine. Here is the full guide.

💡 What You Will Learn

FastChat (39,515 stars) is the open platform for training, serving and evaluating LLMs - including the Vicuna models and a high-performance serving engine. Here is the full guide.

## The short answer **lm-sys/FastChat** (39,515 stars, Python) is an open platform from LMSYS (the team behind Chatbot Arena) for training, serving, and evaluating large language models. It includes the Vicuna model family, an OpenAI-compatible serving engine, and evaluation tools - all in one codebase. ## What you get - **Models**: Vicuna and other fine-tuned chat models - **Serving**: high-throughput inference with the FastChat server - **Evaluation**: pipeline for comparing model outputs (like arena-style) - **Training**: scripts and data for fine-tuning chat models - **Web UI**: a ChatGPT-like interface for demos ## Serve a model with the OpenAI-compatible API ```bash pip install fschat # Serve with the CLI python -m fastchat.serve.cli --model-path lmsys/vicuna-7b-v1.5 # Or start the API server (OpenAI-compatible) python -m fastchat.serve.openai_api_server \n --model-path lmsys/vicuna-7b-v1.5 --port 8000 ``` Then use any OpenAI client: ```python from openai import OpenAI client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY") response = client.chat.completions.create( model="vicuna-7b-v1.5", messages=[{"role": "user", "content": "Hello"}], ) print(response.choices[0].message.content) ``` ## Multi-model serving with workers For production, run one controller + multiple model workers to serve several models behind one API endpoint, with load balancing. ## Practical tips - Use the controller/worker architecture for serving multiple models at scale. - Use the evaluation pipeline to compare fine-tunes before deployment. - Vicuna checkpoints require the base Llama weights for full functionality. ## FAQ **Is it free?** Yes - Apache-2.0 open source (model licenses vary). **Who maintains it?** LMSYS Org - the team behind Chatbot Arena (lmarena.ai). **Can I use it with GPUs?** Yes - it is optimized for GPU inference and supports batching for throughput.