FastChat

What is FastChat?

FastChat is an open-source platform from LMSYS for training, serving and evaluating chatbots built on large language models. It is the software that powers Chatbot Arena, where people compare models side by side and vote, and the README says the Arena has served over 10 million chat requests for 70+ models and gathered more than 1.5M human votes. You install it with pip, then chat with a model from the command line, or run a controller, one or more model workers and a Gradio web server to get a multi-model web interface. It also exposes OpenAI-compatible REST APIs, so existing openai-python code can point at your own models. Supported models include Llama 2, Vicuna, Alpaca, ChatGLM, Falcon, Dolly and many others, and it runs on single or multiple GPUs, CPU, Apple Silicon, Intel XPU and Ascend NPU. The repository also contains the MT-bench evaluation code and Vicuna fine-tuning scripts, and there is no price because you host it yourself.

Use cases

• Serve open LLMs behind a web UI
• Expose models through an OpenAI-compatible API
• Run side-by-side model comparisons
• Benchmark chatbots with MT-bench
• Fine-tune Vicuna on conversation data
• Chat with local models from the terminal
• Host many models with one controller

Who it's for

• ML engineers
• AI researchers
• Developers
• Open-source model enthusiasts

Why use FastChat

• An OpenAI-compatible API lets existing client code work against models you host.
• The same codebase covers training, serving and evaluation, so you do not need to combine separate tools.
• Memory-saving options such as 8-bit loading make it possible to run models on modest hardware.
• You can launch your own Chatbot Arena to compare models side by side.
• Running it yourself keeps data and models under your control and costs only your own compute.

FastChat features

• Command-line chat with supported models
• Distributed serving with a controller, model workers and Gradio web server
• OpenAI-compatible RESTful APIs and SDK support
• Chatbot Arena side-by-side battle interface
• MT-bench multi-turn evaluation with LLM judges
• Vicuna fine-tuning code with multi-turn support
• 8-bit compression and CPU offloading for limited memory
• GPTQ, AWQ and ExLlama V2 quantization support
• vLLM integration for high-throughput serving
• Support for OpenAI, Anthropic, Gemini and Mistral API models

FastChat integrations

• Hugging Face
• OpenAI
• Anthropic
• Gemini
• Mistral
• LangChain
• vLLM
• MLC LLM
• Google Colab
• ModelScope

How to get started with FastChat

1. Install FastChat with pip, using the model_worker and webui extras, or from source.
2. Chat from the terminal with a model from Hugging Face, using the fastchat.serve.cli command.
3. For a web interface, start the controller, then one or more model workers, then the Gradio web server.
4. Optionally start the OpenAI-compatible API server and point your openai-python client at it.
5. Evaluate with MT-bench or fine-tune a model using the training scripts.

FastChat limitations

• You provision GPUs or CPUs yourself, for example about 14GB of GPU memory for Vicuna-7B
• The default Hugging Face worker is compatible but can be slow without vLLM
• Vicuna weights follow Llama's model license
• The ShareGPT training dataset is not released

FastChat pricing

FastChat is open source, so you can use and self-host it for free.

Prices can change and may vary by country or exclude tax. Check the official pricing page.

FastChat alternatives

The main FastChat alternatives listed on Debuly are vLLM, Ollama, LocalAI, llama.cpp, Open WebUI and TextGen (Text Generation Web UI). The table compares pricing and platforms; order is A–Z.

1. 

   llama.cpp

   Open-source engine and desktop app for running LLMs locally, with a built-in OpenAI-compatible API server and web chat UI.

   Developer platformsCodingChat assistants
   Open sourceVisit llama.cpp (official site)
2. 

   LocalAI

   MIT-licensed open-source runtime that serves local models through OpenAI- and Anthropic-compatible APIs for text, images, audio and agents.

   Developer platformsAutomation and agentsChat assistants
   Open sourceVisit LocalAI (official site)
3. 

   Ollama

   Open-source app and CLI for running open AI models locally on macOS, Windows and Linux, with optional paid cloud models.

   Developer platformsCodingChat assistants
   Open sourceVisit Ollama (official site)
4. 

   Open WebUI

   Self-hosted AI chat platform for Ollama and OpenAI-compatible models with RAG, agents, tools, notes, channels and role-based access.

   Chat assistantsDeveloper platformsAutomation and agents
   Open sourceVisit Open WebUI (official site)
5. TT

   TextGen (Text Generation Web UI)

   Open-source desktop and web app for running large language models locally, with chat, vision, tools and an OpenAI-style API.

   Developer platformsChat assistants
   Open sourceVisit TextGen (Text Generation Web UI) (official site)
6. 

   vLLM

   Open-source engine for serving LLMs with PagedAttention, continuous batching and an OpenAI-compatible API.

   Developer platformsCoding
   Open sourceVisit vLLM (official site)

FastChat FAQ

Sources & verification

Facts on this page come from the official sources below, each with the date we read it. Written by Debuly; the vendor did not review this page.

Image credits

• Logo: from avatars.githubusercontent.com · fetched 24 Oct 2026 · Vendor trademark, used only to identify the product; removed on request
• Cover: none collected yet, so a generated tile is shown.

Spot an error? Report an issue with this listing. We also remove a logo or image on the owner's request.