Top AI Repos — open-source AI, indexed and scored

Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.

intel/xFasterTransformer

56Apache-2.0C++

View on GitHub

Stars: 435
Forks: 76
Watchers: 14
Open issues: 20
Contributors: 24

Star history, last 90 days— over 5 snapshots

2026-07-24 · 4352026-08-06 · 435

Star history, last 90 days
Date	Stars
2026-07-24	435
2026-07-25	435
2026-07-28	435
2026-07-30	435
2026-08-06	435

Today

— stars today

This week

— stars this week

This month

— stars this month

Momentum

0.0

growth rate 0.00%/day

From the README

# xFasterTransformer

<p align="center">
  <a href="./README.md">English</a> |
  <a href="./README_CN.md">简体中文</a>
</p>

xFasterTransformer is an exceptionally optimized solution for large language models (LLM) on the X86 platform, which is similar to FasterTransformer on the GPU platform. xFasterTransformer is able to operate in distributed mode across multiple sockets and nodes to support inference on larger models. Additionally, it provides both C++ and Python APIs, spanning from high-level to low-level interfaces, making it easy to adopt and integrate.

## News🔥
-  xFastertransformer supports Qwen3 series models now!
-  [xFastertransformer supports full-power 671B DeepSeek-R1 now! Click for details.](docs/en/deepseek.md)

## Table of Contents
- [xFasterTransformer](#xfastertransformer)
  - [Table of Contents](#table-of-contents)
  - [Models overview](#models-overview)
    - [Model support matrix](#model-support-matrix)
    - [DataType support list](#datatype-support-list)
  - [Documents](#documents)
  - [Installation](#installation)
    - [From PyPI](#from-pypi)
    - [Using Docker](#using-docker)
    - [Built from source](#built-from-source)
      - [Prepare Environment](#prepare-environment)
        - [Manually](#manually)
        - [Install dependent libraries](#install-dependent-libraries)
        - [How to build](#how-to-build)
  - [Models Preparation](#models-preparation)
  - [API usage](#api-usage)
    - [Python API(PyTorch)](#python-apipytorch)
    - [C++ API](#c-api)
  - [How to run](#how-to-run)
    - [Single rank](#single-rank)
    - [Multi ranks](#multi-ranks)
      - [Command line](#command-line)
      - [Code](#code)
        - [Python](#python)
        - [C++](#c)
  - [Web Demo](#web-demo)
  - [Serving](#serving)
    - [vLLM](#vllm)
      - [Install](#install)
      - [OpenAI Compatible Server](#openai-compatible-server)
    - [FastChat](#fastchat)
    - [MLServer](#mlserver)
  - [Benchmark](#benchmark)
  - [Support](#support)
  - [Accepted Papers](#accepted-papers)
  - [Q\&A](#qa)

## Models overview
Large Language Models (LLMs) develops very fast and are more widely used in many AI scenarios. xFasterTransformer is an optimized solution for LLM inference using the mainstream and popular LLM models on Xeon. xFasterTransformer fully leverages the hardware capabilities of Xeon platforms to achieve the high performance and high scalability of LLM inference both on single socket and multiple sockets/multiple nodes.

xFasterTransformer provides a series of APIs, both of C++ and Python, for end users to integrate xFasterTransformer into their own solutions or services directly. Many kinds of example codes are also provided to demonstrate the usage. Benchmark codes and scripts are provided for users to show the performance. Web demos for popular LLM models are also provided.


### Model support matrix

|       Models       | Framework |          | Distribution |
| :----------------: | :-------: | :------: | :----------: |
|                    |  PyTorch  |   C++    |              |
|     DeepSeekR1     | &#10004;  | &#10004; |   &#10004;   |
|     DeepSeekV3     | &#10004;  | &#10004; |   &#10004;   |
|     DeepSeekV2     | &#10004;  | &#10004; |   &#10004;   |
|      ChatGLM       | &#10004;  | &#10004; |   &#10004;   |
|      ChatGLM2      | &#10004;  | &#10004; |   &#10004;   |
|      ChatGLM3      | &#10004;  | &#10004; |   &#10004;   |
|        GLM4        | &#10004;  | &#10004; |   &#10004;   |
|       Llama        | &#10004;  | &#10004; |   &#10004;   |
|       Llama2       | &#10004;  | &#10004; |   &#10004;   |
|       Llama3       | &#10004;  | &#10004; |   &#10004;   |
|     Baichuan       | &#10004;  | &#10004; |   &#10004;   |
|     Baichuan2      | &#10004;  | &#10004; |   &#10004;   |
|        QWen        | &#10004;  | &#10004; |   &#10004;   |
|        QWen2       | &#10004;  | &#10004; |   &#10004;   |
|        QWen3       | &#10004;  | &#10004; |   &#10004;   |
| SecLLM(YaRN-Llama) | &#10004;  | &#

Excerpt of 20,802 characters

Read on GitHub

Top contributors

No contributor data yet — it is fetched during the contributor sync pass.

Quality score

Would you bet a product on this? Bounded 0–100 and slow moving.

56/10056

Maintenance: 20%; How recently the default branch was pushed to
Releases: 10%; Recency and cadence of tagged releases
Community: 80%; Contributor breadth and bus factor
Issue backlog: 60%; Open issues measured against the size of the audience
Documentation: 92%; README depth, plus a homepage and topics
Licensing: 100%; How freely the code can be adopted

What to know before adopting

No commits in 6+ months
No release in over a year
Usable in commercial products with attribution.

Details

Created: 3 years ago
Last push: 11 months ago
Last release: last year
Release tag: v2.1.2
Releases in the last year: 0
Default branch: main
Size: 29k KB
Owner type: Organization
Owner location: —
Discovered via: topic:transformer
First indexed: 13 days ago
Last synced: 12 days ago

intel/xFasterTransformer

56Apache-2.0C++

View on GitHub

Stars: 435
Forks: 76
Watchers: 14
Open issues: 20
Contributors: 24

Star history, last 90 days— over 5 snapshots

2026-07-24 · 4352026-08-06 · 435

Star history, last 90 days
Date	Stars
2026-07-24	435
2026-07-25	435
2026-07-28	435
2026-07-30	435
2026-08-06	435

Today

— stars today

This week

— stars this week

This month

— stars this month

Momentum

0.0

growth rate 0.00%/day

From the README

# xFasterTransformer

<p align="center">
  <a href="./README.md">English</a> |
  <a href="./README_CN.md">简体中文</a>
</p>

xFasterTransformer is an exceptionally optimized solution for large language models (LLM) on the X86 platform, which is similar to FasterTransformer on the GPU platform. xFasterTransformer is able to operate in distributed mode across multiple sockets and nodes to support inference on larger models. Additionally, it provides both C++ and Python APIs, spanning from high-level to low-level interfaces, making it easy to adopt and integrate.

## News🔥
-  xFastertransformer supports Qwen3 series models now!
-  [xFastertransformer supports full-power 671B DeepSeek-R1 now! Click for details.](docs/en/deepseek.md)

## Table of Contents
- [xFasterTransformer](#xfastertransformer)
  - [Table of Contents](#table-of-contents)
  - [Models overview](#models-overview)
    - [Model support matrix](#model-support-matrix)
    - [DataType support list](#datatype-support-list)
  - [Documents](#documents)
  - [Installation](#installation)
    - [From PyPI](#from-pypi)
    - [Using Docker](#using-docker)
    - [Built from source](#built-from-source)
      - [Prepare Environment](#prepare-environment)
        - [Manually](#manually)
        - [Install dependent libraries](#install-dependent-libraries)
        - [How to build](#how-to-build)
  - [Models Preparation](#models-preparation)
  - [API usage](#api-usage)
    - [Python API(PyTorch)](#python-apipytorch)
    - [C++ API](#c-api)
  - [How to run](#how-to-run)
    - [Single rank](#single-rank)
    - [Multi ranks](#multi-ranks)
      - [Command line](#command-line)
      - [Code](#code)
        - [Python](#python)
        - [C++](#c)
  - [Web Demo](#web-demo)
  - [Serving](#serving)
    - [vLLM](#vllm)
      - [Install](#install)
      - [OpenAI Compatible Server](#openai-compatible-server)
    - [FastChat](#fastchat)
    - [MLServer](#mlserver)
  - [Benchmark](#benchmark)
  - [Support](#support)
  - [Accepted Papers](#accepted-papers)
  - [Q\&A](#qa)

## Models overview
Large Language Models (LLMs) develops very fast and are more widely used in many AI scenarios. xFasterTransformer is an optimized solution for LLM inference using the mainstream and popular LLM models on Xeon. xFasterTransformer fully leverages the hardware capabilities of Xeon platforms to achieve the high performance and high scalability of LLM inference both on single socket and multiple sockets/multiple nodes.

xFasterTransformer provides a series of APIs, both of C++ and Python, for end users to integrate xFasterTransformer into their own solutions or services directly. Many kinds of example codes are also provided to demonstrate the usage. Benchmark codes and scripts are provided for users to show the performance. Web demos for popular LLM models are also provided.


### Model support matrix

|       Models       | Framework |          | Distribution |
| :----------------: | :-------: | :------: | :----------: |
|                    |  PyTorch  |   C++    |              |
|     DeepSeekR1     | &#10004;  | &#10004; |   &#10004;   |
|     DeepSeekV3     | &#10004;  | &#10004; |   &#10004;   |
|     DeepSeekV2     | &#10004;  | &#10004; |   &#10004;   |
|      ChatGLM       | &#10004;  | &#10004; |   &#10004;   |
|      ChatGLM2      | &#10004;  | &#10004; |   &#10004;   |
|      ChatGLM3      | &#10004;  | &#10004; |   &#10004;   |
|        GLM4        | &#10004;  | &#10004; |   &#10004;   |
|       Llama        | &#10004;  | &#10004; |   &#10004;   |
|       Llama2       | &#10004;  | &#10004; |   &#10004;   |
|       Llama3       | &#10004;  | &#10004; |   &#10004;   |
|     Baichuan       | &#10004;  | &#10004; |   &#10004;   |
|     Baichuan2      | &#10004;  | &#10004; |   &#10004;   |
|        QWen        | &#10004;  | &#10004; |   &#10004;   |
|        QWen2       | &#10004;  | &#10004; |   &#10004;   |
|        QWen3       | &#10004;  | &#10004; |   &#10004;   |
| SecLLM(YaRN-Llama) | &#10004;  | &#

Excerpt of 20,802 characters

Read on GitHub

Top contributors

No contributor data yet — it is fetched during the contributor sync pass.

Quality score

Would you bet a product on this? Bounded 0–100 and slow moving.

56/10056

Maintenance: 20%; How recently the default branch was pushed to
Releases: 10%; Recency and cadence of tagged releases
Community: 80%; Contributor breadth and bus factor
Issue backlog: 60%; Open issues measured against the size of the audience
Documentation: 92%; README depth, plus a homepage and topics
Licensing: 100%; How freely the code can be adopted

What to know before adopting

No commits in 6+ months
No release in over a year
Usable in commercial products with attribution.

Details

Created: 3 years ago
Last push: 11 months ago
Last release: last year
Release tag: v2.1.2
Releases in the last year: 0
Default branch: main
Size: 29k KB
Owner type: Organization
Owner location: —
Discovered via: topic:transformer
First indexed: 13 days ago
Last synced: 12 days ago

Top AI Repos — open-source AI, indexed and scored

intel/xFasterTransformer

From the README

Top contributors

Quality score

What to know before adopting

Categories

Details

intel/xFasterTransformer

From the README

Top contributors

Quality score

What to know before adopting

Categories

Details

Top AI Repos — open-source AI, indexed and scored

Trend

From the README

Top contributors

Quality score

What to know before adopting

Categories

Details

Trend

From the README

Top contributors

Quality score

What to know before adopting

Categories

Details