Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
WebGLM: An Efficient Web-enhanced Question Answering System (KDD 2023)
| Date | Stars |
|---|---|
| 2026-07-24 | 1602 |
| 2026-07-25 | 1602 |
| 2026-07-28 | 1602 |
| 2026-07-30 | 1602 |
| 2026-08-06 | 1602 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
<h1>WebGLM: Towards An Efficient Web-enhanced Question Answering System with Human Preferences</h1>
<p align="center">📃 <a href="https://arxiv.org/pdf/2306.07906.pdf" target="_blank">Paper (KDD'23)</a> • 🌐 <a href="https://github.com/THUDM/WebGLM/blob/main/README_zh.md" target="_blank">中文 README</a> • 🤗 HF Repo <a href="https://huggingface.co/THUDM/WebGLM" target="_blank">[WebGLM-10B]</a> <a href="https://huggingface.co/THUDM/WebGLM-2B" target="_blank">[WebGLM-2B]</a> • 📚 Dataset <a href="https://huggingface.co/datasets/THUDM/webglm-qa" target="_blank">[WebGLM-QA]</a></p>
This is the official implementation of WebGLM. If you find our open-sourced efforts useful, please 🌟 the repo to encourage our following developement!
<!--https://github.com/THUDM/WebGLM/assets/129033897/d2e1dd35-6340-4175-ac2d-fd585daa17cf-->
**[Please click to watch the demo!]**
[](https://www.youtube.com/watch?v=ohjrlYCLLEU)
_Read this in [中文](README_zh.md)._
## Update
**[2023/06/25]** Release [ChatGLM2-6B](https://github.com/THUDM/ChatGLM2-6B), an updated version of [ChatGLM-6B](https://github.com/THUDM/ChatGLM-6B) which introduces several new features:
1. **Stronger Performance**: we have fully upgraded the ChatGLM2-6B. It uses the hybrid objective function of [GLM](https://github.com/THUDM/GLM), and has undergone pre-training with 1.4T bilingual tokens and human preference alignment training. The [evaluation results](README.md#evaluation-results) show that, compared to the first-generation model, ChatGLM2-6B has achieved substantial improvements in performance on datasets like MMLU (+23%), CEval (+33%), GSM8K (+571%), BBH (+60%), showing strong competitiveness among models of the same size.
2. **Longer Context**: Based on [FlashAttention](https://github.com/HazyResearch/flash-attention) technique, we have extended the context length of the base model from 2K in ChatGLM-6B to 32K, and trained with a context length of 8K during the dialogue alignment, allowing for more rounds of dialogue. However, the current version of ChatGLM2-6B has limited understanding of single-round ultra-long documents, which we will focus on optimizing in future iterations.
3. **More Efficient Inference**: Based on [Multi-Query Attention](http://arxiv.org/abs/1911.02150) technique, ChatGLM2-6B has more efficient inference speed and lower GPU memory usage: under the official implementation, the inference speed has increased by 42% compared to the first generation; under INT4 quantization, the dialogue length supported by 6G GPU memory has increased from 1K to 8K.
More details please refer to [ChatGLM2-6B](https://github.com/THUDM/ChatGLM2-6B)。
<!-- TOC -->
- [Overview](#overview)
- [Features](#features)
- [News](#news)
- [Preparation](#preparation)
- [Prepare Code and Environments](#prepare-code-and-environments)
- [Prepare SerpAPI Key](#prepare-serpapi-key)
- [Prepare Retriever Checkpoint](#prepare-retriever-checkpoint)
- [Try WebGLM](#try-webglm)
- [Export Environment Variables](#export-environment-variables)
- [Run as Command Line Interface](#run-as-command-line-interface)
- [Run as Web Service](#run-as-web-service)
- [Train WebGLM](#train-webglm)
- [Train Generator](#train-generator)
- [Prepare Data](#prepare-data)
- [Training](#training)
- [Train Retriever](#train-retriever)
- [Prepare Data](#prepare-data-1)
- [Training](#training-1)
- [Evaluation](#evaluation)
- [Real Application Cases](#real-application-cases)
- [Citation](#citation)
# Overview
WebGLM aspires to provide an efficient and cost-effective web-enhanced question-answering system using the 10-billion-parameter General Language Model (GLM). It aims to improve real-world application deployment by integrating web search and retrieval capabilities into the pre-trained language model.
## FeatExcerpt of 10,119 characters
Read on GitHubWould you bet a product on this? Bounded 0–100 and slow moving.
matched fp:68ff3f2e6f2d954c, topic:rlhf, readme:alignment
matched fp:68ff3f2e6f2d954c, topic:llm, readme:billion parameter
matched fp:68ff3f2e6f2d954c, topic:chatgpt
