Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
The official implementation of the ICML 2024 paper "MemoryLLM: Towards Self-Updatable Large Language Models" and "M+: Extending MemoryLLM with Scalable Long-Term Memory"
| Date | Stars |
|---|---|
| 2026-07-31 | 318 |
| 2026-08-06 | 318 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
# MemoryLLM & M+ This is the official implementation of paper **MemoryLLM: Towards Self-Updatable Large Language Models** and **M+: Extending MemoryLLM with Scalable Long-Term Memory**. <p align="center" width="100%"> <!-- put the image "memoryllm.png" --> <img src="assets/memoryllm.png" width="80%" height="80%"> </p> ## Official Links [](https://arxiv.org/abs/2402.04624) [](https://arxiv.org/abs/2502.00592) [](https://huggingface.co/YuWangX/memoryllm-7b) [](https://huggingface.co/YuWangX/memoryllm-8b) [](https://huggingface.co/YuWangX/memoryllm-8b-chat) [](https://huggingface.co/YuWangX/mplus-8b) <!-- This is the official code for the paper: **MemoryLLM: Towards Self-Updatable Large Language Models**. The model is open-sourced at https://huggingface.co/YuWangX/memoryllm-7b --> ## Release Notes - [2025/07/27] 🔥 Updated the training code of `mplus-8b` and open-sourced at [mplus-8b-branch](https://github.com/wangyu-ustc/MemoryLLM/tree/mplus). - [2025/02/07] 🔥 The model `mplus-8b` has been uploaded to [mplus-8b](https://huggingface.co/YuWangX/mplus-8b). - [2025/02/01] 🔥 New paper [M+: Extending MemoryLLM with Scalable Long-Term Memory](https://arxiv.org/abs/2502.00592) is on Arxiv! - [2024/08/30] 🔥 We release [memoryllm-8b-chat](https://huggingface.co/YuWangX/memoryllm-8b-chat), the chat model built on top of [memoryllm-8b](https://huggingface.co/YuWangX/memoryllm-8b). - [2024/08/23] 🔥 We release [memoryllm-8b](https://huggingface.co/YuWangX/memoryllm-8b) with 1.67B memory equipped on Llama3! - [2024/06/21] 🔥 Training code is provided in the folder `train`. - [2024/06/02] 🔥 **MemoryLLM** checkpoint is [released](https://huggingface.co/YuWangX/memoryllm-7b)! - [2024/05/02] 🔥 **MemoryLLM** is accepted to ICML 2024! ## Getting Started ### Environment Setup ``` conda create --name memoryllm conda activate memoryllm pip install -r requirements.txt ``` **Note:** In most cases, directly using `requirements.txt` should work well. However, if you encounter any compatibility issues, you can use `requirements_infer_only.txt` which contains locked versions that have been personally tested and verified to work. The testing environment used CUDA version 12.2 with H100-80GB-HBM3 GPUs. ### Load Model First clone the repository and get into the repository: ``` git clone [email protected]:wangyu-ustc/MemoryLLM.git cd MemoryLLM ``` Then to load `MPlus-8B`, please use the following code: ```python import torch from transformers import AutoTokenizer from modeling_mplus import MPlus # load the model mplus-8b (currently we only have the pretrained version) model = MPlus.from_pretrained("YuWangX/mplus-8b", attn_implementation="flash_attention_2", torch_dtype=torch.bfloat16) tokenizer = AutoTokenizer.from_pretrained("YuWangX/mplus-8b") model = model.to(torch.bfloat16) # need to call it again to cast the `inv_freq` in rotary_emb to bfloat16 as well model.put_ltm_to_numpy() # We include ltm as modules so that it can be uploaded to huggingface, but for inference we need to put ltm on CPU and cast ltm_ags to numpy. model = model.cuda() # After this, the usage of MPlus is the same as MemoryLLM-8B, please check "How to use the model" below. ``` To load `MemoryLLM-8B` and `MemoryLLM-8B-chat`, please use the following code: ```python import torch from transformers import AutoTokenizer from modeling_memoryllm import MemoryLLM # load pretrained model model = MemoryLLM.from_pretrained("YuWangX/memoryllm-8b", attn_implementation="flash_attention_2", torch_dtype=torch.bfloat16) tokeni
Excerpt of 10,997 characters
Read on GitHubWould you bet a product on this? Bounded 0–100 and slow moving.
matched fp:f0e9b70c4db9f8a5, desc:long-term memory