Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
LLM finetuned for medical question answering
| Date | Stars |
|---|---|
| 2026-07-31 | 563 |
| 2026-08-06 | 565 |
Today
+2 stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
# medAlpaca: Finetuned Large Language Models for Medical Question Answering
## Project Overview
MedAlpaca expands upon both [Stanford Alpaca](https://github.com/tatsu-lab/stanford_alpaca) and
[AlpacaLoRA](https://github.com/tloen/alpaca-lora) to offer an advanced suite of large language
models specifically fine-tuned for medical question-answering and dialogue applications.
Our primary objective is to deliver an array of open-source language models, paving the way for
seamless development of medical chatbot solutions.
These models have been trained using a variety of medical texts, encompassing resources such as
medical flashcards, wikis, and dialogue datasets. For more details on the data utilized, please consult the data section.
## Getting Started
Create a new virtual environment, e.g. with conda
```bash
conda create -n medalpaca python>=3.9
```
Install the required packages:
```bash
pip install -r requirements.txt
```
## Training of medAlpaca
<img width="256" alt="training your alpaca" src="https://user-images.githubusercontent.com/37253540/229250535-98f28e1c-0a8e-46e7-9e61-aeb98ef115cc.png">
### Memory Requirements
We have benchmarked the needed GPU memory as well as the approximate duration per epoch
for finetuning LLaMA 7b on the Medical Meadow small dataset (~6000 Q/A pairs) on a single GPU:
| Model | 8bit trainig | LoRA | fp16 | bf16 | VRAM Used | Gradient cktp | Duration/epoch |
|----------|--------------|-------|-------|-------|-----------|---------------|----------------|
| LLaMA 7b | True | True | True | False | 8.9 GB | False | 77:30 |
| LLaMA 7b | False | True | True | False | 18.8 GB | False | 14:30 |
| LLaMA 7b | False | False | True | False | OOM | False | - |
| LLaMA 7b | False | False | False | True | 79.5 GB | True | 35:30 |
| LLaMA 7b | False | False | False | False | OOM | True | - |
### Train medAlpaca based on LLaMA
If you have access to the [LLaMA](https://arxiv.org/abs/2302.13971) or [Alpaca](https://crfm.stanford.edu/2023/03/13/alpaca.html)
weights you can finetune the model with the following command.
Just replace `<PATH_TO_LLAMA_WEIGHTS>` with the folder containing you LLaMA or Alpaca weights.
```bash
python medalpaca/train.py \
--model PATH_TO_LLAMA_WEIGHTS \
--data_path medical_meadow_small.json \
--output_dir 'output' \
--train_in_8bit True \
--use_lora True \
--bf16 True \
--tf32 False \
--fp16 False \
--global_batch_size 128 \
--per_device_batch_size 8 \
```
Per default the script performs mixed precision training.
You can toggle 8bit training with the `train_in_8bit` flag.
While 8 bit training currently only works with `use_lora True`, however you can use
LoRA without 8 bit training.
It is also able to train other models such as `facebook/opt-6.7` with the above script.
## Data
<img width="256" alt="Screenshot 2023-03-31 at 09 37 41" src="https://user-images.githubusercontent.com/37253540/229244284-72b00e82-0da1-4218-b08e-63864306631e.png">
To ensure your cherished llamas and alpacas are well-fed and thriving,
we have diligently gathered high-quality biomedical open-source datasets
and transformed them into instruction tuning formats.
We have dubbed this endeavor **Medical Meadow**.
Medical Meadow currently encompasses roughly 1.5 million data points across a diverse range of tasks,
including openly curated medical data transformed into Q/A pairs with OpenAI's `gpt-3.5-turbo`
and a collection of established NLP tasks in the medical domain.
Please note, that not all data is of the same quantitiy and quality and you may need tp subsample
the data for training your own model.
We will persistently update and refine the dataset, and we welcome Excerpt of 11,798 characters
Read on GitHub
72
4
Weirui Kuang · @alibaba · China
1
Raizy Kellerman
1
1
Would you bet a product on this? Bounded 0–100 and slow moving.
matched fp:73d88012535d7e1f, llm:description: 'LLM finetuned for medical question answering'
matched fp:73d88012535d7e1f, llm:description: 'LLM finetuned for medical question answering'
matched fp:73d88012535d7e1f, llm:description: 'LLM finetuned for medical question answering'