Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Fine-tune Gemma 4 and 3n with audio, images and text on Apple Silicon, using PyTorch and Metal Performance Shaders.
| Date | Stars |
|---|---|
| 2026-07-31 | 1493 |
| 2026-08-03 | 1495 |
| 2026-08-06 | 1495 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
# Gemma Multimodal Fine-Tuner  **Fine-tune Gemma on text, images, *and* audio — on your Mac, on data that doesn't fit on your Mac.** - 🖼️ **Image + text LoRA** — captioning and VQA on local CSV. - 🎙️ **Audio + text LoRA** — Apple-Silicon-native, no CUDA required. - 📝 **Text-only LoRA** — instruction or completion on CSV. - ☁️ **Stream from GCS / BigQuery** — train on terabytes without filling your SSD. - 🍎 **Runs on Apple Silicon** — MPS-native, no NVIDIA box required. **Source:** [github.com/mattmireles/gemma-tuner-multimodal](https://github.com/mattmireles/gemma-tuner-multimodal) (public). --- ## Watch your model learn  Loss curve. Attention heatmap. Gradient signal strength. Memory pressure. Token-by-token predictions — all updating in real time, in your browser, while the model trains on your Mac. No TensorBoard. No notebook. One flag in your config, one URL in your terminal. → [Setup takes 30 seconds](#training-visualizer) --- ## LoRA for Gemma 4 & 3n — why not just use…? | | **This** | MLX-LM | Unsloth | axolotl | | --- | :-: | :-: | :-: | :-: | | Fine-tune Gemma (text-only CSV) | ✅ | ✅ | ✅ | ✅ | | Fine-tune Gemma **image + text** (caption / VQA CSV) | ✅ | ⚠️ varies | ⚠️ varies | ⚠️ varies | | Fine-tune Gemma **audio + text** | ✅ | ❌ | ❌ | ⚠️ CUDA only | | Runs on Apple Silicon (MPS) | ✅ | ✅ | ❌ | ❌ | | **Stream training data from cloud** | ✅ | ❌ | ❌ | ⚠️ partial | | No NVIDIA GPU required | ✅ | ✅ | ❌ | ❌ | Fine-tune Gemma on **text, images, or audio** without renting an H100 or copying a terabyte of data to your laptop. All three modalities run on Apple Silicon. **Text-only fine-tuning** (instruction or completion on CSV) is supported: set `modality = text` in your profile and use local CSV splits under `data/datasets/<name>/`. See [Text-only fine-tuning](#text-only-fine-tuning) below. **Image + text fine-tuning** (captioning or VQA on local CSV) uses `modality = image`, `image_sub_mode`, and `image_token_budget`; see [Image fine-tuning](#image-fine-tuning) below. v1 is **local CSV only** (same constraint as text-only). **How it works:** Hugging Face Gemma checkpoints + PEFT LoRA, supervised fine-tuning in [`gemma_tuner/models/gemma/finetune.py`](gemma_tuner/models/gemma/finetune.py), exported as a merged HF / SafeTensors tree by [`gemma_tuner/scripts/export.py`](gemma_tuner/scripts/export.py). For Core ML conversion and GGUF inference tooling, see [`README/guides/README.md`](README/guides/README.md) — this repo's *training* path is Gemma-only by design. **Deeper reading:** [`README/guides/README.md`](README/guides/README.md) · [`README/specifications/Gemma3n.md`](README/specifications/Gemma3n.md) --- ## What you can build with this - **Domain-specific ASR** — fine-tune on medical dictation, legal depositions, call-center recordings, or any field where off-the-shelf Whisper / Gemma mishears the jargon. - **Domain-specific vision** — captioning or VQA on receipts, charts, screenshots, manufacturing defects, medical imagery — any visual domain where generic models hallucinate. - **Document & screen understanding** — train on screenshot → structured-output pairs for UI agents, OCR-adjacent pipelines, or chart QA. - **Accent, dialect, and low-resource language adaptation** — adapt a base Gemma model to underrepresented voices and languages with your own labeled audio. - **Multimodal assistants** — extend Gemma's text reasoning with image *or* audio grounding for transcription, captioning, and Q&A pipelines. - **Private, on-device pipelines** — train and run entirely on your Mac. Data never leaves the machine; weights never touch a third-party API. If your data lives in GCS or BigQuery, you can do al
Excerpt of 20,632 characters
Read on GitHub
215
1
Would you bet a product on this? Bounded 0–100 and slow moving.
matched fp:b0851e798145671d, name:multimodal