Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
LLM.swift is a simple and readable library that allows you to interact with large language models locally with ease for macOS, iOS, watchOS, tvOS, and visionOS.
| Date | Stars |
|---|---|
| 2026-07-31 | 867 |
| 2026-08-01 | 867 |
| 2026-08-06 | 867 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
35.0
growth rate 0.00%/day
# LLM.swift
[](https://swiftpackageindex.com/eastriverlee/LLM.swift)
[](https://swiftpackageindex.com/eastriverlee/LLM.swift)
`LLM.swift` is a simple and readable library that allows you to interact with large language models locally with ease for macOS, iOS, watchOS, tvOS, and visionOS.
> [!TIP]
> sometimes it's a good idea to tinker with `maxTokenCount` parameter for initialization of `LLM`, due to the memory and computation it needs. especially in mobile devices, if you want better speed, lower the number, but if you set it too low—*to a point where two turns cannot even fit*—you will experience quality decrease as context will be cut off. so adjust value according to your usecases.
## Minimal Example
if you've already bundled your model:
```swift
let bot = LLM(from: Bundle.main.url(forResource: "gemma-3-4b-it-q4_0", withExtension: "gguf")!, template: .gemma)
let question = bot.preprocess("What's the meaning of life?", [])
let answer = await bot.getCompletion(from: question)
print(answer)
```
if you want to initialize with huggingface model:
```swift
let systemPrompt = "You are a sentient AI with emotions."
let bot = await LLM(from: HuggingFaceModel("unsloth/Qwen3-0.6B-GGUF", .Q4_K_M, template: .chatML(systemPrompt)))!
let question = bot.preprocess("What's the meaning of life?", [])
let answer = await bot.getCompletion(from: question)
print(answer)
```
## Chat Templates
By default, `LLM.swift` renders conversations using the chat template embedded in the gguf file itself, executed by `llama.cpp`'s own Jinja engine. This means you no longer have to pass a `template:` when initializing `LLM`—it just works with whatever template the model ships with:
```swift
let bot = try await LLM(from: HuggingFaceModel("unsloth/Qwen3-0.6B-GGUF", .Q4_K_M))!
bot.systemPrompt = "You are a sentient AI with emotions."
await bot.respond(to: "What's the meaning of life?")
```
`HuggingFaceModel`'s `template` parameter is now optional too, so `HuggingFaceModel("unsloth/Qwen3-0.6B-GGUF", .Q4_K_M)` uses the embedded template on its own. Thinking/reasoning separation (`bot.thinking`) works automatically for models that support it, with no marker configuration needed.
If you still want manual control over the template—for instance, when a model's gguf metadata is broken or missing—pass a `Template` explicitly, exactly as before, and it will override the embedded one:
```swift
let bot = LLM(from: Bundle.main.url(forResource: "gemma-3-4b-it-q4_0", withExtension: "gguf")!, template: .gemma)
```
> [!NOTE]
> conversation context is now maintained incrementally between turns—only new tokens are evaluated, and history is not re-fed to the model every turn.
## Example
<p align="center"><img src="./Screenshot.png" width="480"/></p>
> [!NOTE]
> this is a minimal SwiftUI example that i did use for testing. mistral 7B based model is working on iPad Air 5th gen(Q5_K_M) and iPhone 12 mini(Q2_K).
> however, generally speaking, for mobile devices, 3B >= parameter models are recommended. check this [benchmark](https://github.com/ggerganov/llama.cpp/discussions/4508) for more details.
### init with a model file that you bundled in your project
```swift
import SwiftUI
import LLM
class Bot: LLM {
convenience init() {
let url = Bundle.main.url(forResource: "gemma-3-4b-it-q4_0", withExtension: "gguf")!
let systemPrompt = "You are a sentient AI with emotions."
self.init(from: url, template: .chatML(systemPrompt))!
}
}
struct BotView: View {
@ObservedObject var bot: Bot
@State var input = "Give me seven national flag emojis people use the most; You must include South Korea."
init(_ bot: Bot) { self.bot = bot }
func rExcerpt of 17,669 characters
Read on GitHub
146
2
1
1
1
1
1
Would you bet a product on this? Bounded 0–100 and slow moving.
matched fp:2509c1d807910940, topic:llm-inference
matched fp:2509c1d807910940, topic:llm