Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
LLM privacy gateway in Go — millisecond-latency PII and secret redaction. Used in production by PackyCode.
| Date | Stars |
|---|---|
| 2026-07-31 | 295 |
| 2026-08-06 | 300 |
Today
+5 stars today
This week
— stars this week
This month
— stars this month
Momentum
5.0
growth rate 0.00%/day
# Privacy Filter (Go)
**English** | [简体中文](README.zh-CN.md)
Strip sensitive user data (PII / secrets) from text before it reaches an LLM.
Pure Go, no model, no GPU, no CGO — a single static binary, millisecond latency on text of any length.
🌐 Running in production at [PackyCode](https://www.packyapi.com) — the privacy-compliance component of an API relay service.
---
## Three ways to use it
1. **Core package**: `import "privacyfilter/filter"` straight into your gateway — redaction is one function call, no HTTP hop.
2. **HTTP service**: `cmd/http`, REST API.
3. **gRPC service**: `cmd/grpc`, interface in `proto/filter.proto`.
The latter two are thin wrappers around the `filter` core package.
---
## Two detection layers
| Layer | Covers | Technique |
|---|---|---|
| Structured PII | Email, phone, national ID, bank card (Luhn-checked), IP | Regex |
| Secrets / credentials | API keys, tokens, private keys, passwords written in prose, unknown high-entropy strings | gitleaks ruleset (keyword pre-filter) + contextual regex + Shannon-entropy fallback |
Each layer emits `(start, end, placeholder)` spans → spans are merged and de-overlapped → the text is rebuilt in a single pass.
Placeholders are typed and carry the entity kind — `[邮箱]` (email), `[电话]` (phone), `[身份证]` (national ID), `[银行卡]` (bank card), `[IP]`, `[密钥]` (secret) — and are irreversible (no un-redaction).
> No person / place / organization name recognition — that needs an NER model, which costs seconds of CPU time on long text and was removed per requirements.
> High-risk identity data (national ID, bank card, secrets, etc.) is fully covered by regex.
---
## Layout
```
privacy-filter/
├── go.mod / go.sum
├── filter/ core package (import directly from a gateway)
│ ├── filter.go Filter / New / Redact
│ ├── pii.go structured PII
│ ├── secrets.go gitleaks + context + entropy
│ └── filter_test.go
├── cmd/
│ ├── http/main.go HTTP service
│ └── grpc/main.go gRPC service
├── proto/filter.proto gRPC interface definition
├── gen/filterpb/ protoc-generated code
├── rules/gitleaks.toml gitleaks ruleset
├── scripts/fetch_rules.sh ruleset update script
└── Dockerfile
```
---
## Build
```bash
go build -o bin/server-http ./cmd/http
go build -o bin/server-grpc ./cmd/grpc
go test ./... # run all tests
```
---
## Usage
### 1. Core package (recommended for gateways)
```go
import "privacyfilter/filter"
// Create once at startup; concurrency-safe, reuse globally.
f, err := filter.New("rules/gitleaks.toml") // pass "" to use the built-in fallback rules
// Per request
res := f.Redact(userPrompt)
forwardToLLM(res.Redacted) // forward the redacted text to the LLM
```
`filter.Result`: `Redacted` (redacted text), `Hit`, `Count`, `Entities` (hit details,
including type and byte offsets).
> To consume this package from your own gateway module: put it in the same monorepo, or add
> `replace privacyfilter => ../privacy-filter` to the gateway's go.mod. The `filter` package
> depends only on `BurntSushi/toml`.
### 2. HTTP service
```bash
./bin/server-http # default :8088
```
```bash
curl http://127.0.0.1:8088/health
curl -X POST http://127.0.0.1:8088/redact -H 'Content-Type: application/json' \
-d '{"text":"我的邮箱是 [email protected]，密码是 Hunter2xy"}'
# {"redacted":"我的邮箱是 [邮箱]，密码是 [密钥]","hit":true,"count":2,"entities":[...],"elapsed_ms":0.08}
```
Endpoints: `GET /health`, `POST /redact`, `POST /redact/batch` (`{"texts":[...]}`).
### 3. gRPC service
```bash
./bin/server-grpc # default :8089
```
Service `filter.v1.PrivacyFilter`, methods `Redact` / `RedactBatch`, defined in `proto/filter.proto`.
Generate a client from that proto on the gateway side. To regenerate the code in this repo:
```bash
protoc -I. --go_out=. --go_opt=module=privacyfilter \
--go-grpc_out=. --go-grpc_opt=module=privacyfilteExcerpt of 5,542 characters
Read on GitHubWould you bet a product on this? Bounded 0–100 and slow moving.
matched fp:7d86e676e9b79654, topic:llm-security, topic:privacy
matched fp:7d86e676e9b79654, topic:ai-gateway