Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
NLP transformers written in Go
| Date | Stars |
|---|---|
| 2026-07-24 | 256 |
| 2026-07-25 | 256 |
| 2026-07-28 | 256 |
| 2026-07-30 | 256 |
| 2026-08-06 | 256 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
# Transformer [](https://opensource.org/licenses/Apache-2.0)[](https://pkg.go.dev/github.com/sugarme/transformer?tab=doc)[](https://travis-ci.org/sugarme/transformer)[](https://goreportcard.com/report/github.com/sugarme/transformer)
## Overview
`transformer` is pure Go package to facilitate applying Natural Language Processing (NLP) models train/test and inference in Go.
This package is in active mode of building and there are many changes ahead. Hence you can use it with your complete own risk. The package will be considered as stable when version 1.0 is released.
`transformer` is heavily inspired by and based on the popular [Python HuggingFace Transformers](https://github.com/huggingface/transformers). It's also influenced by [Rust version - rust-bert](https://github.com/guillaume-be/rust-bert). In fact, all pre-trained models for Rust are compatible to import to this Go `transformer` package as both `rust-bert`'s dependency Pytorch Rust binding - [**`tch-rs`**](https://github.com/LaurentMazare/tch-rs) and Go binding [**`gotch`**](https://github.com/sugarme/gotch) are built with similar principles.
`transformer` is part of an ambitious goal (together with [**tokenizer**](https://github.com/sugarme/tokenizer) and [**gotch**](https://github.com/sugarme/gotch)) to bring more AI/deep-learning tools to Gophers so that they can stick to the language they love and good at and build faster software in production.
## Dependencies
2 main dependencies are:
- `tokenizer`
- `gotch`
## Prerequisites and installation
- As this package depends on `gotch` which is a Pytorch C++ API binding for Go, a pre-compiled Libtorch copy (CPU or GPU) should be installed in your machine. Please see [gotch](https://github.com/sugarme/gotch) installation instruction for detail.
- Install package: `go get -u github.com/sugarme/transformer`
## Basic example
```go
import (
"fmt"
"log"
"github.com/sugarme/gotch"
ts "github.com/sugarme/gotch/tensor"
"github.com/sugarme/tokenizer"
"github.com/sugarme/transformer/bert"
)
func main() {
var config *bert.BertConfig = new(bert.BertConfig)
if err := transformer.LoadConfig(config, "bert-base-uncased", nil); err != nil {
log.Fatal(err)
}
var model *bert.BertForMaskedLM = new(bert.BertForMaskedLM)
if err := transformer.LoadModel(model, "bert-base-uncased", config, nil, gotch.CPU); err != nil {
log.Fatal(err)
}
var tk *bert.Tokenizer = bert.NewTokenizer()
if err := tk.Load("bert-base-uncased", nil); err != nil{
log.Fatal(err)
}
sentence1 := "Looks like one [MASK] is missing"
sentence2 := "It was a very nice and [MASK] day"
var input []tokenizer.EncodeInput
input = append(input, tokenizer.NewSingleEncodeInput(tokenizer.NewInputSequence(sentence1)))
input = append(input, tokenizer.NewSingleEncodeInput(tokenizer.NewInputSequence(sentence2)))
encodings, err := tk.EncodeBatch(input, true)
if err != nil {
log.Fatal(err)
}
var maxLen int = 0
for _, en := range encodings {
if len(en.Ids) > maxLen {
maxLen = len(en.Ids)
}
}
var tensors []ts.Tensor
for _, en := range encodings {
var tokInput []int64 = make([]int64, maxLen)
for i := 0; i < len(en.Ids); i++ {
tokInput[i] = int64(en.Ids[i])
}
tensors = append(tensors, ts.TensorFrom(tokInput))
}
inputTensor := ts.MustStack(tensors, 0).Excerpt of 5,361 characters
Read on GitHubWould you bet a product on this? Bounded 0–100 and slow moving.
matched fp:d3a4e91e3df8ef5c, topic:deep-learning
matched fp:d3a4e91e3df8ef5c, topic:transformer