Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Fast and memory-efficient clustering
| Date | Stars |
|---|---|
| 2026-07-24 | 267 |
| 2026-07-25 | 267 |
| 2026-07-28 | 267 |
| 2026-07-30 | 267 |
| 2026-08-06 | 267 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
# PQk-means
[**Project**](http://yusukematsui.me/project/pqkmeans/pqkmeans.html)
| [**Paper**](https://dl.acm.org/ft_gateway.cfm?id=3123430)
| [**Tutorial**](./tutorial)
A 2D example using both k-means and PQk-means | Large-scale evaluation
:---:|:---:
 | 
[PQk-means [Matsui, Ogaki, Yamasaki, and Aizawa, ACMMM 17]](http://yusukematsui.me/project/pqkmeans/pqkmeans.html) is a Python library for efficient clustering of large-scale data.
By first compressing input vectors into short product-quantized (PQ) codes,
PQk-means achieves fast and memory-efficient clustering, even for
high-dimensional vectors.
Similar to k-means, PQk-means repeats the assignment and update steps,
both of which can be performed in the PQ-code domain.
For a comparison, we provide the ITQ encoding for the binary conversion and
[Binary k-means [Gong+, CVPR 15]](http://www.cv-foundation.org/openaccess/content_cvpr_2015/html/Gong_Web_Scale_Photo_2015_CVPR_paper.html) for the clustering of binary codes.
The library is written in C++ for the main algorithm with wrappers for Python.
All encoding/clustering codes are compatible with scikit-learn.
## Summary of features
- Approximation of k-means
- Tens to hundreds of times faster than k-means
- Tens to hundreds of times more memory efficient than k-means
- Compatible with scikit-learn
- Portable; one-line installation
## Installation
#### Requisites
- CMake
- `brew install cmake` for OS X
- `sudo apt install cmake` for Ubuntu
- OpenMP (Optional)
- If openmp is installed, it will be automatically used to parallelize the algorithm for faster calculation.
#### Build & install
You can install the library from PyPI:
```
pip install pqkmeans
```
Or, if you would like to use the current master version, you can manually build and install the library by:
```
git clone --recursive https://github.com/DwangoMediaVillage/pqkmeans.git
cd pqkmeans
python setup.py install
```
## Run samples
```
# evaluation needs extra texmex package
pip install pqkmeans[texmex]
# with artificial data
python bin/run_experiment.py --dataset artificial --algorithm bkmeans pqkmeans --k 100
# with texmex dataset (http://corpus-texmex.irisa.fr/)
python bin/run_experiment.py --dataset siftsmall --algorithm bkmeans pqkmeans --k 100
```
## Test
```
python setup.py test
```
## Usage
#### For PQk-means
```python
import pqkmeans
import numpy as np
X = np.random.random((100000, 128)) # 128 dimensional 100,000 samples
# Train a PQ encoder.
# Each vector is divided into 4 parts and each part is
# encoded with log256 = 8 bit, resulting in a 32 bit PQ code.
encoder = pqkmeans.encoder.PQEncoder(num_subdim=4, Ks=256)
encoder.fit(X[:1000]) # Use a subset of X for training
# Convert input vectors to 32-bit PQ codes, where each PQ code consists of four uint8.
# You can train the encoder and transform the input vectors to PQ codes preliminary.
X_pqcode = encoder.transform(X)
# Run clustering with k=5 clusters.
kmeans = pqkmeans.clustering.PQKMeans(encoder=encoder, k=5)
clustered = kmeans.fit_predict(X_pqcode)
# Then, clustered[0] is the id of assigned center for the first input PQ code (X_pqcode[0]).
```
Note that an instance of PQ-encoder (`encoder`) and an instance of clustering (`kmeans`) can be pickled and reused later.
```python
import pickle
# An instance of PQ-encoder.
pickle.dump(encoder, open('encoder.pkl', 'wb'))
encoder_dumped = pickle.load(open('encoder.pkl', 'rb'))
# An instance of clustering. This can be reused as a vector quantizer later.
pickle.dump(kmeans, open('kmeans.pkl', 'wb'))
kmeans_dumped = pickle.load(open('kmeans.pkl', 'rb'))
```
#### For Bk-means
In almost the same manner as for PQk-means,
```python
import pqkmeans
import numpy as np
X = np.random.random((100000, 128)) # 128 dimensional 100,000 samples
# Train an ITQ binary encoder
encoder = pqkmeans.encoder.ITQEncoder(num_bit=Excerpt of 5,246 characters
Read on GitHubWould you bet a product on this? Bounded 0–100 and slow moving.
matched fp:701582f7253ac9e5, topic:computer-vision