Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
A scalable machine learning library on Apache Spark
| Date | Stars |
|---|---|
| 2026-07-31 | 797 |
| 2026-08-06 | 797 |
| 2026-08-07 | 797 |
| 2026-08-15 | 797 |
| 2026-08-16 | 797 |
| 2026-08-18 | 797 |
| 2026-09-06 | 796 |
| 2026-09-07 | 795 |
| 2026-09-20 | 795 |
Today
— stars today
This week
— stars this week
This month
-2 stars this month
Momentum
0.0
growth rate 0.00%/day
# Photon Machine Learning (Photon ML)
[](https://travis-ci.org/linkedin/photon-ml)
**Check out our [hands-on tutorial](https://github.com/linkedin/photon-ml/wiki/Photon-ML-Tutorial).**
Photon ML is a machine learning library based on Apache Spark. It was originally developed by the LinkedIn Machine Learning Algorithms Team. Currently, Photon ML supports training different types of [Generalized Linear Models](https://en.wikipedia.org/wiki/Generalized_linear_model)(GLMs) and [Generalized Linear Mixed Models](https://en.wikipedia.org/wiki/Generalized_linear_mixed_model)(GLMMs/GLMix model): logistic, linear, and Poisson.
- [Features](#features)
- [Generalized Linear Models](#generalized-linear-models)
- [GAME - Generalized Additive Mixed Effects](#game---generalized-additive-mixed-effects)
- [Configurable Optimizers](#configurable-optimizers)
- [Regularization](#regularization)
- [Feature scaling and normalization](#feature-scaling-and-normalization)
- [Offset training](#offset-training)
- [Feature summarization](#feature-summarization)
- [Model validation](#model-validation)
- [Warm-start training](#warm-start-training)
- [Partial re-training](#partial-re-training)
- [Experimental Features](#experimental-features)
- [Smoothed Hinge Loss Linear SVM](#smoothed-hinge-loss-linear-svm)
- [Hyperparameter Auto-Tuning](#hyperparameter-auto-tuning)
- [Regularize by Previous Model During Warm-Start Training](#regularize-by-previous-model-during-warm-start-training)
- [How to Build](#how-to-build)
- [How to Use](#how-to-use)
- [Drivers](#drivers)
- [API](#api)
- [Avro Schemas](#avro-schemas)
- [What about other formats?](#what-about-other-formats)
- [Input Data Format](#input-data-format)
- [Models](#models)
- [Shaded Jar](#shaded-jar)
- [Try It Out!](#try-it-out)
- [Install Spark](#install-spark)
- [Get and Build the Code](#get-and-build-the-code)
- [Grab a Dataset](#grab-a-dataset)
- [Train the Model](#train-the-model)
- [Running Photon ML on Cluster Mode](#running-photon-ml-on-cluster-mode)
- [Modules and directories](#modules-and-directories)
- [Source code](#source-code)
- [Other](#other)
- [IntelliJ IDEA setup](#intellij-idea-setup)
- [How to Contribute](#how-to-contribute)
- [Reference](#reference)
## Features
#### Generalized Linear Models
* Linear Regression
* Logistic Regression
* Poisson Regression
#### GAME - Generalized Additive Mixed Effects
The GAME algorithm uses coordinate descent to expand beyond traditional GLMs to further provide per-entity (per-user, per-item, per-country, etc.) coefficients (also known as random effects in statistics literature). It manages to scale model training up to hundreds of billions of coefficients, while remaining solvable within Spark's framework.
For example, a GAME model for movie recommendations can be formulated as (fixed effect model + per-user random effect model + per-movie random effect model + user-movie matrix factorization model). More details on GAME models can be found [here](https://docs.google.com/presentation/d/1vHanpK3KLIVgdDIHYRehUeyb04Hc2AasbBHs4InVPSU).
The type of GAME model currently supported by Photon ML is the GLMM or 'GLMix' model. Many of LinkedIn's core products have adopted GLMix models: jobs search and recommendation, news feed ranking, Ads CTR prediction and "People Also Viewed". More details on GLMix models can be found [here](https://docs.google.com/presentation/d/1tYoelUma9-MMYdteWYS31LqVeoyPEncxJRk-k57gj0A/edit?usp=sharing).
#### Configurable Optimizers
* [LBFGS](https://en.wikipedia.org/wiki/Limited-memory_BFGS)
* [TRON](https://www.csie.ntu.edu.tw/~cjlin/papers/logistic.pdf)
#### Regularization
* L1 (LASSO) regularization
* L2 (Tikhonov) regularization (only type supported by TRON)
* Elastic-net regularization
#### Feature scaling and normalization
* Standardization: Zero-mean, unit-varExcerpt of 18,331 characters
Read on GitHub
137
Josh Fleming · Facebook · United States
125
95
68
63
36
25
21
13
11
6
4
4
3
1
1
Howard Liberty
1
1
Jeremy Kerfs
1
1
Would you bet a product on this? Bounded 0–100 and slow moving.
matched fp:5b6dcbee7ad6f2e4, llm:Repository description: 'A scalable machine learning library on Apache Spark' (linkedin/photon-ml).
matched fp:5b6dcbee7ad6f2e4, llm:Repository description: 'A scalable machine learning library on Apache Spark' (linkedin/photon-ml).
matched fp:5b6dcbee7ad6f2e4, llm:Repository description: 'A scalable machine learning library on Apache Spark' (linkedin/photon-ml).