Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
(ICLR 2025) TabM: Advancing Tabular Deep Learning With Parameter-Efficient Ensembling
| Date | Stars |
|---|---|
| 2026-07-31 | 1091 |
| 2026-08-06 | 1093 |
Today
+2 stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
# TabM: Advancing Tabular Deep Learning With Parameter-Efficient Ensembling" (ICLR 2025)<!-- omit in toc -->
:scroll: [arXiv](https://arxiv.org/abs/2410.24210)
:books: [Other tabular DL projects](https://github.com/yandex-research/rtdl)
This is the official repository of the paper "TabM: Advancing Tabular Deep Learning With
Parameter-Efficient Ensembling".
It consists of two parts:
- [**Python package**](#python-package) described in this document.
- [**Paper-related content**](./paper/README.md) (code, metrics, hyperparameters, etc.) described in `paper/README.md`.
<br>
<details>
<summary>TabM on <b>Kaggle</b> (as of June 2025)</summary>
- TabM was used in [the winning solution](https://www.kaggle.com/competitions/um-game-playing-strength-of-mcts-variants/discussion/549801) in the competition by UM.
- TabM was used in [the winning solution](https://www.kaggle.com/competitions/equity-post-HCT-survival-predictions/discussion/566550), as well as in the top-3, top-4, top-5 and many other solutions in the competition by CIBMTR. Later, it turned out that it was possible to achieve the [25-th place](https://www.kaggle.com/competitions/equity-post-HCT-survival-predictions/discussion/567863) out of 3300+ with only TabM, without ensembling it with other models.
</details>
<details>
<summary>TabM on <b>TabReD</b> (a challenging benchmark)</summary>
[TabReD](https://arxiv.org/abs/2406.19380) is a benchmark based on **real-world industrial datasets** with **time-related distribution drifts** and **hundreds of features**, which makes it more challenging than traditional benchmarks. The figure below shows that TabM achieves higher performance on TabReD (plus one more real-world dataset) compared to prior tabular DL methods.
<img src="images/tabred-and-microsoft.png" width=35% display=block margin=auto>
*One dot represents a performance score on one dataset. For a given model, a diamond represents the mean value across the datasets.*
</details>
<details>
<summary>Training and inference efficiency</summary>
TabM is a simple and reasonably efficient model, which makes it suitable for **real-world applications**, including large datasets. The biggest dataset used in the paper contains **13M objects**, and we are aware of a successful training run on **100M+ objects**, though training takes more time in such cases.
The figure below shows that TabM is relatively slower than MLPs and GBDT, but faster than prior tabular DL methods. Note that (1) the inference throughput was measured on a single CPU thread and *without any optimizations*, in particular without the TabM-specific acceleration technique described later in this document; (2) the left plot uses the *logarithmic* scale.
<img src="images/efficiency.png" display=block margin=auto>
*One dot represents a measurement on one dataset. For a given model, a diamond represents the mean value across the datasets. In the left plot,* $\mathrm{TabM_{mini}^{\dagger*}}$ *denotes* $\mathrm{TabM_{mini}^{\dagger}}$ *trained with mixed precision and `torch.compile`.*
</details>
# TL;DR<!-- omit in toc -->
<img src="images/tabm.png" width=65% display=block margin=auto>
**TabM** (**Tab**ular DL model that makes **M**ultiple predictions) is a simple and powerful tabular DL architecture that efficiently imitates an ensemble of MLPs. The two main differences of TabM compared to a regular ensemble of MLPs:
- **Parallel training** of the MLPs. This allows monitoring the performance of the ensemble during the training and stopping the training when it is optimal for the ensemble, not for individual MLPs.
- **Weight sharing** between the MLPs. In fact, the whole TabM fits in just *one* MLP-like model. Not only this significantly improves the runtime and memory efficiency, but also turns out to be an effective regularization leading to better task performance.
# Reproducing experiments and browsing results<!-- omit in toc -->
> [!IMPORTANT]
> To use TabM in practice and for future work, usExcerpt of 30,478 characters
Read on GitHub
46
4
Would you bet a product on this? Bounded 0–100 and slow moving.
matched fp:2954c827236ed427, desc:parameter-efficient