Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Tensorflow implementation of the HarvardNLP paper - What You Get Is What You See: A Visual Markup Decompiler (https://arxiv.org/pdf/1609.04938v1.pdf)
| Date | Stars |
|---|---|
| 2026-07-24 | 292 |
| 2026-07-25 | 292 |
| 2026-07-28 | 292 |
| 2026-07-30 | 292 |
| 2026-08-06 | 292 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
# im2latex tensorflow implementation
This is a tensorflow implementation of the HarvardNLP paper - What You Get Is What You See: A Visual Markup Decompiler.
This is also a potential solution to OpenAI's Requests For Research Problem - [im2latex](https://openai.com/requests-for-research/#im2latex)
The paper (http://arxiv.org/pdf/1609.04938v1.pdf) provides technical details of the model.
Original Torch implementation of the paper[https://github.com/harvardnlp/im2markup/blob/master/]
What You Get Is What You See: A Visual Markup Decompiler
Yuntian Deng, Anssi Kanervisto, and Alexander M. Rush
http://arxiv.org/pdf/1609.04938v1.pdf
This is a general-purpose, deep learning-based system to decompile an image into presentational markup. For example, we can infer the LaTeX or HTML source from a rendered image.
<p align="center"><img src="http://lstm.seas.harvard.edu/latex/network.png" width="400"></p>
An example input is a rendered LaTeX formula:
<p align="center"><img src="http://lstm.seas.harvard.edu/latex/results/website/images/119b93a445-orig.png"></p>
The goal is to infer the LaTeX formula that can render such an image:
```
d s _ { 1 1 } ^ { 2 } = d x ^ { + } d x ^ { - } + l _ { p } ^ { 9 } \frac { p _ { - } } { r ^ { 7 } } \delta ( x ^ { - } ) d x ^ { - } d x ^ { - } + d x _ { 1 } ^ { 2 } + \; \cdots \; + d x _ { 9 } ^ { 2 }
```
## Sample results from this implementation
For more results, view [results_validset.html](https://rawgit.com/ritheshkumar95/im2markup-tensorflow/master/results_validset.html), [results_testset.html](https://rawgit.com/ritheshkumar95/im2markup-tensorflow/master/results_testset.html) files.
## Prerequsites
Most of the code is written in tensorflow, with Python for preprocessing.
### Preprocess
The proprocessing for this dataset is exactly reproduced as the original torch implementation by the HarvardNLP group
Python
* Pillow
* numpy
Optional: We use Node.js and KaTeX for preprocessing [Installation](https://nodejs.org/en/)
##### pdflatex [Installaton](https://www.tug.org/texlive/)
Pdflatex is used for rendering LaTex during evaluation.
##### ImageMagick convert [Installation](http://www.imagemagick.org/script/index.php)
Convert is used for rending LaTex during evaluation.
##### Webkit2png [Installation](http://www.paulhammond.org/webkit2png/)
Webkit2png is used for rendering HTML during evaluation.
### Preprocessing Instructions
The images in the dataset contain a LaTeX formula rendered on a full page. To accelerate training, we need to preprocess the images.
Please download the training data from https://zenodo.org/record/56198#.WFojcXV94jA and extract into source (master) folder.
```
cd im2markup
```
```
python scripts/preprocessing/preprocess_images.py --input-dir ../formula_images --output-dir ../images_processed
```
The above command will crop the formula area, and group images of similar sizes to facilitate batching.
Next, the LaTeX formulas need to be tokenized or normalized.
```
python scripts/preprocessing/preprocess_formulas.py --mode normalize --input-file ../im2latex_formulas.lst --output-file formulas.norm.lst
```
The above command will normalize the formulas. Note that this command will produce some error messages since some formulas cannot be parsed by the KaTeX parser.
Then we need to prepare train, validation and test files. We will exclude large images from training and validation set, and we also ignore formulas with too many tokens or formulas with grammar errors.
```
python scripts/preprocessing/preprocess_filter.py --filter --image-dir ../images_processed --label-path formulas.norm.lst --data-path ../im2latex_train.lst --output-path train.lst
```
```
python scripts/preprocessing/preprocess_filter.py --filter --image-dir ../images_processed --label-path formulas.norm.lst --data-path ../im2latex_validate.lst --output-path validate.lst
```
```
python scripts/preprocessing/preprocess_filter.py --no-filter --image-dir Excerpt of 6,217 characters
Read on GitHubWould you bet a product on this? Bounded 0–100 and slow moving.
matched fp:7c4a8e7f3cd60ff1, topic:deep-learning, topic:neural-network, topic:tensorflow