Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
PyTorch source code for "Stacked Cross Attention for Image-Text Matching" (ECCV 2018)
| Date | Stars |
|---|---|
| 2026-07-24 | 579 |
| 2026-07-25 | 579 |
| 2026-07-28 | 579 |
| 2026-07-30 | 579 |
| 2026-08-06 | 579 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
# Introduction This is Stacked Cross Attention Network, source code of [Stacked Cross Attention for Image-Text Matching](https://arxiv.org/abs/1803.08024) ([project page](https://kuanghuei.github.io/SCANProject/)) from Microsoft AI and Research. The paper will appear in ECCV 2018. It is built on top of the [VSE++](https://github.com/fartashf/vsepp) in PyTorch. ## Requirements and Installation We recommended the following dependencies. * Python 2.7 * [PyTorch](http://pytorch.org/) 0.3 * [NumPy](http://www.numpy.org/) (>1.12.1) * [TensorBoard](https://github.com/TeamHG-Memex/tensorboard_logger) * Punkt Sentence Tokenizer: ```python import nltk nltk.download() > d punkt ``` ## Download data Download the dataset files and pre-trained models. We use splits produced by [Andrej Karpathy](http://cs.stanford.edu/people/karpathy/deepimagesent/). The raw images can be downloaded from from their original sources [here](http://nlp.cs.illinois.edu/HockenmaierGroup/Framing_Image_Description/KCCA.html), [here](http://shannon.cs.illinois.edu/DenotationGraph/) and [here](http://mscoco.org/). The precomputed image features of MS-COCO are from [here](https://github.com/peteanderson80/bottom-up-attention). The precomputed image features of Flickr30K are extracted from the raw Flickr30K images using the bottom-up attention model from [here](https://github.com/peteanderson80/bottom-up-attention). All the data needed for reproducing the experiments in the paper, including image features and vocabularies, can be downloaded from: https://www.kaggle.com/datasets/kuanghueilee/scan-features We refer to the path of extracted files for `data.zip` as `$DATA_PATH` and files for `vocab.zip` to `./vocab` directory. Alternatively, you can also run vocab.py to produce vocabulary files. For example, ```bash python vocab.py --data_path data --data_name f30k_precomp python vocab.py --data_path data --data_name coco_precomp ``` ## Data pre-processing (Optional) The image features of Flickr30K and MS-COCO are available in numpy array format, which can be used for training directly. However, if you wish to test on another dataset, you will need to start from scratch: 1. Use the `bottom-up-attention/tools/generate_tsv.py` and the bottom-up attention model to extract features of image regions. The output file format will be a tsv, where the columns are ['image_id', 'image_w', 'image_h', 'num_boxes', 'boxes', 'features']. 2. Use `util/convert_data.py` to convert the above output to a numpy array. If downloading the whole data package containing bottom-up image features for Flickr30K and MS-COCO is too slow for you, you can download everything but image features from https://www.kaggle.com/datasets/kuanghueilee/scan-features and compute image features locally from raw images. ## Training new models Run `train.py`: ```bash python train.py --data_path "$DATA_PATH" --data_name coco_precomp --vocab_path "$VOCAB_PATH" --logger_name runs/coco_scan/log --model_name runs/coco_scan/log --max_violation --bi_gru ``` Arguments used to train Flickr30K models: | Method | Arguments | | :-------: | :-------: | | SCAN t-i LSE | `--max_violation --bi_gru --agg_func=LogSumExp --cross_attn=t2i --lambda_lse=6 --lambda_softmax=9` | | SCAN t-i AVG | `--max_violation --bi_gru --agg_func=Mean --cross_attn=t2i --lambda_softmax=9` | | SCAN i-t LSE | `--max_violation --bi_gru --agg_func=LogSumExp --cross_attn=i2t --lambda_lse=5 --lambda_softmax=4` | | SCAN i-t AVG | `--max_violation --bi_gru --agg_func=Mean --cross_attn=i2t --lambda_softmax=4` | Arguments used to train MS-COCO models: | Method | Arguments | | :-------: | :-------: | | SCAN t-i LSE | `--max_violation --bi_gru --agg_func=LogSumExp --cross_attn=t2i --lambda_lse=6 --lambda_softmax=9 --num_epochs=20 --lr_update=10 --learning_rate=.0005` | | SCAN t-i AVG | `--max_violation --bi_gru --agg_func=Mean --cross_attn=t2i --lambda_softmax=9 --num_epochs=20 --lr_update=10 --learning_rate=.0005`
Excerpt of 5,327 characters
Read on GitHub
7
Jamie · Meta Platforms Inc · China
2
Would you bet a product on this? Bounded 0–100 and slow moving.
matched fp:ffad9c65bade5e63, topic:deep-learning, topic:neural-network, topic:pytorch