Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Attention-based multimodal fusion for sentiment analysis
| Date | Stars |
|---|---|
| 2026-07-24 | 367 |
| 2026-07-25 | 367 |
| 2026-07-28 | 367 |
| 2026-07-30 | 367 |
| 2026-08-06 | 367 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
# 🔥 🔥 🔥 Notice: This repository will no longer be maintained. Instead, we are are moving all our multimodal works to this new centralized repository: [https://github.com/declare-lab/multimodal-deep-learning](https://github.com/declare-lab/multimodal-deep-learning). # Attention-based multimodal fusion for sentiment analysis Attention-based multimodal fusion for sentiment analysis Code for the paper [Context-Dependent Sentiment Analysis in User-Generated Videos](http://sentic.net/context-dependent-sentiment-analysis-in-user-generated-videos.pdf) (ACL 2017). [Multi-level Multiple Attentions for Contextual Multimodal Sentiment Analysis](https://ieeexplore.ieee.org/abstract/document/8215597/)(ICDM 2017).  ### Preprocessing **Edit:** the create_data.py is obsolete. The pre-processed datasets have already been provided in the dataset/ folder in the repo. Use them directly. As data is typically present in utterance format, we combine all the utterances belonging to a video using the following code ``` python create_data.py ``` Note: This will create speaker independent train and test splits In dataset/mosei, extract the zip into a folder named 'raw'. Also, extract 'unimodal_mosei_3way.pickle.zip' ### Running the model Sample command: With attention-based fusion: ``` python run.py --unimodal True --fusion True python run.py --unimodal False --fusion True ``` Without attention-based and with concatenation-based fusion: ``` python run.py --unimodal True --fusion False python run.py --unimodal False --fusion False ``` Utterance level attention: ``` python run.py --unimodal False --fusion True --attention_2 True python run.py --unimodal False --fusion True --attention_2 True ``` Note: 1. Keeping the unimodal flag as True (default False) shall train all unimodal lstms first (level 1 of the network mentioned in the paper) 2. Setting --fusion True applies only to multimodal network. ### Datasets: We provide results on the [MOSI](https://arxiv.org/pdf/1606.06259.pdf), [MOSEI ](http://aclweb.org/anthology/P18-1208) and [IEMOCAP](https://sail.usc.edu/iemocap/) datasets.<br> Please cite the creators. We are adding more datasets, stay tuned. Use ```--data [mosi|mosei|iemocap] and --classes [2|3|6]``` in the above commands to test different configurations on different datasets. mosi: 2 classes<br> mosei: 3 classes<br> iemocap: 6 classes<br> Example: ``` python run.py --unimodal False --fusion True --attention_2 True --data mosei --classes 3 ``` #### Dataset details ##### MOSI: 2 classes: Positive/Negative <br> Raw Features: (Pickle files) <br> Audio: dataset/mosi/raw/audio_2way.pickle <br> Text: dataset/mosi/raw/text_2way.pickle <br> Video: dataset/mosi/raw/video_2way.pickle <br> **Each file contains: <br>** train_data, train_label, test_data, test_label, maxlen, train_length, test_length train_data - np.array of dim (62, 63, feature_dim) <br> train_label - np.array of dim (62, 63, 2) <br> test_data - np.array of dim (31, 63, feature_dim) <br> test_label - np.array of dim (31, 63, 2) <br> maxlen - max utterance length int of value 63 <br> train_length - utterance length of each video in train data. <br> test_length - utterance length of each video in test data. <br> Train/Test split: 62/31 videos. Each video has utterances. The videos are padded to 63 utterances. ##### IEMOCAP: 6 classes: happy/sad/neutral/angry/excited/frustrated<br> Raw Features: dataset/iemocap/raw/IEMOCAP_features_raw.pkl (Pickle files) <br> The file contains: videoIDs[vid] = List of utterance IDs in this video in the order of occurance <br> videoSpeakers[vid] = List of speaker turns. e.g. [M, M, F, M, F]. here M = Male, F = Female <br> videoText[vid] = List of textual features for each utterance in video vid. <br> videoAudio[vid] = List of audio features for each utterance in video vid. <br> videoVisual[vid] = List of visual features for each utterance in video vid.
Excerpt of 6,764 characters
Read on GitHubWould you bet a product on this? Bounded 0–100 and slow moving.
matched fp:5cc3177776362e47, topic:natural-language-processing, topic:sentiment-analysis, name:sentiment analysis