Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Code for KDD'20 "Generative Pre-Training of Graph Neural Networks"
| Date | Stars |
|---|---|
| 2026-07-31 | 502 |
| 2026-08-01 | 502 |
| 2026-08-02 | 502 |
| 2026-08-03 | 501 |
| 2026-08-06 | 501 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
# GPT-GNN: Generative Pre-Training of Graph Neural Networks <p align="center"> <img src="images/gpt-intro.png" width="600"> <br /> <br /> </p> GPT-GNN is a pre-training framework to initialize GNNs by generative pre-training. It can be applied to large-scale and heterogensous graphs. You can see our KDD 2020 paper [“**Generative Pre-Training of Graph Neural Networks**”](https://arxiv.org/abs/2006.15437) for more details. ## Overview The key package is GPT_GNN, which contains the the high-level GPT-GNN pretraining framework, base GNN models, and base graph structure and data loader. To illustrate how to apply the GPT_GNN framework for arbitrary graphs, we provide examples of pre-training on both hetergeneous (OAG) and homogeneous graphs (reddit). Both of them are of large-scale. Within each `example_*` package, there is a `pretrain_*.py` file for pre-training a GNN on the given graph, and also multiple `finetune_*.py` files for training and validating on downstream tasks. ## DataSet For **Open Academic Graph (OAG)**, we provide a heterogeneous graph containing highly-cited CS papers (8.1G) spanning from 1900-2020. You can download the preprocessed graph via [this link](https://drive.google.com/open?id=1a85skqsMBwnJ151QpurLFSa9o2ymc_rq). We split the data by their time: Pre-training ( t < 2014 ); Training ( 2014 <= t < 2017); Validation ( t = 2017 ); Testing ( 2018 <= t ). As we use the raw-text as attribute generation task for OAG, we provide a pre-trained word2vec model via [this link](https://drive.google.com/file/d/1ArdaMlPKVqdRGyiw4YSdUOV6CeFb2AmD/view?usp=sharing). If you want to directly process from raw data, you can download via [this link](https://drive.google.com/open?id=1yDdVaartOCOSsQlUZs8cJcAUhmvRiBSz). After downloading it, run `preprocess_OAG.py` to extract features and store them in our data structure. For **Reddit**, we simply download the preprocessed graph using pyG.datasets API, and then turn it into our own data structure using `preprocess_reddit.py`. We randomly split the data into different sets. ## Setup This implementation is based on pytorch_geometric. To run the code, you need the following dependencies: - [Pytorch 1.3.0](https://pytorch.org/) - [pytorch_geometric 1.3.2](https://pytorch-geometric.readthedocs.io/) - torch-cluster==1.4.5 - torch-scatter==1.3.2 - torch-sparse==0.4.3 - [gensim](https://github.com/RaRe-Technologies/gensim) - [sklearn](https://github.com/scikit-learn/scikit-learn) - [tqdm](https://github.com/tqdm/tqdm) - [dill](https://github.com/uqfoundation/dill) - [pandas](https://github.com/pandas-dev/pandas) You can simply run ```pip install -r requirements.txt``` to install all the necessary packages. ## Usage We first introduce the arguments to control hyperparameters. There are mainly three types of arguments, for pre-training; for dataset; for model and optimization. For pre-training, we provide arguments to control different modules for attribute and edge generation tasks: ``` --attr_ratio FLOAT The ratio (0~1) of attribute generation loss . Default is 0.5. --attr_type STR type of attribute decoder ['text' or 'vec'] Default is 'vec' --neg_samp_num BOOL Whether to use layer-norm on the last layer. Default is False. --queue_size INT Max size of adaptive embedding queue. Default is 256. ``` For datasets, we provide arguments to control mini-batch sampling: ``` --data_dir STR The address of preprocessed graph. --pretrain_model_dir STR The address for storing the pre-trained models. --sample_depth INT How many layers within a mini-batch subgraph Default is 6. --sample_width INT How many nodes to be sampled per layer per type Default is 128. ``` For both pre-training and fine-tuning, we provide arguments to co
Excerpt of 6,401 characters
Read on GitHub
94
2
Da Zheng · AWS · United States
1
1
Would you bet a product on this? Bounded 0–100 and slow moving.
matched fp:12eff4e49c79ccf5, desc:pre-training