Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Tensorflow implementation of Product-based Neural Networks. An extended version is at https://github.com/Atomu2014/product-nets-distributed.
| Date | Stars |
|---|---|
| 2026-07-31 | 372 |
| 2026-08-03 | 372 |
| 2026-08-06 | 372 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
# Product-based Neural Networks for User Response Prediction ``Note``: An extended version of the conference paper is https://arxiv.org/abs/1807.00311 , which is accepted by TOIS. Compared with this simple demo, a more detailed implementation of the journal paper is at https://github.com/Atomu2014/product-nets-distributed , which has large-scale data access, multi-gpu support, and distributed training support. ``Note``: I would like to share some intersting and advanced discussions in the [extended version](https://github.com/Atomu2014/product-nets-distributed). ``Note``: Any problems, you can contact me at [email protected]. Through email, you will get my rapid response. This repository maintains the demo code of the paper [Product-based Neural Network for User Response Prediction](https://arxiv.org/abs/1611.00144) and other baseline models, implemented with ``tensorflow``. And this paper has been published on ICDM2016. ## Introduction to User Response Prediction User response prediction takes a fundamental and crucial role in today's business, especially personalized recommender system and online display advertising. Different from traditional machine learning tasks, user response prediction always has ``categorical features`` grouped by different ``fields``, which we call ``multi-field categorical data``, e.g.: ad. request={ 'weekday': 3, 'hour': 18, 'IP': 255.255.255.255, 'domain': xxx.com, 'advertiser': 2997, 'click': 1 } In practice, these categorical features are usually one-hot encoded for training. However, this representation results in sparsity. Challenged by data sparsity, linear models (e.g., ``LR``), latent factor-based models (e.g., ``FM``, ``FFM``), tree models (e.g., ``GBDT``), and DNN models (e.g., ``FNN``, ``DeepFM``) are proposed. A core problem in user response prediction is how to represent the complex feature interactions. Industrial applications prefer feature engineering and simple models. With GPU servers becoming more and more popular, it is promising to design complex models to explore feature interactions automatically. Through our analysis and experiments, we find a ``coupled gradient`` issue of latent factor-based models, and an ``insensitive gradient`` issue of DNN models. Take FM as an example, the gradient of each feature vector is the sum over other feature vectors. Suppose two features are independent, FM can hardly learn two orthogonal feature vectors. The gradient issue of DNNs is discussed in the paper ``Failures of Gradient-based Deep Learning``. <!--Another interesting fact in recommendation or ctr contests is that, winning solutions usually transform discrete features into continuous or vice versa: - Use GBDT to convert continuous features to binary ones, and feed binary features to FM. - Use FM/DNN to convert discrete features to embeddings or interactions, and feed these features to GBDT.--> In order to solve these issues, we propose to use product operators in DNN to help explore feature interactions. We discuss these issues in an extended paper, which is submitted to TOIS at Seq. 2017 and will be released later. Any discussion is welcomed, please contact [email protected]. ## Product-based Neural Networks Through discussion of previous works, we think a good predictor should have a good feature extractor (to convert sparse features into dense representations) as well as a powerful classifier (e.g., DNN as universal approximator). Since FM is good at represent feature interactions, we introduce product operators in DNN. The proposed PNN models follow this architecture: an embedding layer to represent sparse features, a product layer to explore feature interactions, and a DNN classifier. For product layer, we propose 2 types of product operators in the paper: inner product and outer product. These operators output $n(n-1)/2$ feature interactions, which are concatenated with embeddings and fed to
Excerpt of 10,897 characters
Read on GitHubWould you bet a product on this? Bounded 0–100 and slow moving.
matched fp:59199f1d503ad862, topic:deep-learning