Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Train robotic agents to learn to plan pushing and grasping actions for manipulation with deep reinforcement learning.
| Date | Stars |
|---|---|
| 2026-07-24 | 1109 |
| 2026-07-25 | 1109 |
| 2026-07-28 | 1108 |
| 2026-07-30 | 1108 |
| 2026-07-31 | 1108 |
| 2026-08-06 | 1108 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
# Visual Pushing and Grasping Toolbox Visual Pushing and Grasping (VPG) is a method for training robotic agents to learn how to plan complementary pushing and grasping actions for manipulation (*e.g.* for unstructured pick-and-place applications). VPG operates directly on visual observations (RGB-D images), learns from trial and error, trains quickly, and generalizes to new objects and scenarios. <img src="images/teaser.jpg" height=223px align="left"/> <img src="images/self-supervision.gif" height=223px align="left"/> <br> This repository provides PyTorch code for training and testing VPG policies with deep reinforcement learning in both simulation and real-world settings on a UR5 robot arm. This is the reference implementation for the paper: ### Learning Synergies between Pushing and Grasping with Self-supervised Deep Reinforcement Learning [PDF](https://arxiv.org/pdf/1803.09956.pdf) | [Webpage & Video Results](http://vpg.cs.princeton.edu/) [Andy Zeng](http://andyzeng.github.io/), [Shuran Song](http://vision.princeton.edu/people/shurans/), [Stefan Welker](https://www.linkedin.com/in/stefan-welker), [Johnny Lee](http://johnnylee.net/), [Alberto Rodriguez](http://meche.mit.edu/people/faculty/[email protected]), [Thomas Funkhouser](https://www.cs.princeton.edu/~funk/) IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 2018 Skilled robotic manipulation benefits from complex synergies between non-prehensile (*e.g.* pushing) and prehensile (*e.g.* grasping) actions: pushing can help rearrange cluttered objects to make space for arms and fingers; likewise, grasping can help displace objects to make pushing movements more precise and collision-free. In this work, we demonstrate that it is possible to discover and learn these synergies from scratch through model-free deep reinforcement learning. Our method involves training two fully convolutional networks that map from visual observations to actions: one infers the utility of pushes for a dense pixel-wise sampling of end effector orientations and locations, while the other does the same for grasping. Both networks are trained jointly in a Q-learning framework and are entirely self-supervised by trial and error, where rewards are provided from successful grasps. In this way, our policy learns pushing motions that enable future grasps, while learning grasps that can leverage past pushes. During picking experiments in both simulation and real-world scenarios, we find that our system quickly learns complex behaviors amid challenging cases of clutter, and achieves better grasping success rates and picking efficiencies than baseline alternatives after only a few hours of training. We further demonstrate that our method is capable of generalizing to novel objects. <!--  --> <img src="images/method.jpg" width=100%/> #### Citing If you find this code useful in your work, please consider citing: ``` @inproceedings{zeng2018learning, title={Learning Synergies between Pushing and Grasping with Self-supervised Deep Reinforcement Learning}, author={Zeng, Andy and Song, Shuran and Welker, Stefan and Lee, Johnny and Rodriguez, Alberto and Funkhouser, Thomas}, booktitle={IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)}, year={2018} } ``` #### Demo Videos Demo videos of a real robot in action can be found [here](http://vpg.cs.princeton.edu/). #### Contact If you have any questions or find any bugs, please let me know: [Andy Zeng](http://www.cs.princeton.edu/~andyz/) andyz[at]princeton[dot]edu ## Installation This implementation requires the following dependencies (tested on Ubuntu 16.04.4 LTS): * Python 2.7 or Python 3 * [NumPy](http://www.numpy.org/), [SciPy](https://www.scipy.org/scipylib/index.html), [OpenCV-Python](https://docs.opencv.org/3.0-beta/doc/py_tutorials/py_tutorials.html), [Matplotlib](https://matplotlib.org/). Yo
Excerpt of 19,805 characters
Read on GitHub
75
Andrew Hundt · Carnegie Mellon University · United States
2
1
Would you bet a product on this? Bounded 0–100 and slow moving.
matched fp:77a57637cba3f25a, topic:robotics, topic:manipulation, desc:manipulation
matched fp:77a57637cba3f25a, topic:deep-reinforcement-learning, desc:reinforcement learning, readme:reinforcement learning