Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
NetDiscovery 是一款基于 Vert.x、RxJava 2 等框架实现的通用爬虫框架/中间件。
| Date | Stars |
|---|---|
| 2026-07-24 | 647 |
| 2026-07-25 | 647 |
| 2026-07-28 | 647 |
| 2026-07-30 | 647 |
| 2026-08-06 | 647 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
 # NetDiscovery [](http://www.weibo.com/fengzhizi715) [](https://www.apache.org/licenses/LICENSE-2.0.html) [](https://app.codacy.com/app/fengzhizi715/NetDiscovery?utm_source=github.com&utm_medium=referral&utm_content=fengzhizi715/NetDiscovery&utm_campaign=Badge_Grade_Dashboard)  <h3>IPWO爬虫代理资源为采集、跨境与测试项目提供支持(免费试用，流量不过期，爬虫使用保护账号不封号提升爬虫成功率)学习者务必遵循法律！</h3> <h3>顶级真实住宅 IP源于195 + 国家 / 地区精准定位，大幅降低被封禁风险，试用好用再来个优惠折扣码"feng"。</h3> <h3>动态ip不过期，不限最低消耗</h3> <h3>静态ip福利价格9.9</h3> <h3>官网地址</h3> [访问IPWO官网]( https://www.ipwo.net/?ref=githubfengzhizi) ---  # 功能特点： * 轻量级爬虫 * 模块化设计，便于扩展：支持多种消息队列(Disruptor、Redis、Kafka、RabitMQ、RocketMQ)、多种网络框架(Http Client、Okhttp3、Selenium等)，也支持自己实现。 * 支持分布式 * 多线程、异步化：底层使用 RxJava 2 的多线程机制 * 支持线程池隔离：爬虫的 Parser、Pipeline 过程可以放在独立的线程池中进行处理 * 支持 Request 动态添加到正在运行爬虫的 Queue 中 * 支持 Kotlin 协程 * 支持 JS 渲染 * 支持 Request 自定义 header 信息 * 支持 debug 功能：在调试时 Request 可以打开 debug 的开关，使用 Local Cache(RxCache) 存储请求的结果，从而避免多次请求同一个网页。 * 支持失败重试的机制 * 多纬度控制爬取速度（Pipeline、Request、Download、Domain）等等 * 支持代理池、User Agent 池、Cookies 池 * 支持爬虫的深度抓取：能够在 Pipeline 中发起深度抓取 * 支持 URL 去重：使用布隆过滤器 * 支持 Spider 的监控、SpiderEngine 的监控 (基于Etcd、Zookeeper) * agent 模块能够对当前服务器的 CPU 和内存进行实时监控 * SpiderEngine 整合 Quartz * SpiderEngine 提供 http、RPC 接口 # 最新版本 模块名|最新版本| ---|:-------------: netdiscovery-core-core|[  ](https://bintray.com/fengzhizi715/maven/netdiscovery-core-core/_latestVersion) netdiscovery-core-engine|[  ](https://bintray.com/fengzhizi715/maven/netdiscovery-core-engine/_latestVersion) netdiscovery-core-engine-monitor|[  ](https://bintray.com/fengzhizi715/maven/netdiscovery-core-engine-monitor/_latestVersion) netdiscovery-downloader-htmlunit|[  ](https://bintray.com/fengzhizi715/maven/netdiscovery-downloader-htmlunit/_latestVersion) netdiscovery-downloader-httpclient|[  ](https://bintray.com/fengzhizi715/maven/netdiscovery-downloader-httpclient/_latestVersion) netdiscovery-downloader-okhttp|[  ](https://bintray.com/fengzhizi715/maven/netdiscovery-downloader-okhttp/_latestVersion) netdiscovery-downloader-selenium|[  ](https://bintray.com/fengzhizi715/maven/netdiscovery-downloader-selenium/_latestVersion) netdiscovery-pipeline-couchbase|[  ](https://bintray.com/fengzhizi715/maven/netdiscovery-pipeline-couchbase/_latestVersion) netdiscovery-pipeline-elasticsearch|[  ](https://bintray.com/fengzhizi715/maven/netdiscovery-pipeline-elasticsearch/_latestVersion) netdiscovery-pipeline-mongo|[  ](https://bintray.com/fengzhizi715/maven/netdiscovery-pipeline-mongo/_lat
Excerpt of 8,085 characters
Read on GitHub
1.3k
21
6
Codacy Badger · Codacy · Portugal
1
1
Would you bet a product on this? Bounded 0–100 and slow moving.
matched fp:00edc50f6d9a8c12, topic:crawler
matched fp:00edc50f6d9a8c12, topic:selenium