Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
[Open-Source Project] Combining MMOCR with Segment Anything & Stable Diffusion. Automatically detect, recognize and segment text instances, with serval downstream tasks, e.g., Text Removal and Text Inpainting
| Date | Stars |
|---|---|
| 2026-07-31 | 590 |
| 2026-08-06 | 590 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
 # Optical Character Recognition with Segment Anything (OCR-SAM) ## 🐇 Introduction 🐙 Can [SAM](https://github.com/facebookresearch/segment-anything) be applied to OCR? We take a simple try to combine two off-the-shelf OCR models in [MMOCR](https://github.com/open-mmlab/mmocr) with SAM to develop some OCR-related application demos, including **[SAM for Text](#sam-for-text)**, **[Text Removal](#erasing)** and **[Text Inpainting](#inpainting)**. And we also provide a **[WebUI by gradio](#run-webui)** to give a better interaction. ## 📅 Updates 👀 - **2023.08.23**: 🔥 We create a repo **[yeungchenwa/Recommendations-Diffusion-Text-Image](https://github.com/yeungchenwa/Recommendations-Diffusion-Text-Image)** to provide a paper collection of recent diffusion models for text-image generation tasks. - **2023.04.14**: 📣 Our repository is migrated to **[open-mmlab/playground](https://github.com/open-mmlab/playground#-mmocr-sam)**. - **2023.04.12**: Repository Release - **2023.04.12**: Supported the **[Inpainting](#inpainting🥸)** combined with DBNet++, SAM and Stable-Diffusion. - **2023.04.11**: Supported the **[Erasing](#erasing🤓)** combined with DBNet++, SAM and Latent-Diffusion / Stable-Diffusion. - **2023.04.10**: Supported the **[SAM for text](#sam-for-text🧐)** combined tieh DBNet++ and SAM. - **2023.04.09**: How effective is the SAM used on OCR Text Image, we've discussed it in the **[Blog](https://www.zhihu.com/question/593914819/answer/2976012032)**. ## 📸 Demo Zoo 🔥 This project includes: - [x] [SAM for Text](#sam-for-text🧐): DBNet++ + SAM  - [x] [Erasing](#erasing🤓): DBNet++ + SAM + Latent-Diffusion / Stable Diffusion  - [x] [Inpainting](#inpainting🥸): DBNet++ + SAM + Stable Diffusion  ## 🚧 Installation 🛠️ ### Prerequisites(Recommended) - Linux | Windows - Python 3.8 - Pytorch 1.12 - CUDA 11.3 ### Environment Setup Clone this repo: ```bash git clone https://github.com/yeungchenwa/OCR-SAM.git ``` **Step 0**: Download and install Miniconda from the [official website](https://docs.conda.io/en/latest/miniconda.html). **Step 1**: Create a conda environment and activate it. ```bash conda create -n ocr-sam python=3.8 -y conda activate ocr-sam ``` **Step 2**: Install related version Pytorch following [here](https://pytorch.org/get-started/previous-versions/). ```bash # Suggested pip install torch==1.12.1+cu113 torchvision==0.13.1+cu113 --extra-index-url https://download.pytorch.org/whl/cu113 ``` **Step 3**: Install the mmengine, mmcv, mmdet, mmcls, mmocr. ```bash pip install -U openmim mim install mmengine mim install mmocr # In Window, the following symbol ' should be changed to " mim install 'mmcv==2.0.0rc4' mim install 'mmdet==3.0.0rc5' mim install 'mmcls==1.0.0rc5' # Install sam pip install git+https://github.com/facebookresearch/segment-anything.git # Install required packages pip install -r requirements.txt ``` **Step 4**: Prepare for the diffusers and latent-diffusion. ```bash # Install Gradio pip install gradio # Install the diffusers pip install diffusers # Install the pytorch_lightning for ldm pip install pytorch-lightning==2.0.1.post0 ``` ## 📒 Model checkpoints 🖥 We retrain DBNet++ with Swin Transformer V2 as the backbone on a combination of multiple scene text datsets (e.g. HierText, TextOCR). **Checkpoint for DBNet++ on [Google Drive (1G)](https://drive.google.com/file/d/1r3B1xhkyKYcQ9SR7o9hw9zhNJinRiHD-/view?usp=share_link)**. And you should make dir following: ```bash mkdir checkpoints mkdir checkpoints/mmocr mkdir checkpoints/sam mkdir checkpoints/ldm mv db_swin_mix_pretrain.pth checkpoints/mmocr ``` Download the rest of the checkpoints to the related path (If you've done so, ignore the following): ```bash # mmocr recognizer ckpt wget -O checkpoints/mmocr/abinet_20e_st-an_mj_20221005_012617-ead8c139.pth https://download.openmmlab.com/mmocr/textrecog/abinet/abinet_20e_st-an_mj/abinet_20e_st
Excerpt of 7,761 characters
Read on GitHub
Qing Jiang · China
53
42
1
Would you bet a product on this? Bounded 0–100 and slow moving.
matched fp:13ea9b68c1387520, name:ocr
matched fp:13ea9b68c1387520, desc:stable diffusion, desc:inpainting