Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
[ICCV 2023 Oral] Official implementation for "DDFM: Denoising Diffusion Model for Multi-Modality Image Fusion."
| Date | Stars |
|---|---|
| 2026-08-05 | 280 |
| 2026-08-21 | 279 |
| 2026-08-25 | 280 |
| 2026-08-26 | 281 |
| 2026-09-06 | 282 |
| 2026-09-11 | 281 |
| 2026-09-30 | 282 |
| 2026-10-03 | 283 |
| 2026-10-06 | 283 |
| 2026-10-10 | 283 |
| 2026-10-19 | 284 |
| 2026-11-04 | 284 |
Today
— stars today
This week
— stars this week
This month
+1 stars this month
Momentum
0.0
growth rate 0.00%/day
# DDFM (ICCV 2023 Oral)
Codes for ***DDFM: Denoising Diffusion Model for Multi-Modality Image Fusion. (ICCV 2023 Oral)***
[Zixiang Zhao](https://zhaozixiang1228.github.io/), [Haowen Bai](), [Yuanzhi Zhu](https://yuanzhi-zhu.github.io/about/), [Jiangshe Zhang](http://gr.xjtu.edu.cn/web/jszhang), [Shuang Xu](https://shuangxu96.github.io/), [Yulun Zhang](https://yulunzhang.com/), [Kai Zhang](https://cszn.github.io/), [Deyu Meng](https://gr.xjtu.edu.cn/en/web/dymeng), [Radu Timofte](https://www.informatik.uni-wuerzburg.de/computervision/home/) and [Luc Van Gool](https://vision.ee.ethz.ch/people-details.OTAyMzM=.TGlzdC8zMjQ4LC0xOTcxNDY1MTc4.html).
-[*[Project Page]*](https://zhaozixiang1228.github.io/Project/DDFM/)
-[*[Paper]*](https://openaccess.thecvf.com/content/ICCV2023/html/Zhao_DDFM_Denoising_Diffusion_Model_for_Multi-Modality_Image_Fusion_ICCV_2023_paper.html)
-[*[ArXiv]*](https://arxiv.org/abs/2303.06840)
-[*[Supplementary Materials]*](https://openaccess.thecvf.com/content/ICCV2023/supplemental/Zhao_DDFM_Denoising_Diffusion_ICCV_2023_supplemental.pdf)
## Update
- [2023-9] Codes and config files are public available.
## Citation
```
@InProceedings{Zhao_2023_ICCV,
author = {Zhao, Zixiang and Bai, Haowen and Zhu, Yuanzhi and Zhang, Jiangshe and Xu, Shuang and Zhang, Yulun and Zhang, Kai and Meng, Deyu and Timofte, Radu and Van Gool, Luc},
title = {DDFM: Denoising Diffusion Model for Multi-Modality Image Fusion},
booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
month = {October},
year = {2023},
pages = {8082-8093}
}
```
## Abstract
Multi-modality image fusion aims to combine different modalities to produce fused images that retain the complementary features of each modality, such as functional highlights and texture details. To leverage strong generative priors and address challenges such as unstable training and lack of interpretability for GAN-based generative methods, we propose a novel fusion algorithm based on the denoising diffusion probabilistic model (DDPM). The fusion task is formulated as a conditional generation problem under the DDPM sampling framework, which is further divided into an unconditional generation subproblem and a maximum likelihood subproblem. The latter is modeled in a hierarchical Bayesian manner with latent variables and inferred by the expectation-maximization (EM) algorithm. By integrating the inference solution into the diffusion sampling iteration, our method can generate high-quality fused images with natural image generative priors and cross-modality information from source images. Note that all we required is an unconditional pre-trained generative model, and no fine-tuning is needed. Our extensive experiments indicate that our approach yields promising fusion results in infrared-visible image fusion and medical image fusion.
### 🌐 Usage
### ⚙ 1. Virtual Environment
```
# create virtual environment
conda create -n DDFM python=3.8.10
conda activate DDFM
# select pytorch version yourself
# install DDFM requirements
pip install -r requirements.txt
```
### 📃 2. Pre-trained Checkpoint Preparation
From [the link](https://github.com/openai/guided-diffusion), download the checkpoint "256x256_diffusion_uncond.pt" and paste it to ``'./models/'``.
### 🏊 3. Data Preparation
Download the Infrared-Visible Fusion (IVF) and Medical Image Fusion (MIF) dataset and place the paired images in the folder ``'./input/'``.
### 🏄 4. Inference (Sampling)
If you want to infer with our DDFM and obtain the fusion results in our paper, please run
```
python sample.py
```
Then, the fused results will be saved in the ``'./output/recon/'`` folder.
Additionally,
- **[Random seed]:** For the randomly generated seed settings, we adopt the settings [seed = 3407](https://arxiv.org/abs/2109.08203) 😆😆😆(purely for fun, you can change it arbitrarily to obtain different sampling results).
- **[Sampling speedExcerpt of 7,029 characters
Read on GitHubWould you bet a product on this? Bounded 0–100 and slow moving.
matched fp:4b6f84e73d9298fc, desc:diffusion model