From Missing Pieces to Masterpieces: Image Completion With Context-Adaptive Diffusion
Saved in:
| Title: | From Missing Pieces to Masterpieces: Image Completion With Context-Adaptive Diffusion |
|---|---|
| Authors: | Pourya Shamsolmoali, Masoumeh Zareapoor, Huiyu Zhou, Michael Felsberg, Dacheng Tao, Xuelong Li |
| Source: | IEEE Transactions on Pattern Analysis and Machine Intelligence. 47:6073-6087 |
| Publication Status: | Preprint |
| Publisher Information: | Institute of Electrical and Electronics Engineers (IEEE), 2025. |
| Publication Year: | 2025 |
| Subject Terms: | Computer graphics and computer vision, FOS: Computer and information sciences, Computer Vision and Pattern Recognition (cs.CV), Computer Science - Computer Vision and Pattern Recognition, Diffusion models, Solid modeling, Training, Noise reduction, Adaptation models, Image reconstruction, Diffusion processes, Degradation, Coherence, Transformers, Image completion, diffusion models, context-adaptive discrepancy, Datorgrafik och datorseende |
| Description: | Image completion is a challenging task, particularly when ensuring that generated content seamlessly integrates with existing parts of an image. While recent diffusion models have shown promise, they often struggle with maintaining coherence between known and unknown (missing) regions. This issue arises from the lack of explicit spatial and semantic alignment during the diffusion process, resulting in content that does not smoothly integrate with the original image. Additionally, diffusion models typically rely on global learned distributions rather than localized features, leading to inconsistencies between the generated and existing image parts. In this work, we propose ConFill, a novel framework that introduces a Context-Adaptive Discrepancy (CAD) model to ensure that intermediate distributions of known and unknown regions are closely aligned throughout the diffusion process. By incorporating CAD, our model progressively reduces discrepancies between generated and original images at each diffusion step, leading to contextually aligned completion. Moreover, ConFill uses a new Dynamic Sampling mechanism that adaptively increases the sampling rate in regions with high reconstruction complexity. This approach enables precise adjustments, enhancing detail and integration in restored areas. Extensive experiments demonstrate that ConFill outperforms current methods, setting a new benchmark in image completion. Accepted in TPAMI |
| Document Type: | Article |
| File Description: | application/pdf |
| ISSN: | 1939-3539 0162-8828 |
| DOI: | 10.1109/tpami.2025.3558092 |
| DOI: | 10.48550/arxiv.2504.14294 |
| Access URL: | https://pubmed.ncbi.nlm.nih.gov/40184300 http://arxiv.org/abs/2504.14294 http://urn.kb.se/resolve?urn=urn:nbn:se:liu:diva-215351 |
| Rights: | IEEE Copyright CC BY SA |
| Accession Number: | edsair.doi.dedup.....8e1cc106cced3e71af16c99c647bdac7 |
| Database: | OpenAIRE |
Full Text Finder 
Nájsť tento článok vo Web of Science | Abstract: | Image completion is a challenging task, particularly when ensuring that generated content seamlessly integrates with existing parts of an image. While recent diffusion models have shown promise, they often struggle with maintaining coherence between known and unknown (missing) regions. This issue arises from the lack of explicit spatial and semantic alignment during the diffusion process, resulting in content that does not smoothly integrate with the original image. Additionally, diffusion models typically rely on global learned distributions rather than localized features, leading to inconsistencies between the generated and existing image parts. In this work, we propose ConFill, a novel framework that introduces a Context-Adaptive Discrepancy (CAD) model to ensure that intermediate distributions of known and unknown regions are closely aligned throughout the diffusion process. By incorporating CAD, our model progressively reduces discrepancies between generated and original images at each diffusion step, leading to contextually aligned completion. Moreover, ConFill uses a new Dynamic Sampling mechanism that adaptively increases the sampling rate in regions with high reconstruction complexity. This approach enables precise adjustments, enhancing detail and integration in restored areas. Extensive experiments demonstrate that ConFill outperforms current methods, setting a new benchmark in image completion.<br />Accepted in TPAMI |
|---|---|
| ISSN: | 19393539 01628828 |
| DOI: | 10.1109/tpami.2025.3558092 |