Optimizing photo-to-anime translation with prestyled paired datasets

Animation is a widespread artistic expression that holds a special place in people's hearts. Traditionally, animation creation has relied heavily on manual techniques, demanding skilled drawing abilities and a significant amount of time. For instance, many Japanese anime films draw inspiration...

Full description

Saved in:
Bibliographic Details
Published in:Multimedia tools and applications Vol. 83; no. 41; pp. 89393 - 89414
Main Authors: Chang, Chuan-Wang, Dharmawan, Pratamagusta
Format: Journal Article
Language:English
Published: New York Springer US 01.12.2024
Springer Nature B.V
Subjects:
1237: Advanced Deep Learning for Computer Vision and Multimedia Applications
Animation
Anime
Artists
Computer Communication Networks
Computer Science
Data Structures and Information Theory
Datasets
Deep learning
Multimedia
Multimedia Information Systems
Special Purpose and Application-Based Systems
Unsupervised learning
Image style transfer
GAN
U-Net
Pix2pix
Photo-to-anime translation
ISSN:1573-7721, 1380-7501, 1573-7721
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Animation is a widespread artistic expression that holds a special place in people's hearts. Traditionally, animation creation has relied heavily on manual techniques, demanding skilled drawing abilities and a significant amount of time. For instance, many Japanese anime films draw inspiration from real-world settings, requiring access to relevant references and artists capable of translating them into anime visuals. Consequently, the development of technology that automatically converts images into anime holds great significance. Numerous methods for style transfer have been developed using unsupervised learning and have achieved impressive results. However, unsupervised learning methods suffer when the image contains multiple styles within itself because they learn the style of the image globally. To solve this problem, we propose splitting these styles within the image into multiple classes: sky, buildings, greenery, water, and other objects. Then, we style these separated classes using existing image-to-image translation models. Finally, we train a pix2pix model to learn image style transfer in a paired manner. The experimental results show that the images are effectively styled into the resulting anime-styled image domain with comparable results to existing unsupervised learning GAN-based methods. The proposed method can effectively transfer the style from real-world photos into the anime-styled image domain.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:1573-7721
1380-7501
1573-7721
DOI:10.1007/s11042-024-20339-z