Object Detection Using Sim2Real Domain Randomization for Robotic Applications

Robots working in unstructured environments must be capable of sensing and interpreting their surroundings. One of the main obstacles of deep-learning-based models in the field of robotics is the lack of domain-specific labeled data for different industrial applications. In this article, we propose...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on robotics Vol. 39; no. 2; pp. 1225 - 1243
Main Authors: Horvath, Daniel, Erdos, Gabor, Istenes, Zoltan, Horvath, Tomas, Foldi, Sandor
Format: Journal Article
Language:English
Published: New York IEEE 01.04.2023
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subjects:
Adaptation models
Artificial neural networks
Computational modeling
Computer vision for automation
Datasets
Deep learning
deep learning in robotics and automation
Industrial applications
localization
Object detection
Object recognition
Randomization
Robotics
Robots
Service robots
sim2real knowledge transfer
Synthetic data
Task analysis
Training
ISSN:1552-3098, 1941-0468
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Robots working in unstructured environments must be capable of sensing and interpreting their surroundings. One of the main obstacles of deep-learning-based models in the field of robotics is the lack of domain-specific labeled data for different industrial applications. In this article, we propose a sim2real transfer learning method based on domain randomization for object detection with which labeled synthetic datasets of arbitrary size and object types can be automatically generated. Subsequently, a state-of-the-art convolutional neural network, YOLOv4, is trained to detect the different types of industrial objects. With the proposed domain randomization method, we could shrink the reality gap to a satisfactory level, achieving 86.32% and 97.38% <inline-formula><tex-math notation="LaTeX">\mathrm{{mAP}}_{50}</tex-math></inline-formula> scores, respectively, in the case of zero-shot and one-shot transfers, on our manually annotated dataset containing 190 real images. Our solution fits for industrial use as the data generation process takes less than 0.5 s per image and the training lasts only around 12 h, on a GeForce RTX 2080 Ti GPU. Furthermore, it can reliably differentiate similar classes of objects by having access to only one real image for training. To our best knowledge, this is the only work thus far satisfying these constraints.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:1552-3098
1941-0468
DOI:10.1109/TRO.2022.3207619