Automatic segmentation of overlapped poplar seedling leaves combining Mask R-CNN and DBSCAN

•Combination of Mask R-CNN and DBSCAN segments single leaves accurately.•A new approach to fuse color and depth features in the FPN structure is proposed.•Manifold distance is better than Euclidean distance for segmenting curled leaves.•Pixel-wise IoU evaluates the results more accurately than tradi...

Full description

Saved in:
Bibliographic Details
Published in:Computers and electronics in agriculture Vol. 178; p. 105753
Main Authors: Liu, Xuan, Hu, Chunhua, Li, Pingping
Format: Journal Article
Language:English
Published: Amsterdam Elsevier B.V 01.11.2020
Elsevier BV
Subjects:
agriculture
Algorithms
cameras
Clustering
Contact stresses
electronics
Feature extraction
Feature fusion
Genotype & phenotype
Heavy metals
Leaf segmentation
Leaves
Manifold distance
Manifolds
Mask R-CNN
Performance evaluation
phenotype
Poplar
Populus
RGB-D camera
seedlings
Segmentation
Segments
Feature fusion
Leaf segmentation
RGB-D camera
Manifold distance
Mask R-CNN
ISSN:0168-1699, 1872-7107
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:•Combination of Mask R-CNN and DBSCAN segments single leaves accurately.•A new approach to fuse color and depth features in the FPN structure is proposed.•Manifold distance is better than Euclidean distance for segmenting curled leaves.•Pixel-wise IoU evaluates the results more accurately than traditional IoU. Effective segmentation of plant leaves is very necessary for non-contact extraction of plant leaf phenotype, especially leaf phenotype under environmental stress. However, the phenotype of leaves will change due to the influence of the environment, which increases the difficulty of detection. In this study, we proposed an accurate automatic segmentation method that combines Mask R-CNN with Density-Based Spatial Clustering of Applications with Noise (DBSCAN) clustering algorithm based on RGB-D camera to segment overlapped poplar seedling leaves under heavy metal stress. Firstly, an effective encoding method of depth information was used to facilitate the feature extraction of depth information. Next, we deployed Mask R-CNN to train the RGB-D data and fuse their features in the FPN structure to obtain more accurate leaf areas. Based on the detected leaf areas and depth data, DBSCAN based on manifold distance was then applied to segment a single leaves from overlapping leaves in the detected areas. Several analyses were performed to evaluate the performance of the proposed method, including the comparison of our network with classic Mask R-CNN and the comparison of DBSCAN based on manifold distance with other classic clustering methods. We used the pixel-wise Intersection over Union (p-IoU) to evaluate the detection results more accurately. In the experiments, the obtained p-IoU of normal and stressed leaves was 0.885 and 0.874, respectively, with corresponding mean accuracy values of 0.897 and 0.888. From our experimental results, it can be concluded that the proposed method can automatically detect leaves with high accuracy, which can be applied to 3-D leaf phenotype research and automatic plant de-leafing.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
content type line 23
ISSN:0168-1699
1872-7107
DOI:10.1016/j.compag.2020.105753