Whole-cell segmentation of tissue images with human-level performance using large-scale data annotation and deep learning

A principal challenge in the analysis of tissue imaging data is cell segmentation—the task of identifying the precise boundary of every cell in an image. To address this problem we constructed TissueNet, a dataset for training segmentation models that contains more than 1 million manually labeled ce...

Full description

Saved in:
Bibliographic Details
Published in:Nature biotechnology Vol. 40; no. 4; pp. 555 - 565
Main Authors: Greenwald, Noah F., Miller, Geneva, Moen, Erick, Kong, Alex, Kagel, Adam, Dougherty, Thomas, Fullaway, Christine Camacho, McIntosh, Brianna J., Leow, Ke Xuan, Schwartz, Morgan Sarah, Pavelchek, Cole, Cui, Sunny, Camplisson, Isabella, Bar-Tal, Omer, Singh, Jaiveer, Fong, Mara, Chaudhry, Gautam, Abraham, Zion, Moseley, Jackson, Warshawsky, Shiri, Soon, Erin, Greenbaum, Shirley, Risom, Tyler, Hollmann, Travis, Bendall, Sean C., Keren, Leeat, Graf, William, Angelo, Michael, Van Valen, David
Format: Journal Article
Language:English
Published: New York Nature Publishing Group US 01.04.2022
Nature Publishing Group
Subjects:
631/114/1564
631/114/794
631/1647/245
Agriculture
Algorithms
Annotations
Bioinformatics
Biomedical and Life Sciences
Biomedical Engineering/Biotechnology
Biomedicine
Biotechnology
Cell lineage
Cell morphology
Cytology
Data Curation
Datasets
Deep Learning
Feature extraction
Human performance
Humans
Image processing
Image Processing, Computer-Assisted - methods
Image segmentation
Life Sciences
Localization
Machine learning
Tissue analysis
Training
ISSN:1087-0156, 1546-1696, 1546-1696
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:A principal challenge in the analysis of tissue imaging data is cell segmentation—the task of identifying the precise boundary of every cell in an image. To address this problem we constructed TissueNet, a dataset for training segmentation models that contains more than 1 million manually labeled cells, an order of magnitude more than all previously published segmentation training datasets. We used TissueNet to train Mesmer, a deep-learning-enabled segmentation algorithm. We demonstrated that Mesmer is more accurate than previous methods, generalizes to the full diversity of tissue types and imaging platforms in TissueNet, and achieves human-level performance. Mesmer enabled the automated extraction of key cellular features, such as subcellular localization of protein signal, which was challenging with previous approaches. We then adapted Mesmer to harness cell lineage information in highly multiplexed datasets and used this enhanced version to quantify cell morphology changes during human gestation. All code, data and models are released as a community resource. Deep learning algorithms perform as well as humans in identifying cells in tissue images.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
content type line 23
Authorship Contributions
These authors contributed equally to this work
N.F.G., L.K., M.A., and D.V.V. conceived the project. E.M. and D.V.V. conceived the human-in-the-loop approach. L.K. and M.A. conceived the whole-cell segmentation approach. G.M., T.D., E.M., W.G., and D.V.V. developed DeepCell Label. G.M., N.F.G., E.M., I.C., W.G., and D.V.V. developed the human-in-the-loop pipeline. M.S., C.P., W.G., and D.V.V. developed Mesmer’s deep learning architecture. W.G., N.F.G., and D.V.V. developed model training software. C.P. and W.G. developed cloud deployment. M.S., S.C., W.G., and D.V.V. developed metrics software. W.G. developed plug-ins. N.F.G., A.Kong, A.Kagel, J.S., and O.B-T. developed the multiplex image analysis pipeline. A.Kagel and G.M. developed the pathologist evaluation software. N.F.G., G.M., and T.H. supervised training data creation. N.F.G., C.C.F., B.M., K.L., M.F., G.C., Z.A., J.M., and S.W. performed quality control on the training data. E.S., S.G., and T.R. generated MIBI-TOF data for morphological analyses. S.B. helped with experimental design. N.F.G., W.G., and D.V.V. trained the models. N.F.G., W.G., G.M., and D.V.V. performed data analysis. N.F.G., G.M., M.A., and D.V.V. wrote the manuscript. M.A. and D.V.V. supervised the project. All authors provided feedback on the manuscript.
ISSN:1087-0156
1546-1696
1546-1696
DOI:10.1038/s41587-021-01094-0