Simultaneous Super-Resolution and Cross-Modality Synthesis of 3D Medical Images Using Weakly-Supervised Joint Convolutional Sparse Coding

Magnetic Resonance Imaging (MRI) offers high-resolution in vivo imaging and rich functional and anatomical multimodality tissue contrast. In practice, however, there are challenges associated with considerations of scanning costs, patient comfort, and scanning time that constrain how much data can b...

Full description

Saved in:
Bibliographic Details
Published in:2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) pp. 5787 - 5796
Main Authors: Huang, Yawen, Shao, Ling, Frangi, Alejandro F.
Format: Conference Proceeding
Language:English
Published: IEEE 01.07.2017
Subjects:
Biomedical imaging
Convolutional codes
Image coding
Image reconstruction
Image resolution
Three-dimensional displays
Training
ISSN:1063-6919, 1063-6919
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Abstract Magnetic Resonance Imaging (MRI) offers high-resolution in vivo imaging and rich functional and anatomical multimodality tissue contrast. In practice, however, there are challenges associated with considerations of scanning costs, patient comfort, and scanning time that constrain how much data can be acquired in clinical or research studies. In this paper, we explore the possibility of generating high-resolution and multimodal images from low-resolution single-modality imagery. We propose the weakly-supervised joint convolutional sparse coding to simultaneously solve the problems of super-resolution (SR) and cross-modality image synthesis. The learning process requires only a few registered multimodal image pairs as the training set. Additionally, the quality of the joint dictionary learning can be improved using a larger set of unpaired images. To combine unpaired data from different image resolutions/modalities, a hetero-domain image alignment term is proposed. Local image neighborhoods are naturally preserved by operating on the whole image domain (as opposed to image patches) and using joint convolutional sparse coding. The paired images are enhanced in the joint learning process with unpaired data and an additional maximum mean discrepancy term, which minimizes the dissimilarity between their feature distributions. Experiments show that the proposed method outperforms state-of-the-art techniques on both SR reconstruction and simultaneous SR and cross-modality synthesis.
AbstractList Magnetic Resonance Imaging (MRI) offers high-resolution in vivo imaging and rich functional and anatomical multimodality tissue contrast. In practice, however, there are challenges associated with considerations of scanning costs, patient comfort, and scanning time that constrain how much data can be acquired in clinical or research studies. In this paper, we explore the possibility of generating high-resolution and multimodal images from low-resolution single-modality imagery. We propose the weakly-supervised joint convolutional sparse coding to simultaneously solve the problems of super-resolution (SR) and cross-modality image synthesis. The learning process requires only a few registered multimodal image pairs as the training set. Additionally, the quality of the joint dictionary learning can be improved using a larger set of unpaired images. To combine unpaired data from different image resolutions/modalities, a hetero-domain image alignment term is proposed. Local image neighborhoods are naturally preserved by operating on the whole image domain (as opposed to image patches) and using joint convolutional sparse coding. The paired images are enhanced in the joint learning process with unpaired data and an additional maximum mean discrepancy term, which minimizes the dissimilarity between their feature distributions. Experiments show that the proposed method outperforms state-of-the-art techniques on both SR reconstruction and simultaneous SR and cross-modality synthesis.
Author Huang, Yawen
Frangi, Alejandro F.
Shao, Ling
Author_xml – sequence: 1
  givenname: Yawen
  surname: Huang
  fullname: Huang, Yawen
  email: yhuang36@sheffield.ac.uk
  organization: Dept. of Electron. & Electr. Eng., Univ. of Sheffield, Sheffield, UK
– sequence: 2
  givenname: Ling
  surname: Shao
  fullname: Shao, Ling
  email: a.frangi@sheffield.ac.uk
  organization: Sch. of Comput. Sci., Univ. of East Anglia, Norwich, UK
– sequence: 3
  givenname: Alejandro F.
  surname: Frangi
  fullname: Frangi, Alejandro F.
  email: ling.shao@uea.ac.uk
  organization: Dept. of Electron. & Electr. Eng., Univ. of Sheffield, Sheffield, UK
BookMark eNpNjEFPwjAYhqvBRECPnrz0Dwzbdeu2o5miGIiGiR7J1_UbVkdL1o1kP8F_LVEOnt4nb_I8IzKwziIhV5xNOGfZTf72spyEjCcTycUJGfFYpJJFcRKdkiFnUgQy49ngH5-TkfefjIUiCdmQfBdm29UtWHSdp0W3wyZYond11xpnKVhN88Z5Hyychtq0PS16236gN566ioo7ukBtSqjpbAsb9HTljd3Qd4Svug9-e3vjUdMnZ2xLc2f3x_ZBKXbQeDyc-uBckLMKao-Xxx2T1fT-NX8M5s8Ps_x2HhiexG2QAYSR4oqHoFMNJZRMRBhhqVTMtE5AyxQ4SqFUprlIeBgJKTGEDOJKqUqMyfVf1yDieteYLTT9OuWMsUyKHyQIaDg
CODEN IEEPAD
ContentType Conference Proceeding
DBID 6IE
6IH
CBEJK
RIE
RIO
DOI 10.1109/CVPR.2017.613
DatabaseName IEEE Electronic Library (IEL) Conference Proceedings
IEEE Proceedings Order Plan (POP) 1998-present by volume
IEEE Xplore All Conference Proceedings
IEEE/IET Electronic Library
IEEE Proceedings Order Plans (POP) 1998-present
DatabaseTitleList
Database_xml – sequence: 1
  dbid: RIE
  name: IEL
  url: https://ieeexplore.ieee.org/
  sourceTypes: Publisher
DeliveryMethod fulltext_linktorsrc
Discipline Applied Sciences
Computer Science
EISBN 1538604574
9781538604571
EISSN 1063-6919
EndPage 5796
ExternalDocumentID 8100096
Genre orig-research
GroupedDBID 23M
29F
29O
6IE
6IH
6IK
ABDPE
ACGFS
ALMA_UNASSIGNED_HOLDINGS
CBEJK
IPLJI
M43
RIE
RIO
RNS
ID FETCH-LOGICAL-i175t-9aa24b1b12ad8dacac034e4ecbb50dd7ad68a1e63bb9d137124366e2a9a5fbbf3
IEDL.DBID RIE
ISICitedReferencesCount 153
ISICitedReferencesURI http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000418371405092&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
ISSN 1063-6919
IngestDate Wed Aug 27 03:06:19 EDT 2025
IsPeerReviewed false
IsScholarly true
Language English
LinkModel DirectLink
MergedId FETCHMERGED-LOGICAL-i175t-9aa24b1b12ad8dacac034e4ecbb50dd7ad68a1e63bb9d137124366e2a9a5fbbf3
PageCount 10
ParticipantIDs ieee_primary_8100096
PublicationCentury 2000
PublicationDate 2017-July
PublicationDateYYYYMMDD 2017-07-01
PublicationDate_xml – month: 07
  year: 2017
  text: 2017-July
PublicationDecade 2010
PublicationTitle 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
PublicationTitleAbbrev CVPR
PublicationYear 2017
Publisher IEEE
Publisher_xml – name: IEEE
SSID ssj0023720
ssj0003211698
Score 2.4755492
Snippet Magnetic Resonance Imaging (MRI) offers high-resolution in vivo imaging and rich functional and anatomical multimodality tissue contrast. In practice, however,...
SourceID ieee
SourceType Publisher
StartPage 5787
SubjectTerms Biomedical imaging
Convolutional codes
Image coding
Image reconstruction
Image resolution
Three-dimensional displays
Training
Title Simultaneous Super-Resolution and Cross-Modality Synthesis of 3D Medical Images Using Weakly-Supervised Joint Convolutional Sparse Coding
URI https://ieeexplore.ieee.org/document/8100096
WOSCitedRecordID wos000418371405092&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
link http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV07T8MwELZKxcDEoyDe8sCIaRIncTwXKkBQVRRKt8qOz1IEJFWTVupP4F9ju2nLwMJmnWTr5Ni57-678yF0xb1UBwZ5k1B5moQQgblzXmKZQhqwKOXaMbrDJ9brJaMR7zfQ9boWBgBc8hnc2KHj8lWRzmyorJ34DnJvoS3G4mWt1jqeQo0nE_M1gxDY7iuO6YwpibnPN-9rtjvD_otN6mI3rqvBr64qzqh0d_-nzh463FTn4f7a7uyjBuQHaLeGk7i-rKURrTo2rGQt9D3IbAahyME4_Hgwm8CU2AD-8vhhkSvcsWaTPBfKAXQ8WOQGIpZZiQuN6S2uiR388GV-RCV2GQf4HcTH54K49eZZabR4LLK8wkbheb22mTKYGCcajNCqfYjeunevnXtSN2MgmUEYFeFCBKH0pR8IlSiRitSjIYSQShl5SjGh4kT4EFMpufIpM7iBxjEEgotIS6npEWrmRQ7HCEttfDQQHpWBCJn2eaI8rqNUqzCAkMkT1LKbPZ4s39sY1_t8-rf4DO3Yb7lMoT1HzWo6gwu0nc6rrJxeukPyA7cBvnw
linkProvider IEEE
linkToHtml http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LTwIxEG4QTfSEisa3PXi0uLvdV88o8QGECKI30m6nyUbdJSyQ8BP817ZlQQ9evG0m2WbSbXe-6TfTD6Er5iTK08ib-NJRxIcA9J5zYsMUUi8KEqYsoztsR91u_PbGehV0ve6FAQBbfAYN82i5fJknM3NUdhO7FnJvoE2jnFV2a61PVKjOZUK25hA8o79iuc6QkpC57OeGzZvmsPdsyrqihtU1-KWrYsNKq_Y_h3bRwU9_Hu6tI88eqkC2j2oloMTldi20aaXZsLLV0Vc_NTWEPAOd8uP-bAwTYo7wlwsQ80zipgmcpJNLC9Fxf5FpkFikBc4Vpre4pHbww6f-FRXY1hzgV-DvHwtix5unhfbiMU-zKdYOz8ux9Sv9sU6jQRuN2wfopXU3aN6TUo6BpBpjTAnj3POFK1yPy1jyhCcO9cGHRIjAkTLiMoy5CyEVgkmXRho50DAEjzMeKCEUPUTVLM_gCGGhdJYG3KHC436kXBZLh6kgUdL3wI_EMaqbyR6NlzdujMp5PvnbfIm27wed9qj90H06RTvmuy4Las9QdTqZwTnaSubTtJhc2AXzDcnlwcU
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=proceeding&rft.title=2017+IEEE+Conference+on+Computer+Vision+and+Pattern+Recognition+%28CVPR%29&rft.atitle=Simultaneous+Super-Resolution+and+Cross-Modality+Synthesis+of+3D+Medical+Images+Using+Weakly-Supervised+Joint+Convolutional+Sparse+Coding&rft.au=Huang%2C+Yawen&rft.au=Shao%2C+Ling&rft.au=Frangi%2C+Alejandro+F.&rft.date=2017-07-01&rft.pub=IEEE&rft.issn=1063-6919&rft.eissn=1063-6919&rft.spage=5787&rft.epage=5796&rft_id=info:doi/10.1109%2FCVPR.2017.613&rft.externalDocID=8100096
thumbnail_l http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=1063-6919&client=summon
thumbnail_m http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=1063-6919&client=summon
thumbnail_s http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=1063-6919&client=summon