Multi-Modal Transformer for Compressive LiDARs Using Hyperspectral Imaging Side-Information

Compressive satellite LiDAR (CS-LiDAR) has been recently introduced as a radically different computational sensing and reconstruction approach for LiDAR sensing of Earth. It is based on NASA's adaptive wavelength scanning LiDAR (AWSL) system. Unlike conventional 1D LiDAR methods, CS-LiDAR utili...

Full description

Saved in:
Bibliographic Details
Published in:IEEE International Geoscience and Remote Sensing Symposium proceedings pp. 2451 - 2454
Main Authors: Porras-Diaz, N., Ramirez-Jaime, A., Arce, G. R., Vargas, R., Harding, D., Stephen, M., MacKinnon, J.
Format: Conference Proceeding
Language:English
Published: IEEE 07.07.2024
Subjects:
Adaptation models
Earth
Hyperspectral
Image coding
Imaging
Laser radar
LiDAR
Machine Learning
Side information
Three-dimensional displays
Training
Transformers
ISSN:2153-7003
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Abstract Compressive satellite LiDAR (CS-LiDAR) has been recently introduced as a radically different computational sensing and reconstruction approach for LiDAR sensing of Earth. It is based on NASA's adaptive wavelength scanning LiDAR (AWSL) system. Unlike conventional 1D LiDAR methods, CS-LiDAR utilizes sparse coded laser illumination across a 2D field-of-view. The aim is to compressively capture Earth from hundreds of kilometers above, enabling computational 3D imagery reconstruction with resolution that is comparable to that attained with data collected from just hundreds of meters. The forward imaging model captures the light propagation phenomena affecting the photon pulses transmitted from the sensor to the Earth's surface and back. This work enhances CS-LiDAR by integrating imaging spectroscopy into a multimodal system and employing a transformer network for the inverse imaging problem, driven by multimodal attention mechanisms. Emulations enabled by enormous observational LiDAR data of Earth, available from NASA's G-LiHT imaging observatory, highlight the efficacy of methods developed.
AbstractList Compressive satellite LiDAR (CS-LiDAR) has been recently introduced as a radically different computational sensing and reconstruction approach for LiDAR sensing of Earth. It is based on NASA's adaptive wavelength scanning LiDAR (AWSL) system. Unlike conventional 1D LiDAR methods, CS-LiDAR utilizes sparse coded laser illumination across a 2D field-of-view. The aim is to compressively capture Earth from hundreds of kilometers above, enabling computational 3D imagery reconstruction with resolution that is comparable to that attained with data collected from just hundreds of meters. The forward imaging model captures the light propagation phenomena affecting the photon pulses transmitted from the sensor to the Earth's surface and back. This work enhances CS-LiDAR by integrating imaging spectroscopy into a multimodal system and employing a transformer network for the inverse imaging problem, driven by multimodal attention mechanisms. Emulations enabled by enormous observational LiDAR data of Earth, available from NASA's G-LiHT imaging observatory, highlight the efficacy of methods developed.
Author Arce, G. R.
Porras-Diaz, N.
Vargas, R.
Ramirez-Jaime, A.
Harding, D.
Stephen, M.
MacKinnon, J.
Author_xml – sequence: 1
  givenname: N.
  surname: Porras-Diaz
  fullname: Porras-Diaz, N.
  organization: University of Delaware, ECE
– sequence: 2
  givenname: A.
  surname: Ramirez-Jaime
  fullname: Ramirez-Jaime, A.
  organization: University of Delaware, ECE
– sequence: 3
  givenname: G. R.
  surname: Arce
  fullname: Arce, G. R.
  organization: University of Delaware, ECE
– sequence: 4
  givenname: R.
  surname: Vargas
  fullname: Vargas, R.
  organization: University of Delaware, PaSS
– sequence: 5
  givenname: D.
  surname: Harding
  fullname: Harding, D.
  organization: NASA, GSFC
– sequence: 6
  givenname: M.
  surname: Stephen
  fullname: Stephen, M.
  organization: NASA, GSFC
– sequence: 7
  givenname: J.
  surname: MacKinnon
  fullname: MacKinnon, J.
  organization: NASA, GSFC
BookMark eNo1kMtOwzAQRQ0Cibb0D1iED0gZe-zEXlYF2kitkPpYsajcZFIZNQ_ZAal_31TAZs7oXs1ZzJDd1U1NjD1zmHAO5iWbT9ebjUKZqokAISccEsmlNDdsbFKjUQEmgELdsoHgCuMUAB_YMISvftECYMA-V9-nzsWrprCnaOttHcrGV-SjHtGsqVpPIbgfipbudboO0S64-hgtzi350FLe-f4sq-zxmm5cQXFWXwW2c039yO5Lewo0_uOI7d7ftrNFvPyYZ7PpMnY8TbrYpsDzhKPNe5jSaM25yukgDCUorJSQHFRZoMEDJ6JSadRQaE39KDgKHLGnX6_r633rXWX9ef__DLwABvpXBA
ContentType Conference Proceeding
DBID 6IE
6IH
CBEJK
RIE
RIO
DOI 10.1109/IGARSS53475.2024.10641449
DatabaseName IEEE Electronic Library (IEL) Conference Proceedings
IEEE Proceedings Order Plan (POP) 1998-present by volume
IEEE Xplore All Conference Proceedings
IEEE Electronic Library (IEL)
IEEE Proceedings Order Plans (POP) 1998-present
DatabaseTitleList
Database_xml – sequence: 1
  dbid: RIE
  name: IEEE Electronic Library (IEL)
  url: https://ieeexplore.ieee.org/
  sourceTypes: Publisher
DeliveryMethod fulltext_linktorsrc
Discipline Geology
EISBN 9798350360325
EISSN 2153-7003
EndPage 2454
ExternalDocumentID 10641449
Genre orig-research
GroupedDBID 6IE
6IF
6IH
6IK
6IL
6IM
6IN
AAJGR
AAWTH
ADZIZ
ALMA_UNASSIGNED_HOLDINGS
BEFXN
BFFAM
BGNUA
BKEBE
BPEOZ
CBEJK
CHZPO
IPLJI
OCL
RIE
RIL
RIO
RNS
ID FETCH-LOGICAL-i176t-a701c613ac1c69f988115ceb29e632a4406b5fd393b1eeef58380d88e0d8d1323
IEDL.DBID RIE
ISICitedReferencesCount 3
ISICitedReferencesURI http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=001316158502192&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
IngestDate Wed Aug 27 02:03:01 EDT 2025
IsPeerReviewed false
IsScholarly false
Language English
LinkModel DirectLink
MergedId FETCHMERGED-LOGICAL-i176t-a701c613ac1c69f988115ceb29e632a4406b5fd393b1eeef58380d88e0d8d1323
PageCount 4
ParticipantIDs ieee_primary_10641449
PublicationCentury 2000
PublicationDate 2024-July-7
PublicationDateYYYYMMDD 2024-07-07
PublicationDate_xml – month: 07
  year: 2024
  text: 2024-July-7
  day: 07
PublicationDecade 2020
PublicationTitle IEEE International Geoscience and Remote Sensing Symposium proceedings
PublicationTitleAbbrev IGARSS
PublicationYear 2024
Publisher IEEE
Publisher_xml – name: IEEE
SSID ssj0038200
Score 1.8979386
Snippet Compressive satellite LiDAR (CS-LiDAR) has been recently introduced as a radically different computational sensing and reconstruction approach for LiDAR...
SourceID ieee
SourceType Publisher
StartPage 2451
SubjectTerms Adaptation models
Earth
Hyperspectral
Image coding
Imaging
Laser radar
LiDAR
Machine Learning
Side information
Three-dimensional displays
Training
Transformers
Title Multi-Modal Transformer for Compressive LiDARs Using Hyperspectral Imaging Side-Information
URI https://ieeexplore.ieee.org/document/10641449
WOSCitedRecordID wos001316158502192&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
link http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1bS8MwFD64oeKTt4l3Ivia2TZt0zwOdRfQMbYJAx9G2iRQ0E26C_jvPclaxQcffGlK6KE0h_TLSc53PoBbrljKIxFQlUlDQ64klWnCqJEiDmOTesIzTmyC9_vJZCIGJVndcWG01i75TDftrTvLV_NsZbfKcIbHIQYAogY1zuMNWav67TKEMm8Xbsoimne9Tms4GkUs5BFGgUHYrIx_yag4FGnv__P9B9D44eORwTfSHMKWnh3BTseJ8n4ew6tj0dLnuZJvZFytRHVBsCF2wrtc17UmT_lDa7ggLk2AdDEC3RAtCzTrvTu5IjLKlaYlR8n6rAEv7cfxfZeWogk093m8pJJ7foYYLTNshBFJgmu-DONnoWMWyBABPI2MYoKlPn6gscemnkoSjReFoSk7gfpsPtOnQAQ-6aGNsUX3Uz-WImOpxr4gUVFoojNo2DGafmzqYkyr4Tn_o_8C9qwnXLIrv4T6sljpK9jO1st8UVw7b34BWEqhFQ
linkProvider IEEE
linkToHtml http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1bS8MwFA46r0_eJt6N4Gu0bdKmeRzqLriNsU0Y-DDSJoGCbtJdwH_vSdYqPvjgS1NCD7Q5pF9Ocr7zIXTLFU14KAKiUmkI40oSmcSUGCkiFpnEE55xYhO8241HI9EryOqOC6O1dsln-s7eurN8NU0XdqsMZnjEIAAQ62gjZCzwVnSt8sdLAcy8bXRTlNG8bzVq_cEgpIyHEAcG7K40_yWk4nCkvvfPN9hH1R9GHu59Y80BWtOTQ7TVcLK8n0fo1fFoSWeq5BselmtRnWNosJ3yLtt1qXE7e6z1Z9glCuAmxKArqmUOZq13J1iEB5nSpGApWa9V0Uv9afjQJIVsAsl8Hs2J5J6fAkrLFBphRBzDqi-FCFroiAaSAYQnoVFU0MSHDzT24NRTcazhoiA4pceoMplO9AnCAp70wMbYsvuJH0mR0kRDXxCrkJnwFFXtGI0_VpUxxuXwnP3Rf412msNOe9xudZ_P0a71ikt95ReoMs8X-hJtpst5NsuvnGe_AOj2pFw
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=proceeding&rft.title=IEEE+International+Geoscience+and+Remote+Sensing+Symposium+proceedings&rft.atitle=Multi-Modal+Transformer+for+Compressive+LiDARs+Using+Hyperspectral+Imaging+Side-Information&rft.au=Porras-Diaz%2C+N.&rft.au=Ramirez-Jaime%2C+A.&rft.au=Arce%2C+G.+R.&rft.au=Vargas%2C+R.&rft.date=2024-07-07&rft.pub=IEEE&rft.eissn=2153-7003&rft.spage=2451&rft.epage=2454&rft_id=info:doi/10.1109%2FIGARSS53475.2024.10641449&rft.externalDocID=10641449