DeepTaste: Augmented Reality Gustatory Manipulation with GAN-Based Real-Time Food-to-Food Translation

We have been studying augmented reality (AR)-based gustatory manipulation interfaces and previously proposed a gustatory manipulation interface using generative adversarial network (GAN)-based real time image-to-image translation. Unlike three-dimensional (3D) food model-based systems that only chan...

Full description

Saved in:
Bibliographic Details
Published in:2019 IEEE International Symposium on Mixed and Augmented Reality (ISMAR) pp. 212 - 223
Main Authors: Nakano, Kizashi, Horita, Daichi, Sakata, Nobuchika, Kiyokawa, Kiyoshi, Yanai, Keiji, Narumi, Takuji
Format: Conference Proceeding
Language:English
Published: IEEE 01.10.2019
Subjects:
Human centered computing Human computer interaction (HCI) Interaction paradigms Mixed / augmented reality Computing methodologies Computer graphics Graphics systems and interfaces Perception Computing methodologies Machine learning Machine learning approaches Neural networks
Image color analysis
Modulation
Real-time systems
Resists
Solid modeling
Three-dimensional displays
Visualization
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Abstract We have been studying augmented reality (AR)-based gustatory manipulation interfaces and previously proposed a gustatory manipulation interface using generative adversarial network (GAN)-based real time image-to-image translation. Unlike three-dimensional (3D) food model-based systems that only change the color or texture pattern of a particular type of food in an inflexible manner, our GAN-based system changes the appearance of food into multiple types of food in real time flexibly, dynamically, and interactively. In the present paper, we first describe in detail a user study on a vision-induced gustatory manipulation system using a 3D food model and report its successful experimental results. We then summarize identified problems of the 3D model-based system and describe implementation details of the GAN-based system. We finally report in detail the main user study in which we investigated the impact of the GAN-based system on gustatory sensations and food recognition when somen noodles were turned into ramen noodles or fried noodles, and steamed rice into curry and rice or fried rice. The experimental results revealed that our system successfully manipulates gustatory sensations to some extent and that the effectiveness seems to depend on the original and target types of food as well as the experience of each individual with the food.
AbstractList We have been studying augmented reality (AR)-based gustatory manipulation interfaces and previously proposed a gustatory manipulation interface using generative adversarial network (GAN)-based real time image-to-image translation. Unlike three-dimensional (3D) food model-based systems that only change the color or texture pattern of a particular type of food in an inflexible manner, our GAN-based system changes the appearance of food into multiple types of food in real time flexibly, dynamically, and interactively. In the present paper, we first describe in detail a user study on a vision-induced gustatory manipulation system using a 3D food model and report its successful experimental results. We then summarize identified problems of the 3D model-based system and describe implementation details of the GAN-based system. We finally report in detail the main user study in which we investigated the impact of the GAN-based system on gustatory sensations and food recognition when somen noodles were turned into ramen noodles or fried noodles, and steamed rice into curry and rice or fried rice. The experimental results revealed that our system successfully manipulates gustatory sensations to some extent and that the effectiveness seems to depend on the original and target types of food as well as the experience of each individual with the food.
Author Nakano, Kizashi
Narumi, Takuji
Yanai, Keiji
Horita, Daichi
Kiyokawa, Kiyoshi
Sakata, Nobuchika
Author_xml – sequence: 1
  givenname: Kizashi
  surname: Nakano
  fullname: Nakano, Kizashi
  organization: Nara Institute of Science and Technology
– sequence: 2
  givenname: Daichi
  surname: Horita
  fullname: Horita, Daichi
  organization: The University of Electro-Communications
– sequence: 3
  givenname: Nobuchika
  surname: Sakata
  fullname: Sakata, Nobuchika
  organization: Nara Institute of Science and Technology
– sequence: 4
  givenname: Kiyoshi
  surname: Kiyokawa
  fullname: Kiyokawa, Kiyoshi
  organization: Nara Institute of Science and Technology
– sequence: 5
  givenname: Keiji
  surname: Yanai
  fullname: Yanai, Keiji
  organization: The University of Electro-Communications
– sequence: 6
  givenname: Takuji
  surname: Narumi
  fullname: Narumi, Takuji
  organization: The University of Tokyo
BookMark eNotjL1OwzAURo0EA5TuSCx-AZdrJ41ttlBoqNSCVMJcucm9YCl_ShyhvD2gdjrDd75zwy6btkHG7iQspAT7sPnYpfuFAmkXACDkBZtbbaRW5m812l4zfEbscjcEfOTp-FVjE7Dke3SVDxPPxiG40PYT37nGd2Plgm8b_uPDN8_SN_HkhrMtcl8jX7dtKUIr_snz3jXD6XHLrshVA87PnLHP9Uu-ehXb92yzSrfCK4iCoOIIRZHEUpWKwNAytgZLS0QWdEGYoAaVJDHRkiDBmI5UIhUaInAyQRXN2P2p6xHx0PW-dv10MDaOtLbRL6--VJ4
ContentType Conference Proceeding
DBID 6IE
6IL
CBEJK
RIE
RIL
DOI 10.1109/ISMAR.2019.000-1
DatabaseName IEEE Electronic Library (IEL) Conference Proceedings
IEEE Proceedings Order Plan All Online (POP All Online) 1998-present by volume
IEEE Xplore All Conference Proceedings
IEEE Electronic Library (IEL)
IEEE Proceedings Order Plans (POP All) 1998-Present
DatabaseTitleList
Database_xml – sequence: 1
  dbid: RIE
  name: IEEE Electronic Library (IEL)
  url: https://ieeexplore.ieee.org/
  sourceTypes: Publisher
DeliveryMethod fulltext_linktorsrc
EISBN 9781728109879
1728109876
EndPage 223
ExternalDocumentID 8943779
Genre orig-research
GroupedDBID 6IE
6IL
CBEJK
RIE
RIL
ID FETCH-LOGICAL-i203t-fcb0cc6412d2f08f5498ed9fff907cfe6e702664ff5f06e4fbfdefc7030a16e23
IEDL.DBID RIE
ISICitedReferencesCount 28
ISICitedReferencesURI http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000525841300022&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
IngestDate Mon Jul 08 05:38:45 EDT 2024
IsPeerReviewed false
IsScholarly true
Language English
LinkModel DirectLink
MergedId FETCHMERGED-LOGICAL-i203t-fcb0cc6412d2f08f5498ed9fff907cfe6e702664ff5f06e4fbfdefc7030a16e23
PageCount 12
ParticipantIDs ieee_primary_8943779
PublicationCentury 2000
PublicationDate 2019-Oct.
PublicationDateYYYYMMDD 2019-10-01
PublicationDate_xml – month: 10
  year: 2019
  text: 2019-Oct.
PublicationDecade 2010
PublicationTitle 2019 IEEE International Symposium on Mixed and Augmented Reality (ISMAR)
PublicationTitleAbbrev ISMAR
PublicationYear 2019
Publisher IEEE
Publisher_xml – name: IEEE
Score 2.2470634
Snippet We have been studying augmented reality (AR)-based gustatory manipulation interfaces and previously proposed a gustatory manipulation interface using...
SourceID ieee
SourceType Publisher
StartPage 212
SubjectTerms Human centered computing Human computer interaction (HCI) Interaction paradigms Mixed / augmented reality Computing methodologies Computer graphics Graphics systems and interfaces Perception Computing methodologies Machine learning Machine learning approaches Neural networks
Image color analysis
Modulation
Real-time systems
Resists
Solid modeling
Three-dimensional displays
Visualization
Title DeepTaste: Augmented Reality Gustatory Manipulation with GAN-Based Real-Time Food-to-Food Translation
URI https://ieeexplore.ieee.org/document/8943779
WOSCitedRecordID wos000525841300022&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
link http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1NTwIxEG2AePCkBozf6cGjlW53abve8AP0ACGICTdS2hnDZSG4-Pttu4gx8eKpTdOmSSfN6-vMmyHk2qLxnCsDz01yxzKdSGZQ5yxNBaZWeMwzNhabUMOhnk7zUY3c7LQwABCDz-A2dKMv3y3tJnyVtUOucKXyOqkrJSut1rfnkeftl9dBdxyCtUIGSs6SX_VSIlz0Dv630SFp_eju6GiHKEekBkWTwCPAamK8Ne5od_Mek2g6Oob4gKb9IIAKjnI6MMXiuxgXDd-rtN8dsnuPUtVsFtQetBfi1MslCy2NQFWtaJG33tPk4ZltiyOwheBpydDOubUyS4QTyDV6nqfB5Yjo6a5FkKA8vZIZYge5hAzn6ABtuOAmkSDSY9IolgWcECq9PYwVCjIVXkdSOy2t6Tg-N8K4jj4lzXBEs1WV_2K2PZ2zv4fPyX6wQRXwdkEa5XoDl2TPfpaLj_VVNNoXGm6cxg
linkProvider IEEE
linkToHtml http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LTwMhECa1muhJTWt8y8Gj2F2WZVlv9dFHbDdNrUlvDYXB9LJt6tbfL7BtjYkXTxACIWFCPj5mvhmEbpWRlnMxsNwk1YSJkBNpREqiiJpIUYt5UvliE0mWifE4HVTQ3VYLAwA--AzuXdf78vVcrdxXWcPlCk-SdAftxozRoFRrbXyPQdrovvWbQxeu5XJQBiT8VTHFA0br8H9bHaH6j_IOD7aYcowqkNcQPAMsRtLa4wE3Vx8-jabGQ_BPaNx2EijnKsd9mc825biw-2DF7WZGHi1OlbOJ03vglotUL-bEtdhDVbmijt5bL6OnDlmXRyAzGkQFMWoaKMVZSDU1gTCW6QnQqTHGEl5lgENiCRZnxsQm4MDM1Ggwyl1xGXKg0Qmq5vMcThHm1iJS0QRY4t5HXGjBlYx1MJVU6licoZo7osmizIAxWZ_O-d_DN2i_M-r3Jr1u9nqBDpw9yvC3S1Qtliu4Qnvqq5h9Lq-9Ab8BbPWgDQ
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=proceeding&rft.title=2019+IEEE+International+Symposium+on+Mixed+and+Augmented+Reality+%28ISMAR%29&rft.atitle=DeepTaste%3A+Augmented+Reality+Gustatory+Manipulation+with+GAN-Based+Real-Time+Food-to-Food+Translation&rft.au=Nakano%2C+Kizashi&rft.au=Horita%2C+Daichi&rft.au=Sakata%2C+Nobuchika&rft.au=Kiyokawa%2C+Kiyoshi&rft.date=2019-10-01&rft.pub=IEEE&rft.spage=212&rft.epage=223&rft_id=info:doi/10.1109%2FISMAR.2019.000-1&rft.externalDocID=8943779