FaceForge: Markerless Non-Rigid Face Multi-Projection Mapping

Recent publications and art performances demonstrate amazing results using projection mapping. To our knowledge, there exists no multi-projection system that can project onto non-rigid target geometries. This constrains the applicability and quality for live performances with multiple spectators. Gi...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on visualization and computer graphics Vol. 23; no. 11; pp. 2440 - 2446
Main Authors: Siegl, Christian, Lange, Vanessa, Stamminger, Marc, Bauer, Frank, Thies, Justus
Format: Journal Article
Language:English
Published: United States IEEE 01.11.2017
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subjects:
Augmented reality
Cameras
Computational modeling
Face Projection
Face recognition
Image color analysis
Mapping
Mixed Reality
Multi-Projection Mapping
Non-Rigid Face Tracking
Object tracking
Projection
Quality
Spectators
Target tracking
ISSN:1077-2626, 1941-0506, 1941-0506
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Abstract Recent publications and art performances demonstrate amazing results using projection mapping. To our knowledge, there exists no multi-projection system that can project onto non-rigid target geometries. This constrains the applicability and quality for live performances with multiple spectators. Given the cost and complexity of current systems, we present a low-cost easy-to-use markerless non-rigid face multi-projection system. It is based on a non-rigid, dense face tracker and a real-time multi-projection solver adapted to imprecise tracking, geometry and calibration. Using this novel system we produce compelling results with only consumer-grade hardware.
AbstractList Recent publications and art performances demonstrate amazing results using projection mapping. To our knowledge, there exists no multi-projection system that can project onto non-rigid target geometries. This constrains the applicability and quality for live performances with multiple spectators. Given the cost and complexity of current systems, we present a low-cost easy-to-use markerless non-rigid face multi-projection system. It is based on a non-rigid, dense face tracker and a real-time multi-projection solver adapted to imprecise tracking, geometry and calibration. Using this novel system we produce compelling results with only consumer-grade hardware.
Recent publications and art performances demonstrate amazing results using projection mapping. To our knowledge, there exists no multi-projection system that can project onto non-rigid target geometries. This constrains the applicability and quality for live performances with multiple spectators. Given the cost and complexity of current systems, we present a low-cost easy-to-use markerless non-rigid face multi-projection system. It is based on a non-rigid, dense face tracker and a real-time multi-projection solver adapted to imprecise tracking, geometry and calibration. Using this novel system we produce compelling results with only consumer-grade hardware.Recent publications and art performances demonstrate amazing results using projection mapping. To our knowledge, there exists no multi-projection system that can project onto non-rigid target geometries. This constrains the applicability and quality for live performances with multiple spectators. Given the cost and complexity of current systems, we present a low-cost easy-to-use markerless non-rigid face multi-projection system. It is based on a non-rigid, dense face tracker and a real-time multi-projection solver adapted to imprecise tracking, geometry and calibration. Using this novel system we produce compelling results with only consumer-grade hardware.
Author Siegl, Christian
Lange, Vanessa
Stamminger, Marc
Bauer, Frank
Thies, Justus
Author_xml – sequence: 1
  givenname: Christian
  surname: Siegl
  fullname: Siegl, Christian
  email: Christian.Siegl@fau.de
– sequence: 2
  givenname: Vanessa
  surname: Lange
  fullname: Lange, Vanessa
  email: Vanessa.Lange@fau.de
– sequence: 3
  givenname: Marc
  surname: Stamminger
  fullname: Stamminger, Marc
  email: Marc.Stamminger@fau.de
– sequence: 4
  givenname: Frank
  surname: Bauer
  fullname: Bauer, Frank
  email: Frank.Bauer@fau.de
– sequence: 5
  givenname: Justus
  surname: Thies
  fullname: Thies, Justus
  email: Justus.Thies@fau.de
BackLink https://www.ncbi.nlm.nih.gov/pubmed/28809691$$D View this record in MEDLINE/PubMed
BookMark eNp9kU1PGzEQhi1ExVf5AVUlFIkLl03HH7u2kTigiAAStFVFuVpeZzZyulkHe_fAv8erhB44cJqR5nlH8857TPa70CEh3yhMKQX94-l5djtlQOWUSS4EU3vkiGpBCyih2s89SFmwilWH5DilFQAVQukDcsiUAl1pekSu5tbhPMQlXk4ebfyHscWUJj9DV_zxS7-YjPPJ49D2vvgdwwpd70OX0c3Gd8uv5Etj24Snu3pC_s5vnmZ3xcOv2_vZ9UPhuNB9UVqJUjb1AnQDaGtlGSig3FaOIta14xLqRqMo0amyyVZQKsFtma1BvXD8hFxs925ieBkw9Wbtk8O2tR2GIRmqmVZKSlZl9PwDugpD7PJ1mRIVVxwqkamzHTXUa1yYTfRrG1_N-2MyILeAiyGliI1xvrej9z5a3xoKZozAjBGYMQKziyAr6Qfl-_LPNN-3Go-I_3kFIDll_A0Vr489
CODEN ITVGEA
CitedBy_id crossref_primary_10_1109_TVCG_2023_3247104
crossref_primary_10_1109_TVCG_2024_3372031
crossref_primary_10_1109_TVCG_2024_3372097
crossref_primary_10_3390_en13236442
crossref_primary_10_1111_cgf_13382
crossref_primary_10_1109_TVCG_2018_2868530
crossref_primary_10_3390_app11093753
crossref_primary_10_2183_pjab_100_012
crossref_primary_10_1109_TVCG_2022_3203085
crossref_primary_10_1109_TVCG_2024_3456170
crossref_primary_10_1109_TVCG_2022_3150488
crossref_primary_10_1111_cgf_13387
crossref_primary_10_1016_j_cag_2022_01_004
crossref_primary_10_1109_TVCG_2023_3320256
crossref_primary_10_1166_jmihi_2022_3944
crossref_primary_10_1109_TVCG_2025_3554037
crossref_primary_10_1145_3292039
crossref_primary_10_1109_TVCG_2025_3527203
crossref_primary_10_1145_3182644
crossref_primary_10_3390_s18093063
crossref_primary_10_1109_TVCG_2021_3106511
Cites_doi 10.1145/1667239.1667251
10.1007/s10055-010-0175-5
10.1145/2897824.2925873
10.1145/2010324.1964972
10.1145/2601097.2601204
10.1109/CVPR.2015.7298776
10.1145/2816795.2818056
10.1145/2461912.2461976
10.1145/2816795.2818111
10.1109/MC.2012.154
10.1145/2929464.2929475
10.1145/311535.311556
10.1109/3DIMPVT.2012.77
10.1145/1401132.1401239
10.1145/2766943
10.1109/VR.2015.7223330
10.1145/2806173.2806188
10.1007/978-3-7091-6242-2_9
10.1145/2461912.2462019
10.1145/383259.383317
10.1201/b10624
10.1145/2508363.2508416
10.1111/cgf.13128
10.1007/s11263-010-0380-4
ContentType Journal Article
Copyright Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2017
Copyright_xml – notice: Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2017
DBID 97E
RIA
RIE
AAYXX
CITATION
NPM
7SC
7SP
8FD
JQ2
L7M
L~C
L~D
7X8
DOI 10.1109/TVCG.2017.2734428
DatabaseName IEEE All-Society Periodicals Package (ASPP) 2005–Present
IEEE All-Society Periodicals Package (ASPP) 1998–Present
IEEE/IET Electronic Library (IEL) (UW System Shared)
CrossRef
PubMed
Computer and Information Systems Abstracts
Electronics & Communications Abstracts
Technology Research Database
ProQuest Computer Science Collection
Advanced Technologies Database with Aerospace
Computer and Information Systems Abstracts – Academic
Computer and Information Systems Abstracts Professional
MEDLINE - Academic
DatabaseTitle CrossRef
PubMed
Technology Research Database
Computer and Information Systems Abstracts – Academic
Electronics & Communications Abstracts
ProQuest Computer Science Collection
Computer and Information Systems Abstracts
Advanced Technologies Database with Aerospace
Computer and Information Systems Abstracts Professional
MEDLINE - Academic
DatabaseTitleList
PubMed
Technology Research Database
MEDLINE - Academic
Database_xml – sequence: 1
  dbid: NPM
  name: PubMed
  url: http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed
  sourceTypes: Index Database
– sequence: 2
  dbid: RIE
  name: IEEE Xplore
  url: https://ieeexplore.ieee.org/
  sourceTypes: Publisher
– sequence: 3
  dbid: 7X8
  name: MEDLINE - Academic
  url: https://search.proquest.com/medline
  sourceTypes: Aggregation Database
DeliveryMethod fulltext_linktorsrc
Discipline Engineering
EISSN 1941-0506
EndPage 2446
ExternalDocumentID 28809691
10_1109_TVCG_2017_2734428
8007312
Genre orig-research
Research Support, Non-U.S. Gov't
Journal Article
GrantInformation_xml – fundername: DFG
  grantid: GRK-1773
  funderid: 10.13039/100004807
GroupedDBID ---
-~X
.DC
0R~
29I
4.4
53G
5GY
5VS
6IK
97E
AAJGR
AARMG
AASAJ
AAWTH
ABAZT
ABQJQ
ABVLG
ACGFO
ACIWK
AENEX
AETIX
AGQYO
AGSQL
AHBIQ
AI.
AIBXA
AKJIK
AKQYR
ALLEH
ALMA_UNASSIGNED_HOLDINGS
ATWAV
BEFXN
BFFAM
BGNUA
BKEBE
BPEOZ
CS3
DU5
EBS
EJD
F5P
HZ~
H~9
IEDLZ
IFIPE
IFJZH
IPLJI
JAVBF
LAI
M43
O9-
OCL
P2P
PQQKQ
RIA
RIE
RNI
RNS
RZB
TN5
VH1
AAYXX
CITATION
AAYOK
NPM
RIG
7SC
7SP
8FD
JQ2
L7M
L~C
L~D
7X8
ID FETCH-LOGICAL-c349t-5a7e77fbd09f0eab8a208013a6c1eebbc370bf9e45ec85f344e7843a50170bdc3
IEDL.DBID RIE
ISICitedReferencesCount 27
ISICitedReferencesURI http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000412214000010&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
ISSN 1077-2626
1941-0506
IngestDate Thu Oct 02 11:27:30 EDT 2025
Sun Nov 09 06:48:00 EST 2025
Thu Apr 03 07:08:33 EDT 2025
Sat Nov 29 06:05:37 EST 2025
Tue Nov 18 21:21:07 EST 2025
Wed Aug 27 02:47:57 EDT 2025
IsPeerReviewed true
IsScholarly true
Issue 11
Language English
License https://ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html
LinkModel DirectLink
MergedId FETCHMERGED-LOGICAL-c349t-5a7e77fbd09f0eab8a208013a6c1eebbc370bf9e45ec85f344e7843a50170bdc3
Notes ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
content type line 23
PMID 28809691
PQID 1946383064
PQPubID 75741
PageCount 7
ParticipantIDs proquest_miscellaneous_1929887726
proquest_journals_1946383064
pubmed_primary_28809691
crossref_citationtrail_10_1109_TVCG_2017_2734428
crossref_primary_10_1109_TVCG_2017_2734428
ieee_primary_8007312
PublicationCentury 2000
PublicationDate 2017-11-01
PublicationDateYYYYMMDD 2017-11-01
PublicationDate_xml – month: 11
  year: 2017
  text: 2017-11-01
  day: 01
PublicationDecade 2010
PublicationPlace United States
PublicationPlace_xml – name: United States
– name: New York
PublicationTitle IEEE transactions on visualization and computer graphics
PublicationTitleAbbrev TVCG
PublicationTitleAlternate IEEE Trans Vis Comput Graph
PublicationYear 2017
Publisher IEEE
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher_xml – name: IEEE
– name: The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
References ref13
ref12
ref14
ref10
ref1
ref17
ref16
lange (ref15) 0
ref19
ref18
asai (ref2) 2016
cao (ref11) 2014; 20
ref24
ref23
ref26
ref25
ref20
ref22
ref21
(ref28) 2015
ref27
ref8
ref7
ref9
ref4
ref3
ref6
ref5
References_xml – ident: ref1
  doi: 10.1145/1667239.1667251
– ident: ref17
  doi: 10.1007/s10055-010-0175-5
– ident: ref12
  doi: 10.1145/2897824.2925873
– ident: ref27
  doi: 10.1145/2010324.1964972
– ident: ref10
  doi: 10.1145/2601097.2601204
– ident: ref14
  doi: 10.1109/CVPR.2015.7298776
– year: 2016
  ident: ref2
– year: 2015
  ident: ref28
  publication-title: Kat von ds live face projection mapping
– ident: ref25
  doi: 10.1145/2816795.2818056
– ident: ref8
  doi: 10.1145/2461912.2461976
– ident: ref23
  doi: 10.1145/2816795.2818111
– ident: ref18
  doi: 10.1109/MC.2012.154
– ident: ref26
  doi: 10.1145/2929464.2929475
– ident: ref7
  doi: 10.1145/311535.311556
– ident: ref19
  doi: 10.1109/3DIMPVT.2012.77
– ident: ref5
  doi: 10.1145/1401132.1401239
– ident: ref9
  doi: 10.1145/2766943
– ident: ref24
  doi: 10.1109/VR.2015.7223330
– ident: ref13
  doi: 10.1145/2806173.2806188
– ident: ref21
  doi: 10.1007/978-3-7091-6242-2_9
– ident: ref16
  doi: 10.1145/2461912.2462019
– ident: ref20
  doi: 10.1145/383259.383317
– volume: 20
  start-page: 413
  year: 2014
  ident: ref11
  article-title: Facewarehouse: A 3D facial expression database for visual computing
  publication-title: IEEE TVCG
– ident: ref6
  doi: 10.1201/b10624
– year: 0
  ident: ref15
  article-title: Robust blending and occlusion compensation in dynamic multi-projection mapping
  publication-title: EUROGRAPHICS
– ident: ref4
  doi: 10.1145/2508363.2508416
– ident: ref3
  doi: 10.1111/cgf.13128
– ident: ref22
  doi: 10.1007/s11263-010-0380-4
SSID ssj0014489
Score 2.3616893
Snippet Recent publications and art performances demonstrate amazing results using projection mapping. To our knowledge, there exists no multi-projection system that...
SourceID proquest
pubmed
crossref
ieee
SourceType Aggregation Database
Index Database
Enrichment Source
Publisher
StartPage 2440
SubjectTerms Augmented reality
Cameras
Computational modeling
Face Projection
Face recognition
Image color analysis
Mapping
Mixed Reality
Multi-Projection Mapping
Non-Rigid Face Tracking
Object tracking
Projection
Quality
Spectators
Target tracking
Title FaceForge: Markerless Non-Rigid Face Multi-Projection Mapping
URI https://ieeexplore.ieee.org/document/8007312
https://www.ncbi.nlm.nih.gov/pubmed/28809691
https://www.proquest.com/docview/1946383064
https://www.proquest.com/docview/1929887726
Volume 23
WOSCitedRecordID wos000412214000010&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
journalDatabaseRights – providerCode: PRVIEE
  databaseName: IEEE Xplore
  customDbUrl:
  eissn: 1941-0506
  dateEnd: 99991231
  omitProxy: false
  ssIdentifier: ssj0014489
  issn: 1077-2626
  databaseCode: RIE
  dateStart: 19950101
  isFulltext: true
  titleUrlDefault: https://ieeexplore.ieee.org/
  providerName: IEEE
link http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LS8QwEB5UPOjB96M-lgqexGjfSQQPsrh6WkRU9laSdLIsSFfcXX-_mbZbFFTwVsi0DfNNkkkm8w3AaYGR0s6tZ4kMFEssGqYymzJa7IVzgSl4VRWb4P2-GAzkwwKct7kwiFhdPsMLeqxi-cXYzOio7FJQXIlKCi9yzutcrTZi4LYZsr5fyFnkvPQmghkG8vLppXtHl7j4BXG5OH-bGICd3cpMht-Wo6q-yu-uZrXk9Nb_19kNWGtcS_-mtoVNWMByC1a_EA5uw3VPGezRKfiVT1k6SKH2id8fl-xxNBwVPrX7VVIue6jPaBxuTpRYHIY78Ny7feres6aAAjNxIqcsVRw5t7oIpA1QaaEi5yCGscpMiKi1iXmgrcQkRSNS63SDXCSxSolURxcm3oWlclziPvixEFpYB1-RaYeoVUGiVSGLVFqRqIx7EMz1mJuGXZyKXLzm1S4jkDmhkBMKeYOCB2ftK281tcZfwtuk4law0a4HR3Ow8mbwTfJQJm5Woa2VBydtsxs2FAtRJY5nJBNJN7_yKPNgrwa5_fbcNg5-_uchrFDP6oTEI1iavs_wGJbNx3Q0ee842xyITmWbn4xu2_g
linkProvider IEEE
linkToHtml http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV3dSxwxEB_ECtWHVuvXtn6s0Ccx535kN4ngg4inpXpIOYtvS5KdyIHsiXfn39_M7t7SghV8W8jsbpjfJPklk5kB-F5ioo2n9YyrSDPu0DKdu4zRYi89BSbnVV1sQgwG8v5e3S7AURcLg4j15TPs0WPtyy_HdkZHZceS_EpUUvhDxnkSN9Fanc_AbzRUc8NQsMTz9NaHGUfqePj7_JKucYkeZXPxjJtyAHvLVbmK_1mQ6gor_yeb9aLT__y-7q7Cp5ZchmeNNazBAlZfYOWvlIPrcNrXFvt0Dn4SUpwOkrN9Eg7GFfs1ehiVIbWHdVguu21OaTxyXpTyODxswF3_Ynh-xdoSCsymXE1ZpgUK4UwZKRehNlInniLGqc5tjGiMTUVknEKeoZWZ87pBIXmqM0qrY0qbbsJiNa5wG8JUSiOdB7DMjcfU6YgbXaoyU05ynYsAorkeC9vmF6cyF49Fvc-IVEEoFIRC0aIQwGH3ylOTXOMt4XVScSfYajeAnTlYRTv8JkWsuJ9XaHMVwEHX7AcOeUN0heMZySTKz7AiyQPYakDuvj23ja-v_3MfPl4Nb66L6x-Dn99gmXrZhCfuwOL0eYa7sGRfpqPJ815toX8A90_eVw
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=FaceForge%3A+Markerless+Non-Rigid+Face+Multi-Projection+Mapping&rft.jtitle=IEEE+transactions+on+visualization+and+computer+graphics&rft.au=Siegl%2C+Christian&rft.au=Lange%2C+Vanessa&rft.au=Stamminger%2C+Marc&rft.au=Bauer%2C+Frank&rft.date=2017-11-01&rft.issn=1077-2626&rft.volume=23&rft.issue=11&rft.spage=2440&rft.epage=2446&rft_id=info:doi/10.1109%2FTVCG.2017.2734428&rft.externalDBID=n%2Fa&rft.externalDocID=10_1109_TVCG_2017_2734428
thumbnail_l http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=1077-2626&client=summon
thumbnail_m http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=1077-2626&client=summon
thumbnail_s http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=1077-2626&client=summon