Lips Don't Lie: A Generalisable and Robust Approach to Face Forgery Detection
Although current deep learning-based face forgery detectors achieve impressive performance in constrained scenarios, they are vulnerable to samples created by unseen manipulation methods. Some recent works show improvements in generalisation but rely on cues that are easily corrupted by common post-...
Uloženo v:
| Vydáno v: | Proceedings (IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Online) s. 5037 - 5047 |
|---|---|
| Hlavní autoři: | , , , |
| Médium: | Konferenční příspěvek |
| Jazyk: | angličtina |
| Vydáno: |
IEEE
01.06.2021
|
| Témata: | |
| ISSN: | 1063-6919 |
| On-line přístup: | Získat plný text |
| Tagy: |
Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
|
| Abstract | Although current deep learning-based face forgery detectors achieve impressive performance in constrained scenarios, they are vulnerable to samples created by unseen manipulation methods. Some recent works show improvements in generalisation but rely on cues that are easily corrupted by common post-processing operations such as compression. In this paper, we propose LipForensics, a detection approach capable of both generalising to novel manipulations and withstanding various distortions. LipForensics targets high-level semantic irregularities in mouth movements, which are common in many generated videos. It consists in first pretraining a spatio-temporal network to perform visual speech recognition (lipreading), thus learning rich internal representations related to natural mouth motion. A temporal network is subsequently finetuned on fixed mouth embeddings of real and forged data in order to detect fake videos based on mouth movements without overfitting to low-level, manipulation-specific artefacts. Extensive experiments show that this simple approach significantly surpasses the state-of-the-art in terms of generalisation to unseen manipulations and robustness to perturbations, as well as shed light on the factors responsible for its performance. |
|---|---|
| AbstractList | Although current deep learning-based face forgery detectors achieve impressive performance in constrained scenarios, they are vulnerable to samples created by unseen manipulation methods. Some recent works show improvements in generalisation but rely on cues that are easily corrupted by common post-processing operations such as compression. In this paper, we propose LipForensics, a detection approach capable of both generalising to novel manipulations and withstanding various distortions. LipForensics targets high-level semantic irregularities in mouth movements, which are common in many generated videos. It consists in first pretraining a spatio-temporal network to perform visual speech recognition (lipreading), thus learning rich internal representations related to natural mouth motion. A temporal network is subsequently finetuned on fixed mouth embeddings of real and forged data in order to detect fake videos based on mouth movements without overfitting to low-level, manipulation-specific artefacts. Extensive experiments show that this simple approach significantly surpasses the state-of-the-art in terms of generalisation to unseen manipulations and robustness to perturbations, as well as shed light on the factors responsible for its performance. |
| Author | Vougioukas, Konstantinos Petridis, Stavros Haliassos, Alexandros Pantic, Maja |
| Author_xml | – sequence: 1 givenname: Alexandros surname: Haliassos fullname: Haliassos, Alexandros email: alexandros.haliassos14@imperial.ac.uk organization: Imperial College London – sequence: 2 givenname: Konstantinos surname: Vougioukas fullname: Vougioukas, Konstantinos email: k.vougioukas@imperial.ac.uk organization: Imperial College London – sequence: 3 givenname: Stavros surname: Petridis fullname: Petridis, Stavros email: stavros.petridis04@imperial.ac.uk organization: Imperial College London – sequence: 4 givenname: Maja surname: Pantic fullname: Pantic, Maja email: m.pantic@imperial.ac.uk organization: Imperial College London |
| BookMark | eNotzLFOwzAUQFGDQKIt_QIYvDGlvGfHdsxWtbQgBYEqYK0c5xmMQhwl6dC_BwmmO507ZWdtaomxa4QFItjb1fvLLte5NAsBAhcACuCETVFrlecKrDhlEwQtM23RXrD5MHwBgBSI2hYT9lTGbuDr1N6MvIx0x5d8Sy31romDqxrirq35LlWHYeTLruuT8598THzjPPFN6j-oP_I1jeTHmNpLdh5cM9D8vzP2trl_XT1k5fP2cbUssyhAjpm0TtZkAipjbBEK6Wr0AmUQVai0B_KkTKEFaFUobQyS9igKGYwKdfAoZ-zq7xuJaN_18dv1x739RRZB_gDFSU7l |
| CODEN | IEEPAD |
| ContentType | Conference Proceeding |
| DBID | 6IE 6IH CBEJK RIE RIO |
| DOI | 10.1109/CVPR46437.2021.00500 |
| DatabaseName | IEEE Electronic Library (IEL) Conference Proceedings IEEE Proceedings Order Plan (POP) 1998-present by volume IEEE Xplore All Conference Proceedings IEEE Electronic Library (IEL) IEEE Proceedings Order Plans (POP) 1998-present |
| DatabaseTitleList | |
| Database_xml | – sequence: 1 dbid: RIE name: IEEE Electronic Library (IEL) url: https://ieeexplore.ieee.org/ sourceTypes: Publisher |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Applied Sciences |
| EISBN | 1665445092 9781665445092 |
| EISSN | 1063-6919 |
| EndPage | 5047 |
| ExternalDocumentID | 9578910 |
| Genre | orig-research |
| GroupedDBID | 6IE 6IH 6IL 6IN AAWTH ABLEC ADZIZ ALMA_UNASSIGNED_HOLDINGS BEFXN BFFAM BGNUA BKEBE BPEOZ CBEJK CHZPO IEGSK IJVOP OCL RIE RIL RIO |
| ID | FETCH-LOGICAL-i203t-39a3de7f157798f83ad1c213f2bfb6c0ece57862065856771e6c1283f75fdfc13 |
| IEDL.DBID | RIE |
| ISICitedReferencesCount | 321 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000739917305024&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| IngestDate | Wed Aug 27 02:28:29 EDT 2025 |
| IsPeerReviewed | false |
| IsScholarly | true |
| Language | English |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-i203t-39a3de7f157798f83ad1c213f2bfb6c0ece57862065856771e6c1283f75fdfc13 |
| PageCount | 11 |
| ParticipantIDs | ieee_primary_9578910 |
| PublicationCentury | 2000 |
| PublicationDate | 2021-June |
| PublicationDateYYYYMMDD | 2021-06-01 |
| PublicationDate_xml | – month: 06 year: 2021 text: 2021-June |
| PublicationDecade | 2020 |
| PublicationTitle | Proceedings (IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Online) |
| PublicationTitleAbbrev | CVPR |
| PublicationYear | 2021 |
| Publisher | IEEE |
| Publisher_xml | – name: IEEE |
| SSID | ssj0003211698 |
| Score | 2.6587799 |
| Snippet | Although current deep learning-based face forgery detectors achieve impressive performance in constrained scenarios, they are vulnerable to samples created by... |
| SourceID | ieee |
| SourceType | Publisher |
| StartPage | 5037 |
| SubjectTerms | Face recognition Meetings Mouth Perturbation methods Semantics Speech recognition Visualization |
| Title | Lips Don't Lie: A Generalisable and Robust Approach to Face Forgery Detection |
| URI | https://ieeexplore.ieee.org/document/9578910 |
| WOSCitedRecordID | wos000739917305024&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV3NS8MwFA9zePDkxyZ-k4PgxW5N0iaNt7FZPMwxhspuo0leoCDtWDv_fpOuTAQv3kIuebwkvM_f7yF0H8VZIhNuAuASgkhpFmTShoGgsQWiFSWmmVoyFbNZslzKeQc97rEwANA0n8HAL5tavin11qfKhtI9L-nxVAdCiB1Wa59PYS6S4TJp0XEklMPxx3wR-bqUiwIpGXimk_DXDJXGhKTH_zv8BPV_sHh4vrcyp6gDxRk6bp1H3H7Nqodep_m6wpOyeKjxNIcnPMItpbRn0f0EnBUGL0q1rWo8aonEcV3iNHMHpGUDjsYTqJvWrKKP3tPnt_FL0M5KCHIasjpgMmMGhCWxEDKxCcsM0ZQwS5VVXIegwcnPqfc4Yi4EAa6daWJWxNZYTdg56hZlARcIJ1rGRhOIQgtRBlQKJ4hWyjoPl0sVXqKe185qvaPDWLWKufp7-xodefXvuqtuULfebOEWHeqvOq82d80dfgNitp0d |
| linkProvider | IEEE |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV3fS8MwEA5jCvo0dRN_mwfBF7s1SZs0vo3NMbEbY0zZ22jTCxSkHWvn32_SlYngi28hL3fcJdxdct93CD14fhTIgCcOcAmOFyvmRFK7jqC-BqJiSpJqakkoptNguZSzBnraY2EAoGo-g65dVn_5Sa629qmsJ83xkhZPdeB7HiU7tNb-RYWZWobLoMbHEVf2Bh-zuWd_pkwdSEnXcp24v6aoVEFk1Pqf-BPU-UHj4dk-zpyiBmRnqFWnj7i-nEUbTcJ0XeBhnj2WOEzhGfdxTSpteXQ_AUdZgud5vC1K3K-pxHGZ41FkBIzyCh6Nh1BWzVlZB72PXhaDsVNPS3BS6rLSYTJiCQhNfCFkoAMWJURRwjSNdcyVCwqM_pzanMPnQhDgygQnpoWvE60IO0fNLM_gAuFAST9RBDxXgxcBlcIoouJYmxyXy9i9RG1rndV6R4ixqg1z9ff2PToaLybhKnydvl2jY-uKXa_VDWqWmy3cokP1VabF5q7y5ze_kKBk |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=proceeding&rft.title=Proceedings+%28IEEE+Computer+Society+Conference+on+Computer+Vision+and+Pattern+Recognition.+Online%29&rft.atitle=Lips+Don%27t+Lie%3A+A+Generalisable+and+Robust+Approach+to+Face+Forgery+Detection&rft.au=Haliassos%2C+Alexandros&rft.au=Vougioukas%2C+Konstantinos&rft.au=Petridis%2C+Stavros&rft.au=Pantic%2C+Maja&rft.date=2021-06-01&rft.pub=IEEE&rft.eissn=1063-6919&rft.spage=5037&rft.epage=5047&rft_id=info:doi/10.1109%2FCVPR46437.2021.00500&rft.externalDocID=9578910 |