An Identity-Preserved Framework for Human Motion Transfer
Human motion transfer (HMT) aims to generate a video clip for the target subject by imitating the source subject's motion. Although previous methods have achieved good results in synthesizing good-quality videos, they lose sight of individualized motion information from the source and target mo...
Uložené v:
| Vydané v: | arXiv.org |
|---|---|
| Hlavní autori: | , , |
| Médium: | Paper |
| Jazyk: | English |
| Vydavateľské údaje: |
Ithaca
Cornell University Library, arXiv.org
22.02.2024
|
| Predmet: | |
| ISSN: | 2331-8422 |
| On-line prístup: | Získať plný text |
| Tagy: |
Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
|
| Abstract | Human motion transfer (HMT) aims to generate a video clip for the target subject by imitating the source subject's motion. Although previous methods have achieved good results in synthesizing good-quality videos, they lose sight of individualized motion information from the source and target motions, which is significant for the realism of the motion in the generated video. To address this problem, we propose a novel identity-preserved HMT network, termed \textit{IDPres}. This network is a skeleton-based approach that uniquely incorporates the target's individualized motion and skeleton information to augment identity representations. This integration significantly enhances the realism of movements in the generated videos. Our method focuses on the fine-grained disentanglement and synthesis of motion. To improve the representation learning capability in latent space and facilitate the training of \textit{IDPres}, we introduce three training schemes. These schemes enable \textit{IDPres} to concurrently disentangle different representations and accurately control them, ensuring the synthesis of ideal motions. To evaluate the proportion of individualized motion information in the generated video, we are the first to introduce a new quantitative metric called Identity Score (\textit{ID-Score}), motivated by the success of gait recognition methods in capturing identity information. Moreover, we collect an identity-motion paired dataset, \(Dancer101\), consisting of solo-dance videos of 101 subjects from the public domain, providing a benchmark to prompt the development of HMT methods. Extensive experiments demonstrate that the proposed \textit{IDPres} method surpasses existing state-of-the-art techniques in terms of reconstruction accuracy, realistic motion, and identity preservation. |
|---|---|
| AbstractList | Human motion transfer (HMT) aims to generate a video clip for the target subject by imitating the source subject's motion. Although previous methods have achieved good results in synthesizing good-quality videos, they lose sight of individualized motion information from the source and target motions, which is significant for the realism of the motion in the generated video. To address this problem, we propose a novel identity-preserved HMT network, termed \textit{IDPres}. This network is a skeleton-based approach that uniquely incorporates the target's individualized motion and skeleton information to augment identity representations. This integration significantly enhances the realism of movements in the generated videos. Our method focuses on the fine-grained disentanglement and synthesis of motion. To improve the representation learning capability in latent space and facilitate the training of \textit{IDPres}, we introduce three training schemes. These schemes enable \textit{IDPres} to concurrently disentangle different representations and accurately control them, ensuring the synthesis of ideal motions. To evaluate the proportion of individualized motion information in the generated video, we are the first to introduce a new quantitative metric called Identity Score (\textit{ID-Score}), motivated by the success of gait recognition methods in capturing identity information. Moreover, we collect an identity-motion paired dataset, \(Dancer101\), consisting of solo-dance videos of 101 subjects from the public domain, providing a benchmark to prompt the development of HMT methods. Extensive experiments demonstrate that the proposed \textit{IDPres} method surpasses existing state-of-the-art techniques in terms of reconstruction accuracy, realistic motion, and identity preservation. |
| Author | Ma, Jingzhe Zhang, Xiaoqing Yu, Shiqi |
| Author_xml | – sequence: 1 givenname: Jingzhe surname: Ma fullname: Ma, Jingzhe – sequence: 2 givenname: Xiaoqing surname: Zhang fullname: Zhang, Xiaoqing – sequence: 3 givenname: Shiqi surname: Yu fullname: Yu, Shiqi |
| BookMark | eNotzc1KAzEUQOEgCtbaB3AXcD1jcvO_LMXaQkXB7ks6uYGpNtFkpurbK-jq7L5zRc5TTkjIDWettEqxO1---lMLwGTLtNVwRiYgBG-sBLgks1oPjDHQBpQSE-Lmia4DpqEfvpvnghXLCQNdFn_Ez1xeacyFrsajT_QxD31OdFt8qhHLNbmI_q3i7L9T8rK83y5WzebpYb2YbxqvQDbOoZbOQ-hctPvOdE7GaJR1GhgPaI02XbQ2Msv3TqAWHgOowCNqzg0XU3L7p76X_DFiHXaHPJb0O9yBVkyAYkqKH2LASVE |
| ContentType | Paper |
| Copyright | 2024. This work is published under http://creativecommons.org/licenses/by/4.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License. |
| Copyright_xml | – notice: 2024. This work is published under http://creativecommons.org/licenses/by/4.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License. |
| DBID | 8FE 8FG ABJCF ABUWG AFKRA AZQEC BENPR BGLVJ CCPQU DWQXO HCIFZ L6V M7S PHGZM PHGZT PIMPY PKEHL PQEST PQGLB PQQKQ PQUKI PRINS PTHSS |
| DOI | 10.48550/arxiv.2204.06862 |
| DatabaseName | ProQuest SciTech Collection ProQuest Technology Collection Materials Science & Engineering Collection ProQuest Central (Alumni) ProQuest Central UK/Ireland ProQuest Central Essentials ProQuest One Academic Technology Collection ProQuest One ProQuest Central Korea SciTech Premium Collection ProQuest Engineering Collection Engineering Database ProQuest Central Premium ProQuest One Academic Publicly Available Content Database ProQuest One Academic Middle East (New) ProQuest One Academic Eastern Edition (DO NOT USE) One Applied & Life Sciences ProQuest One Academic (retired) ProQuest One Academic UKI Edition ProQuest Central China Engineering Collection |
| DatabaseTitle | Publicly Available Content Database Engineering Database Technology Collection ProQuest One Academic Middle East (New) ProQuest Central Essentials ProQuest One Academic Eastern Edition ProQuest Central (Alumni Edition) SciTech Premium Collection ProQuest One Community College ProQuest Technology Collection ProQuest SciTech Collection ProQuest Central China ProQuest Central ProQuest One Applied & Life Sciences ProQuest Engineering Collection ProQuest One Academic UKI Edition ProQuest Central Korea Materials Science & Engineering Collection ProQuest Central (New) ProQuest One Academic ProQuest One Academic (New) Engineering Collection |
| DatabaseTitleList | Publicly Available Content Database |
| Database_xml | – sequence: 1 dbid: PIMPY name: ProQuest Publicly Available Content Database url: http://search.proquest.com/publiccontent sourceTypes: Aggregation Database |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Physics |
| EISSN | 2331-8422 |
| Genre | Working Paper/Pre-Print |
| GroupedDBID | 8FE 8FG ABJCF ABUWG AFKRA ALMA_UNASSIGNED_HOLDINGS AZQEC BENPR BGLVJ CCPQU DWQXO FRJ HCIFZ L6V M7S M~E PHGZM PHGZT PIMPY PKEHL PQEST PQGLB PQQKQ PQUKI PRINS PTHSS |
| ID | FETCH-LOGICAL-a524-99e649a2dc9f8bc7c94ff75896201de8767cf88f081b93e63aed25d1fe611713 |
| IEDL.DBID | BENPR |
| IngestDate | Mon Jun 30 09:19:34 EDT 2025 |
| IsDoiOpenAccess | true |
| IsOpenAccess | true |
| IsPeerReviewed | false |
| IsScholarly | false |
| Language | English |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-a524-99e649a2dc9f8bc7c94ff75896201de8767cf88f081b93e63aed25d1fe611713 |
| Notes | SourceType-Working Papers-1 ObjectType-Working Paper/Pre-Print-1 content type line 50 |
| OpenAccessLink | https://www.proquest.com/docview/2650325054?pq-origsite=%requestingapplication% |
| PQID | 2650325054 |
| PQPubID | 2050157 |
| ParticipantIDs | proquest_journals_2650325054 |
| PublicationCentury | 2000 |
| PublicationDate | 20240222 |
| PublicationDateYYYYMMDD | 2024-02-22 |
| PublicationDate_xml | – month: 02 year: 2024 text: 20240222 day: 22 |
| PublicationDecade | 2020 |
| PublicationPlace | Ithaca |
| PublicationPlace_xml | – name: Ithaca |
| PublicationTitle | arXiv.org |
| PublicationYear | 2024 |
| Publisher | Cornell University Library, arXiv.org |
| Publisher_xml | – name: Cornell University Library, arXiv.org |
| SSID | ssj0002672553 |
| Score | 1.8612721 |
| SecondaryResourceType | preprint |
| Snippet | Human motion transfer (HMT) aims to generate a video clip for the target subject by imitating the source subject's motion. Although previous methods have... |
| SourceID | proquest |
| SourceType | Aggregation Database |
| SubjectTerms | Coders Gait recognition Human motion Realism Synthesis Video |
| Title | An Identity-Preserved Framework for Human Motion Transfer |
| URI | https://www.proquest.com/docview/2650325054 |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwpV09T8MwED1BCxIT3-KjVB5YTRvHceIJAWoFQ6uIMpSp8qfUJZSkVPDvsY0LAxILY-Qlsq17787v7gFcSilklvctFoK5BIVphguaEGy0zayRjvFbGcwm8vG4mE55GQtuTZRVrmNiCNT6RfkaeY84KpF6vKbXi1fsXaP862q00NiEtp9URlvQvh2My8fvKgthuePM6ddzZhje1RP1-3x1RYgfcOrbI34F4YAsw93__tMetEuxMPU-bJjqALaDolM1h8BvKhS7cD-wF1p4aaNGw7UYCzm2ikIJH42Ckw8KsGVNfQST4eDp7h5HnwQsMkIx54ZRLohW3BZS5YpTa10awJkDd21cuMuVLQrrwF_y1LBUGE0ynVjDksTlqMfQql4qcwLIOsTk_Uw7FmJpLlKeKOoZhSm0pH2lTqGz3odZvOrN7GcTzv5ePocd4hhB6AcnHWgt6zdzAVtqtZw3dTeeXNeLLyfuq3wYlc-fROKmwg |
| linkProvider | ProQuest |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMw1V3LTsJAFL1B0OjKd3ygdqHLCp1Op52FMUYlEB4hgQUryTwTNgULovyTH-nMQHVh4o6F6yZNO49zzp177lyAa84Zj-Kq9hkjJkAhkvgJDpCvpI604kbxa-6aTcSdTjIY0G4BPvNaGGurzDHRAbUcC3tGXkFGSoSWr_H95NW3XaNsdjVvobFcFk21eDch2_Su8WTm9wah2nP_se6vugr4LELYp1QRTBmSguqEi1hQrLURzZQYKpTKgEMsdJJoQ5WchoqETEkUyUArEgQmojNv3YASttjvjIK97xMdRGKjz8Nl6tRdFFZh2cdofouQvUzVlmL8AnzHYrXd__X_e1DqsonK9qGg0gPYcl5VMT0E-pB6q_rihW8tJNa0Kb1abjPzjA73XHLCa7seRZ4jZK2yI-it4VuPoZiOU3UCnjZagFYjafSVxjELaSCw1UoqkRxXhTiFcj7qw9Umng5_hvzs78dXsF3vt1vDVqPTPIcdZHSPq3pHZSjOsjd1AZtiPhtNs0u3Yjx4We8EfQHLKf1n |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=An+Identity-Preserved+Framework+for+Human+Motion+Transfer&rft.jtitle=arXiv.org&rft.au=Ma%2C+Jingzhe&rft.au=Zhang%2C+Xiaoqing&rft.au=Yu%2C+Shiqi&rft.date=2024-02-22&rft.pub=Cornell+University+Library%2C+arXiv.org&rft.eissn=2331-8422&rft_id=info:doi/10.48550%2Farxiv.2204.06862 |