Soundprism: An Online System for Score-Informed Source Separation of Music Audio
Soundprism, as proposed in this paper, is a computer system that separates single-channel polyphonic music audio played by harmonic sources into source signals in an online fashion. It uses a musical score to guide the separation process. To the best of our knowledge, this is the first online system...
Uloženo v:
| Vydáno v: | IEEE journal of selected topics in signal processing Ročník 5; číslo 6; s. 1205 - 1215 |
|---|---|
| Hlavní autoři: | , |
| Médium: | Journal Article |
| Jazyk: | angličtina |
| Vydáno: |
New York
IEEE
01.10.2011
The Institute of Electrical and Electronics Engineers, Inc. (IEEE) |
| Témata: | |
| ISSN: | 1932-4553, 1941-0484 |
| On-line přístup: | Získat plný text |
| Tagy: |
Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
|
| Abstract | Soundprism, as proposed in this paper, is a computer system that separates single-channel polyphonic music audio played by harmonic sources into source signals in an online fashion. It uses a musical score to guide the separation process. To the best of our knowledge, this is the first online system that addresses score-informed music source separation that can be made into a real-time system. The proposed system consists of two parts: 1) a score follower that associates a score position to each time frame of the audio performance; 2) a source separator which reconstructs the source signals for each time frame, informed by the score. The score follower uses a hidden Markov approach, where each audio frame is associated with a 2-D state vector (score position and tempo). The observation model is defined as the likelihood of observing the frame given the pitches at the score position. The score position and tempo are inferred using particle filtering. In building the source separator, we first refine the score-informed pitches of the current audio frame by maximizing the multi-pitch observation likelihood. Then, the harmonics of each source's fundamental frequency are extracted to reconstruct the source signal. Overlapping harmonics between sources are identified and their energy is distributed in inverse proportion to the square of their respective harmonic number. Experiments on both synthetic and human-performed music show both the score follower and the source separator perform well. Results also show that the proposed score follower works well for highly polyphonic music with some degree of tempo variations. |
|---|---|
| AbstractList | Soundprism, as proposed in this paper, is a computer system that separates single-channel polyphonic music audio played by harmonic sources into source signals in an online fashion. It uses a musical score to guide the separation process. To the best of our knowledge, this is the first online system that addresses score-informed music source separation that can be made into a real-time system. The proposed system consists of two parts: 1) a score follower that associates a score position to each time frame of the audio performance; 2) a source separator which reconstructs the source signals for each time frame, informed by the score. The score follower uses a hidden Markov approach, where each audio frame is associated with a 2-D state vector (score position and tempo). The observation model is defined as the likelihood of observing the frame given the pitches at the score position. The score position and tempo are inferred using particle filtering. In building the source separator, we first refine the score-informed pitches of the current audio frame by maximizing the multi-pitch observation likelihood. Then, the harmonics of each source's fundamental frequency are extracted to reconstruct the source signal. Overlapping harmonics between sources are identified and their energy is distributed in inverse proportion to the square of their respective harmonic number. Experiments on both synthetic and human-performed music show both the score follower and the source separator perform well. Results also show that the proposed score follower works well for highly polyphonic music with some degree of tempo variations. |
| Author | Zhiyao Duan Pardo, B. |
| Author_xml | – sequence: 1 surname: Zhiyao Duan fullname: Zhiyao Duan email: zhiyao-duan00@gmail.com organization: Dept. of Electr. Eng. & Comput. Sci., Northwestern Univ., Evanston, IL, USA – sequence: 2 givenname: B. surname: Pardo fullname: Pardo, B. organization: Dept. of Electr. Eng. & Comput. Sci., Northwestern Univ., Evanston, IL, USA |
| BookMark | eNp9kM1P9CAQh4nRxM9_QC_Ei6euDNCWvreN8TMaTapnwtJpgmlhX2gP_veyrvHgwRMTeH4zzHNIdn3wSMgpsAUAay4f2tf2ZcEZwIJD2dQMdsgBNBIKJpXc3dSCF7IsxT45TOmdsbKuQB6QlzbMvltHl8Z_dOnpsx-cR9p-pAlH2odIWxsiFvc-1yN2NPPRZgDXJprJBU9DT5_m5Cxdzp0Lx2SvN0PCk-_ziLzdXL9e3RWPz7f3V8vHwgpeTYUpmeVccN5VgKLDpgfRG5C1NaY2KyXypeIlMJDGiFVdmqbLLwi96irBVuKIXGz7rmP4P2Oa9OiSxWEwHsOcdMMrIUBymcnzX-R73sHnz2nVMKFqDiJDfAvZGFKK2OvsZDTxQwPTG8X6S7HeKNbfinNI_QpZN31JmaJxw9_Rs23UIeLPrFKpWiguPgFMGIsU |
| CODEN | IJSTGY |
| CitedBy_id | crossref_primary_10_1186_1687_6180_2013_184 crossref_primary_10_1109_TASLP_2013_2285484 crossref_primary_10_1002_cmm4_1040 crossref_primary_10_1109_TASLP_2016_2611938 crossref_primary_10_1186_s13636_019_0168_6 crossref_primary_10_1007_s11227_016_1865_x crossref_primary_10_1007_s11042_013_1398_8 crossref_primary_10_1109_TASLP_2014_2355772 crossref_primary_10_1109_TASLP_2015_2507862 crossref_primary_10_1109_TASLP_2023_3277290 crossref_primary_10_1186_1687_6180_2014_23 crossref_primary_10_1007_s11227_018_2703_0 crossref_primary_10_1016_j_patcog_2017_09_020 crossref_primary_10_1007_s11227_020_03282_2 crossref_primary_10_1007_s11227_018_2265_1 crossref_primary_10_1109_TASLP_2024_3356980 crossref_primary_10_1162_COMJ_a_00286 crossref_primary_10_1038_s41467_020_15367_w crossref_primary_10_1186_s13636_020_00190_4 crossref_primary_10_1109_TMM_2018_2856090 crossref_primary_10_1007_s11042_018_6349_y crossref_primary_10_1109_TASLPRO_2025_3571294 crossref_primary_10_1080_09298215_2014_989174 crossref_primary_10_1109_TASLP_2015_2412464 crossref_primary_10_1007_s11227_016_1647_5 crossref_primary_10_1109_TASLP_2016_2598323 crossref_primary_10_1109_MSP_2013_2296076 crossref_primary_10_1145_2926717 crossref_primary_10_1109_TASLP_2021_3121991 crossref_primary_10_1016_j_engappai_2013_03_010 crossref_primary_10_1109_LSP_2018_2847236 |
| Cites_doi | 10.1007/s10994-006-8415-3 10.1109/TSA.2005.858005 10.1109/ICASSP.2011.5946324 10.1162/comj.2008.32.1.51 10.1109/TASL.2009.2020886 10.1109/TSA.2005.857574 10.1109/TASL.2008.919073 10.1109/TPAMI.2009.106 10.1109/78.978374 10.1109/34.761266 10.1007/978-1-4757-3437-9 10.1109/ICASSP.2009.4959972 10.1155/2007/48317 10.1109/TASL.2011.2134092 10.1109/ICASSP.2010.5496224 10.1109/ASPAA.2003.1285862 10.1109/TASL.2009.2030006 10.1109/TSA.2003.815516 10.1109/TASL.2010.2042119 10.1109/ICASSP.2006.1661258 |
| ContentType | Journal Article |
| Copyright | Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) Oct 2011 |
| Copyright_xml | – notice: Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) Oct 2011 |
| DBID | 97E RIA RIE AAYXX CITATION 7SP 8FD H8D L7M |
| DOI | 10.1109/JSTSP.2011.2159701 |
| DatabaseName | IEEE All-Society Periodicals Package (ASPP) 2005–Present IEEE All-Society Periodicals Package (ASPP) 1998–Present IEEE/IET Electronic Library (IEL) (UW System Shared) CrossRef Electronics & Communications Abstracts Technology Research Database Aerospace Database Advanced Technologies Database with Aerospace |
| DatabaseTitle | CrossRef Aerospace Database Technology Research Database Advanced Technologies Database with Aerospace Electronics & Communications Abstracts |
| DatabaseTitleList | Technology Research Database Aerospace Database |
| Database_xml | – sequence: 1 dbid: RIE name: IEEE/IET Electronic Library url: https://ieeexplore.ieee.org/ sourceTypes: Publisher |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Engineering Music |
| EISSN | 1941-0484 |
| EndPage | 1215 |
| ExternalDocumentID | 2456647341 10_1109_JSTSP_2011_2159701 5887382 |
| Genre | orig-research |
| GroupedDBID | -~X 0R~ 29I 4.4 5GY 5VS 6IK 97E AAJGR AARMG AASAJ AAWTH ABAZT ABQJQ ABVLG ACIWK AENEX AETIX AGQYO AGSQL AHBIQ AKJIK AKQYR ALMA_UNASSIGNED_HOLDINGS ATWAV BEFXN BFFAM BGNUA BKEBE BPEOZ CS3 DU5 EBS EJD F5P HZ~ IFIPE IPLJI JAVBF LAI M43 O9- OCL RIA RIE RNS AAYXX CITATION 7SP 8FD H8D L7M RIG |
| ID | FETCH-LOGICAL-c326t-a50c22322d61e3de9f13fa147caa7ab83e3d8251014aa3b75a9daa7e1f8d630b3 |
| IEDL.DBID | RIE |
| ISICitedReferencesCount | 64 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000295012900011&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| ISSN | 1932-4553 |
| IngestDate | Tue Sep 30 20:10:03 EDT 2025 Mon Jun 30 10:17:39 EDT 2025 Sat Nov 29 03:55:50 EST 2025 Tue Nov 18 21:29:46 EST 2025 Tue Aug 26 17:17:22 EDT 2025 |
| IsPeerReviewed | true |
| IsScholarly | true |
| Issue | 6 |
| Language | English |
| License | https://ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-c326t-a50c22322d61e3de9f13fa147caa7ab83e3d8251014aa3b75a9daa7e1f8d630b3 |
| Notes | ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 14 ObjectType-Article-2 ObjectType-Feature-1 content type line 23 |
| PQID | 890387213 |
| PQPubID | 75721 |
| PageCount | 11 |
| ParticipantIDs | proquest_miscellaneous_926331424 crossref_primary_10_1109_JSTSP_2011_2159701 ieee_primary_5887382 proquest_journals_890387213 crossref_citationtrail_10_1109_JSTSP_2011_2159701 |
| PublicationCentury | 2000 |
| PublicationDate | 2011-Oct. 2011-10-00 20111001 |
| PublicationDateYYYYMMDD | 2011-10-01 |
| PublicationDate_xml | – month: 10 year: 2011 text: 2011-Oct. |
| PublicationDecade | 2010 |
| PublicationPlace | New York |
| PublicationPlace_xml | – name: New York |
| PublicationTitle | IEEE journal of selected topics in signal processing |
| PublicationTitleAbbrev | JSTSP |
| PublicationYear | 2011 |
| Publisher | IEEE The Institute of Electrical and Electronics Engineers, Inc. (IEEE) |
| Publisher_xml | – name: IEEE – name: The Institute of Electrical and Electronics Engineers, Inc. (IEEE) |
| References | ref35 ref13 orio (ref9) 2001 ref15 orio (ref20) 2001 ref36 ref14 every (ref30) 2004 ref11 ref32 ref10 ref2 ref1 cano (ref12) 1999 raphael (ref21) 2001 macrae (ref7) 2010 goto (ref39) 2002 vercoe (ref17) 1984 grubb (ref22) 1994 yeh (ref33) 2010; 18 li (ref29) 2009; 17 ref24 ref23 ref26 cont (ref38) 2007 dannenberg (ref16) 1984 woodruff (ref4) 2006 ganseman (ref5) 2010 ref25 puckette (ref18) 1995 grubb (ref19) 1997 dixon (ref6) 2005 vincent (ref37) 0 virtanen (ref31) 2003 ref28 ref27 ref8 ref3 ganseman (ref34) 2010 |
| References_xml | – start-page: 35 year: 2003 ident: ref31 article-title: Algorithm for the separation of harmonic sounds with time-frequency smoothness constraint publication-title: Proc Int Conf Digital Audio Effects (DAFx) – ident: ref14 doi: 10.1007/s10994-006-8415-3 – start-page: 94 year: 1994 ident: ref22 article-title: Automated accompaniment of musical ensembles publication-title: Proc 10th Nat Conf Artificial Intell (AAAI-92) – start-page: 315 year: 2007 ident: ref38 article-title: Evaluation of real-time audio-to-score alignment publication-title: Proc Int Conf Music Inf Retrieval (ISMIR) – start-page: 193 year: 1984 ident: ref16 article-title: An on-line algorithm for real-time accompaniment publication-title: Proc Int Comput Music Conf (ICMC) – year: 0 ident: ref37 publication-title: BSS Oracle Toolbox Version 2 1 – ident: ref36 doi: 10.1109/TSA.2005.858005 – start-page: 199 year: 1984 ident: ref17 article-title: The synthetic performer in the context of live performance publication-title: Proc Int Comput Music Conf (ICMC) – ident: ref3 doi: 10.1109/ICASSP.2011.5946324 – ident: ref2 doi: 10.1162/comj.2008.32.1.51 – volume: 17 start-page: 1361 year: 2009 ident: ref29 article-title: Monaural musical sound separation based on pitch and common amplitude modulation publication-title: IEEE Trans Audio Speech Lang Process doi: 10.1109/TASL.2009.2020886 – start-page: 314 year: 2006 ident: ref4 article-title: Remixing stereo music with score-informed source separation publication-title: Proc Int Conf Music Inf Retrieval (ISMIR) – year: 2010 ident: ref5 article-title: Source separation by score synthesis publication-title: Proc Int Comput Music Conf (ICMC) – start-page: 197 year: 2004 ident: ref30 article-title: A spectral-filtering approach to music signal separation publication-title: Proc Int Conf Digital Audio Effects (DAFx) – ident: ref32 doi: 10.1109/TSA.2005.857574 – start-page: 155 year: 2001 ident: ref9 article-title: Alignment of monophonic and polyphonic music to a score publication-title: Proc Int Comput Music Conf (ICMC) – ident: ref1 doi: 10.1109/TASL.2008.919073 – ident: ref24 doi: 10.1109/TPAMI.2009.106 – ident: ref28 doi: 10.1109/78.978374 – year: 2001 ident: ref21 article-title: A Bayesian network for real-time musical accompaniment publication-title: Proc Adv Neural Inf Process Syst (NIPS) – start-page: 199 year: 1995 ident: ref18 article-title: Score following using the sung voice publication-title: Proc Int Comput Music Conf (ICMC) – start-page: 441 year: 1999 ident: ref12 article-title: Score-performance matching using HMMs publication-title: Proc Int Comput Music Conf (ICMC) – ident: ref13 doi: 10.1109/34.761266 – ident: ref27 doi: 10.1007/978-1-4757-3437-9 – ident: ref11 doi: 10.1109/ICASSP.2009.4959972 – start-page: 219 year: 2010 ident: ref34 article-title: Evaluation of a score-informed source separation system publication-title: Proc Int Symp Music Inf Retrieval (ISMIR) – start-page: 301 year: 1997 ident: ref19 article-title: A stochastic method of tracking a vocal performer publication-title: Proc Int Comput Music Conf (ICMC) – ident: ref25 doi: 10.1155/2007/48317 – ident: ref15 doi: 10.1109/TASL.2011.2134092 – ident: ref35 doi: 10.1109/ICASSP.2010.5496224 – ident: ref10 doi: 10.1109/ASPAA.2003.1285862 – start-page: 423 year: 2010 ident: ref7 article-title: Accurate real-time windowed time warping publication-title: Proc Int Conf Music Inf Retrieval (ISMIR) – volume: 18 start-page: 1116 year: 2010 ident: ref33 article-title: Multiple fundamental frequency estimation and polyphony inference of polyphonic music signals publication-title: IEEE Trans Audio Speech Lang Process doi: 10.1109/TASL.2009.2030006 – ident: ref26 doi: 10.1109/TSA.2003.815516 – year: 2001 ident: ref20 article-title: Score following using spectral analysis and hidden markov models publication-title: Proc Int Comput Music Conf (ICMC) – start-page: 287 year: 2002 ident: ref39 article-title: RWC music database: Popular, classical, and jazz music databases publication-title: Proc Int Conf Music Inf Retrieval (ISMIR) – ident: ref8 doi: 10.1109/TASL.2010.2042119 – ident: ref23 doi: 10.1109/ICASSP.2006.1661258 – start-page: 92 year: 2005 ident: ref6 article-title: Live tracking of musical performances using on-line time warping publication-title: Proc Int Conf Digital Audio Effects (DAFx) |
| SSID | ssj0057614 |
| Score | 2.3111217 |
| Snippet | Soundprism, as proposed in this paper, is a computer system that separates single-channel polyphonic music audio played by harmonic sources into source signals... |
| SourceID | proquest crossref ieee |
| SourceType | Aggregation Database Enrichment Source Index Database Publisher |
| StartPage | 1205 |
| SubjectTerms | Followers Frames Harmonic analysis Harmonics Hidden Markov models Instruments Mathematical model Multi-pitch estimation Music On-line systems Online online algorithm Real time systems score following Separation Separators Source separation Studies |
| Title | Soundprism: An Online System for Score-Informed Source Separation of Music Audio |
| URI | https://ieeexplore.ieee.org/document/5887382 https://www.proquest.com/docview/890387213 https://www.proquest.com/docview/926331424 |
| Volume | 5 |
| WOSCitedRecordID | wos000295012900011&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| journalDatabaseRights | – providerCode: PRVIEE databaseName: IEEE/IET Electronic Library customDbUrl: eissn: 1941-0484 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0057614 issn: 1932-4553 databaseCode: RIE dateStart: 20070101 isFulltext: true titleUrlDefault: https://ieeexplore.ieee.org/ providerName: IEEE |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LS8QwEB7W1YMefOwqri9y8KZxm6ZtGm-LuHiQZaEqeytpksKCtLIPf795tIugCN5Kk4Yyk8xMMpnvA7iWXBVlzDkmghU4YkxiETGKeSKKQAZKei6Ct2c2maSzGZ924HZTC6O1dpfP9J19dLl8Vcu1PSobxmZF0NQY3C3GEl-r1VpdEzaTJoMc4iiOaVsgE_ChmeLZ1KN1GgfHWUMA0zohx6rywxQ7_zI--N-fHcJ-E0eikVf8EXR01YO9b-iCPdh2HM59mGaWOsnhHd6jUYU8uijyWOXIBK0os1iW2FcmaYUyd6CPMu1xwesK1SVyg6HRWs3rY3gdP748POGGSAEbUScrLOJAmjAgDFVCNFWal4SWgkRMCsFEkVLz0pawmu2SELRgseDKtGhSpiqhQUFPoFvVlT4FFJOypGFh4jotokKHnJj-qaAmVOCWaXMApJVsLhuUcUt28Z673UbAc6eN3Gojb7QxgJvNNx8eY-PP3n0r_03PRvQDOG8VmDfLcJmn3GbnQ0IHgDatZv3YpIiodL1e5jxMKLXlfme_j3sOu2F77Y9cQHe1WOtL2JGfq_lyceXm4BdAptfj |
| linkProvider | IEEE |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV3dS-QwEB9EBfXh_Dpxz688-OblbJK2aXxbRFFcl4Wq-BbSJAVBWnF37--_fLSL4CH4Vpo0lJlkZpLJ_H4Ap1qYqs6EwETxCqeca6xSzrDIVZXoxOjIRfA04uNx8fwsJkvwe1ELY60Nl8_sH_8Ycvmm1XN_VHaeuRXBCmdwV7I0pUms1urtrgucSZdDpjjNMtaXyCTi3E3ychLxOp2LE7yjgOndUOBV-WSMg4e53vzev23Bjy6SRMOo-m1Yss0ObHzAF9yBlcDivAuT0pMnBcTDCzRsUMQXRRGtHLmwFZUezRLH2iRrUBmO9FFpIzJ426C2RmEwNJybl_YnPF5fPVze4I5KATth5zOsskS7QIBSkxPLjBU1YbUiKddKcVUVzL30Raxuw6QUq3imhHEtltSFyVlSsT1YbtrG7gPKSF0zWrnIzqq0slQQ179QzAULwnNtDoD0kpW6wxn3dBevMuw3EiGDNqTXhuy0MYCzxTdvEWXjy967Xv6Lnp3oB3DQK1B2C3EqC-Hz85SwAaBFq1tBPi2iGtvOp1LQnDFf8Pfr_-OewNrNw_1Ijm7HdwewTvtLgOQQlmfvc3sEq_rv7GX6fhzm4z-k5dsq |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Soundprism%3A+An+Online+System+for+Score-Informed+Source+Separation+of+Music+Audio&rft.jtitle=IEEE+journal+of+selected+topics+in+signal+processing&rft.au=Zhiyao+Duan&rft.au=Pardo%2C+B.&rft.date=2011-10-01&rft.pub=IEEE&rft.issn=1932-4553&rft.volume=5&rft.issue=6&rft.spage=1205&rft.epage=1215&rft_id=info:doi/10.1109%2FJSTSP.2011.2159701&rft.externalDocID=5887382 |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=1932-4553&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=1932-4553&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=1932-4553&client=summon |