Soundprism: An Online System for Score-Informed Source Separation of Music Audio

Soundprism, as proposed in this paper, is a computer system that separates single-channel polyphonic music audio played by harmonic sources into source signals in an online fashion. It uses a musical score to guide the separation process. To the best of our knowledge, this is the first online system...

Full description

Saved in:
Bibliographic Details
Published in:IEEE journal of selected topics in signal processing Vol. 5; no. 6; pp. 1205 - 1215
Main Authors: Zhiyao Duan, Pardo, B.
Format: Journal Article
Language:English
Published: New York IEEE 01.10.2011
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subjects:
Followers
Frames
Harmonic analysis
Harmonics
Hidden Markov models
Instruments
Mathematical model
Multi-pitch estimation
Music
On-line systems
Online
online algorithm
Real time systems
score following
Separation
Separators
Source separation
Studies
ISSN:1932-4553, 1941-0484
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Soundprism, as proposed in this paper, is a computer system that separates single-channel polyphonic music audio played by harmonic sources into source signals in an online fashion. It uses a musical score to guide the separation process. To the best of our knowledge, this is the first online system that addresses score-informed music source separation that can be made into a real-time system. The proposed system consists of two parts: 1) a score follower that associates a score position to each time frame of the audio performance; 2) a source separator which reconstructs the source signals for each time frame, informed by the score. The score follower uses a hidden Markov approach, where each audio frame is associated with a 2-D state vector (score position and tempo). The observation model is defined as the likelihood of observing the frame given the pitches at the score position. The score position and tempo are inferred using particle filtering. In building the source separator, we first refine the score-informed pitches of the current audio frame by maximizing the multi-pitch observation likelihood. Then, the harmonics of each source's fundamental frequency are extracted to reconstruct the source signal. Overlapping harmonics between sources are identified and their energy is distributed in inverse proportion to the square of their respective harmonic number. Experiments on both synthetic and human-performed music show both the score follower and the source separator perform well. Results also show that the proposed score follower works well for highly polyphonic music with some degree of tempo variations.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ObjectType-Article-2
ObjectType-Feature-1
content type line 23
ISSN:1932-4553
1941-0484
DOI:10.1109/JSTSP.2011.2159701