Kevin H. Knuth This site is an updated
version of my older BSE site, which had derivations
that can be found in the publications. Last updated: 6 November 2004 

SOURCE
SEPARATION The problem of source separation is a ubiquitous problem in the sciences where there are multiple signal sources s(t), which are recorded by multiple detectors. In this scenario, each detector records a mixture x(t) of the original signals. The goal is to recover estimates of the original signals y(t). 


SOURCE SEPARATION AS INFERENCE THE BAYESIAN APPROACH
EXAMPLES


Bayesian derivation of ICA In this paper, I demonstrated how one can derive the Bell & Sejnowski Infomax ICA algorithm in the Bayesian framework. The advantage here is that the prior probabilities and the signal model can be varied at will to produce a host of source separation algorithms specifically tailored to suit the needs of a particular problem. My subsequent work demonstrated this approach on varies applications. Knuth K.H. 1999. A Bayesian approach to source separation. In: J.F. Cardoso, C. Jutten and P. Loubaton (eds.), Proceedings of the First International Workshop on Independent Component Analysis and Signal Separation: ICA'99, Aussios, France, Jan. 1999, pp. 283288. [arXiv link] [pdf (218 kb)]


ICA with source position priors 

In this picture, the sensors around the bridge recorded sounds from each of the characters during one of the crew's weekly catastrophic events. Since we know that the Starship Enterprise officers won't wander far from their posts, we can use their approximate locations to help separate their recorded speech signals from the mayhem. 


In the SPIE98 paper below,
I considered an example where the source positions are known with some
accuracy. This combined with the propagation laws of the signal (inverse
square) leads to a prior probability on the values of the mixing matrix,
which, in general, improves the separation. The results aren't perfect
however, because I use a prior on the source amplitude histograms that
is inappropriate for some of the other recorded signals, such as the photon
torpedo blast. These difficulties are discussed in the MaxEnt97 paper
below, although in a different context. More detailed information can
be found at my old BSE site, and in the papers
below. Knuth K.H. 1998. Bayesian source separation and localization. In: A. MohammadDjafari (ed.), SPIE'98 Proceedings: Bayesian Inference for Inverse Problems, San Diego, July 1998, pp. 147158. [arXiv link] [pdf (363 kb)] Knuth K.H. 1998. Difficulties applying recent blind source separation techniques to EEG and MEG. In: G.J. Erickson, J.T. Rychert and C.R. Smith (eds.), Maximum Entropy and Bayesian Methods, Boise 1997, Kluwer, Dordrecht, pp. 209222. [pdf (421 kb)] 

EEG/MEG Source Separation and Localization 

Introducing information about the signal propagation and modeling the source locations as well as their signals, leads naturally to joint source localization and separation. We investigated this briefly in this paper where we demonstrated that the selected model of the physical system is all that separates source separation problems from source localization problems. The basic problem considered in this paper was estimation of neural sources. (Yes, that is my brain in the picture) 


Knuth K.H., Vaughan H.G., Jr. 1999. Convergent Bayesian formulations of blind source separation and electromagnetic source estimation. In: W. von der Linden, V. Dose, R. Fischer and R. Preuss (eds.), Maximum Entropy and Bayesian Methods, Munich 1998, Dordrecht. Kluwer, pp. 217226. [pdf (204 kb)] 

Differentially Variable Component Analysis (dVCA) This is a highly specialized algorithm that takes into account the fact that EEG/MEG experiments record data in a finite number of experimental trials. Since the activity produced by neural ensembles varies from trial to trial, our signal model accounts for this by allowing the source waveshape to vary in both amplitude and latency. These effects are estimated for each trial along with the stereotypic source waveshape. The relevant publications are below: Knuth K.H., Shah A.S., Truccolo W.A., Bressler S.L., Ding M., Schroeder C.E. 2005. Differentially variable component analysis (dVCA): Identifying multiple evoked components using trialtotrial variability. Submitted. [on request] Truccolo W.A., Knuth K.H., Shah A.S., Bressler S.L., Schroeder C.E., and Ding M. 2003. Estimation of singletrial multicomponent ERPs: Differentially variable component analysis. Biol. Cybern. 89(6): 42638. [PubMed link] [pdf (717kb)] Shah A.S., Knuth K.H., Lakatos P., Schroeder C.E. 2003. Lessons from applying differentially variable component analysis (dVCA) to electroencephalographic activity. In: G.J. Erickson, Y. Zhai (eds.), Bayesian Inference and Maximum Entropy Methods in Science and Engineering, Jackson Hole WY 2003, AIP Conference Proceedings 707, American Institute of Physics, Melville NY, pp. 167181. [pdf (445 kb)] Shah A.S., Knuth K.H., Truccolo W.A., Ding M., Bressler S.L., Schroeder C.E. 2002. A Bayesian approach to estimating coupling between neural components: evaluation of the multiple component event related potential (mcERP) algorithm. In: C. Williams (ed.), Bayesian Inference and Maximum Entropy Methods in Science and Engineering, Moscow ID 2002, AIP Conference Proceedings 659, American Institute of Physics, Melville NY, pp. 2338. [pdf (1.05 mb)] Truccolo, W.A, Ding M., Knuth K.H., Nakamura, R. and Bressler S.L. 2002. Variability of cortical evoked responses: implications for the analysis of functional connectivity. Clinical Neurophysiol. 113(2):20626. [PubMed link ] [pdf (433 kb)] Truccolo, W.A., Knuth K.H., Ding M., Bressler S.L. 2001. Bayesian estimation of amplitude, latency and waveform of single trial cortical evoked components. In: R.L. Fry and M. Bierbaum (eds.), Bayesian Inference and Maximum Entropy Methods in Science and Engineering, Baltimore 2001, AIP Conference Proceedings 617, American Institute of Physics, Melville NY, pp. 6473. [pdf (100 kb)] Knuth K.H., Truccolo, W.A., Bressler S.L., Ding M. 2001. Separation of multiple evoked responses using differential amplitude and latency variability. Proceedings of the Third International Workshop on Independent Component Analysis and Blind Signal Separation (ICA 2001), San Diego CA. [arXiv link] [pdf (248 kb)] 
