Re: DM: Imputation of binary-valued features

[David L Dowe]

M> From owner-datamine-l@nessie.crosslink.net Fri Aug 15 09:28:38 1997
M> Date: Fri, 15 Aug 1997 10:24:57 +1200
M> From: Murray Jorgensen <maj@waikato.ac.nz>
M> Subject: Re: DM: Imputation of binary-valued features
M> To: datamine-l@nautilus-sys.com

At 21:22 14/08/97 +0100, Richard Dybowski <richard@n-space.co.uk> 
wrote:
>Hi
>
>I have a dataset in which all the variables (features) are binary, 
>however,
>some of the rows of the dataset have at least one value missing. Can 
>anyone
>give me details of an E-M algorithm (for which convergence is 
>guaranteed)
>that will enable me to model the underlying probability mass 
>distribution
>thus enabling me to perform imputation? There is an established 
>method of
>doing this when the variables are real-valued (i.e. by using a 
>Gaussian
>mixture model of a multivariate pdf), but what is the approved 
>method when
>the variables are binary-valued (or a mixture of real- and 
>binary-valued
>variables)?
>
>Thanking you in advance,
>
>Richard


M> To answer Richard's question, the answer for binary or 
multi-category
M> variables is known as Latent Class Analysis and the answer for when
M> variables are both continuous and categorical is our MULTIMIX.
M> 
M> Our earlier announcement follows:
M> 
M> 
-------------------------------------------------------------------------


Hi, Murray.

Richard and any other interested readers,
I have mailed to this list before the existence of my mixture 
modelling
(or clustering) page,
http://www.cs.monash.edu.au/~dld/mixture.modelling.page.html 

One of the reasons that I created this page (other than to publicise 
my
own work :-) ) was that questions such as Richard's are asked so 
often.


This WWW page lists three packages for doing mixture models for 
discrete data:


<B>Mixture modellers of Multi-nomial (or Bernoulli or multi-category) 
distributi
ons</B>
<BR>
<a href="ftp://ftp.cs.monash.edu.au/software/snob/">Snob</A>, by 
Chris Wallace a
nd <a href="/~dld/">David Dowe</A>   (see above).
<BR>
<a href="http://www.cs.waikato.ac.nz/stats/Staff/maj.html">Murray 
Jorgensen</A>'
s home page (see above, or link to MULTIMIX).
<BR>
<a href="http://markov.commerce.ubc.ca/marty/">Martin Puterman</A>'s 
home page,
with several of his papers, data and codes.
M. Puterman has worked on mixture models for discrete data.  
<I>Method</I>: Maxi
mum Likelihood and penalised likelihood.
</UL>


Two of these, Murray (Jorgensen)'s MULTIMIX and Chris Wallace's and my
Snob also deal with Gaussian (and other) distributions.


Please link there for more information.
Contact Murray Jorgensen for further info re MULTIMIX,
and please feel free to contact me either about Snob or about
being listed on my mixture modelling page.


Regards.         - David.

(Dr.) David Dowe, Dept of Computer Science, Monash University, 
Clayton,
Victoria 3168, Australia  dld@cs.monash.edu.au     Fax:+61 3 9905-5146
http://www.cs.monash.edu.au/~dld/
http://www.cs.monash.edu.au/~dld/mixture.modelling.page.html