DM: RE: Principal Components and Correlations

[Cunningham, Scott W]

Krishnadas -

I know of no way to standardize principal components to ensure they
correlate in the expected, meaningful direction.   As you noted there 
is
a strong element of subjectivity in assigning the "meaningful"
direction.

The loading of the factor (positive or negative) is mathematically
arbitrary; both loadings explain the same amount of variance, and 
result
in the same prediction of the original correlation matrix.  
Multiplying
the loadings by negative one is therefore an appropriate means of
switching loadings to the expected direction.  You can quickly assess
whether the factor loaded in the expected direction by examining the
factor loadings.  I 

-       Scott

Scott Cunningham, D.Phil.
Human Interface Technology Center
NCR Corporation


        -----Original Message-----
        From:   C. K. Krishnadas [SMTP:ckkrish@cyberspace.org]
        Sent:   Thursday, January 29, 1998 5:20 AM
        To:     Datamining Mailinng List
        Subject:        DM: Principal Components and Correlations


        Hi,

        I  am having trouble with principal components and their
correla-
        tions with the original  variables.

        Suppose I have 10 variables, many of which move together.  I
have
        taken  principal components.  The first principal component
which
        accounts for a large chunk of the variance shows a negative
cor-
        relation  with  most of the variables, including the set of
vari-
        ables which are known to be moving together.  The  variables
are
        standardized  before  computing their variance-covariance
matrix.
        It is also expected that the  first  principal  component
should
        have  a  significant  (+ve) correlation with the set of
variables
        mentioned before.  But the correlations turn out to  be
negative
        and  significant.  In the computation, since the eigen vectors
of
        the variance-covariance matrix are chosen so as to maximize
vari-
        ability  in their direction, with orthogonality imposed with
each
        other, the correlations of variables of the  variables  with
the
        principal  components  can have signs contrary to common
expecta-
        tions.  Since the eigen vectors can be multiplied by  -1,  I
can
        get  a  new  set of eigen vectors which can be used to 
generate
a
        new set of principal components which can show correlations
with
        the  expected sign.  But this would involve compution of
correla-
        tion of the principal components with the original variables
and
        a  subjective  examination depending on the nature of data or
do-
        main knowledge (of application).

        Is there a standard method of choosing the eigen vectors or
prin-
        cipal components in such a way that they have correlations of
the
        expected (and subjectively meaningful) sign with the 
variables?


        Thanks

          -- Krishnadas


-----------------------------------------------------------------
        C. K. Krishnadas                c k krish at cyberspace dot o 
r
g
        ckkrish@cyberspace.org
http://www.cyberspace.org/~ckkrish
        na.kck@na-net.ornl.gov

-----------------------------------------------------------------