Re: DM: Genetic Algorithms

[Warren Sarle]

Sarab asks: 
> I found Warrens e-mail very interesting. It raises some very 
>interesting questions - 
> What is the definition of a "good algorithm"? 
> Also, how can we identify "bad initial values" so that we can steer 
>clear of them?

The definition of "good algorithm" for optimization will naturally be
somewhat subjective and application-dependent, but I would think the
basic requirements would be that a good algorithm should find an
acceptably accurate solution within an acceptable amount of time, and
should not be significantly slower and less accurate than other
algorithms.

As to the question of which algorithms are good, that definitely
depends on the problem (there are theorems to that effect, such as
Wolpert's No Free Lunch theorem). For neural network training (the
area with which I have been most concerned for the past few years),
there are some algorithms that are clearly bad (standard backprop),
and others that seem to be good for a wide variety of networks (such
as Levenberg-Marquardt). There are references and more info in the
neural net FAQ, part 2, at ftp://ftp.sas.com/pub/neural/FAQ2.html

Methods for finding good initial values are also very 
problem-dependent,
although it is almost always possible to find bad initial values!
Neural nets such as MLPs are quite difficult to initialize well, but
there are somm guidelines. For example, see:

   G.P.Drago, S.Ridella "Statistically Controlled Activation
   Weight Initialization (SCAWI)" IEEE Trans. on NN, Vol.3,No.4,
   July 1992, pp. 627-631.
   
   Bishop, C.M. (1995), Neural Networks for Pattern Recognition,
   Oxford: Oxford University Press, section 7.4. 

-- 

Warren S. Sarle       SAS Institute Inc.   The opinions expressed here
saswss@unx.sas.com    SAS Campus Drive     are mine and not 
necessarily
(919) 677-8000        Cary, NC 27513, USA  those of SAS Institute.
* Do not send me unsolicited commercial, political, or religious 
email *