Re: DM: Mining longitudinal data

[Lynd Bacon]

Rajesh, here are a few ideas about dm and longitudinal customer data.
Maybe some other list subscribers will add to them.

Seems to me your approach depends on how you want to use time as a 
variable
in your models.

In the simplest case, you could ignore it entirely.  You could 
describe
associations between purchases of different products within a 
specified
interval.   Methods like market basket analysis and link analysis, as 
well
as other techniques for summarizing the structure of associations 
between
discrete entities, might be applicable. 

Or, you could try to predict purchase of a particular product using
purchases of other products.  The interval over which you consider
purchases might be defined based on the Allstate calendar, based on
customers' tenures as Allstate customers, or based on "customer" 
calendars. 

You can represent time as a discrete or a continuous variable.  In
practice, however, most variables are sampled, or binned, in discrete 
time
intervals.  Picking the width of the interval is an important
warehousing/dm decision, since it affects what kinds of models you 
might
apply, and what kind of results you're likely to observe.  Make the
intervals too narrow, and the events of interest are rare within 
intervals.
 Make them too wide, and you get too coarse a view of temporal
dependencies.  Because the within-customer purchase frequency for 
insurance
products is low (at least compared to package goods purchases), you 
may
find that fairly wide intervals are the most useful, say quarterly,
semi-annual, or annual.

Modeling approaches would include examining associations over 
successive
intervals, predicting over intervals.  Most methods that can be 
applied to
a single time interval's worth of data can be used for successive
intervals, although not always so conveniently.

One thing you have to watch out for is that the purchases are 
endogenous
variables.  Associations between purchases might be due to factors 
other
than stable customer needs.  The marketing efforts of Allstate or its
competitors, for example, might cause them.

You might want to consider segmenting your customers based on 
household
characteristics and/or developmental stage, and looking to see how 
segment
membership is associated with purchase patterns over time.  Also,
distinguishing between new purchase and renewals might be important.

Should you also be considering customer attrition, one thing you've 
got
working for you is that most insurance products must be renewed an a
regular basis, so it's easy to tell if you've lost a customer or not. 
 For
most products that are purchased irregularly, it's hard to tell 
whether a
customer will indeed purchase again.  Such purchase data are right 
censored
in time.

Finally, in terms of "success stories," IMS America has been modeling 
the
prescribing behavior of physicians over time in order to detect 
physicians
for whom sales tactics need to be changed.  They have been using
transaction data from several thousand pharmacies.  Their approach has
included (1) defining interesting temporal patterns (e.g. decreasing 
or
oscillating prescription rates); (2) developing models to predict 
who's
likely to show these patterns; (3) providing the predictions for use 
by
sales personnel. 

-lynd bacon

At 11:22 AM 9/14/98 -0600, Rajesh Parekh wrote:
>Hello everyone,
>
>I am working on a project involving data mining of longitudinal 
>data. Our data contains customer insurance purchase history
>over the years and our goal is to  design suitable models for 
>cross-selling applications.
>
>I am interested in knowing about research papers and/or application
>success stories that describe techniques for dealing with 
>longitudinal
>data. 
>
>Thanks,
>- Rajesh Parekh
>
>Datamining Research Group
>Allstate Research and Planning Center
>Menlo Park, CA 94025
>e-mail: rpare@allstate.com
>
>
>
>
/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
|~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~|
| LYND BACON & ASSOCIATES, LTD.       http://www.lba.com |
| marketing and management science    mr.daemon@lba.com  |
| Homewood IL USA                     +1.708.957.0883    |
|                ---------------------                   |            
| 
| Find out about the Chicago ASA monthly speaker series  |
|         at http://www.lba.com/asa-lunch.html           |
\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/