Customer Conversion Prediction with Markov Chain Classifier

For on line users, conversion generally refers to the user action that results in some tangible gain for a business e.g., an user opening an account or an user making his or her first purchase. Next to drawing large number of users to a web site, getting an user to convert is the most critical event in an user’s relationship with on line business. Being able to predict when an user will convert to become a customer should be an important tool that on line businesses should have at their disposal. A business could intiate targeted marketing campaign based on the prediction result.

In this post, I will be using user online behavior data to predict whether an user will  convert using Markov Chain Classifier. The Hadoop based implementation is part of my OSS project avenir.

Markov Chain Classification for Sequence Data

Depending on whether the data is instance or sequence , there two classes of learning algorithms. With sequence data, the order of the data points is considered an important aspect of the data and the learning algorithms will exploit the order in the data. Markov Chain Classification is a  supervised learning algorithm for sequential data.

Sequence data with a temporal context is called time series data. For many learning problems, sequence data is more effective. When we use instance data, the  order between the data points, temporal or something else, is lost.

For our example, each web site visit or session will be considered a data point in a sequence of data points. Session data ordered by time will be our sequence data. The data elements in the sequence i.e. session data will a vector of several attributes.

There are many relevant attributes for web session. I will be considering only the following.  I am not implying that these are only feature attributes that matter.

1. Time elapsed since the last visit
2. Time spent in the session

Other feature attributes that could be relevant in the predictive model are as follows. It requires something called feature analysis to determine the relevant sub set of features for any predictive analytic problem.

1. Referrer for the first visit
2. Hour of the day for visit
3. Day of the week for visit
4. Number of pages visited in a session

Building Markov Model

The first step towards building the complete solution is to build the Markov State Transition model. It’s a state transition  probability matrix. If a problem is modeled with n states, the matrix will be n x n. A matrix element value in the matrix is the probability of transitioning from one state to another.

Going back to our problem, we are are considering only two session attributes. To keep the state transition matrix manageable, we will discretize  the attributes into 3 levels; High, Medium and Low. With two attributes, we will end up with 9 states in out problem. Each session will be characterized with two symbols, which stands for a state. For example, HM will imply that time elapsed since last session is high and time spent in the current session is medium.

Each line in our input will capture the data about all consecutive past sessions for a given user. It will consist of the following

1. Cookie ID (or User ID)
2. Class variable indicating whether the user converted or not (True or False)
3. Sequence of session data where each element of the sequence is a 2 alphabet symbol

This is essentially our training data. Here is some sample data

4F014156K07N,F,LL,ML,HH,HL,LL,HM,HL,LH,ML,HH,HL,LH
G7C0M9H5SUZ1,F,HL,LM,HL,MH,HH,HH,ML,HL
GWBX875AD31D,F,LL,HM,HL,HL,HM
KRO2F24JUDE5,T,HL,HM,HM,HL,HM,MH,HM,HL,HL
3J0G4BB9BI1Q,F,LM,LH,LH,MH,LM,MH,LH

This data is ingested by the map reduce class MarkovStateTransitionModel. The output will consist of 2 matrices, each being a 9 x 9 state transition probability matrix. Here is the output. The first line contains the list of states.

LL,LM,LH,ML,MM,MH,HL,HM,HH
classLabel:T
85,103,219,81,46,105,159,94,105
100,142,294,55,61,127,79,49,89
102,142,311,54,61,135,45,46,100
100,93,155,88,61,93,214,118,72
98,105,218,93,55,115,120,100,91
98,124,299,55,39,136,91,47,107
121,51,81,121,61,75,276,101,108
108,84,150,110,62,67,201,100,115
87,108,208,74,59,114,166,74,106
classLabel:F
106,51,50,129,66,53,300,130,110
128,53,69,124,60,57,276,125,102
105,59,89,109,52,62,296,111,113
108,46,46,130,59,54,289,147,116
108,49,51,154,43,57,294,125,115
115,64,55,120,58,60,276,131,115
111,52,41,135,64,55,288,137,111
112,52,43,133,54,55,288,136,122
117,48,50,143,62,56,270,137,112

The cell values in the matrix are not actual probability, but normalized frequency count. As we will see later, since we will take ratio of the matrix cell values, actual probability is not necessary.

Predicting with Markov Model

Our test data format will be similar to the training data, except that the class variable is missing, which is going to be predicted. The test data and the Markov State Transition matrix produced by the first map reduce is processed by the map reduce class MarkovModelClassifier. For each sequence we calculate log odd ratio as below.

logOdds = Σlog(tp(i,j) / tn(i,j))
where
tp(i,j) = transition probability for transition from state i to j for positive (T) class value
tn(i,j) = transition probability for transition from state i to j for negative (F) class value

From the sequence, we take contiguous pairs of data points by sliding an window of size 2 over the sequence. For each state pair, we read two normalized frequency count values from the two matrices and calculate the log odds ratio as above. If the ratio is greater than 0, then the user is likely to convert.

In plain language, we determine whether the sequence of states for an user is more likely for positive class values or negative class values and make the prediction accordingly. The prediction score i.e. log odds ratio changes with time based on the sequence of states at any given time.

Instead of 0, a threshold value can be provided through a configuration parameter. For example, a small positive threshold value could be provided to keep us way from the class separation boundary and reduce the number of false positives. Here is some sample output, for a log odds threshold of 0.5. In this output snippet, one user has been predicted for conversion

6F1RIJZ882GA,F,-0.9419187552056059
R393GY9VWM9P,F,-1.0870003657324971
X573L7KOBRR0,F,-0.01828609983127566
KMUY5UN2M7TY,T,1.1076440756594925
2VBVQLCDDYJE,F,-4.542646122905596

All the predictive model does is that it identifies users that have the potential to convert. A business could target those users with marketing campaign, hopefully to turn that potential into reality.

The marketing could specifically target users that are close to the border line i.e., log odds ratio is slightly above the threshold. Those are the users that need to be nurtured into paying customers. Those with high log odds ratio, are more likely to convert  any way and may not need any special attention.

Meta Learning and Model Complexity

Meta learning is about understanding and learning about the learning process. In machine learning context, typically it involves learning about and selecting some higher order parameters that drive the learning algorithm performance. In our case, we are concerned about two meta learning parameters.

We would like find out the optimum sub set of feature attributes among all feature attributes listed earlier. We would also like to  learn whether or not a higher order Markov Model will  have more predictive accuracy and if so what order.

It’s good to remind ourselves that making a model more complex will not necessarily yield the best result, because of bias variance trade off. Generally there is an optimum level of complexity when the generalization error is minimum.

Feature Set Selection

We could increase the complexity of our model by including more feature attributes.We have used only two session related variable, with each one having 3 possible values. We ended up with 9 x 9 state transition probability matrices.

If we included some of the other attributes mentioned earlier, the state transition probability matrices would have been much bigger.

These are some of the ways we could explore the feature space looking for an optimum sub set of features providing best prediction accuracy.

1. Find relevance of each feature attribute with respect to the class atribute and other feature attributes
2. Brute force way of trying different feature sub sets with learning algorithm being used

Details of the first approach can be find in my earlier post on feature sub set selection. The solution is based on calculating certain statistical parameters based entropy and mutual information.

Higher Order Markov Model

We used first order Markov Model i.e., the state probability is only conditioned on the immediately earlier state. As a result, we have two dimensional state transition probability matrices. If the probability was conditional on n earlier states, we would have used (n +1) dimensional state transition probability matrices.

The question is how do we select the number conditioning data points n for best predictive accuracy. Here are are couple of approaches.

1. For different values of n, construct higher order conditional probability matrix. Find the conditional entropy for each case. Choose n based on lowest conditional entropy.
2. Brute force way of trying different n with learning algorithm being used

Summing Up

We have explored Markov Model based predictive analytic technique that operates on sequence data. The tutorial for  preparing data and  executing the map reduce jobs are available.

Python Version

A Python version of the implementation is now available. Please follow the Python section of the tutorial document.