Business Goal Infused Recommendation


The output of a recommendation engine,  whether based on collaborative filtering or some other techniques reflects consumer’s interest in products or services. However a business may have some goals that may be at odds with the items recommended by  the recommendation engine. For example, a business may be more interested in selling items with large inventory  or items that are being promoted with discounted pricing. How do we reconcile these conflicting interests.

My open source recommendation engine sifarish is no exception to this problem. . So I decided to do something about it. We need to find score for recommended items that reflect a compromise between consumer interest and business interest. Here is my solution. For every item scores are assigned for one or more business goals.  These scores are combined with the recommendation score using a weighted average algorithm.

Recommendation Engine Processing

As outlined in my earlier blog, there are three MR jobs that run in tandem to generate predicted rating for each user, item pair. The three MR jobs do the following

  1. Create rating matrix from available rating data and generate correlation between items
  2. For all users, using available rating data as input and the correlation coefficient from the first job generate predicted rating
  3. For a given item and user, aggregate across all input rating to generate final predicted rating for an user and item

Business Goal Processor Map reduce

The scores for different business goals along with recommendation output are fed into the MR class BusinessGoalInjector  along with the output of the last MR from the list above to generate the final rating score. Here is some sample business score data

W2EIVQXWU5,568,21
M7LIBV4M4P,897,681
FN95R3KA36,298,185
1Z12NT7OVE,692,622
YCEM2VGI7K,632,933
1XJME7Y3V3,332,670
Y49XI0HKKZ,803,691

The fields for the business goal in our example input are

  1. Item ID
  2. Score for inventory level
  3. Score for promotion.

The magnitude  of score reflects the importance of the particular business goal. For example, a score of 1000 for inventory  implies highest level of inventory. Actual inventory level is transformed to a scale of 0 to 1000.

The mapper output key is the itemID.  There are two kinds of values associated with each key; one business score record and multiple user ID, rating score records. We are using secondary sorting so that the business score record appears first on the reducer side followed by multiple userID, rating score pairs for a given itemID which is the key

Relative Weighting

Different weights can be can be assigned to different business goal through a config parameter as follows

 biz.goal.weights=25,15

The remaining weight of 60 in this case is assigned to the basic recommendation rating score. The total adds up to 100. The reducer computes the weighted score and outputs (userID, itemID, score).

Some business goals may force a recommended item to be excluded, based on the score of that business goal. For example, when the score for inventory for an item is below some predefined threshold, that item does not make it to the final recommendation list. This threshold is defined through the following configuration for the two business goals we have in our example. The value of -1 implies no such filtering is applied.

biz.goal.min.threshold=5,-1

Results

Finally, I run the output through a MR called TextSorter, which sorts the output based on userID, so that I have all the predicted score for different items for a given user together. Here is some sample out. The first set is the basic recommendation engine output. The second set is the output after business goal processing

T32JFQSFM45T,VU9D1W970J,1000
T32JFQSFM45T,A3AS3B4KQT,250
T32JFQSFM45T,2AC311PMPT,1000

T32JFQSFM45T,VU9D1W970J,786
T32JFQSFM45T,A3AS3B4KQT,386
T32JFQSFM45T,2AC311PMPT,731

The fields are userID, itemID and score. As you can see, in some cases the score has dropped and in some cases gone up.

Contextual Recommendation

In contextual recommendation, additional constraints are place on the recommendations results based on various other criteria. Here is an example of contextual recommendation that can be handled using the same technique.

In recommending web content, the freshness of the content could be incorporated by assigning score inversely proportional to the time elapsed since the content was published. Content could also be assigned score based on advertisement revenue potential .  These scores will impact the final recommendation score and how they are ranked when presented to the user.

Summing Up

The processing presented here is simple and straightforward. However, the feature is important for any recommendation engine being used to solve real world problem. I have updated the collaborative filtering based recommendation tutorial document to include these two post processing MR.

Advertisements

About Pranab

I am Pranab Ghosh, a software professional in the San Francisco Bay area. I manipulate bits and bytes for the good of living beings and the planet. I have worked with myriad of technologies and platforms in various business domains for early stage startups, large corporations and anything in between. I am an active blogger and open source project owner. I am passionate about technology and green and sustainable living. My technical interest areas are Big Data, Distributed Processing, NOSQL databases, Machine Learning and Programming languages. I am fascinated by problems that don't have neat closed form solution.
This entry was posted in Big Data, Collaborative Filtering, Hadoop and Map Reduce, Predictive Analytic, Recommendation Engine and tagged , , . Bookmark the permalink.

One Response to Business Goal Infused Recommendation

  1. Pingback: Positive Feedback Driven Recommendation Rank Reordering | Mawazo

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s