When I teach data analysis, I always consider what the best application of statical models to the real businesses is. I researched it several weeks and I found a recommender engine is one of the best applications to explain how statistical models or machine learning work in the real world. Now that most people get the recommendations about products, services, news through emails and websites. One of the famous examples is the recommendation by Amazon.com. So it is easy to understand what recommendations are and how useful they are.
I provided recommendations to my customers manually when I used to be an account executive at the branch of the security company more than 20 years ago. I had more than 200 customers there and sold the financial products to them. It was an interesting job as financial markets have been moving every day, every second. But there were problems about the way of marketing at that time.
1. I could not take care of every customer effectively
I could contact 20 or 30 customers by phone on a daily basis (there were no e-mails at that time). It was impossible, however, to contact more than 200 customers so I might miss or overlook the needs of customers because I could not understand who the customers were and what they wanted within limited time. It led to opportunity cost for me.
2 . I could not understand all products effectively
When I used to be an account executive, financial innovation was going on in Japan. It means that not only traditional products, such as stocks and bonds, but also derivatives and options were available to retail investors. There was not enough time for me to understand every product in detail. So I might fail to satisfy customers’ needs due to lack of knowledge of products which were available at that time.
If I could have a recommender at that time, these problems above could be solved as recommender engine could make the most of the information about both customers and products at once, in a timely manner. In order to provide good recommendations, it is clear that information about both customers and products are needed. It might be time-consuming and require human resources if we manage this information manually. But recommender engine can process it quick enough to provide recommendations in a timely manner. When companies, such as SME have small sales force, recommender engines are critically important because it enables such companies to provide recommendations to customers effectively. This is one of the best ways to communicate to customers with reasonable cost as well.
So I want to develop a recommender engine to provide good recommendations based on information from customers and products. My company is starting an open project to develop a recommender engine with R. This will be open source and get public so that everyone can learn how it works if he/she is interested in it. I will report the progress of the project on this blog going forward. I hope you can enjoy it!