“Unsupervised learning” is a powerful weapon to unknown worlds, isn’t it?

I have almost finished my MOOCs of machine learning in Coursera.   The algorithm to be learned now is “Unsupervised learning”.  It is the first time for me to learn unsupervised learning algorithms. It must be excited! What is the difference between  “Supervised learning” and “Unsupervised learning”?   Unlikely supervised learning algorithm,  we do not need to have the results of occurring events in the past. For example, when we try to apply the logistic regression model in the predictions of  defaults of customers,  we need to have results of defaults to train the models so that  the model can classify “Who will be in a default or not” effectively.  But  “Unsupervised learning”  does not need to have such results in advance. When I heard that, I was very surprised because I only knew supervised learning.  When I used to be a credit risk manager in a consumer finance company,  I should consider how we could obtain the data about customers which included data about “who was in default in the past?”.  However, unsupervised learning does not need to have the results of classifications in advance. Without the result of that, unsupervised learning algorithm can capture the structures of data. It enables us to jump into unknown worlds because unsupervised learning algorithm works in such an area because it does not need to have the results to train the models.

Then what kind of problems we can apply this powerful algorithm? Let us stretch our  imaginations here!

1.  Social structure

This is related to microeconomics. As there is a lot of social classes or group in our society.  There are many ways to make clusters in it.  For examples,  based on age, sex, annual salary, educations, address, industries,  cities, countries, etc.  Each class may behave differently to the economic events. Therefore, we may predict the sequences after the events, such as a tax increase when we can make clustering of our society. If unsupervised learning can make new clusters in our society on a real-time basis, it must be good for economic analysis as this is based on the latest information.

2. Banking and financial system

The banking system is critically important for the economy of each country.  It sometimes fails and malfunctions, however,  the economy has serious damage for long periods. Everyone knows what happened after Lehman crisis, which was one of the biggest financial crisis in history. There are a lot of players in the banking and financial systems, such as commercial banks, investment banks, credit card companies, asset management companies, consumer finance companies, etc. This  system is sometimes unstable due to massive lending activities. Usually it is difficult to understand what is going on there in real-time basis. If an unsupervised learning algorithm can capture change of structure in banking and financial system in advance, it may enable us to take action before the problem arises, rather than after.

Unsupervised learning may provide us new insight of our society as there is no need to obtain the result of events in advance.  It is good because the future is generally different from the past.  Unsupervised learning must be a powerful weapon to analysis new, unknown worlds  as our society has been changing everyday basis and sometimes no one knows what is going on there.

How does economics work on a shrinking population in Japan?

Bank of Japan may cut its growth forecast for this fiscal year to see the result of GDP growth in Q2 after an increase of consumption tax in Japan, according to Bloomberg on 15 Aug 2014.   Is it too early to increase taxes? Or is it inevitable to decrease the fiscal deficit?  Let me consider a little bit here because this is very important not only for Japanese people, but also other aging societies which follow Japan.

 

Fundamental problem in Japan, I think, is shrinking populations. The population of Japan is decreasing at a rapid pace.  More than 200,000 populations are lost in Japan every year. It must be very sad that if we can see a 200,000-living city is disappearing from our sight every year.  Although it is almost impossible to see what happens in the population every day,  I am sure it is not good for economic growth, investment strategy. The richer countries are, the more population they can sustain in them.  Therefore, economics implicitly assume populations in countries are not decreasing at least unless there are disasters or epidemic. Unfortunately, it is not the case in Japan.  The population has been shrinking even though it is the third biggest economy in the world.

 

Aggregate demand

Shrinking population has a negative impact against aggregate demand as fewer people buy goods and services. Therefore tax increase may discourage consumer confidence more than in a normal economy in which populations are increasing. In addition to that older people consume fewer goods and services than younger people do.  There will be no need for new shopping malls, convenience stores, gasoline stands, schools and kindergartens anymore in such situations. Even thought we would need new hospitals to take care older people and funeral ceremony services when they pass away,  I do not think these services can compensate lost demands due to shrinking populations.

 

Export

How about exports to grow GDP?  When JPY is weakening, exports used to be picked up.  The current situation,  however, a little different.  Since the big earthquake hit Japan in March 2011,  trend of trade balance has been negative, even though JPY has been weakening.  One of the reasons is the importing energy to replace nuclear power plants.  Another is that Japanese consumer goods are less competitive than they were in 2000s.  So Japan cannot rely on exports to offset shrinking domestic demand.  What should we do? 

 

Human capital

One way to revive the Japanese economy is that bringing up high profit and productive industries.  The key is human capital in Japan to achieve that. When we focus on how older people should be cared after their retirement, however, people tend to forget how we should bring up younger people , who are the next generation of workforces. This is a kind of problem about optimization of our society.  How we should allocate our resources between older people and younger people.  Which should come first, schools or hospitals?  In terms of education in Japan,  I am not so confident to say that the Japanese education system enables its children to compete global competition to obtain skilled jobs. English, math and programming will be critical to raise employability in the future, however, it seems there is no change in the Japanese educational system to teach them effectively.  In the long run, I am afraid Japan can not raise productivity because its workforce lack fundamental skills.

 

Immigration

People who have never been to Japan, may not understand why Japan does not have immigrations from overseas to compensate shrinking populations.  In my view, Japan is not ready to have immigrations from overseas as it is culturally homogeneous.  People share the same language and the same experience.  It enables them to do “non-verbal communication” which is difficult to understand from the standpoint of foreign people. This is an obstacle to live with foreigners.  It takes longer time for Japan to accept immigrations as few people has experience of “living with foreigners”.

 

 

I must say there is no easy way out of this difficult situation.  Although tax increase from 8% to 10% is needed to decrease the fiscal deficit of Japan, it is very difficult to keep the best timing for the Japanese government to introduce it.  Japan has only limited time to make its fiscal balance to be sustainable.

Three reasons why I recommend the MOOCs of Machine Learing in Coursera

There is no doubt that machine learning is a hot topic in 2014.  It is difficult, however, to answer what machine learning is.  I think the best way to understand machine learning is taking MOOCs by Andrew Ng in Stanford university at Coursera.  Actually, I am taking this course now and I am able to understand how to program complex mechanism in machine learning. This is amazing as it is free and available to everyone.  I would like to present three reasons why I recommend this course to everyone.

 

1.  Several models are explained and compared each other in the course

In this course, we can learn regression models, support vector machines, neural networks, unsupervised learning algorithms.  In addition, we can compare them each other.  So it is good to obtain an overview of machine learning.  Of course we can implement algorithm of each method by using MATLAB/Octave. You can be an expert of machine learning after completion of this course.

 

2.  The math theories behind the algorithms are also explained in it.

When each algorithm is introduced,  math theory behind them are also explained.  Even thought it is high level,  rather than proof of them,  it is useful because  it can help us to understand what the theory behind the algorithms is and we do not need to treat machine learning as “black box”.  I do not think math at postgraduate level is needed in this course.  If you are not familiar with math, there is no need to worry as basic linear algebra lecture is available in the course. This linear algebra lecture starts with high school level and covers everything which is needed in the course. I like it very much!

 

3.  Best practice in the industry is also presented

Dr.Andew Ng, who is the instructor of this course, has been working in the IT industries and currently works for Baidu as their Chief Scientist . So he provides us a lot of best practice in analyzing data by machine learning. It is very important because I assume that most participants in this course are practitioners rather than theorists.  I hope a lot of  startups will be born based on the best practice presented in this course.

 

These three points above keep a good balance in the course. Therefore, even beginners of machine Learning can understand how it works in theory and practice. I cannot say it is easy to pass the program exercises in this course if you are not familiar with MATLAB/Octave, however, I can say it is worth just looking at the video lectures of this course. Although it takes 10 weeks to complete,  you can learn anytime and anywhere through internet once you register the course. I want to give a strong recommendation to this course  for your career development. It might be also good for high school students who is interested in programming. I would like to appreciate Dr. Andew Ng, who prepares this great course.  This is a treasure for everyone!

 

NASDAQ Global Index Family now available to everyone for free !

Is this also a game changer ? I think it is!  Quandl, which is a data gathering platform, announced that indexes data from NASDAQ OMX are provided and users can download and use them for data analysis without paying any fees.  These indexes are called ” The Global Index Family “. According to the website of Quandl, “The Global Index Family is comprehensive: it covers international securities segmented by geography, sector, and size. NASDAQ OMX’s transparent and rules-based selection method results in a complete representation of the global investable equity marketplace. The indexes cover 45 individual countries within Developed and Emerging Markets, and facilitate a multitude of tracking, trading, and investing opportunities.” This is amazing. When I worked in investment banks before,  each division of the banks has Bloomberg terminals.  However, I could not have it by myself because it was too expensive to me. Now 40,000 indexes are available for me through Quadl.   Then what can we do with it?

 

1.  Research by region

We can analyze markets by country.  For example, there are more than 400 indexes about Malaysia.  These indexes are enough to analyze the market in Malaysia. Before the Global Index Family is available, it is difficult to do that as only a small number of indexes were available for me.  These indexes cover 45 countries so far. It means that we can cover major economic power globally.  It is fantastic!

 

2. Research by sector

If you are interested in relationship market behaviors and the real economy,  indexes by sectors are very useful.  When the price of energy goes up,   each sector must have different behavior.  How about interest rate appreciation?  What if geopolitical issues happen?  When you would like to know what happened in the market at the time of major events in the economy,  you can analyze data by sector with Quadl now.

 

3. Research by size

If you are in the management of SME.  You may be interested in the behavior of small-size listed companies in equity markets. It is possible to analyze that because the Global Index Family includes mid cap and small cap indexes. I would like to analyze markets based on the index data by size.

 

The Global Index Family is comprehensive. So once I am familiar with that,  it is easy to handle the data around the globe. In addition to that,  combination between these indexes and other economic data enable me to produce new research and analysis with different angles, which has been impossible for me before these indexes are available. It must be exciting!  It should be noted that all indexes in Quandl can be downloaded directly into the major data analysis tools such as  Python, R  and MATLAB, etc. For example, if you want to download data of Nikkei 225 into R,  You just type command like ” Quandl(“NIKKEI/INDEX”, trim_start=”1955-01-17″, trim_end=”2015-06-21″, collapse=”monthly”) “.  Then data you want are downloaded into your R . It is simple and easy to obtain data in order to analyze it in statistical tools.  Why don’t you start data analysis with Quandl ?