Facebook, Twitter, Google and “new wave” of economic analysis


On Saturday, I found that the report from Bank of England.  This report is about economic analysis in central banks with Big data such as social network services. It is good not only for economic researchers, but also business personnels to consider how Big data should be used. So I would like to consider it based on this report for a while.

Before considering usage of Big data, I would like to define “Big data”. Big data is data sets that are granular, in real time basis and  non-numeric data as well as  numeric one.   These data are completely opposite in nature compared with data which are currently analyzed in Central banks.  Because such data are usually “aggregated,  periodical and numeric”.  One of the examples is financial statements of companies.  Big data are different from such data.  For example Twitter are generated by individuals in real time. These are usually text, images and video. Then the questions come.


1. Can we build up macro economic models based on big data?

Central banks are responsible for the stability of the financial system in the country.  Is it possible for central banks to collect data of each loan from private banks and assess credit risk of each, then confirm financial stability as a whole country?  It can be applied to private companies, too. Is it possible that the company collect data of each customer, forecast the amount of purchase by each customer and predict the revenue of the company next fiscal year?  Big data may enable us to do so even though it takes time.


2. Is the method used “theory based” or “data driven”?

Even though they cannot be clearly distinguished in practice,  these are two approaches to analyze Big data in economic analysis. Someone puts importance to economic theories. Let us call it “theory based”.  Others take another approach of “Let the data speak for themselves”.   We may call it “data driven”.  Their opinions are sometimes against each other even though they analyze the same data. So we should have well-balanced approach between them.


3. Should we change the processes to make business decisions?

Big data comes to us in a real time basis.  But our decision making process in organizations is usually periodical. For example, board of directors meetings and executive committees in companies are generally held on a monthly basis.  Should they be held more flexibly in a timely manner based on outputs from analysis of Big data, rather than periodical one?  The bigger companies become, the more difficult it is to change the process in practice.


FRB in the US is currently wondering when they should raise the interest rate of the US.  Chairwoman of FRB has been always saying  “It is based on economic data“.  But I am not sure she cares about data (conversations) on social networking services in the US. What do you think?

Do you know how computers can read e-mails instead of us?


Hello, friends. I am Toshi. Today I update my weekly letter. This week’s topic is “e-mail”.   Now everyone uses email to communicate with customers, colleagues and families. It is useful and efficient. However, if you try to read massive amounts of e-mails at once manually, it takes a lot of time.  Recently, computers can read e-mail and classify potentially relevant e-mail from others instead of us. So I am wondering how computers can do that. Let us consider it a little.

1.  Our words can become “data”

When we hear the word “data”,  we imagine numbers in spreadsheets.  This is a kind of “traditional” data.  Formally, it is called “structured data”. On the other hand, text such as words in e-mail, Twitter, Facebook can be “data”, too.  This kind of data is called “unstructured data“. Most of our data exist as “unstructured data” around us.  However, computers can transform these data into data that can be analyzed. This is generally an automated process. So we do not need to check each of them one by one. Once we can create these new data, computers can analyze them at astonishing speed.  It is one of the biggest advantages to use computers in analyzing e-mails.

2. Classification comes again

Actually, there are many ways for computers to understand e-mails. These methods are sometimes called Natural language processing (NLP)“.  One of the most sophisticated one is a method using machine learning and understanding the meaning of sentences by looking at the structures of sentences. Here I would like to introduce one of the simplest methods so that everyone can understand how it works.  It is easy to imagine that the “number of each word” can be data.  For example, ” I want to meet you next week.”.  In this case, (I,1), (want,1),(to,1), (meet,1),(you,1), (next,1),(week,1) are data to be analyzed. The longer sentences are, the more words appear as data. For example, we try to analyze e-mails from customers to assess who are satisfied with our products. If the number of positive words, such as like, favorite, satisfy, are high,  it might mean customers are satisfied with the products, vice versa.  This is a problem of “classification“.  So we can apply the same method as I explained before. The “target” is “customers satisfied” or “not satisfied” and “features” are the number of each word. 

3. What’s the impact to businesses?

If computers understand what we said in text such as e-mails,  we can make the most out of it in many fields. For the marketing, we can analyze the voices of customers from the massive amount of e-mails. For the legal services, computers identify what e-mails are potentially relevant as evidences for litigations.  It is called “e-discovery“.  In addition to that, I found that Bank of England started monitoring social networks such as Twitter and Facebook in order to research economies.  This is a kind of “new-wave” of economic analysis.  These are just examples. I think  you can create many examples of applications for businesses by yourself because we are surrounded by a lot of e-mails now.  

In my view, natural language processing (NLP) will play a major role in the digital economy.   Would you like to exchange e-mail with computers?

Can Abenomics achieve its objects after winning the election?


Liberal Democratic party (LDP) won the election in Japan yesterday. Then Abenomics is going to fight against financial markets.  Financial market is tougher than other political parties as it reacts very quickly. If LDP can not convince the financial market that financial condition of Japan can be improved,  interest rate of Japanese government bond (JGB) will increase rapidly and price of JGB will be plunged. It means collapse of Japanese fiscal condition as Japanese government might not repay interest rare of debts. This is the race against time.

Moody’s Investors Service downgraded the Government of Japan’s debt rating by one notch to A1 from Aa3 at 1st December 2014.  The outlook is stable. This is the beginning of the story.  From now on,  market participants will focus on how Abenomics work in Japan after LDP won the election.  Rise of consumption tax is postponed until April 2017.  So only less than two and half years are left for Japan. Can Abenomics turn deflation to inflation during such a short period?  Can productivity of Japan be improved strong enough to rise the consumption tax?

Debt to GDP ratio already exceeds 200%.  Japan has no experience of such heavy burden except the time after world war 2. Ray Dalio, founder of Bridgewater Associates, explains that we need policy mix below to solve the heavy burden of national debt .

1. Wealth redistribution

2. Spending cut

3. Debt restructuring

4. Debt Monetization

In short,  tax rate will be  increased to wealthy people,  Japanese capital expenditure and social welfare will be cut and Bank of Japan will finance JGB.   This plocy mix is needed to avoid JGB default. LDP should convince Japanese people that this policy mix is needed and should be exercised in a timely manner.  This is the toughest task for LDP.  But it can not be postponed because it allows financial market to trigger the interest hike.

Productivity may increase gradually by Abenomics however it can not offset hike of consumption tax rate.  At the digital era,  knowledge of software engineering,  machine learning and artificial intelligence are critically important.  These are key in order to optimize the systems,  streamline the processes and improve the productivity. Unfortunately management of Japanese big companies are not familiar with fields of software . So Japanese companies are generally shy and away the innovations to improve productivity by big change.  I do see a few managements whose majors are software engineering and compter science in Japan.  In my view,  it is too late to change this situations because it takes ten years to train new managements within companies.  But we cannot wait such a long period anymore.

Anyway,  we should listen carefully to what the financial  market says.  I am not sure whether LDP wins over the financial market or not. All I can say is that Japan should be changed by herself, rather than being forced to change  by the market.

“Unsupervised learning” is a powerful weapon to unknown worlds, isn’t it?


I have almost finished my MOOCs of machine learning in Coursera.   The algorithm to be learned now is “Unsupervised learning”.  It is the first time for me to learn unsupervised learning algorithms. It must be excited! What is the difference between  “Supervised learning” and “Unsupervised learning”?   Unlikely supervised learning algorithm,  we do not need to have the results of occurring events in the past. For example, when we try to apply the logistic regression model in the predictions of  defaults of customers,  we need to have results of defaults to train the models so that  the model can classify “Who will be in a default or not” effectively.  But  “Unsupervised learning”  does not need to have such results in advance. When I heard that, I was very surprised because I only knew supervised learning.  When I used to be a credit risk manager in a consumer finance company,  I should consider how we could obtain the data about customers which included data about “who was in default in the past?”.  However, unsupervised learning does not need to have the results of classifications in advance. Without the result of that, unsupervised learning algorithm can capture the structures of data. It enables us to jump into unknown worlds because unsupervised learning algorithm works in such an area because it does not need to have the results to train the models.

Then what kind of problems we can apply this powerful algorithm? Let us stretch our  imaginations here!

1.  Social structure

This is related to microeconomics. As there is a lot of social classes or group in our society.  There are many ways to make clusters in it.  For examples,  based on age, sex, annual salary, educations, address, industries,  cities, countries, etc.  Each class may behave differently to the economic events. Therefore, we may predict the sequences after the events, such as a tax increase when we can make clustering of our society. If unsupervised learning can make new clusters in our society on a real-time basis, it must be good for economic analysis as this is based on the latest information.

2. Banking and financial system

The banking system is critically important for the economy of each country.  It sometimes fails and malfunctions, however,  the economy has serious damage for long periods. Everyone knows what happened after Lehman crisis, which was one of the biggest financial crisis in history. There are a lot of players in the banking and financial systems, such as commercial banks, investment banks, credit card companies, asset management companies, consumer finance companies, etc. This  system is sometimes unstable due to massive lending activities. Usually it is difficult to understand what is going on there in real-time basis. If an unsupervised learning algorithm can capture change of structure in banking and financial system in advance, it may enable us to take action before the problem arises, rather than after.

Unsupervised learning may provide us new insight of our society as there is no need to obtain the result of events in advance.  It is good because the future is generally different from the past.  Unsupervised learning must be a powerful weapon to analysis new, unknown worlds  as our society has been changing everyday basis and sometimes no one knows what is going on there.

How does economics work on a shrinking population in Japan?


Bank of Japan may cut its growth forecast for this fiscal year to see the result of GDP growth in Q2 after an increase of consumption tax in Japan, according to Bloomberg on 15 Aug 2014.   Is it too early to increase taxes? Or is it inevitable to decrease the fiscal deficit?  Let me consider a little bit here because this is very important not only for Japanese people, but also other aging societies which follow Japan.


Fundamental problem in Japan, I think, is shrinking populations. The population of Japan is decreasing at a rapid pace.  More than 200,000 populations are lost in Japan every year. It must be very sad that if we can see a 200,000-living city is disappearing from our sight every year.  Although it is almost impossible to see what happens in the population every day,  I am sure it is not good for economic growth, investment strategy. The richer countries are, the more population they can sustain in them.  Therefore, economics implicitly assume populations in countries are not decreasing at least unless there are disasters or epidemic. Unfortunately, it is not the case in Japan.  The population has been shrinking even though it is the third biggest economy in the world.


Aggregate demand

Shrinking population has a negative impact against aggregate demand as fewer people buy goods and services. Therefore tax increase may discourage consumer confidence more than in a normal economy in which populations are increasing. In addition to that older people consume fewer goods and services than younger people do.  There will be no need for new shopping malls, convenience stores, gasoline stands, schools and kindergartens anymore in such situations. Even thought we would need new hospitals to take care older people and funeral ceremony services when they pass away,  I do not think these services can compensate lost demands due to shrinking populations.



How about exports to grow GDP?  When JPY is weakening, exports used to be picked up.  The current situation,  however, a little different.  Since the big earthquake hit Japan in March 2011,  trend of trade balance has been negative, even though JPY has been weakening.  One of the reasons is the importing energy to replace nuclear power plants.  Another is that Japanese consumer goods are less competitive than they were in 2000s.  So Japan cannot rely on exports to offset shrinking domestic demand.  What should we do? 


Human capital

One way to revive the Japanese economy is that bringing up high profit and productive industries.  The key is human capital in Japan to achieve that. When we focus on how older people should be cared after their retirement, however, people tend to forget how we should bring up younger people , who are the next generation of workforces. This is a kind of problem about optimization of our society.  How we should allocate our resources between older people and younger people.  Which should come first, schools or hospitals?  In terms of education in Japan,  I am not so confident to say that the Japanese education system enables its children to compete global competition to obtain skilled jobs. English, math and programming will be critical to raise employability in the future, however, it seems there is no change in the Japanese educational system to teach them effectively.  In the long run, I am afraid Japan can not raise productivity because its workforce lack fundamental skills.



People who have never been to Japan, may not understand why Japan does not have immigrations from overseas to compensate shrinking populations.  In my view, Japan is not ready to have immigrations from overseas as it is culturally homogeneous.  People share the same language and the same experience.  It enables them to do “non-verbal communication” which is difficult to understand from the standpoint of foreign people. This is an obstacle to live with foreigners.  It takes longer time for Japan to accept immigrations as few people has experience of “living with foreigners”.



I must say there is no easy way out of this difficult situation.  Although tax increase from 8% to 10% is needed to decrease the fiscal deficit of Japan, it is very difficult to keep the best timing for the Japanese government to introduce it.  Japan has only limited time to make its fiscal balance to be sustainable.

My memory of bubble economy in Japan

DataHero Nikkei 225 (Max in the year)

Stock markets in the US are very active recently.  The Dow industrial average went beyond 17,000 and hit the record high last week.  Stock market in Kuala Lumpur is also active, FTSE Bursa Malaysia KLCI Index was about to reach the highest last week. It reminds me the bubble economy of Japan in the end of 1980s. It was definitely enthusiasm at that time. Stocks would be going up tomorrow because it went up today. This is a story in Japan,1989.


1. Situations at the end of 1989 in Japan.

Nikkei 225 hit a historical all-time high of 38,957.44 in December 29, 1989.  When I saw the screen board,  I wondered why it happened because I did not find the reason for Nikkei 225 to reach the highest.  In 1989, the stock market went up from approximately 30,000  to nearly 40,000. But I did not see any big news to justify that in 1989.   Without a economic mechanism changed,  I thought that just the only price of stocks had been going up.  It was strange, definitely.  At the beginning of year,  a lot of economists and think tanks predicted movement of the stock markets in the year,  I remembered one of economic research centers said “Nikkei 225 will reach 100,000 in the near future.” at the beginning of 1990.  However, current Nikkei 225 is around 15,000, as you know. What’s the difference it is!


2.  How it happened.

In 1989, I worked at one of the four biggest Japanese investment banks as a stockbroker.  My main clients were retail investors, such as owners of small companies,  doctors,  professionals and housewives!  There were a lot of economic research to analyze Japanese economy and  justify this crazy appreciation of the stock market in Japan. But the reason why it appreciated was very simple.   The stock market went up because everyone bought it.  I talked a lot of retail investors everyday.  Beginners are usually quite conservative at first,  however once they realized every stock was going up and most people around them bought stocks this year, beginners also start investing aggressively although they had little experience.  It is a little difficult to know how it works because it is not a normal situation.  Can you imagine every housewife starts investing stocks today?   In 1989, they did.   I heard this phrase many times in 1989,  ” He does not invest stocks as he is foolish”.


3.  What happened after bubble burst?

It was miserable after the bubble burst as you know.  Most investors lost their money and only debts remained.  A lot of companies, including listed companies went out of the markets.  The most important thing,  I think, is that Japanese people lost confidence to go forward with taking risks.  Emotionally, this impact was strong enough to discourage the economic growth.  Everyone in Japan felt to get richer and richer until the bubble burst, but suddenly realized it was a bubble and their wealth disappeared from sight.  This is the beginning of a lost decade in Japan.


It is said that human’s memory is short-lived, so bubble will appear again every 20-30 years.  I would like to monitor when it happens again.