Sunday, 26 August 2012

Kaggle Prospect - Harvard Business Review

This post is meant for submitting visual analysis for the Harvard Business Review Contest on Kaggle

I used the subject lines for all the articles and all the years and mapped the articles into one of the following 18 categories


  1.  Business Ethics
  2.  Business Management
  3.  Crisis
  4.  Emerging Markets
  5.  Financial Performance
  6.  Health Care
  7.  Information Technology
  8.  Labor
  9.  Leadership
  10.  Management Systems
  11.  Marketing Strategy
  12.  Regulation
  13.  Social Media
  14.  Stock Market
  15.  Strategic Planning
  16.  Supply Chain
  17.  United States & World
  18.  Women & Management


Changes in popularity of these topics were visualized using the googleVis package for R. This visualization is available here (I could not figure out how to upload it on Kaggle).

Observations:


  1. The average number of pages per article has gone down steadily since the 1950s falling to below 5 pages per article in for the first time in 1981 and then staying pretty much below that mark. Could this partly be attributed to the internet revolution?
  2. Recent trending topics are related to Emerging Markets & China, and Social Media.
  3. Some evergreen topics in HBR include - Business Management, Employees/Workforce, Labor, Marketing Strategy, and Strategic Planning.
  4. The lengthiest articles are on issues concerning United States & the World, Regulation and Management Systems



Wednesday, 25 July 2012

Measuring persistence in a time series : Application of rolling window regression


During my final semester at IGIDR I did a project paper in macroeconomics involving timeseries econometrics. The concept that I focused on my study was unit root, which I have touched upon in my earlier posts. This study presents a novel applicative aspect of unit root test called persistence. We investigate the level of persistence exhibited by the inflation rate series in India and also see how this level has changed over time. 

Background and definition:

Persistence level is an important dynamic property of any timeseries that gives us an overview understanding of the series in question. Persistence, in English language, is defined as "continuance of an effect after the cause is removed". This pretty much captures the econometric definition too. If a series is given an external shock, the level of persistence would give us an idea as to what the impact of that shock will be on that series, will it soon revert to its mean path or will it be further pushed away from the mean path. In case of a highly persistence series, a shock to the series tends to persist for long and the series drifts away from its 
historical mean path. Opposite is case of a series with low level of persistence, post a shock to the series it has a tendency to get back to its historical mean path. 


Inflation is measured as percentage rise in the price index or, informally speaking, a general rise in the prices of all goods and services in the economy. Its important to note that rise in prices of just few commodities could be due to market conditions in that particular sector and might not cause a general rise in the price index. For a non-econ student, inflation could be thought of as the rate at which wealth is losing its value. If the inflation rate in your economy is 10% year on year (Y-o-Y) it means that what you can buy for 100 INR today would be worth 110 INR in the next year. There are a plenty of reasons why keeping the inflation level in check is an important proposition for any economy's policy makers but I shall not elaborate on that in this post.


Why is measuring the level of persistence in inflation series important? Well, a simple commonsense reason that one can think of is that the level of persistence would play a monumental role in the RBI's decision of tackling inflation. If the inflation series is highly persistent then a shock to the inflation series would have to be dealt with in a much more stringent manner as the shock might tend to last for a really long time with detrimental impacts. Those familiar with macroeconomics would be able to relate this to the concept of Taylor's rule that governs most of the central bankers ideologies about weighted importance of inflation and growth. This is of course one of the many reasons why it might be important, there could be others you could think of.

In a timeseries econometricians world there is a formal mathematical(or rather empirical) definition of persistence. The definition of persistence is intimately related to the concept of unit root that I have discussed in my earlier posts. I would assume some prior knowledge of timeseries going forward readers are requested to improve upon wherever I go overboard. Our predefined augmented dickey fuller(ADF) test uses the ADF test regression and compute the coefficient on z(t-1) and tests whether it is statistically different than 1. Now empirically, a series that has a unit root is supposed to be highly persistent. What we intent to do is to do a rolling regression and compute the persistence coefficient for each regression and plot the persistence values over time along with the 95% confidence interval band.(For a detailed explanation of the maths behind ADF test refer to Dr. Krishnan's notes here, refer to pg. 13 for the ADF test regression equation)


Methodology:

Let me try and explain the rolling window regression that I have used in my analysis here. Rolling window regression for a timeseries data is basically running multiple regression with different overlapping (or non-overlapping) window of values at a time. For example, if your dataset has values on a timeseries with 100 observations and you want to perform rolling regression, or for that matter any operation on a rolling window, the idea is to start with an initial window of say 40 values(1st to the 40th observation) perform the operation that you wish to and then roll the window with some values, lets say we roll the window by 5. Now, the second window of data would be the next 40 observations starting from the 5th observation (5th to the 45th observation). Similarly, the third window will be the next 40 values starting from the 10th value, and so on. The advantage of using this technique is basically to look at any changing property of a series over time. You will get an estimate of the property over time instead of one single constant measure for the entire period. 

I have used this above discussed idea to look at the persistence level of the inflation series over time. Using a  rolling window ADF test regression to compute the persistence parameter and plotting it over time along with the 95% confidence band. I would confess here that the codes that I have used are not the best that one can work with. I would be grateful to reader who could suggest better way of going about this exercise otherwise I might be convinced of this "timeseries handling shortcoming" with R.

Codes:




Plots generated from the codes above:





Persistence level in CPI over time




Persistence level in WPI over time


Key observations:

We have some interesting observations when we look at the plots. The persistence level is generally on a  lower level when we look at the consumer price index(CPI) series, over time we see that the level of persistence has been constant. The story remains same for the wholesale price index(WPI) series. We see that the level of persistence is on the lower level where even the 95% confidence band is close to 0. There have been recent arguments about how sticky inflation rate series is for India, but the empirical investigation above does not confirm with that hypothesis. There also are arguments towards central banks faulty measure of WPI targeting. Currently, the RBI looks at the WPI series for keeping the inflation in check, but when we have a divergence in the dynamic property of the CPI and WPI inflation series it becomes difficult for the policy makers to decide to what series to target. There are research papers that throw some light on this recent divergence in the different measures of inflation in India and what importance it holds for policy makers. However, persistence seems to be a property that exhibits somewhat a similar feature across the 2 series. 


The results that we obtain above are consistent with this working paper at the RBI(the Indian central bank) which illustrates  low persistence level across various measure of inflation. We have essentially replicated this methodology using R codes and updated data till present to affirm the argument in this paper.

Data used: 


If you wish to replicate the exercise above the data can be obtained form here, CPI data, WPI data. The files contain 2 columns one with the raw data and the other with seasonally adjusted data. I have used the seasonally adjusted data for the analysis here. The seasonal adjustment was done using the X-12 ARIMA filter in EVIEWS. For more about seasonally adjusted and unadjusted data refer to my previous post here.


Readers critiques/feedback are welcome.

Friday, 1 June 2012

Term structure of interest rate spread volatility : Unit root test

Recently, I was working on my master's thesis and came across an interesting observation regarding the term structure of interest rate spread volatility that I wish to share. Let me first try and throw some light on the jargon that I have used. To begin with, term structure of interest rates is basically a curve plotting the market expectation of nominal interest for different maturities. How do we get this term structure? Well, the trading of bonds (i.e fixed income securities) happen in the secondary market for debt instruments. Since, bonds have fixed coupon payments at fixed period of time, we can actually look at combinations of discount rates for these different (but fixed) time periods and discount these future coupon payments by their respective discount rates to arrive at the current price, i.e the one discovered in the market (simple application of the present value principal). Now the beauty is that we have a number of securities trading in the debt market, each with different maturity and coupon payment. So the discovered price of all these securities can be used to fit a function to the rate at which different coupon payments have been discounted, this results in what we called the term structure of interest rates or sometimes referred to as zero coupon yield curve (zcyc). The most popular functional form adopted by applied researchers, and also the one adopted by the Clearing Corporation of India ltd (CCIL) is the Nelson-Siegel-Svensson (NSS) equation. But I shall not delve into that in detail. My observation pertains to the dynamics of the volatility of the term structure of interest rate spread in India.

Let me start by plotting the daily interest rates from the term structure starting February 2003.

Daily term structure rates
Next we simply plot the monthly standard deviation of the interest rates by at different maturities.



Things are straightforward till now but when I plot the standard deviation of the interest rate spread things get a little strange. Well, to start with we know that spread is already a first differenced variable, in the sense that it is basically the difference between interest rate at 2 different maturities. So, apriori we would not expect it to be a non-stationary series. Let us examine how the plot of the interest rate spread volatility looks like. For simplicity and sake of exposition we plot only the (3-1) month spread volatility.

(3-1) month interest rate spread volatility

Now this series turns out to be non-stationary when passed through the adf.test(). This counter-intuitive observation was difficult for me to digest so I delved into the problem to figure out what could possibly be going wrong. To start with my sample size was not very large (~90 obs) which could lead to the wrong rejection of the null of stationarity. Secondly, there is a structural change to be suspected in the series approximately near September 09. Now keeping this in mind when we ran the adf.test() for the sample before Sept'09 and after Sept'09, surprisingly,  we find that both the samples are non-stationary. Looking at the plot one would not suspect such a result. From a casual visual inspection it seems that the series is stationary before Sept'09 but could be non-stationary post Sept'09. So to solve this puzzle, we break the sample into pre-Sept'09 and post Sept'09 and do some simple visualizations.

(3-1) month Interest rate spread volatility for the entire sample 
Now if we look at the adf.test() results the high p-value suggests that the series, both pre and post the break, is no where near to being an I(0) series.


Interest rate spread volatility for the split sample
When we visualize the series pre and post break the intuition becomes clear as the movement of the series appear to be quite random and the constant mean property of a stationary series seems to be violated (Refer to this previous post for a similar discussion). The reason why a split sample visualization made more sense here was because the movements post Sept'09 were so large that the actual dynamics (which are actually quite random) were not visible in the same plot, thus giving the false impression of stationarity in the pre break period. However, this simple split sample visualization makes the results seem quite intuitive.

Thursday, 12 April 2012

M-O-M vs Y-O-Y inflation rates in India

Some time back I came across this article, by one of our alumni who is currently at NIPFP, which presents some caveats and shortcomings of different measure of inflation in India in a concise manner. Since my master's thesis topic revolves around inflation; I was in a better position to appreciate the author's insightful critique. It might have been the space constrain that held the author from graphically illustrating his point in his article, I'll try and substantiate his arguments using some graphs and R.

There is a recent concern about the rising levels of inflation in India; refer to this post for some numbers that support this claim. However, in this post I wish to illustrate the crucial differences in M-O-M vs Y-O-Y measure for inflation using elementary visualizations in R. From a researchers point of view the choice of variables is something that has to be meticulously addressed and dealt with; especially if one wishes to tread the ambitious path of "inflation forecasting". Before any theoretical mumbo-jumbo one needs to decide whether to work with M-O-M or Y-O-Y rates, or the kind of seasonal adjustment that is required for the data.

Some basic definitions

Inflation rate : Is the percentage change in the consumer price index(CPI).

M-O-M or Month on month inflation rate: Is defined as the monthly percentage change in the CPI series. For eg. January 2010 inflation rate will be the %age change in CPI from December 2009.

Y-O-Y or Year on year inflation rates : Is defined as the yearly percentage change in the CPI series. For eg. January 2010 inflation rate will be the %age change in the CPI from January 2009.

Seasonally adjusted CPI: Any macroeconomic time series, which has a frequency of less than an year, has to be de-seasonalized before being fed into any model. This is basically done to eliminate the seasonal/cyclical effects that might influence the analysis.

The monthly CPI-IW(industrial worker) series was obtained from the RBI handbook of statistics and the data spans from1990M4 : 2011M6. The seasonal adjustment has been done using the X-12 ARIMA filter in EVIEWS.(I am not very proud of this confession, but there were some issues with the "x12" package in R that I could not resolve). 

The below figure shows the Y-O-Y and M-O-M measure of inflation using the seasonally adjusted and unadjusted CPI-IW series.






The difference in the seasonally adjusted and unadjusted series is not evident from the above plots. However, the difference between the M-O-M and Y-O-Y measure for inflation is surely visible by eye balling the plots. To illustrate the difference between the seasonally adjusted and unadjusted measures we plot the same measures with and without the adjustment.





The difference in the two series is not evident in the Y-O-Y rates but we can clearly see that the in M-O-M rates the dynamics gets substantially altered after the seasonal adjustment. This is fairly intuitive since computing the Y-O-Y rate is analogous to passing the series through a (1-B^12) filter(where "B" is the lag operator) which discounts the annual seasonal effect.

Another interesting observation that I made about the 2 series is that Y-O-Y rate is not I(0)(at 5% significance), i.e it has a unit root. The M-O-M rate, on the other hand, comes out clean after being fed into the adf.test() command. Now, I(0)ness is a wonderful property which brings a wry smile to every time series econometricians' face when the p-values drops below 0.01 in the adf.test(). But this apparent conundrum, where the Y-O-Y and M-O-M yield different order of integration, poses a dilemma for the researcher.

The M-O-M inflation captures the short run inflation dynamics, which has less variance and has high forecast ability, that might be helpful in portfolio adjustment for active financial investors. The Y-O-Y rates capture the long run dynamics of inflation, which has high variance and low forecast ability, that might be helpful for the central banker to track.

Here are the codes that I used for the above exercise:


Friday, 23 March 2012

Project Euler: Problem 20



n! means n x (n - 1) x ... x 3 x 2 x 1
For example, 10! = 10 x 9 x ... x 3 x 2 x 1 = 3628800,
and the sum of the digits in the number 10! is 3 + 6 + 2 + 8 + 8 + 0 + 0 = 27.
Find the sum of the digits in the number 100!

The approach to this problem is quite similar to problem 16 which we can see here. We just need to deal with a really large number here. So, GMP again to the rescue.

library(gmp)

x <- factorialZ(100)

sum(as.numeric(unlist(strsplit(as.character(x), split=""))))
# as.character(x) coverts the big number into a string
# strsplit() extracts individual elements of the string. This can be ensured by using the split = "" argument, which splits the string one element at a time
# strsplit() returns a list. To access the elements of this list, use unlist()
# convert the individual characters to numeric using as.numeric()
# sum the elements

Ans: 648

Thursday, 15 March 2012

Project Euler: Problem 16


215 = 32768 and the sum of its digits is 3 + 2 + 7 + 6 + 8 = 26.
What is the sum of the digits of the number 21000?

Handling large numbers or rather, very large numbers, can be a pain at times. But have no fear, for GMP is here.

GMP  makes the solution quite simple.

library(gmp)

x <- as.bigz(2^1000)
# as.bigz() is a function in the gmp library that helps dealing with large numbers quite easily 

This is the approach we take
  1. Convert the number to a string so that we can extract the individual elements of the number
  2. Extract the individual elements or digits
  3. Create a vector of these digits
  4. Sum the vector
  5. Rejoice

sum(as.numeric(unlist(strsplit(as.character(x), split=""))))
# as.character(x) coverts the big number into a string
# strsplit() extracts individual elements of the string. This can be ensured by using the split = "" argument, which splits the string one element at a time
# strsplit() returns a list. To access the elements of this list, use unlist()
# convert the individual characters to numeric using as.numeric()
# sum the elements

Ans: 1366

Saturday, 10 March 2012

"Fear of floating exchange rate" or "fear of losing international reserves".

We were recently required to do an assignment for the International Finance course where we had to investigate the policy that the emerging economies adopt towards holding international reserves. A recent research paper at the NBER by Joshua Aizenman and Yi Sun demonstrated that many emerging economies are averse to the idea of losing international reserves in times to crisis in order to maintain their exchange rates (against $). They have termed this phenomenon as "fear of losing international reserves". Let me test my understand on this subject (which is limited) and try illustrating in simple language what we did in our assignment. Readers familiar with international economics might relate to it easily, but I would try to make it decipherable for the novice economists too.

Foreign exchange reserves and Foreign exchange market:
In the globalized world that we live, trade relation between countries have become very strong and the quantum of trade has boomed by a considerable amount in the past few decades. In the historical gold-era the trading countries made the respective payments for imports/exports in gold, but with the increasing volume of trade and the Bretton Woods system, countries moved to US $ as the major trading and reserve currency. To put it simply, post the 1960's when the $ became the official reserve currency countries started trading with $ as the universal currency and also started keeping reserves mostly in the form of US currency.

International reserves are nothing but foreign currency deposits and bonds that are held by the central bank or the monetary authority of the country. Its important for countries as they use these reserves to pay for their import. Countries accumulate reserves when they export goods (as it results in inflows of foreign currency) and use up their reserves to pay for imports. So we see why it is important for countries to maintain their reserve levels. But interestingly this is not the only function that the reserves perform. Since we have a foreign exchange market where different currencies are traded, these reserves are also used by the countries' central bank in order to avoid excess volatility in exchange rates. For eg. if the demand for a currency increases its value appreciates and if its demand falls its value depreciates, both these movements are not very desirable if in large magnitude ("fear of floating exchange rate"). So in case of these excess movements the central bank can kick in with a huge buy/sell order and stabilize this movement in the currency exchange rate. This kind of an exchange rate policy is called a "managed float" since the exchange rate is not completely market determined but instead is managed in a band by the central bank.

"Fear of losing reserve":
The recent global financial crisis of 2008 provided some evidence that suggested that the countries are now less worried about their exchange rate volatility and are more concerned about the foreign reserve levels. This is precisely what Joshua Aizenman and Yi Sun investigated in their research paper for 21 emerging economies and found that many economies refrained from using up their reserves during the 2008 crisis. For the ones that did, the loss was not substantial and even for the ones that lost their reserves by a considerable amount they quickly regained it back. This phenomenon was termed as "fear of losing reserves". More than its currency depreciation/appreciation a country might be bothered about the signalling effect that a depleting level of reserves might create. Losing reserves could increase the countries' probability of defaulting on foreign debt which might hamper its trading relation with other nations, or it could lead to a balance of payment crisis. Some prominent figures in the academics field, for instance Dr Ajay Shah, argue that given the large trading volume in the foreign exchange market, it has become a highly liquid market, and as a result the magnitude of orders needed to move the currency rates in your favor are tremendously large. We can see this as one possible reason for emerging economies not to intervene too frequently in the FX markets in times of crisis, but more research is needed to establish whether these explanations hold water.

Evidence from 4 South Asian and 2 Non-oil exporting Middle East economies:
For our assignment, we had to check whether a similar trend in foreign exchange reserve is seen during the Euro-zone debt crisis that started in mid-2010. The countries that we used for our study were India, Pakistan, Sri Lanka, Bangladesh, Jordan and Egypt. We simply plotted a few graphs and visually commented on the trends that we saw. First we plotted the exchange rates of the respective countries' currencies to see if there was some excess volatility in the rates during the time of crisis and subsequently we checked if this volatility was reflected in the foreign reserves levels.

Exchange rates w.r.t US $ of South Asian and middle east emerging economies over time
In order to make these graphs comparable we plotted the [exchange rates/max(exchange rates)] which normalized the plots to values between 0 and 1. The time series has been plotted with the starting month as January 2002 and end month as September 2011. The fi rst vertical line (at Months = 79) indicates the global financial crisis of 2008. Similarly the second dotted vertical line (at Months = 102) indicates the advent of the Euro-zone crisis.


Apart from Jordan, which has its exchange rates pegged to the US $, we see that the exchange rates have been fairly stable since 2002, and that the volatility has increased for most of them in the 2008 crisis. This volatility has been very drastic for India, Pakistan and Sri Lanka in particular. Let us now see whether their trend of international reserves corroborate to this excess volatility in exchange rates.

The below graph shows how these emerging economies have accumulated reserves over the past decade.


Foreign reserves of South Asian and middle east emerging economies over time


The codes to replicate the graphs are fairly simple:


You can do a similar thing with the exchange rates.


The reserve accumulation for all the economies has seen a steady rise from 2002 to 2008 (till the  financial crisis) but post the crisis there has been a decline in the reserves for all these economies except Jordan, which has its exchange rate pegged to the US $. The decline has been marginal for Bangladesh and Egypt, but India, Sri Lanka and Pakistan lost a substantial proportion of their reserves in the 2008 crisis, which corroborates to our earlier fi nding of excess volatility in these countries. Despite this loss of international reserves, all the countries bounced back soon within 10 months time to an accelerating trend of reserve accumulation. This idea seems to be consistent with the hypothesis that countries were adamant not to lose their reserves at times of crisis, instead they chose to play safe by shedding some reserves but quickly regaining them back. The depletion of reserves is viewed by the countries as a sign of vulnerability, and hence they refrain from depleting them in a rush.


When we look at the trend of international reserves post the advent of 2010 Euro-zone crisis the trend seems to be on a rising for most of the countries and stable for some. There has been no depletion of reserves per se, except in Egypt. One reason that can be attributed to this is that the volatility of these countries respective exchange rates w.r.t US $ has not be comparable to the 2008 levels. Srinivasan et al. (2009) have argued in the context of Indian central bank that there is a zone or band of exchange rate in which the central bank does not intervene and its only when the rate exceeds the band does the CB intervene in the forex market. Using a similar argument we can say that since the movement of exchange rates has been within that zone of "no-action" for most of these countries post the Euro-zone crisis the CBs have refrained from intervening in the forex market.

Apart from the above visualization we also plotted these countries trading relation (import and export) with the European union and found a steady decline. (I am not posting those graphs here for sake of precision but the ones interested can view the entire document from here.)

Conclusion:
In this study we find evidence of "fear of depleting reserves" for most of the above discussed economies in the context of the 2008 global financial crisis which is consistent with the Joshua Aizenman & Yi Sun (2009) findings. However, the same cannot be concluded for the Euro-zone debt crisis yet. Since the crisis is still unfolding and its full repercussions is unclear, we will have to wait for the crisis to unfold to see what are the adverse e ects it has on the emerging economies. The evidences so far suggest that the trading relation of these emerging economies with the EU nations has declined post the 2010 crisis, but the international reserve position has not seen a substantial decline yet.

P.S: The data for the above exercise has been extracted from the IFS database. Feedback are welcome.