Thursday, 25 May 2017

Residual Plot in Regression, ACF, PACF in ARIMA




What is Constant Mean for a time series- If we draw mean value line on time series, half of the points are above live and half of the points are below line, so we can say that mean is constant over the period of time for this series.



What is the important of residual plot in modelling linear relationshipThe first plot shows a random pattern, indicating a good fit for a linear model. The other plot patterns are non-random (U-shaped and inverted U), suggesting a better fit for a non-linear model.

What is constant variance of a time-series-  
  1.       The mean E(xt) is the same for all t.
  2.       The variance of xt is the same for all t.

      In other words, Mean of series x_t and x_t-h is same.
      Standard deviation of series x_t is same as standard deviation of series x_t-h.
An interesting property of a stationary series is that theoretically it has the same structure forwards as it does backwards.

what is ACF for a time series-  The correlation between xt to all xt-h is ACF. If there is .3 correlation between xt anf xt-1 then by simple multiplicative rule, there will be .3*.3 correlation between xt and xt-2 and .3*.3*.3 between xt and xt-3.


in the above image correlation with lag 1 is .6, with lag 2 is .36 and so on. Based on above graph we can say that it is AR(1) process where y=.6*yt-1 +constant +error.


What is Moving Average and how is it related to PACF -  MA is time series of past errors(multiplied by some constant). PACF is more difficult to understand. It is a conditional correlation between variables( series).It is correlation between 2 variables conditioning that the correlation is coming from some other variables.

If we have regression y= x1 and x2 so PACF between y and x2 will be-
covariance(y,x2/x1)/sd(y/x1)*sd(x2/x1)

For time series, PACF between yt and yt-2 is given by-
covariance (yt, yt-2/yt-1)/sd(yt,yt-1)* sd(yt-2.yt-1)

ACF is used to identify order of MA and PACF is used to identify order of AR terms in stationary time series.

know more about the relation between time-series and regression-
Regression and time series





Tuesday, 23 May 2017

Before ABC of Deep Learning.



Mark Cuban has rightly said above lines for deep learning.First of all, when was it coined first time? Deep Learning is used by Google in its voice and image recognition algorithms, Netflix and Amazon say that they have used in their recommendation engine, researchers at MIT say they are relying more on deep learning now.



According to Jack Rae; Google DeepMind Research Engineer, Deep learning refers to artificial neural networks that are composed of many layers. Essentially Deep Learning involves feeding a computer system a lot of data, which it can use to make decisions about other data. This data is fed through neural networks, as is the case in machine learning. These networks – logical constructions which ask a series of binary true/false questions, or extract a numerical value, of every bit of data which pass through them, and classify it according to the answers received. If we define in language of data scientists, It uses a cascade of many layers of nonlinear processing units for feature extraction and transformation. It is based on the (unsupervised) learning of multiple levels of features or representations of the data and learns multiple levels of representations that correspond to different levels of abstraction. 

According to Jack Rae; Google DeepMind Research Engineer, Deep learning refers to artificial neural networks that are composed of many layers. Essentially Deep Learning involves feeding a computer system a lot of data, which it can use to make decisions about other data. This data is fed through neural networks, as is the case in machine learning. These networks – logical constructions which ask a series of binary true/false questions, or extract a numerical value, of every bit of data which pass through them, and classify it according to the answers received. If we define in language of data scientists, It uses a cascade of many layers of nonlinear processing units for feature extraction and transformation. It is based on the (unsupervised) learning of multiple levels of features or representations of the data and learns multiple levels of representations that correspond to different levels of abstraction. 


Deep learning methods are often looked at as a black box, with most confirmations done empirically, rather than theoretically.

Deep learning is used across all industries for a number of different tasks. Commercial apps that use image recognition, open source platforms with consumer recommendation apps, navigation of self-driving cars, re-colouring black and white images, automated analysis and reporting and medical research tools that explore the possibility of reusing drugs for new ailments are the examples of deep learning.

There are many available software libraries for deep learning-

Deeplearning4j - Written in Java and algorithms can be integrated with Hadoop, Spark. It was developed mainly by a machine learning group in San Francisco led by Adam Gibson.

Torch - Open Source and written in Lua language. It is presently used by Facebook, IBM and Yandex.

Theano- Open source library for python and developed by Université de Montréal.

TensorFlow - Developed by Google Brain team and used by google in their products. It is Google's second generation machine learning system.

PaddlePaddle - Baidu's deep learning platform.

Keras- It is an open source neural network library written in Python. It is capable of running on top of Deeplearning4jTensorflow or Theano.

CNTK - It is deep learning framework developed by Microsoft Research. Also know as Microsoft's Cognitive tool-kit.


Another article on advance- machine learning by Russian Andrey Markov-

http://machinelearningstories.blogspot.in/2017/02/hidden-markov-model-session-1.html

start you first feed-forward newral network from here-




Monday, 15 May 2017

Facebook's Prophet Model for forecasting

Forecasting is central to data science activities.Facebook's open source forecasting tool 'PROPHET' is available in R and Python. This has been very useful in Web-sites' page view forecasting, road traffic forecasting and in the areas where there is multiple level of seasonality. Prophet is useful in below scenarios-


  • Seasonal data ( Hourly, daily,weekly, monthly).
  • Data having outliers.
  • Data with holidays' information.
  • Having multiple trend change points.

Prophet works on additive regressive model with four component.

  1. A piece-wise linear or logistic growth curve trend. Prophet automatically detects changes in trends by selecting change points from the data.
  2. A yearly seasonal component modelled using Fourier series.
  3. A weekly seasonal component using dummy variables.
  4. A user-provided list of important holidays.
Here is the example of using prophet model in R.
we take model_data dataset. It must have ds column which is to have dates and other column 'y' which is the values to forecast ( Uni-variate time series). This is how data look like -


running the model - 
mod <- prophet::prophet(model_data, weekly.seasonality =F)
There is no weekly seasonality so one of the argument is false.

Preparing op data-set to store result -
future <- make_future_dataframe(mod, periods = 4, freq = 'month')
we want to predict for next 4 data points and on monthly basis. This can be day, week etc.

predicting future values -
forecast <- predict(mod, future)

Visualising result-
forecast <- predict(mod, future)





Here are the forecasted values for the coming months- Yhat are output values with the range of yhat_lower and yhat_upper.


Different component of forecasts can also be plotted like this-
prophet_plot_components(m, forecast) 




Impact of confidence interval in forecasting-


m1 <- prophet::prophet(n_train, weekly.seasonality =F, interval.width = 0.99)
m2 <- prophet::prophet(n_train, weekly.seasonality =F, interval.width = 0.80)

I have run above 2 models to see impact of having different confidence interval in forecasting-
Here is the result of forecasting using above two-



mean values are same but the variation increases with increase in confidence interval.


This is how, we can use prophet model for time series forecasting!

Here is another time series technique. Which is more related to pure linear regression  and good to know-
Time-Series and Regression Relation


MBA is no longer Masters in business analytics. it's something else. ↙😋 If you want to know-
Market- Basket Analysis