Slippage and Root Mean Squared Error in Model performance

REAL traders all know how important slippage is.  You wanted to buy at 20.05 but got filled at 20.07.  Then you wanted to exit at 20.55 but got filled at 20.52.  The $0.02 on the went in and the $0.03 on the way out is what is referred to as slippage or skid. Total slippage $0.05

When building mix models I always incorporate slippage but where has become an increasingly important question in my work.  It’s similar to when people run a regression analysis on data that has no linear time factor built it.  You have no idea if Y preceded X or X preceded Y.  Running a linear regression doesn’t make this so.

Root Mean Squared Error or RMSE is one of many ways to compare regression models.  The formula is SQRT(mean((Observed_data- Predicted_Value)^2)).

When building models I often have a maximum profit from the trigger point (trade entry).  Back to our example. If I bought in at $20.07 and the highest print on the chart that day was 20.77. Max Profit would be $0.70.  My question to answer is typically what features will allow me to predict that $0.70 with the least amount of error & risk.

SLIPPAGE MATTERS! BIG TIME! I ran a Monte Carlo. 10,000 iterations. Running the same model where slippage was not subtracted from max Profit and when it was.  These were the results.

There are MANY interpretations of this.  I’ll offer a few.

  • RMSE is sensitive to large numbers creating the flaccid bi-nomial distribution in the bottom graph.  Large numbers potentially produced by small max profits and large spreads. IE CMG’s spread is about $0.20 where could be Max Profit = 0.05.
  • The top graph RMSE, max profit, is normally distributed in this model because individual stock differences for max profit are not enough to disrupt the distribution.  Which suggests that my ability to predict a stock is potentially easier.
  • SPREADS MATTER! The difference between the two graphs is caused by individual stock characteristics in the spread.  Like, two people, no spreads are ever really the same.  The error for my prediction goes up as we account for individual differences.

I think this illustrates something we experience every day in life. We have general predictions about what people will do in a given context (features) but everyone often does something unique which can not be accounted for.  I believe the bottom graph illustrates this.  It also illustrates why your real life trading profits may not reflect model performance if slippage is not included.  Because what happens. When I thought I was going to make $50.00 I only made $35.00…. Slippage!

 

Eastern Psychological Association 2010 Conference

Sayeed, Dr. Gorman and I had a successful presentation on March 5th at the Eastern Psychological Association 2010 Conference presenting “An Exploratory Qualitative Analysis of the 2008 Presidential Campaign.” You can read about my current research, including the work on Presidential Leadership on the research section of my website. Below is the short abstract along with Scribd version of our paper. In addition, all graphs are included for your viewing pleasure

An Exploratory Qualitative Analysis of the 2008 Presidential Campaign.
Short Abstract: A content analysis using Hart’s DICTON program was performed on the 2008 Obama vs. McCain presidential campaign speeches. It was found that the content of the speeches varied over time on the DICTION dimensions of certainty, activity, optimism, realism, and commonality. Obama consistently demonstrated higher levels of communality throughout the campaign. Implications for dynamic, time series content analyses are discussed.

Scribd link to our paper entitled: An Exploratory Qualitative Analysis of the 2008 Presidential Campaign.

Realism Scores – Graph
Activity Scores – Graph
Certainty Scores – Graph
Commonality Scores – Graph
Optimism Scores – Graph

Where are we taking this research next? We are exploring machine learning techniques to evaluate factors of leadership.

The New Blog @ AndrewIlardi.com

Hello!  I want to welcome you to my new blog.  Here I will post information regarding psychology, statistics, the stock market & their ever interchanging relation to one another.  You can visit my old blog here for older posts from Andrew Ilardi.  In addition, there are additional pages to peak into the other areas of my work.  At the top you’ll notice 5 tabs:

  • Home: The Main Blogging Area.
  • About: Information regarding my background.
  • Research: Current research I’m working on at the academic level.
  • Services: Information on what I provide and contact.
  • Capital Markets: Posts regarding daily research I do into the markets.