post list
QuantDare
categories
artificial intelligence

What is the difference between Artificial Intelligence and Machine Learning?

ogonzalez

artificial intelligence

Random forest: many is better than one

xristica

artificial intelligence

Non-parametric Estimation

T. Fuertes

artificial intelligence

Classification trees in MATLAB

xristica

artificial intelligence

Applying Genetic Algorithms to define a Trading System

aparra

artificial intelligence

Graph theory: connections in the market

T. Fuertes

artificial intelligence

Data Cleansing & Data Transformation

psanchezcri

artificial intelligence

Learning with kernels: an introductory approach

ogonzalez

artificial intelligence

SVM versus a monkey. Make your bets.

P. López

artificial intelligence

Clustering: “Two’s company, three’s a crowd”

libesa

artificial intelligence

Euro Stoxx Strategy with Machine Learning

fjrodriguez2

artificial intelligence

Visualizing Fixed Income ETFs with T-SNE

j3

artificial intelligence

Hierarchical clustering, using it to invest

T. Fuertes

artificial intelligence

Markov Switching Regimes say… bear or bullish?

mplanaslasa

artificial intelligence

“K-Means never fails”, they said…

fjrodriguez2

artificial intelligence

What is the difference between Bagging and Boosting?

xristica

artificial intelligence

Outliers: Looking For A Needle In A Haystack

T. Fuertes

artificial intelligence

Machine Learning: A Brief Breakdown

libesa

artificial intelligence

Stock classification with ISOMAP

j3

artificial intelligence

Returns clustering with k-Means algorithm

psanchezcri

artificial intelligence

Confusion matrix & MCC statistic

mplanaslasa

artificial intelligence

Reproducing the S&P500 by clustering

fuzzyperson

artificial intelligence

Random forest vs Simple tree

xristica

artificial intelligence

Clasificando el mercado mediante árboles de decisión

xristica

artificial intelligence

Árboles de clasificación en Matlab

xristica

artificial intelligence

Redes Neuronales II

alarije

artificial intelligence

Análisis de Componentes Principales

j3

artificial intelligence

Vecinos cercanos en una serie temporal

xristica

artificial intelligence

Redes Neuronales

alarije

artificial intelligence

Caso Práctico: Multidimensional Scaling

rcobo

artificial intelligence

Sir Bayes: all but not naïve!

mplanaslasa

21/10/2015

7
Sir Bayes: all but not naïve!

Is it possible to classify and predict (yes, predict!) if market trends will be bullish, bear or ranged by using a method called “naïve” and based on something as simple as Bayes’ theorem? Let’s see!

Our main objective

Here, we’re looking to explore machine learning techniques that can help us not only to label series in a posteriori analysis, but also to predict to which class a new value given of the serie belongs.

The Naïve Bayesian Classifier is a supervised learning method  of machine learning, as well as a statistical method for classification.

Although this method includes the word “naïve” in its name, it will be our chosen tool to predict different trends of an index-represented market. Bayesian classification provides practical learning algorithms where prior knowledge and observed data can be combined. It calculates explicit probabilities for hypothesis, and theory maintains that it’s robust to noise in input data.

An extremely brief lesson of history of statistics…

This classification is named after Thomas Bayes (1702-1761), who proposed the famous Bayes Theorem, which constitutes the guts of the issue concerned:

bayes_theorem

This theorem is only telling us how we can transform the conditional probability of two events, assuming that the conditional event, B, has probability of occurrence not null. This means P(B)!=0.

We can trivially prove that if {Cj} are a set of n exclusive and exhaustive events, this formula becomes:

bayes_formula_derivada

From now on,  C will be referred to as the class of a variable (this is what we want to predict) and A will represent the different attributes or characteristics measured that can be known every day, unlike variable C.

We want to maximize the probability P(Ci|A) for each class i. Therefore, the Bayesian Naïve Classifier solves the next problem, which can be derived from the previous formula (take into account that the divisor of the formula is common for each class):

argmax

where Ci is the class (i denotes the possible label it can show) and Aj is the j-attribute.

But why “naïve“? This name comes from the fact that we are assuming (to get the previous formula) that:

why_naive

Keep in mind the basic idea:

Find out the probability of the previously unseen instance belonging to each class, then simply pick the most probable class.

cartapacios

The key to all the work behind this post

We are going to classify and predict classes of trends (Upward, Downward and Ranged Market, which could be the possible values of our variable C in the above formula) by using historical data mixing how the real market was in the past (variable C) and what some models (attributes) were indicating to us. They could be betting for bullish market, down market, or they could be positioning out of the market.

More details are given to understand our application of this classifier:

  • Attributes Aj : The attributes will be the models mentioned before. But which models and how many? We have developed a program built with 30 models based on different philosophies, which quantify different aspects and leads us to a position (-1, 0 or 1) in the market.  These models are assumed to be independent. Therefore, we will have 30 different positions each day.
  • Variable C: This variable has to be estimated carefully. We want a measure to predict trends, because today we don’t know which trend we are living. We’re going to make predictions by using a window of historical data. The “real values” of variable C (Upward, Downward, Ranged Market) in the window have been collected using an algorithm which works with future information, so every day, when we estimate probabilities, we have to delete “advanced” (unknown at the time) information. This means that every day, we have a dynamic historical window, the length of which depends on the information we have to throw out in order to make fair estimations.
  • Which probabilities do we have to estimate? If today we have a new 30-tuple of positions of our attributes (models) and the tuple shows k different values (k<=3 because possible values are -1, 0 or 1),  we have to estimate (every day) the next probabilities for each class i (i=Upward, Downward or Ranged Market):

elementos_argmax

These probabilities are estimated by simply counting correspondent frequencies in the historic window.

ultima_formula

  • And, finally… for each class we calculate, the class that shows the maximum value will constitute our bet. In other words, our  prediction.

We have applied this to guess the trends of the stock Google Inc from October of 2006, and the results look like this, (much better than we expected!):

trends_sin_adelanto_97431

Taking a quick look, we notice that some downward trends have been predicted as upward, but although we are not magicians, the predictions obtained should not be disregarded. We cannot throw away the idea of using this method yet, because the coincidence of the prediction based on future information is around 51% value, which is not an insignificant result since this application does not cheat, as the other method does.

Tweet about this on TwitterShare on LinkedInShare on FacebookShare on Google+Email this to someone

add a comment

Congrats! We really think that Attributes Aj are the key to not falling into a curse GIGO

Thanks for your comment and for showing interest.

I have classified the price movement using an own algorithm based on the serie seen a posteriori as I have indicated in the post. The key is to use this tool without taking into account the part of information that could be unknown at this moment.

[…] Sir Bayes: all but not na ve! [Quant Dare] Is it possible to classify and predict (yes, predict!) if market trends will be bullish, bear or ranged by using a method called nave and based on something as simple as Bayes theorem is? Lets see! Predicting trends with nave Bayesian classifier Our main objective is to explore techniques of machine learning that can help us not only to label series in a posteriori analysis, but […]

Thanks for the post. Very informative to someone who has never dealt with Machine learning.

Could you please share your thoughts on how you classified the price movement into Upward, Downward and Ranged Market?

wpDiscuz