For those of us working with time series, the autocorrelation function (ACF) is a fundamental tool to understand how the values in a series correlate with others certain distance away.

Indeed, we could even say that autocorrelation plots (a.k.a correlogram) are probably the most common visualizations in econometrics and time series analysis. This is why functions to compute and plot the ACF are readily available in every software package somehow devoted to time series analysis: Matlab’s autocorr, R’s acf, Python’s plot_acf, SPSS’ acf, etc. It is worth noting that when we speak of “computing the ACF” what we really mean is “computing the sample ACF” or, in other words, computing an estimate of the population autocorrelation function from a sample of the actual process generating the time series. Therefore, as a consequence of it being a sample statistic, it is a random variable itself, and it has a probability distribution that we like to characterise. It does not make sense to say “this is the value of the autocorrelation function” without an assessment of the uncertainty behind that statement.

Good news is that all the software packages above have you covered. At least in principle…

To illustrate why consider the following toy example in which we generate (using Python) a length-100 sample of a synthetic moving average process of order 2 with Gaussian innovations. Then, we estimate the autocorrelation function for that sample. As expected, the largest estimates correspond to the first two taps and they are relatively close to their theoretical counterparts. The advantage of using synthetic data is that we can compute the deterministic autocorrelation of the moving average filter coefficients and use it as our ground truth value.

Note, also, that we have drawn the 95% confidence interval as provided by the library but… how exact is it? do we know the implicit assumptions behind that grayed area?

Comparison of theoretical autocorrelation and sample autocorrelation function with 95% confidence intervals based on Bartlett’s formula.

Well, in this case the confidence interval is calculated by means of the popular Bartlett’s formula and these are the underlying assumptions:

• It is a model-dependent asymptotic approximation. That is, given that the series is a Gaussian moving average model, it can be established that the difference between the sample ACF and the population ACF is increasingly normal as the sample size grows to infinity. Therefore, it is not exact for a small sample as ours.
• The variance of that normal distribution depends on the values of the population ACF. Once the variance is determined and the distribution is known it is straightforward to derive confidence intervals for any significance level. The center of the confidence interval depends again on the population ACF. As the population ACF is not available, it is replaced by the sample ACF. Recall that even though in this synthetic example we are lucky to know the population ACF, this is hardly ever the case.

In the case at hand, this may be useful as an approximation but it could also fail in many different ways. For example, if we didn’t have a moving average model, if we wanted to compute a different statistic for which an analytic formula is not available or it is difficult to evaluate, etc.

Bootstrapping

Bootstrapping is a well-known technique used to estimate the properties of an statistic. It was developed by Bradley Efron in 1979. The most common use cases include estimating variances and/or confidence intervals. Also, we have already seen how to apply it to portfolio management here on Quantdare. The technique is conceptually very simple: it relies on random sampling with replacement. The general idea is that by doing so we are effectively sampling from a distribution that matches the empirical distribution of the current sample, which can be seen as an approximation of sampling from the actual population distribution. It is a very simple and powerful approach that allows approximating the solution of problems that would be, otherwise, impossible or very tedious to solve.

According to this, we may be tempted to use Efron’s bootstrap but we will soon realise it fails miserably:

Comparison of theoretical autocorrelation and sample autocorrelation function with 95% confidence intervals obtained via IID bootstrapping.

As you can see, the returned confidence interval is the same for all lags, resembling what we would expect if our time series were white noise.
And that is exactly what we are seeing; by sampling randomly without constraints, we are destroying the time-dependence structure in the time series. This is the main limitation of the tradicional bootstrapping method and, to make it explicit, it is sometimes referred to as independent and identically distributed (IID) bootstrap.

Block bootstrapping

Nonetheless, there are ways around it, and the idea is, again, really simple: every single time you pick a value from the original sample, a set of adjacent samples must be also picked in order not to break temporal dependencies among series values. This subclass of bootstrap methods are known as block bootstrap. They are generally used when the data, or the errors in a model, are correlated. In this case, the conventional IID approach or residual resampling will fail, as it will not be able to replicate the correlation in the data. The block bootstrap tries to replicate the correlation by resampling blocks of data instead of individual values. It can be used with data correlated in time (i.e. time series) but can also be used with data correlated in space, or among groups.

Let’s see how even a simple approach to block bootstrapping (using length-5 blocks) can change the result:

Comparison of theoretical autocorrelation and sample autocorrelation function with 95% confidence intervals obtained via block bootstrapping.

Now the confidence interval estimation is much more stable and consistent with the input model.

Further ideas

This has been just one example of the kind of things that can be done with bootstrapping, however there are many more applications. Some of them include variance estimation, bias reduction, etc. Furthermore, there exist many other variants to the block bootstrapping concept such as circular block bootstrapping, stationary bootstrapping and related resampling techniques such as subsampling. If you are interesting in knowing more consider having a look at the arch Python package by Kevin Sheppard and the “Subsampling” book by Politis, Romano and Wolf.

Possibilities are virtually endless. I would love to hear about other use cases, so, feel free to drop a note below!