Long memory

• Long memory, or long-range dependence, in time series analysis deals with the notion that data may have a strong dependence on past values.

• In particular, the autocorrelation function of the data may decay slower than what ARMA models can capture.

• Long memory models are used in climate, finance, biology, economics, and many other fields.

• Today we will discuss the concept of long memory, why it may occur, and how to model it.

Long memory

Nile River annual flows is the classical example.

using Pkg; Pkg.activate(pwd())
using LongMemory
NileDataPlot()

Long memory

using Plots
autocorrelation_plot(NileData().NileMin, 50)
plot!(1:51, [0.4^i for i in 0:50],linewidth=3,linestyle=:dash)
title!("AR(1) with α=0.4")

Long memory

autocorrelation_plot(NileData().NileMin, 50)
plot!(1:51, [0.4^i for i in 0:50],linewidth=3,linestyle=:dash)
plot!(1:51, [0.8^i for i in 0:50],linewidth=3,linestyle=:dot)
title!("AR(1) with α=0.8")

Long memory

autocorrelation_plot(NileData().NileMin, 50)
plot!(1:51, [0.4^i for i in 0:50],linewidth=3,linestyle=:dash)
plot!(1:51, [0.8^i for i in 0:50],linewidth=3,linestyle=:dot)
plot!(1:51, [0.9^i for i in 0:50],linewidth=3,linestyle=:dashdot)
title!("AR(1) with α=0.9")

Long memory

Definition

We say that a time series \(x_t\) has long memory if: \[\begin{equation}\label{def:cov} \gamma_x(k) \approx C_x k^{2d-1}\quad \text{as}\quad k\to\infty, \end{equation}\] where \(\gamma_x(k)\) is the autocovariance function and \(C_x\in\mathbb{R}\).

Above, \(g(x)\approx h(x)\) as \(x\to x_0\) means that \(g(x)/h(x)\) converges to \(1\) as \(x\) tends to \(x_0\).

Long memory examples

• The Nile River annual flows.

• Stock volatilities; for example the (Chicago board options exchange) Volatility Index, or VIX.

• Temperature series.

• Network traffic.

Long memory origins

Reasons for long memory in time series include:

• The series is a result of cross-sectional aggregation, Granger (1980) and Haldrup and Vera‐Valdés (2017).

• The series is a result of shocks of stochastic duration, Parke (1999).

• The series was fractionally differenced, Granger and Joyeux (1980) and Hosking (1981).

Fractional differencing

• It is the most common approach to model long memory in time series.

• It is based on the idea of extending the concept of differencing to non-integer orders.

• Was first proposed by Granger and Joyeux (1980) and Hosking (1981).

Fractional differencing

Definition

The series \(x_t\) is said to be integrated of order \(d\), denoted \(I(d)\), if \[ (1-L)^dx_t = \varepsilon_t,\] where \(L\) is the lag operator, and \(\varepsilon_t\) is a white noise process with variance \(\sigma^2\).

Fractional differencing

• The fractional difference operator is decomposed using the binomial expansion:

\[(1-L)^d = \sum_{k=0}^{\infty} \begin{pmatrix} d \\ k \end{pmatrix} (-L)^k,\]

where \(\begin{pmatrix} d \\ k \end{pmatrix}\) are the (generalized) binomial coefficients.

Fractional differencing

• The series can be written as

\[x_t = \sum_{k=0}^{\infty} \pi_k \varepsilon_{t-k},\]

where \(\pi_k=\frac{(-1)^k\Gamma(1-d)}{\Gamma(k+1)\Gamma(1-k-d)}\), and \(\Gamma()\) is the gamma function.

Fractional differencing

• The autocovariance function, \(\gamma(k)\), is given by: \[\gamma(k) = \sigma^2\frac{(-1)^k\Gamma(1-2d)}{\Gamma(1+k-d)\Gamma(1-k-d)}.\]

Fractional differencing

autocorrelation_plot(NileData().NileMin, 50)
plot!(1:51, [0.4^i for i in 0:50],linewidth=3,linestyle=:dash)
plot!(1:51, [0.8^i for i in 0:50],linewidth=3,linestyle=:dot)
plot!(1:51, [0.9^i for i in 0:50],linewidth=3,linestyle=:dashdot)
plot!(1:51, fi_cor_vals( 51, 0.34 ), linewidth = 3, line = :solid)
title!("I(d) [fractional differencing] with d=0.34")

Long memory estimation

• There are several methods to estimate the fractional differencing parameter \(d\).

• The most common are the semi-parametric methods, such as the log-periodogram estimator, and the parametric methods, such as the maximum likelihood estimator.

• Heuristic methods, such as the log-variance plot, are also used.

Log-variance plot

• Consider the variance of the sample mean given by

\[\text{Var}(\bar{x}_n) = \frac{\gamma(0)}{n} + 2\sum_{k=1}^{n-1}\left(1-\frac{k}{n}\right)\gamma(k).\]

Log-variance plot

• For long memory series, the following relationship holds: \[Var\left(\bar{x}\right) \approx C_v n^{2d-1}.\]

• This suggest a way to determine if the series has long memory by plotting the log-variance of the sample mean against the log of the sample size.

Log-variance plot

log_variance_plot( NileData().NileMin; m=300, slope = true, slope2 = true )

Log-variance plot properties

Maximum likelihood estimator

• The MLE is a parametric estimator based on fitting the autocovariance function of the time series.

• Let \(X=[x_0,\cdots,x_{T-1}]'\) be a sample of size \(T\) of a fractionally differenced time series, and let \(\theta = [d,\sigma^2]'\).

• Under the assumption that \(\varepsilon_t\) follows a normal distribution, \(X\) follows a normal distribution.

Maximum likelihood estimator

• The probability density of \(X\) is given by: \[ f(X|\theta) = (2\pi)^{-T/2}|\Sigma|^{-1/2}\exp\left(-\frac{1}{2}X^\top\Sigma^{-1}X\right),\] where \(\Sigma\) is the covariance matrix defined as: \[\Sigma = [\gamma(|j-k|)].\]

MLE properties

Log-periodogram estimators

• The log-periodogram esimator is a semiparametric estimator in the frequency domain.

• It was originally proposed by Geweke and Porter-Hudak (1983), hence it is also known as the Geweke-Porter-Hudak estimator or GPH estimator.

• Alternative versions are known as Whittle estimators.

Frequency domain

• The frequency domain refers to the analysis of series (or signals) with respect to frequency rather than time.

• The idea is that any time series can be decomposed into a sum of sinusoidal functions of different frequencies.

Frequency domain

The time series next is given by \[x_t = x_{1,t}+x_{2,t}+x_{3,t},\]

for \(t=1,\cdots,100\), where

\[\begin{align} x_{1,t} &= 3\cos\left(2\pi\frac{t}{100}\right) + 4\sin\left(2\pi\frac{t}{100}\right),\\ x_{2,t} &= 4\cos\left(2\pi\frac{5t}{100}\right) + 2\sin\left(2\pi\frac{5t}{100}\right),\\ x_{3,t} &= \cos\left(2\pi\frac{40t}{100}\right) + \sin\left(2\pi\frac{40t}{100}\right). \end{align}\]

Frequency domain

Frequency domain

\[x_t = \sum_{j=1}^n a_j\cos(2\pi \lambda_j t) + b_j\sin(2\pi \lambda_j t) + \varepsilon_t\]

• The contribution of frequency \(\lambda_j\) is \(a_j^2+b_j^2\).

• This measures the sample variance at each frequency component and is then an estimate of \(\sigma_j^2\) for frequency \(\lambda_j\).

Log-periodogram estimator

• The low-frequency components of the periodogram are those associated to the longer cycles.

• For long memory series, the periodogram satisfies:

\[I_x(\lambda)\approx C_f\lambda^{-2d}\quad \text{as}\quad \lambda\to 0.\]

Log-periodogram estimator

• The log-periodogram regression is given by:

\[\log(I(\lambda_k)) = c-2d \log(\lambda_k)+u_k,\quad k = 1,\cdots,m,\]

where \(u_k\) is the error term, and \(m\) is a bandwidth parameter that determines the number of frequencies used for estimation.

• The estimate of \(d\) is retrieved from the slope of the regression line.

Log-periodogram estimator

periodogram_plot( NileData().NileMin; slope = true)

Log-periodogram estimator properties

Whittle estimator

• An alternative semiparametric formulation was developed by Künsch (1987).

• The author proposed to estimate the parameter as the minimiser of the local Whittle likelihood function given by \[R(d) = \log\left(\frac{1}{m}\sum_{k=1}^{m}\lambda_k^{2d}I(\lambda_k)\right)-\frac{2d}{m}\sum_{k=1}^{m}\log(\lambda_k),\] where \(m\) is the bandwidth parameter.

Whittle estimator properties

ARFIMA model

• The ARFIMA model is a generalization of the ARIMA model that includes fractional differencing.

• The ARFIMA model is defined by the following equation: \[\phi(L)(1-L)^d x_t = \theta(L)\varepsilon_t,\]

where \(\phi(L)\) and \(\theta(L)\) are the autoregressive and moving average polynomials, respectively.

FIGARCH model

• The FIGARCH model is a generalization of the GARCH model that includes fractional differencing.

• The FIGARCH model is defined by: \[[1-\alpha(L)-\beta(L)](1-L)^d\varepsilon_t^2=\alpha_0+[1-\beta(L)]\nu_t,\]

where \(\alpha(L)\) and \(\beta(L)\) are lag polynomials.