An Introduction to Bootstrap Theory in Time Series Econometrics

Giuseppe Cavaliere; Heino Bohn Nielsen; Anders Rahbek

doi:10.1093/acrefore/9780190625979.013.493

Summary

While often simple to implement in practice, application of the bootstrap in econometric modeling of economic and financial time series requires establishing validity of the bootstrap. Establishing bootstrap asymptotic validity relies on verifying often nonstandard regularity conditions. In particular, bootstrap versions of classic convergence in probability and distribution, and hence of laws of large numbers and central limit theorems, are critical ingredients. Crucially, these depend on the type of bootstrap applied (e.g., wild or independently and identically distributed (i.i.d.) bootstrap) and on the underlying econometric model and data. Regularity conditions and their implications for possible improvements in terms of (empirical) size and power for bootstrap-based testing differ from standard asymptotic testing, which can be illustrated by simulations.

Bootstrap in Time Series Econometrics

Bootstrap in econometrics is frequently applied in the context of estimation and testing (Berkowitz & Kilian, 2000; Cavaliere & Rahbek, 2020; Davidson & MacKinnon, 2006; Horowitz, 2001, 2003; MacKinnon, 2009). As an example, consider the case where some test statistic, $τ_{n}$ say, is of interest given a sample of (time-series) data $x_{1},..., x_{n}$ with initial values $x_{0}, x_{- 1},.., x_{- p}$ for $p \geq 0$ , ${x_{t}}_{t = - p}^{n}$ with $x_{t} \in ℝ$ . Under suitable regularity conditions, including typically (a) stationarity and ergodicity of the $x_{t}$ process and (b) finite moments conditions on the form $E | x_{t}^{2} |^{k} < \infty$ for some $k \geq 1$ , it holds that under the null hypothesis of interest, $H_{0}$ say,

τ_{n} \overset{d}{\to} χ_{q}^{2} as n \to \infty,

where $q$ denotes the degrees of freedom and “ $\overset{d}{\to}$ ” denotes convergence in distribution. Moreover, under the alternative, or when $H_{0}$ is not true, the statistic $τ_{n}$ diverges. In standard asymptotic testing, the $χ_{q}^{2}$ distributional approximation of the test statistic $τ_{n}$ is frequently applied using the $1 - α$ quantile of the $χ_{q}^{2}$ distribution or, equivalently, by calculating the p-value.

A bootstrap-based test is often motivated by noting that the asymptotic approximation may not be good in finite samples, which may lead, for a finite number of observations $n$ , to an actual size larger, or smaller, than the nominal level $α$ . This is often corrected by applying even simple bootstrap schemes. An additional motivation for the bootstrap is that the underlying model used for estimation—and on which the derivation of $τ_{n}$ is based—may be misspecified. A typical example is the assumption of homoskedasticity of model innovations, which often in practice is challenged in the modeling of macro and financial data, where (conditional and unconditional) heteroskedasticity is typically present. In this case, the so-called wild bootstrap may correct for such heteroskedasticity. Likewise, for other types of misspecification and model departures, different bootstrap schemes may be applied depending on which type of misspecification is of concern. In addition, in some cases the limiting distribution of $τ_{n}$ cannot be tabulated, for example, because it is a function of unknown nuisance parameters. The bootstrap may resolve such issues. However, it should be emphasized that any bootstrap-based test is—as for the original asymptotic test—only valid under certain regularity conditions. These regularity conditions are important to check and understand as in many cases application of what may be thought of as a “ standard bootstrap” may be invalid and misleading, despite its popularity in many empirical applications. Typical situations in which a standard bootstrap does not work can be found in the context of nonstationary variables and when data exhibit heavy-tailed distributions, as is typical in financial data. When application of existing, or standard, bootstrap schemes fails to work, these may be corrected by more elaborate bootstrap schemes, as is richly documented in recent research in econometrics (see inter alia Cavaliere & Rahbek, 2020, and the references therein).

The bootstrap is a simulation-based approach that is typically simple to apply in the context of inference and testing. Based on some bootstrap scheme, the bootstrap is based on generating new bootstrap samples of data and the test statistic of interest, denoted by ${x_{t}^{*}}_{t = - p}^{n}$ and $τ_{n}^{*}$ , respectively. The (re-)generation of such bootstrap data is based on keeping the original sample ${x_{t}}_{t = - p}^{n}$ fixed.

If the limiting distribution (in the bootstrap sense) of the bootstrap test statistic $τ_{n}^{*}$ is identical to the limiting distribution of the original statistic $τ_{n}$ under the null of $H_{0}$ , such as the $χ_{q}^{2}$ distribution noted earlier, this implies validity under the null of the bootstrap. If, in addition, under the alternative when $H_{0}$ is not true, the bootstrap statistic $τ_{n}^{*}$ does not diverge as the sample size $n$ increases (or, at most it diverges at a slower rate than the original statistic), the bootstrap is asymptotically valid. This is the case when under the alternative the bootstrap-limiting distribution of $τ_{n}^{*}$ is identical to the limiting distribution of the original statistic or when $τ_{n}^{*}$ is bounded in (bootstrap) probability.

As a leading example throughout, and to fix ideas, we consider the first-order autoregressive (AR) model. The section “Autoregressive Model” provides a summary of standard econometric nonbootstrap analysis of the AR model before turning to the discussion of the implementation and (asymptotic) theory of the bootstrap in the section “Bootstrap in the Autoregressive Model.” Next, a Monte Carlo study is used to discuss finite sample behavior of different bootstrap schemes (see “Finite Sample Behavior”) and recent advances in econometric time series bootstrap analysis are provided (see “Selection of Further Topics”). The conclusion provides an overview of some important further topics and approaches to bootstrap inference.

The Autoregressive Model

In order to present bootstrap theory and arguments, a brief summary of results and arguments for the (nonbootstrap) AR model of order one is provided. The results and asymptotic arguments presented here are well known (see, e.g., Hamilton, 1994; Hayashi, 2000).

Consider the AR model of order one

x_{t} = ρ x_{t - 1} + ε_{t}, t = 1,2,..., n,

(1)

with the initial value $x_{0}$ fixed in the statistical analysis and the innovations $ε_{t}$ are assumed to be independently and identically distributed (i.i.d). $N (0, σ^{2})$ (see also Remark 2 regarding relaxing the assumption of Gaussianity). The parameter of the model is $θ = {(ρ, σ^{2})}^{'}$ with $θ_{0} = {(ρ_{0}, σ_{0}^{2})}^{'}$ denoting the true value. For estimation, the parameter space is given by

θ \in Θ = ℝ \times (0, \infty) .

For the true parameter $θ_{0}$ , we assume $θ_{0} \in Θ_{0} \subset Θ$ , with $Θ_{0} = {θ \in Θ | | ρ_{0} | < 1}$ , such that $x_{t}$ in (1) for $θ_{0} \in Θ_{0}$ has a stationary and geometrically ergodic solution given by

x_{t} = \sum_{i = 0}^{\infty} ρ_{0}^{i} ε_{t - i} .

With $\bar{ρ}$ some fixed value, consider testing the hypothesis $H_{0}$ given by

H_{0} : ρ = \bar{ρ} .

The test is based on the likelihood ratio test statistic, $τ_{n}$ , defined in terms of the Gaussian log-likelihood function

l_{n} (θ) = - \frac{n}{2} (\log σ^{2} + n^{- 1} \sum_{t = 1}^{n} ε_{t}^{2} (ρ) / σ^{2}),

(2)

with $x_{0}$ fixed and $ε_{t} (ρ) = x_{t} - ρ x_{t - 1}$ . Standard optimization gives the unrestricted maximum likelihood estimator (MLE), ${\hat{θ}}_{n} = ({\hat{ρ}}_{n}, {\hat{σ}}_{n}^{2})^{'}$ , where

{\hat{ρ}}_{n} = n^{- 1} \sum_{t = 1}^{n} x_{t} x_{t - 1} {(n^{- 1} \sum_{t = 1}^{n} x_{t - 1}^{2})}^{- 1} and {\hat{σ}}_{n}^{2} = n^{- 1} \sum_{t = 1}^{n} ε_{t}^{2} ({\hat{ρ}}_{n}) .

(3)

The restricted estimator ${\tilde{θ}}_{n} = {({\tilde{ρ}}_{n}, {\tilde{σ}}_{n}^{2})}^{'}$ is obtained by maximization under $H_{0}$ , and is given by

{\tilde{ρ}}_{n} = \bar{ρ} and {\tilde{σ}}_{n}^{2} = n^{- 1} \sum_{t = 1}^{n} ε_{t}^{2} (\bar{ρ}) .

It follows that the likelihood ratio (LR) test statistic is given by

τ_{n} = {LR}_{n} (ρ = \bar{ρ}) = 2 (l_{n} ({\hat{θ}}_{n}) - l_{n} ({\tilde{θ}}_{n})) = n \log ({\tilde{σ}}_{n}^{2} / {\hat{σ}}_{n}^{2}) .

(4)

Moreover, under $H_{0}$ and regularity conditions detailed in the section “Asymptotics for the Autoregressive Model,” with $\bar{ρ} = ρ_{0}$ and $θ_{0} \in Θ_{0}$ it holds that $τ_{n} \overset{d}{\to} χ_{1}^{2}$ . Note in this respect that as long as the true $ρ_{0}$ is not “ too large” the asymptotic $χ_{1}^{2}$ approximation is a good approximation even for small samples. In contrast, and as exemplified in Table 1, if either $ε_{t}$ is not i.i.d. $N (0, σ^{2})$ or if $ρ_{0} = 0.9$ the $χ_{1}^{2}$ distribution is not a good approximation of the actual distribution of $τ_{n}$ for small, or even moderate, finite samples of size $n$ .

Remark 1 (Wald and t statistics). Although focus is on the likelihood ratio test statistics, other statistics could be considered as well. For instance, the classic Wald statistic ( $W_{n}$ ) is given by

W_{n} (ρ = \bar{ρ}) = {({\hat{ρ}}_{n} - \bar{ρ})}^{2} \sum_{t = 1}^{n} x_{t - 1}^{2} / {\hat{σ}}_{n}^{2}

and, under the same regularity conditions required for the likelihood ratio test, when $\bar{ρ} = ρ_{0}$ and $θ_{0} \in Θ_{0}$ , it holds that $W_{n} (ρ = \bar{ρ}) \overset{d}{\to} χ_{1}^{2}$ . It is also customary to consider the $t$ -ratio statistic, defined as

t_{n} (ρ = \bar{ρ}) = \frac{{\hat{ρ}}_{n} - \bar{ρ}}{s e ({\hat{ρ}}_{n})}

where the so-called standard error $se ({\hat{ρ}}_{n})$ is defined as ${({\hat{V}}_{n} / n)}^{1 / 2}$ , with

{\hat{V}}_{n} = {\hat{σ}}_{n}^{2} {(n^{- 1} \sum_{t = 1}^{n} x_{t - 1}^{2})}^{- 1}

an unrestricted (i.e., obtained without imposing the null hypothesis) estimator of the asymptotic variance of $n^{1 / 2} ({\hat{ρ}}_{n} - ρ_{0})$ . Notice that $t_{n} (ρ = \bar{ρ}) = sign ({\hat{ρ}}_{n} - \bar{ρ}) {(W_{n} (ρ = \bar{ρ}))}^{1 / 2}$ and, when $\bar{ρ} = ρ_{0}$ and $θ_{0} \in Θ_{0}$ , $t_{n} (ρ = \bar{ρ}) \overset{d}{\to} N (0,1)$ .

Remark 2 (Quasi likelihood). Often if the $ε_{t}$ sequence is assumed to be i.i.d., Gaussian is relaxed to i.i.d. $(0, σ^{2})$ for some unknown distribution. When the Gaussian log-likelihood function in (2) in this case is used to obtain estimators and test statistics, these are referred to as (Gaussian) quasi MLE and quasi LR statistics, respectively. This is also the case when $ε_{t}$ is assumed to be mean zero and (conditionally or unconditionally) heteroskedastic, as for example with some autoregressive conditional heteroskedastic (ARCH) specification or with a structural break in the variance (see the section “Finite Sample Behavior”).

Asymptotics for the Autoregressive Model

To ease the presentation of the key arguments for the standard nonbootstrap asymptotic analysis, assume here without loss of generality that $σ^{2}$ is fixed at the true value—that is, $σ^{2} = σ_{0}^{2}$ . Accordingly, the parameter is $θ = ρ$ and the likelihood function in (2) simplifies as

l_{n} (θ) = l_{n} (ρ) = - \frac{n}{2} (\log σ_{0}^{2} + n^{- 1} \sum_{t = 1}^{n} ε_{t}^{2} (ρ) / σ_{0}^{2}) .

The MLE ${\hat{ρ}}_{n}$ is given by (3), while the $L R_{n} (ρ = \bar{ρ})$ statistic in (4) in this case is identical to the Wald statistic $(W_{n})$

τ_{n} = W_{n} (ρ = \bar{ρ}) = {({\hat{ρ}}_{n} - \bar{ρ})}^{2} \sum_{t = 1}^{n} x_{t - 1}^{2} / σ_{0}^{2} .

(5)

By construction, $τ_{n}$ is a simple function of $n^{1 / 2} ({\hat{ρ}}_{n} - \bar{ρ})$ and $n^{- 1} \sum_{t = 1}^{n} x_{t - 1}^{2}$ , and the limiting distribution is found by applying a law of large numbers (LLN) to the average $n^{- 1} \sum_{t = 1}^{n} x_{t - 1}^{2}$ , as well as a central limit theorem (CLT) to $n^{1 / 2} ({\hat{ρ}}_{n} - \bar{ρ})$ .

With $\bar{ρ} = ρ_{0} \in Θ_{0}$ , the AR process $x_{t}$ is stationary and ergodic, and has finite variance, $V (x_{t}) = ω_{0} = σ_{0}^{2} / (1 - ρ_{0}^{2})$ . In particular, it follows by the LLN for stationary and ergodic processes that

n^{- 1} \sum_{t = 1}^{n} x_{t - 1}^{2} \overset{p}{\to} V (x_{t}) = ω_{0} .

Next, by definition of the MLE,

n^{1 / 2} ({\hat{ρ}}_{n} - ρ_{0}) = n^{- 1 / 2} \sum_{t = 1}^{n} ε_{t} x_{t - 1} {(n^{- 1} \sum_{t = 1}^{n} x_{t - 1}^{2})}^{- 1} .

Here $m_{t} = ε_{t} x_{t - 1}$ is a martingale difference sequence (mds) with respect to the filtration $F_{t}$ , where $F_{t} = σ (x_{t}, x_{t - 1} ...)$ , as $E | m_{t} | < \infty$ , and $E (m_{t} | F_{t - 1}) = 0$ . Moreover, the conditional second order moment converges in probability,

n^{- 1} \sum_{t = 1}^{n} E (m_{t}^{2} | F_{t - 1}) = σ_{0}^{2} n^{- 1} \sum_{t = 1}^{n} x_{t - 1}^{2} \overset{p}{\to} σ_{0}^{2} ω_{0} .

This implies by the CLT for mds that $n^{- 1 / 2} \sum_{t = 1}^{n} m_{t} \overset{d}{\to} N (0, σ_{0}^{2} ω_{0})$ (Hall & Heyde, 1980; Hamilton, 1994) and such that

n^{1 / 2} ({\hat{ρ}}_{n} - ρ_{0}) \overset{d}{\to} N (0, σ_{0}^{2} / ω_{0}) \overset{d}{=} N (0,1 - ρ_{0}^{2}) .

Remark 3 (Lindeberg condition). Note that for a CLT for mds in general to hold (Hall & Heyde, 1980), a Lindeberg-type condition of the form $γ_{n} = n^{- 1} \sum_{t = 1}^{n} E (m_{t}^{2} I (| m_{t} | > δ n^{1 / 2})) \to 0$ for any $δ > 0$ must hold, where $I (\cdot)$ is the indicator function. This holds here as, by stationarity,

γ_{n} = n^{- 1} \sum_{t = 1}^{n} E (m_{t}^{2} I (| m_{t} | > δ n^{1 / 2})) = E (m_{t}^{2} I (| m_{t} | > δ n^{1 / 2})),

which tends to zero as $n \to \infty$ by dominated convergence under the moment condition $E (m_{t}^{2}) < \infty$ as implied by $E (ε_{t}^{2}) < \infty$ and independence of $ε_{t}$ and $x_{t - 1}$ .

Collecting terms, it follows that we have the following result.

Lemma 1. For the AR(1) model in (1) with $θ_{0} \in Θ_{0}$ and $ε_{t}$ i.i.d. $(0, σ_{0}^{2})$ , it follows that with $τ_{n}$ given by (5), under the null $τ_{n} \overset{d}{\to} χ_{1}^{2}$ as $n \to \infty$ .

Note that in Lemma 1, it is not assumed that the $ε_{t}$ sequence is Gaussian, but instead the lemma is formulated in terms of the milder sufficient condition that $ε_{t}$ is an i.i.d. $(0, σ^{2})$ sequence (Remark 2). Thus the essential regularity conditions for the lemma to hold are that $| ρ_{0} | < 1$ and $ε_{t}$ i.i.d. $(0, σ^{2})$ .

Remark 4 (Heteroskedasticity). Note that if $ε_{t} = σ_{t} z_{t}$ , with $z_{t}$ i.i.d. $N (0,1)$ and $σ_{t}^{2}$ given by an ARCH process with $E (σ_{t}^{2}) = σ_{ε}^{2} < \infty$ , it follows that $τ_{n} \overset{d}{\to} c χ_{1}^{2}$ , with $c = σ_{ε}^{2} / σ_{0}^{2}$ . This reflects in general that if $ε_{t}$ are (conditionally, or unconditionally) heteroskedastic, the limiting distribution of the test statistic $τ_{n}$ is not $χ_{1}^{2}$ . This is wellknown in the context of regression models, and corrections of the test statistic (White, 1980) are typically applied to ensure valid asymptotic inference; the bootstrap, or more precisely, the wild bootstrap discussed later, may correct for such misspecification without the need to correct the original test statistic (cf. Gonçalves & Kilian, 2004).

Remark 5 (Test statistic). For $σ^{2}$ an unknown parameter to be estimated, such that $θ = {(σ^{2}, ρ)}^{'}$ , note that the LR statistic in (4) can be written as

τ_{n} = L R_{n} (ρ = \bar{ρ}) = - n \log (1 - n^{- 1} W_{n} (ρ = \bar{ρ}) \frac{σ_{0}^{2}}{{\tilde{σ}}_{n}^{2}}),

with ${\tilde{σ}}_{n}^{2} = n^{- 1} \sum_{t = 1}^{n} ε_{t}^{2} (\bar{ρ})$ . By Lemma 1, $w_{n} = n^{- 1} W_{n} (ρ = \bar{ρ}) \overset{p}{\to} 0$ , and by a Taylor expansion, $- \log (1 - w) = w + o (w)$ , with $o (w)$ a term that tends to zero as $w \to 0$ . Hence,

τ_{n} = W_{n} (ρ = \bar{ρ}) + o_{p} (1),

where $o_{p} (1)$ denotes a term that converges to zero in probability as $n \to \infty$ (see, e.g., van der Vaart, 2000, Lemma 2.12, for details on $o_{p} (\cdot)$ (and $O_{p} (\cdot)$ notation and the stochastic [Taylor] expansion as applied here).

Bootstrap in the Autoregressive Model

The general idea behind a bootstrap algorithm as implemented in the context of likelihood-based testing in the first order AR model can be summarized as follows:

Step A. With the original data ${x_{t}}_{t = 0}^{n}$ fixed, generate a sample of bootstrap data ${x_{t}^{*}}_{t = 0}^{n}$ using some bootstrap scheme with bootstrap true parameter, $θ_{n}^{*} = {(ρ_{n}^{*}, σ_{n}^{* 2})}^{'}$ . Specifically, for the AR model, set $x_{0}^{*} = x_{0}$ and generate $x_{t}^{*}$ recursively by

x_{t}^{*} = ρ_{n}^{*} z_{t} + ε_{t}^{*},

with $z_{t} = x_{t - 1}^{*}$ for a recursive bootstrap scheme, while $z_{t} = x_{t - 1}$ for a fixed design bootstrap scheme. As to the choice of bootstrap innovations $ε_{t}^{*}$ in terms of the original data ${x_{t}}_{t = 0}^{n}$ and a bootstrap sampling distribution as detailed after step C.

Step B. Compute the bootstrap (quasi) MLE ${\hat{ρ}}_{n}^{*}$ and the bootstrap LR statistic $τ_{n}^{*} = L R_{n}^{*} (ρ = ρ_{n}^{*})$ , where the bootstrap log-likelihood function $l_{n}^{*} (θ)$ as a function of $θ$ is given by

l_{n}^{*} (θ) = - \frac{n}{2} (\log σ^{2} + n^{- 1} \sum_{t = 1}^{n} ε_{t}^{* 2} (ρ) / σ^{2}), with ε_{t}^{*} (ρ) = x_{t}^{*} - ρ z_{t} .

Step C. Generate ${x_{t, b}^{*}}_{t = 0}^{n}$ , ${\hat{θ}}_{n, b}^{*}$ and $τ_{n, b}^{*}$ for $b = 1,2,..., B$ by repeating Steps $A$ and $B$ , and use the empirical distribution of ${τ_{n, b}^{*}}_{b = 1}^{B}$ for testing based on the original statistic $τ_{n} = L R_{n} (ρ = ρ_{0})$ . Precisely, with the bootstrap p-value set to $p_{n, B}^{*} = B^{- 1} \sum_{b = 1}^{B} I (τ_{n} \geq τ_{n, b}^{*})$ , $H_{0}$ is rejected when $p_{n, B}^{*} < α$ , with $α$ the nominal level. (For the choice of bootstrap repetitions $B$ , see Remark 9).

As to the definition in step $A$ of the bootstrap innovations $ε_{t}^{*}$ , usually these are obtained by i.i.d. draws with replacement from re-centered estimated model residuals; henceforth referred to as iid bootstrap, or iid resampling. The model residuals can be either estimated under the null ( $H_{0}$ imposed) or without imposing the null. That is, the $ε_{t}^{*}$ are in the first case resampled from centered residuals, ${{\tilde{ε}}_{t}^{c}}_{t = 1}^{n}$ , where

{\tilde{ε}}_{t}^{c} = ε_{t} (\bar{ρ}) - n^{- 1} \sum_{t = 1}^{n} ε_{t} (\bar{ρ}), with ε_{t} (\bar{ρ}) = x_{t} - \bar{ρ} x_{t - 1} .

(8)

Using unrestricted residuals, the bootstrap $ε_{t}^{*}$ innovations are resampled from ${{\hat{ε}}_{t}^{c}}_{t = 1}^{n}$ , where

{\hat{ε}}_{t}^{c} = ε_{t} ({\hat{ρ}}_{n}) - n^{- 1} \sum_{t = 1}^{n} ε_{t} ({\hat{ρ}}_{n}), with ε_{t} ({\hat{ρ}}_{n}) = x_{t} - {\hat{ρ}}_{n} x_{t - 1} .

(9)

Centering is required because by doing so both of the residual series ${\hat{ε}}_{t}^{c}$ and ${\tilde{ε}}_{t}^{c}$ have empirical mean zero, $n^{- 1} \sum_{t = 1}^{n} {\hat{ε}}_{t}^{c} = 0$ and $n^{- 1} \sum_{t = 1}^{n} {\tilde{ε}}_{t}^{c} = 0$ , and ideally “ mimic” the true $ε_{t}$ . In particular, under $H_{0}$ , $ε_{t} (\bar{ρ}) = ε_{t} (ρ_{0}) = ε_{t}$ , and, moreover, ${\hat{ρ}}_{n} \overset{p}{\to} ρ_{0}$ . In contrast, under the alternative, when $H_{0}$ does not hold, while $ε_{t} (\bar{ρ}) \neq ε_{t}$ , it still holds that ${\hat{ρ}}_{n} \overset{p}{\to} ρ_{0}$ . Hence, while one may expect that the unrestricted residuals ${\hat{ε}}_{t}^{c}$ perform better under the alternative, at least asymptotically, in practice little difference is found between the two.

An alternative to iid resampling is the so-called wild bootstrap, where either $ε_{t}^{*} = {\tilde{ε}}_{t}^{c} w_{t}^{*},$ or $ε_{t}^{*} = {\hat{ε}}_{t}^{c} w_{t}^{*}$ , with $w_{t}^{*}$ an auxiliary i.i.d. sequence, independent of the original data, with $E (w_{t}^{*}) = 0$ and $V (w_{t}^{*}) = 1$ . A simple example is $w_{t}^{*}$ i.i.d. $N (0,1)$ (see Remark 10 for alternative specifications). The wild bootstrap is typically motivated by its potential ability to allow for possible model misspecification in the sense that it allows for (conditional and unconditional) heteroskedasticity in the innovations $ε_{t}$ . To see this (e.g., $ε_{t}^{*} = {\tilde{ε}}_{t}^{c} w_{t}^{*}$ ), it follows that (conditionally on the data ${x_{t}}_{t = 0}^{n}$ ), the $ε_{t}^{*}$ are i.i.d. distributed with mean zero and with time-varying variance,

V (ε_{t}^{*} | {x_{t}}_{t = 0}^{n}) = {({\tilde{ε}}_{t}^{c})}^{2} .

This way, the variation in $ε_{t}^{*}$ from the wild bootstrap “in essence reflects the heteroskedasticity of the original data” (Liu, 1988, p. 1704) as is also illustrated in the Monte Carlo simulations in the section on “Finite Sample Behavior.” This differs from the iid bootstrap, where the bootstrap innovations $ε_{t}^{*}$ (conditionally on the data) are i.i.d., with a discrete distribution given by $P (ε_{t}^{*} = {\tilde{ε}}_{j}^{c} | {x_{t}}_{t = 0}^{n}) = n^{- 1}$ , for $j = 1,2,..., n$ . Specifically, as mentioned earlier, the (empirical, or conditional) mean is zero, $E (ε_{t}^{*} | {x_{t}}_{t = 0}^{n}) = 0$ , and the variance is given by

V (ε_{t}^{*} | {x_{t}}_{t = 0}^{n}) = n^{- 1} \sum_{t = 1}^{n} {({\tilde{ε}}_{t}^{c})}^{2} .

That is, for fixed $n$ , the variance for the iid bootstrap innovations $ε_{t}^{*}$ is constant (and equal to the empirical variance of the estimated residuals), while depending on time $t$ for the wild bootstrap.

In short, the bootstrap scheme in (6), $x_{t}^{*} = ρ_{n}^{*} z_{t} + ε_{t}^{*}$ , with $z_{t} = x_{t - 1}$ or $z_{t} = x_{t - 1}^{*}$ , depends on two types of randomness: (a) the variation of the original data ${x_{t}}_{t = 0}^{n}$ ; and (b) the bootstrap resampling for ${ε_{t}^{*}}_{t = 1}^{n}$ (wild or iid resampling). This is important for the application of the LLN and the CLT to establish bootstrap validity as demonstrated in the next.

Some further remarks are in order.

Remark 6 (t statistics and bootstrap standard errors). When interest is in bootstrapping the $t$ ratio statistic (or Wald-type statistics), then steps $A - C$ can be used with $τ_{n}^{*}$ , now defined as (taking the case of unknown $σ^{2}$ to illustrate)

τ_{n}^{*} = t_{n}^{*} (ρ = \bar{ρ}) = \frac{{\hat{ρ}}_{n}^{*} - \bar{ρ}}{{se}^{*} ({\hat{ρ}}_{n}^{*})}

where the so-called bootstrap standard error ${se}^{*} ({\hat{ρ}}_{n}^{*})$ , the bootstrap analog of $se ({\hat{ρ}}_{n})$ , given by

{se}^{*} ({\hat{ρ}}_{n}^{*}) = (n^{- 1} {\hat{V}}_{n}^{*})^{1 / 2}

with $V_{n}^{*}$ being an estimator of the variance of ${\hat{ρ}}_{n}^{*} - ρ_{0}$ , such as

{\hat{V}}_{n}^{*} = {\hat{σ}}_{n}^{* 2} {(n^{- 1} \sum_{t = 1}^{n} x_{t - 1}^{* 2})}^{- 1}

where ${\hat{σ}}_{n}^{* 2}$ is the sample variance of the bootstrap residuals, ${\hat{σ}}_{n}^{* 2} = n^{- 1} \sum_{t = 1}^{n} ε_{t}^{*} ({\hat{ρ}}_{n}^{*})$ , $ε_{t}^{*} ({\hat{ρ}}_{n}^{*}) = x_{t}^{*} - {\hat{ρ}}_{n}^{*} x_{t - 1}^{*}$ . It is worth noticing that the bootstrap standard error could also be computed without exploiting the autoregressive structure of the model; this can be done by simply resorting to the $B$ bootstrap realizations of ${\hat{ρ}}_{n}^{*}$ , say ${\hat{ρ}}_{n, b}^{*}$ , $b = 1,..., B$ . Specifically, one could consider the following bootstrap standard error

{se}_{B}^{*} ({\hat{ρ}}_{n}^{*}) = {(\frac{1}{B - 1} \sum_{b = 1}^{B} {({\hat{ρ}}_{n, b}^{*} - \bar{{\hat{ρ}}_{n, b}^{*}})}^{2})}^{1 / 2}, \bar{{\hat{ρ}}_{n, b}^{*}} = \frac{1}{B} \sum_{b = 1}^{B} {\hat{ρ}}_{n, b}^{*},

which is straightforward to compute once the $B$ realizations of ${\hat{ρ}}_{n}^{*}$ have been obtained.

Remark 7 ([Un-]restricted bootstrap). As to the choice in step $A$ of the bootstrap true value $θ_{n}^{*} = {(ρ_{n}^{*}, σ_{n}^{* 2})}^{'}$ (or simply, $ρ_{n}^{*}$ for the AR model), one may set $θ_{n}^{*}$ to the value of the unrestricted estimator ${\hat{θ}}_{n}$ , $θ_{n}^{*} = {\hat{θ}}_{n}^{*}$ , which is referred to as unrestricted bootstrap. If $θ_{n}^{*} = {\tilde{θ}}_{n}$ , the restricted estimator, the bootstrap is referred to as restricted. It should be emphasized that for the unrestricted bootstrap, the bootstrap likelihood ratio statistic $τ_{n}^{*}$ is derived for the hypothesis $ρ = {\hat{ρ}}_{n}$ , while the original hypothesis $H_{0} : ρ = \bar{ρ}$ is considered for the restricted bootstrap. While both choices are widely applied in existing literature, the restricted bootstrap in the context of testing is more popular in econometrics (see, e.g., Davidson & MacKinnon, 2000).

Remark 8 (Recursive and fixed design bootstrap). In step $B$ of the algorithm in equation (6), with $z_{t} = x_{t - 1}^{*}$ , this is an example of a recursive bootstrap. That is, the original autoregressive structure for $x_{t}$ is replicated for the bootstrap process $x_{t}^{*}$ . However, with $z_{t} = x_{t - 1}$ , the original data $x_{t - 1}$ are used as lagged value of $x_{t}^{*}$ , such that $x_{t}^{*}$ is not an autoregressive process even conditionally on the data. The fixed design bootstrap typically simplifies (some of) the asymptotic arguments, and is often found to behave as well as the recursive bootstrap (see, e.g., Gonçalves & Kilian, 2004, 2007, for general AR models, and Cavaliere, Pedersen, & Rahbek, 2018, for ARCH models).

Remark 9 (Bootstrap p-value). In step $C$ , the bootstrap p-value $p_{n, B}^{*}$ is defined as $p_{n, B}^{*} = B^{- 1} \sum_{b = 1}^{B} I (τ_{n} \geq τ_{n, b}^{*})$ , with $B$ the number of bootstrap repetitions (see also Remark 14). Typical choices are $B = 199$ , $399$ , or $999$ (see also Andrews & Buchinsky, 2000, and Davidson & MacKinnon, 2000, for details on the choice of $B$ ).

Remark 10 (Choice of $w_{t}^{*}$ ). With respect to the choice of distribution of the i.i.d. sequence $w_{t}^{*}$ for the wild bootstrap, Liu (1988) provides a detailed discussion of various choices based on so-called Edgeworth expansions (see, e.g., Hall, 1992; van der Vaart, 2000, for an introduction) of test statistics similar to $τ_{n}$ . In particular, Liu (1988), with $ξ_{k} = E (w_{t}^{k}), k \geq 1$ , emphasizes $ξ_{1} = 0$ , $ξ_{2} = 1$ as well as $ξ_{3} = 1$ as important. For the case of $ξ_{3} = 1$ , emphasis is on possible skewness, while $ξ_{3} = 0$ works well in the case of symmetry. In applications, standard choices for $w_{t}^{*}$ , all with $ξ_{1} = 0$ and $ξ_{2} = 1$ , include the Gaussian, Rademacher, and Mammen distributions.¹ It follows that $ξ_{3} = 0$ for the first two (with $ξ_{4} = 3$ and $1$ , respectively), while $ξ_{3} = 1$ (and $ξ_{4} = 2$ ) for the Mammen distribution.

Remark 11 (Parametric bootstrap). In step $A$ of the bootstrap scheme, one may also use a so-called parametric bootstrap, where the bootstrap innovations $ε_{t}^{*}$ are generated as i.i.d. $N (0, σ_{n}^{* 2})$ . While this parametric bootstrap performs well in the case where the true innovations $ε_{t}$ are Gaussian, this may not be the case when the distribution of $ε_{t}$ is non-Gaussian (Horowitz, 2001).

Asymptotic Theory for the Recursive Bootstrap

In order to discuss regularity conditions under which bootstrap-based testing holds, consider here the details of verification of the asymptotic validity of the recursive bootstrap for the AR model.

Thus, with the AR model given in (1), consider here the recursive restricted bootstrap scheme as defined by setting $z_{t} = x_{t - 1}^{*}$ and $ρ_{n}^{*} = \bar{ρ}$ in step $A$ of the bootstrap algorithm. In short, the $x_{t}^{*}$ bootstrap sequence is here generated as

x_{t}^{*} = \bar{ρ} x_{t - 1}^{*} + ε_{t}^{*},

(10)

with $x_{0}^{*} = x_{0}$ and $\bar{ρ} = ρ^{*}$ as the bootstrap true value. Moreover, we consider the classic case of iid resampling from the autoregressive residuals obtained under $H_{0}$ , that is, from ${{\tilde{ε}}_{t}^{c}}_{t = 1}^{n}$ defined in (8). The statistic of interest is $τ_{n} = W_{n} (ρ = \bar{ρ})$ in (5), which is computed using the bootstrap sample ${x_{t}^{*}}_{t = 0}^{n}$ as

τ_{n}^{*} = W_{n}^{*} (ρ = \bar{ρ}) = {({\hat{ρ}}_{n}^{*} - \bar{ρ})}^{2} \sum_{t = 1}^{n} {(x_{t - 1}^{*})}^{2} / σ_{0}^{2} and {\hat{ρ}}_{n}^{*} = \sum_{t = 1}^{n} x_{t}^{*} x_{t - 1}^{*} {(\sum_{t = 1}^{n} x_{t - 1}^{* 2})}^{- 1} .

(11)

While $x_{t}^{*}$ clearly has some features similar to $x_{t}$ , one cannot apply standard concepts such as stationarity and ergodicity when analyzing the asymptotic behavior of $τ_{n}^{*}$ , due to two types of randomness: the bootstrap resampling distribution and the distribution of the original data, ${x_{t}}_{t = 0}^{n}$ . Introduce therefore the bootstrap equivalent concepts of convergence in probability and distribution, which reflect the fact that inference is based on conditioning on the original data that are themselves random.

Bootstrap Probability, Expectation, and Convergence

With $P^{*} (\cdot)$ denoting the bootstrap probability—that is, the probability conditional on the data—the iid bootstrap innovations $ε_{t}^{*}$ are by definition i.i.d. distributed with

P^{*} (ε_{t}^{*} = {\tilde{ε}}_{j}^{c}) = P (ε_{t}^{*} = {\tilde{ε}}_{j}^{c} | {x_{t}}_{t = 0}^{n}) = n^{- 1} for j = 1,2,..., n .

Similarly, the expectation $E^{*} (\cdot)$ is defined by $E^{*} (\cdot) = E (\cdot | {x_{t}}_{t = 0}^{n})$ . As an example, consider the expectation of $ε_{t}^{*}$ conditionally on the data. It follows that, as already discussed, $E^{*} (ε_{t}^{*}) = 0$ as

E^{*} (ε_{t}^{*}) = \sum_{j = 1}^{n} P^{*} (ε_{t}^{*} = {\tilde{ε}}_{j}^{c}) {\tilde{ε}}_{j}^{c} = n^{- 1} \sum_{j = 1}^{n} {\tilde{ε}}_{j}^{c} = 0,

by the definition in (8). Next, consider the variance of $ε_{t}^{*}$ conditionally on the data, $V^{*} (ε_{t}^{*})$ . Again, by definition, $V^{*} (ε_{t}^{*}) = E^{*} (ε_{t}^{* 2}) - {(E^{*} (ε_{t}^{*}))}^{2}$ and hence as $E^{*} (ε_{t}^{*}) = 0$ , it follows that

V^{*} (ε_{t}^{*}) = \sum_{j = 1}^{n} P^{*} (ε_{t}^{*} = {\tilde{ε}}_{j}^{c}) {({\tilde{ε}}_{j}^{c})}^{2} = n^{- 1} \sum_{j = 1}^{n} {({\tilde{ε}}_{j}^{c})}^{2} .

That is, the variance conditional on the data is equal to the sample variance of the original estimated residuals under $H_{0}$ . In particular, $V^{*} (ε_{t}^{*})$ is a random variable (in terms of the original probability measure) and moreover, by the LLN for i.i.d. variables, under $H_{0}$ with $\bar{ρ} = ρ_{0}$ ,

V^{*} (ε_{t}^{*}) = E^{*} (ε_{t}^{* 2}) \overset{p}{\to} V (ε_{t}) = σ_{0}^{2} .

(12)

Note that for the wild bootstrap, $V^{*} (ε_{t}^{*}) = V^{*} ({\tilde{ε}}_{t}^{c} w_{t}^{*}) = ({\tilde{ε}}_{t}^{c})^{2}$ , emphasizing that the wild bootstrap indeed “mimics” heteroskedasticity and the iid bootstrap does not.

Similar to the definition of $P^{*} (\cdot)$ and $E^{*} (\cdot)$ this motivates the definition of the bootstrap equivalent of convergence in probability, denoted “ ${\overset{p^{*}}{\to}}_{p}$ .” Formally, a sequence of stochastic variables $X_{n}^{*}$ is said to converge in probability conditional on the data (or, to converge in $P^{*}$ -probability, in probability) to $c$ (possibly random), if $P^{*} (| X_{n}^{*} - c | > δ)$ converge in probability to zero. This can be stated as

X_{n}^{*} - c {\overset{p^{*}}{\to}}_{p} 0 if P^{*} (| X_{n}^{*} - c | > δ) \overset{p}{\to} 0 for any δ > 0.

As an example, it follows that $X_{n}^{*} = n^{- 1} \sum_{t = 1}^{n} ε_{t}^{*} {\overset{p^{*}}{\to}}_{p} 0$ as by the bootstrap equivalent of Markov’s inequality² one has,

P^{*} (| X_{n}^{*} | > δ) \leq E^{*} {(X_{n}^{*})}^{2} / δ^{2} .

By definition,

E^{*} {(X_{n}^{*})}^{2} = n^{- 2} E^{*} (\sum_{t = 1}^{n} ε_{t}^{* 2} + 2 \sum_{t = 1}^{n} \sum_{s = t + 1}^{n} ε_{t}^{*} ε_{s}^{*}),

with $E^{*} (ε_{t}^{* 2}) = V^{*} (ε_{t}^{*})$ , and for $s \neq t$ ,

E^{*} (ε_{s}^{*} ε_{t}^{*}) = E^{*} (ε_{s}^{*}) E^{*} (ε_{t}^{*}) = 0.

Hence, since $E^{*} {(X_{n}^{*})}^{2} = n^{- 1} V^{*} (ε_{t}^{*})$ , we conclude that

X_{n}^{*} = n^{- 1} \sum_{t = 1}^{n} ε_{t}^{*} {\overset{p^{*}}{\to}}_{p} 0.

(13)

A key ingredient in the asymptotic analysis of the nonbootstrap AR model is the CLT, and we need a bootstrap equivalent of the CLT and a bootstrap equivalent of convergence in distribution. By definition, $X_{n} \overset{d}{\to} X$ if $F_{X_{n}} (x) = P (X_{n} \leq x) \to F_{X} (x) = P (X \leq x)$ at all continuity points of $F_{X} (\cdot)$ . Likewise, $X_{n}^{*}$ converge in distribution to $X$ conditional on the data (or, as sometimes used, $X_{n}^{*}$ converges “weakly in probability”); that is, $X_{n}^{*} {\overset{d^{*}}{\to}}_{p} X$ , if the bootstrap cumulative distribution function converges in probability. Specifically, $X_{n}^{*} {\overset{d^{*}}{\to}}_{p} X$ if

F_{X_{n}^{*}}^{*} (x) = P^{*} (X_{n}^{*} \leq x) \overset{p}{\to} F_{X} (x),

at all continuity points of $F_{X} (\cdot)$ . Alternatively, weak convergence in probability may be defined in terms of convergence in probability of the bootstrap characteristic function as in (15).

The next lemma illustrates that, as one might expect, the sum of bootstrap innovations $ε_{t}^{*}$ is asymptotically Gaussian (in probability).

Lemma 2 (van der Vaart, 2000, Theorem 23.4). With $X_{n}^{*} = n^{- 1 / 2} \sum_{t = 1}^{n} ε_{t}^{*}$ , then

X_{n}^{*} {\overset{d^{*}}{\to}}_{p} X \overset{d}{=} N (0, σ_{0}^{2}) .

(14)

The proof, as for most bootstrap CLTs, is based on applying a CLT for triangular arrays, as ${ε_{t}^{*}}_{t = 1}^{n}$ are sampled from ${\tilde{ε}}_{t}^{c}$ , which depends on $n$ .

To give an idea of the underlying theory, consider here verifying (14) using a classic approach based on the characteristic function (Durret, 2019, proof of Theorem 3.4.10). For a random variable $X$ , the characteristic function defines uniquely the distribution of $X$ and is defined by $ϕ (s) = E (\exp (i s X))$ . Here, $i$ is the complex (unit imaginary) number that satisfies $i^{2} = - 1$ and $s \in ℝ$ , and for $X \overset{d}{=} N (0, σ_{0}^{2})$ it holds that $ϕ (s) = \exp (- \frac{s^{2}}{2} σ_{0}^{2})$ .

With the bootstrap characteristic function of $X_{n}^{*}$ defined by $ϕ_{n}^{*} (s) = E^{*} (\exp (i s X_{n}^{*}))$ , it follows that (14) holds if

ϕ_{n}^{*} (s) \overset{p}{\to} \exp (- \frac{s^{2}}{2} σ_{0}^{2}) = ϕ (s) .

(15)

Note first, as $ε_{t}^{*}$ are i.i.d. conditionally on the data,

\begin{matrix} E^{*} (\exp (i s X_{n}^{*})) = E^{*} (\prod_{t = 1}^{n} \exp (i s X_{n}^{*})) = \prod_{t = 1}^{n} E^{*} (\exp (i s n^{- 1 / 2} ε_{t}^{*})) \\ = {(E^{*} (\exp (i s n^{- 1 / 2} ε_{t}^{*})))}^{n} . \end{matrix}

Next, a Taylor expansion of $\exp (\cdot)$ at $s = 0$ gives

\begin{matrix} E^{*} (\exp (i s n^{- 1 / 2} ε_{t}^{*})) = 1 + i s n^{- 1 / 2} E^{*} (ε_{t}^{*}) - \frac{1}{2} s^{2} n^{- 1} E^{*} (ε_{t}^{* 2}) + o_{p} (n^{- 1}) \\ = 1 - n^{- 1} (\frac{1}{2} s^{2} σ_{0}^{2}) + o_{p} (n^{- 1}), \end{matrix}

using $E^{*} (ε_{t}^{*}) = 0$ and $E^{*} (ε_{t}^{* 2}) \overset{p}{\to} σ_{0}^{2}$ (see [12]; Durret, 2019, Lemma 3.3.19). It therefore follows as desired that

E^{*} (\exp (i s X_{n}^{*})) = {(1 - n^{- 1} (\frac{1}{2} s^{2} σ_{0}^{2}) + o_{p} (n^{- 1}))}^{n} \overset{p}{\to} ϕ (s),

as for any sequence $c_{n}$ , with $c_{n} \overset{p}{\to} c \in ℂ$ as $n \to \infty$ , then similar to Durret (2019, Theorem 3.4.2), ${(1 - n^{- 1} c_{n})}^{n} \overset{p}{\to} \exp (- c)$ .

Remark 12 (Lindeberg condition). The CLTs in Durret (2019, Theorem 3.4.10) and van der Vaart (2000, Theorem 23.4) for triangular arrays follow by verifying

E^{*} (X_{n}^{* 2}) = n^{- 1} \sum_{t = 1}^{n} E^{*} (ε_{t}^{* 2}) \overset{p}{\to} σ_{0}^{2},

in addition to the bootstrap Lindeberg condition,

γ_{n}^{*} = n^{- 1} \sum_{t = 1}^{n} * E^{*} (ε_{t}^{* 2} I (| ε_{t}^{*} | > δ n^{1 / 2})) = n^{- 1} \sum_{t = 1}^{n} {({\tilde{ε}}_{t}^{c})}^{2} I (| {\tilde{ε}}_{t}^{c} | > δ n^{1 / 2}) \overset{p}{\to} 0.

A simple way to see that $γ_{n}^{*} \overset{p}{\to} 0$ is for example to note that if (the rather strong moment condition) $E (ε_{t}^{4}) < \infty$ holds, the LLN applies to $n^{- 1} \sum_{t = 1}^{n} {({\tilde{ε}}_{t}^{c})}^{4}$ and hence,

γ_{n}^{*} = n^{- 1} \sum_{t = 1}^{n} {({\tilde{ε}}_{t}^{c})}^{2} I (| {\tilde{ε}}_{t}^{c} | > δ n^{1 / 2}) \leq \frac{1}{n δ^{2}} n^{- 1} \sum_{t = 1}^{n} {({\tilde{ε}}_{t}^{c})}^{4} \overset{p}{\to} 0.

Bootstrap Validity Under $H_{0}$

As briefly mentioned in the introduction, it is important for the application of the bootstrap that the limiting distribution (in probability) of the bootstrap test statistic has the same limiting distribution as the original test statistic when the null is true. Stated differently, we wish here to establish that under $H_{0}$ with $\bar{ρ} = ρ_{0},$

τ_{n}^{*} = W_{n}^{*} (ρ = ρ_{0}) = {({\hat{ρ}}_{n}^{*} - ρ_{0})}^{2} \sum_{t = 1}^{n} x_{t - 1}^{* 2} / σ_{0}^{2} {\overset{d^{*}}{\to}}_{p} χ_{1}^{2} .

By definition, the bootstrap estimator ${\hat{ρ}}_{n}^{*}$ is given by

{\hat{ρ}}_{n}^{*} = n^{- 1} \sum_{t = 1}^{n} x_{t}^{*} x_{t - 1}^{*} {(n^{- 1} \sum_{t = 1}^{n} x_{t - 1}^{* 2})}^{- 1},

such that by the bootstrap scheme employed, that is $x_{t}^{*} = ρ_{0} x_{t - 1}^{*} + ε_{t}^{*}$ , it follows that

n^{1 / 2} ({\hat{ρ}}_{n}^{*} - ρ_{0}) = \underset{(i)}{\underset{}{\underset{︸}{n^{- 1 / 2} \sum_{t = 1}^{n} ε_{t}^{*} x_{t - 1}^{*}}}} \underset{(i i)}{\underset{}{\underset{︸}{{(n^{- 1} \sum_{t = 1}^{n} x_{t - 1}^{* 2})}^{- 1}}}} .

Here a bootstrap CLT should be used for the first term (Should be (i) as refers to equation just above!)

a), and a bootstrap LLN for the second term (b) (Should be (ii) as refers to equation) to find the limiting behavior of the bootstrap estimator and, hence, of the test statistic $τ_{n}^{*}$ . Consider first (b), which is an average of lagged $x_{t}^{*}$ squared, with

x_{t}^{*} = ρ_{0} x_{t - 1}^{*} + ε_{t}^{*} = \sum_{i = 0}^{t - 1} ρ_{0}^{i} ε_{t - i}^{*} + ρ_{0}^{t} x_{0} .

(16)

As $ε_{t}^{*}$ depends on $n$ , and the data ${x_{t}}_{t = 0}^{n}$ , the concepts of stationarity and ergodicity—while applying to $x_{t}$ —do not apply to $x_{t}^{*}$ . However, the following lemma holds, which establishes that the LLN holds for the average of $x_{t}^{*}$ and $x_{t}^{* 2}$ .

Lemma 3. Suppose that ${x_{t}}_{t = 0}^{n}$ is given by (1) with $| ρ_{0} | < 1$ and $ε_{t}$ i.i.d. $(0, σ_{0}^{2})$ . Assume furthermore, with $ε_{t}^{*}$ iid sampled with replacement from ${{\tilde{ε}}_{t}^{c}}_{t = 1}^{n}$ and $x_{t}^{*}$ given by (10). Then, as $n \to \infty$ ,

n^{- 1} \sum_{t = 1}^{n} x_{t - 1}^{*} {\overset{p^{*}}{\to}}_{p} 0 a n d n^{- 1} \sum_{t = 1}^{n} x_{t - 1}^{* 2} {\overset{p^{*}}{\to}}_{p} ω_{0} = σ_{0}^{2} {(1 - ρ_{0}^{2})}^{- 1} .

The proof of Lemma 3 is given in the appendix. Note that for the case of $ρ_{0} = 0$ the arguments are similar to the arguments used to establish $n^{- 1} \sum_{t = 1}^{n} ε_{t}^{*} {\overset{p^{*}}{\to}}_{p} 0$ .

Remark 13 (LLN triangular arrays). Lemma 3 is the bootstrap equivalent of the weak law of large numbers for triangular arrays (see also Durret, 2019, Theorem 2.2.6).

Next, consider the CLT candidate term (b),

n^{- 1 / 2} \sum_{t = 1}^{n} ε_{t}^{*} x_{t - 1}^{*} .

As for the nonbootstrap case, with $F_{t}^{*} = σ (x_{t}^{*}, x_{t - 1}^{*},..., x_{0}^{*})$ , then (conditionally on the data) a bootstrap CLT for martingale difference arrays (mda) can be applied. In particular, $E^{*} (ε_{t}^{*} x_{t - 1}^{*} | F_{t - 1}^{*}) = x_{t - 1}^{*} E^{*} (ε_{t}^{*}) = 0$ , while for the conditional second order moment (conditional on the data), it follows by application of Lemma 3 that

n^{- 1} \sum_{t = 1}^{n} E^{*} ({(ε_{t}^{*} x_{t - 1}^{*})}^{2} | F_{t - 1}^{*}) = n^{- 1} \sum_{t = 1}^{n} x_{t - 1}^{* 2} E^{*} (ε_{t}^{* 2}) \overset{p^{*}}{\to_{p}} σ_{0}^{2} ω_{0} .

It remains to establish the bootstrap Lindeberg condition, $γ_{n}^{*} {\overset{p^{*}}{\to}}_{p} 0$ , where

γ_{n}^{*} = n^{- 1} \sum_{t = 1}^{n} E^{*} ({(ε_{t}^{*} x_{t - 1}^{*})}^{2} I (| ε_{t}^{*} x_{t - 1}^{*} | > δ \sqrt{n}) | F_{t - 1}^{*}) .

(17)

Similar to Remark 12, this follows by using that for some (arbitrarily small) $η > 0$ ,

\begin{matrix} γ_{n}^{*} \leq \frac{1}{n^{1 + η / 2} δ^{η}} \sum_{t = 1}^{n} E^{*} (| ε_{t}^{*} x_{t - 1}^{*} |^{2 + η} | F_{t - 1}^{*}) \\ = \frac{1}{n^{η / 2} δ^{η}} n^{- 1} \sum_{t = 1}^{n} | x_{t - 1}^{*} |^{2 + η} E^{*} | ε_{t}^{*} |^{2 + η} {\overset{p^{*}}{\to}}_{p} 0, \end{matrix}

which holds provided $E | ε_{t} |^{2 + η} < \infty$ , using arguments as in Lemma 3. (Note that in Remark 12 the same argument is used for $η = 2$ .)

Hence, with $ε_{t}^{*}$ iid resampled from ${{\tilde{ε}}_{t}^{c}}_{t = 1}^{n}$ , $ε_{t}$ i.i.d. $(0, σ_{0}^{2})$ with $E | ε_{t} |^{2 + η} < \infty$ , it follows that the limiting distribution (in probability) of the bootstrap MLE is given by

n^{1 / 2} ({\hat{ρ}}_{n}^{*} - ρ_{0}) {\overset{d^{*}}{\to}}_{p} N (0, σ_{0}^{2} ω_{0}^{- 1}) .

(18)

This immediately leads to the desired result:

Theorem 1. Under $H_{0}$ with $| ρ_{0} | < 1$ , and with $ε_{t}^{*}$ iid resampled from ${{\tilde{ε}}_{t}^{c}}_{t = 1}^{n}$ , with $ε_{t}$ being i.i.d. $(0, σ_{0}^{2})$ with $E | ε_{t} |^{2 + η} < \infty$ for some $η > 0$ , it holds that

τ_{n}^{*} = W_{n}^{*} (ρ = ρ_{0}) {\overset{d^{*}}{\to}}_{p} χ_{1}^{2} .

(19)

Remark 14 (Bootstrap p-value). The bootstrap p-value $p_{n, B}^{*}$ in step $C$ of the bootstrap algorithm is an approximation to the “true” bootstrap p-value $p_{n}^{*}$ , where $p_{n}^{*} = P^{*} (τ_{n}^{*} > τ_{n})$ , in the sense that $p_{n, B}^{*} \overset{p}{\to} p_{n}^{*}$ as $B$ tends to infinity. (For details in terms of the stronger concept of “almost sure” convergence, see, e.g., Cavaliere, Nielsen, & Rahbek, 2015, Remark 2.)

Remark 15 (Validity under $H_{0}$ ). Note that by Cavaliere et al. (2015, Corollary 1) for the bootstrap p-value $p_{n}^{*}$ (see Remark 14), it follows that as the limiting $χ_{1}^{2}$ distribution has a continuous distribution function, under the conditions of Theorem 1, $p_{n}^{*} \overset{d}{\to} U$ , with $U$ uniformly distributed on $[0,1]$ (Hansen, 1996, 2000).

Remark 16 (Moments of $ε_{t}$ ). Note that $E | ε_{t} |^{2 + η} < \infty$ for some $η$ , or $E (ε_{t}^{4}) < \infty$ as is often used, is required. This reflects that, as typically found for the bootstrap, further moment restrictions are used to prove bootstrap validity than in the nonbootstrap case. This is because complexities arise when applying bootstrap LLNs and arguments in connection to establishing bootstrap Lindeberg-type conditions (Cavaliere & Rahbek, 2020). However, notably, while the higher order moment conditions are sufficient for the mathematical arguments, their necessity is often not reflected in bootstrap simulations (see “Finite Sample Behavior”).

Remark 17 (Introducing $σ^{2}$ ). As for the nonbootstrap case, the result in Theorem 1 also holds for the case where $σ^{2}$ is treated as a parameter.

Remark 18 (Consistency of bootstrap standard errors). It is important to notice that the fact that $n^{1 / 2} ({\hat{ρ}}_{n}^{*} - ρ_{0})$ converges in conditional distribution (in probability) to the Gaussian distribution does not imply that the bootstrap standard error is consistent. Intuitively, this happens because convergence in distribution alone does not imply convergence of moments. Hence, the econometrician must evaluate on a case-by-case basis whether the bootstrap standard errors are consistent. An interesting result is provided by Hahn and Liao (2019), who report that the bootstrap standard error computed by simulation using $B$ bootstrap repetitions does not underestimate the population standard error (see Remark 6), and hence bootstrap inference based on the $t$ statistic coupled with the numerically computed standard errors is conservative (i.e., the type-one error associated to the bootstrap test does not exceed the user-chosen significance level in large samples).

Bootstrap Validity Under the Alternative

Consider here the convergence of the bootstrap statistic $τ_{n}^{*}$ in (11) when the alternative holds. That is, assume here that the original data are generated with true value $θ_{0} = (ρ_{0}, σ_{0}^{2})^{'}$ , but the hypothesis tested is as before $H_{0} : ρ = \bar{ρ}$ with $\bar{ρ} \neq ρ_{0}$ . As argued below, Theorem 1 holds under the alternative as well, such that

τ_{n}^{*} {\overset{d^{*}}{\to}}_{p} χ_{1}^{2} .

(20)

For the application of bootstrap-based testing, this implies that under the alternative, as $W_{n} (ρ = \bar{ρ})$ diverges while $W_{n}^{*} (ρ = \bar{ρ})$ converges in distribution, the bootstrap-based test will reject with probability tending to one. That is, asymptotic bootstrap validity holds since by Cavaliere et al. (2015, Corollary 1) the bootstrap p-value $p_{n}^{*}$ , defined in Remark 15, tends to zero in probability under the alternative, $p_{n}^{*} \overset{p}{\to} 0$ .

A key argument for (20) to hold is to note that the identity,

ε_{t}^{*} = x_{t}^{*} - \bar{ρ} x_{t - 1}^{*},

holds independently of whether $\bar{ρ}$ is the data true value $ρ_{0}$ or not. That is, $\bar{ρ}$ is by construction the bootstrap true value, such that under the null and also under the alternative, the bootstrap estimator can be rewritten as

n^{1 / 2} ({\hat{ρ}}_{n}^{*} - \bar{ρ}) = n^{- 1 / 2} \sum_{t = 1}^{n} ε_{t}^{*} x_{t - 1}^{*} {(n^{- 1} \sum_{t = 1}^{n} x_{t - 1}^{* 2})}^{- 1} .

What differs is that $ε_{t}^{*}$ under the alternative is resampled from recentered residuals ${\tilde{ε}}_{t}^{c}$ , with

{\tilde{ε}}_{t} = ε_{t} (\bar{ρ}) = x_{t} - \bar{ρ} x_{t - 1} = ε_{t} + (\bar{ρ} - ρ_{0}) x_{t - 1} \neq ε_{t} .

That is, while the identity ${\tilde{ε}}_{t} = ε_{t}$ holds under the null hypothesis when $\bar{ρ} = ρ_{0}$ , this is not the case under the alternative. Hence, to establish (20), a repeated application of the bootstrap LLN (applied to $\sum_{t = 1}^{n} x_{t - 1}^{* 2}$ ) and CLT (applied to $\sum_{t = 1}^{n} ε_{t}^{*} x_{t - 1}^{*}$ ) is needed under the alternative. For the AR process of order one considered here, the arguments are based on simple modifications of the theory under $H_{0}$ .

Remark 19 (Theory for the wild bootstrap). The same results can be shown to apply for the wild bootstrap in the case of conditional heteroskedasticity (Gonçalves & Kilian, 2004, 2007).

Finite Sample Behavior

Throughout, the focus has been on establishing asymptotic validity. This was done by verifying that the bootstrap statistic $τ_{n}^{*}$ has the same limiting distribution (in probability) as the original statistic $τ_{n}$ under the null hypothesis. Moreover, the same was argued to hold under the alternative. To illustrate the finite sample performance of the iid and wild bootstraps for the AR model, this section highlights some selected typical findings for the bootstrap based on a small and simple (to replicate) Monte Carlo study. Thus the Monte Carlo study here is not meant to be elaborate; exhaustive and detailed bootstrap Monte Carlo-based investigations are given in several papers (see, e.g., Gonçalves & Kilian, 2004, with special attention to higher order AR models, as well as the references in Cavaliere & Rahbek, 2020).

The Monte Carlo results reported here highlight the importance of the assumptions for the established validity of the bootstrap-based test of $H_{0} : ρ = \bar{ρ}$ in the autoregressive model. Specifically, it was argued that the true value of the autoregressive root $ρ_{0}$ for $x_{t}$ should satisfy $| ρ_{0} | < 1$ , and it was emphasized that $ε_{t}$ is an i.i.d. $(0, σ_{0}^{2})$ sequence, such that $E | ε_{t} |^{2 + η} < \infty$ , or rather, $E (ε_{t}^{4}) < \infty$ .

With details of the Monte Carlo designs and consideration given below, we initially mention the following findings for bootstrap simulations in the AR model with a constant term. The findings are typical for existing applications of the bootstrap and are standard in time series contexts.

(1)

With $ε_{t}$ i.i.d. $N (0, σ_{0}^{2})$ the $χ_{1}^{2}$ -based asymptotic test performs well for even small samples of size $n$ in terms of empirical rejection frequencies, or empirical size, for $ρ_{0} = 0.5$ , while for $ρ_{0} = 0.9$ the asymptotic test fails as its empirical size is not close to the nominal level $α$ . In comparison, the iid (and wild) bootstrap-based test has empirical size close to the nominal level in both cases, see Table 1 columns A and B for $ρ_{0} \in {0.5,0.9}$ , $σ_{0}^{2} = 1$ , and $n \in {15,25,...,1000}$ with $α = 0.05$ .

(2)

With $ε_{t}$ independently $N (0, σ_{t}^{2})$ distributed with $σ_{t}^{2}$ time-varying (heteroskedasticity), neither the $χ_{1}^{2}$ -based asymptotic nor the iid bootstrap-based tests have empirical size close to the nominal level $α$ , which contrasts with the wild bootstrap-based test. This is illustrated in Table 1 columns C–F for a time-changed volatility, $σ_{t}^{2} = σ_{0,1}^{2} + σ_{0,2}^{2} I (t \geq n / 2])$ , with $σ_{0,1}^{2} = 1 < σ_{0,2}^{2} = 15$ , and as before $ρ_{0} = {0.5,0.9}$ and $n \in {15,25,...,1000}$ with $α = 0.05$ .

(3)

In terms of empirical rejection frequencies under the alternative, bootstrap-based tests and the asymptotic test are comparable. This is illustrated in Table 1, with $ε_{t}$ i.i.d. $N (0, σ_{0}^{2})$ for $n = 250$ and $ρ_{0} = 0.9$ , and the test of $H_{0} : ρ = \bar{ρ}$ is considered for values of $\bar{ρ}$ ranging from $0.70$ to $0.875$ (with $\bar{ρ} = ρ_{0}$ included as a benchmark).

(4)

The wild bootstrap is often motivated by its ability to replicate underlying heteroskedasticity (see the section “Bootstrap in the Autoregressive Model” and Remark 10). This is illustrated in Figure 1, where panel A shows the empirical residuals ${\tilde{ε}}_{t}$ from one of the draws in Table 1 with $σ_{t}^{2} = σ_{0,1}^{2} + σ_{0,2}^{2} I (t \geq n / 2])$ . Figure 1, in panels B, C, and D, illustrates that, while $ε_{t}^{*}$ replicates the heteroskedasticity for the wild bootstrap, this is not the case for the iid bootstrap.

(5)

In terms of requirements for finite moments of the i.i.d. sequence $ε_{t}$ , the section “Bootstrap Validity Under $H_{0}$ ” discussed sufficiency and necessity of the condition $E | ε_{t} |^{2 + η} < \infty$ for some $η > 0$ , and it was conjectured that $E (ε_{t}^{2}) < \infty$ was sufficient. This is illustrated in Table 2, which shows that when $ε_{t}$ does not have a finite variance, then the asymptotic test, as well as the wild and iid bootstrap-based tests, fail to have correct empirical size. However, when $ε_{t}$ has a finite variance, while the asymptotic test has empirical size far from the nominal, the bootstraps work despite the lack of fourth order moments, for example. Also, Table 2 shows results from a so-called permutation bootstrap (see the section “Moment Condition, $E | ε_{t} |^{k} < \infty$ for $k \geq 2$ ”).

Asymptotic Test

We consider $x_{t}$ as given by the AR model of order one, with a constant term $δ$ included,

x_{t} = δ + ρ x_{t - 1} + ε_{t}, t = 1,2,..., n,

(21)

with $ε_{t}$ i.i.d. $N (0, σ^{2})$ and $x_{0}$ fixed. The parameters are given by $θ = (δ, ρ, σ^{2})^{'} \in Θ = ℝ^{2} \times (0, \infty)$ , with $θ_{0} = (δ_{0}, ρ_{0}, σ_{0}^{2})^{'}$ the true value, where $θ_{0} \in Θ_{0} = {θ \in Θ | | ρ_{0} | < 1}$ and the hypothesis of interest is given by $H_{0} : ρ = \bar{ρ}$ .

The unrestricted and restricted (Gaussian likelihood-based) estimators that maximize

l_{n} (θ) = - \frac{n}{2} (\log σ^{2} + n^{- 1} \sum_{t = 1}^{n} ε_{t}^{2} (ρ, δ) / σ^{2}),

with $x_{0}$ fixed and $ε_{t} (ρ, δ) = x_{t} - ρ x_{t - 1} - δ$ , are given by

{\hat{θ}}_{n} = ({\hat{δ}}_{n}, {\hat{ρ}}_{n}, {\hat{σ}}_{n}^{2})^{'} and {\tilde{θ}}_{n} = ({\tilde{δ}}_{n}, ρ_{0}, {\tilde{σ}}_{n}^{2})^{'},

respectively. The theory from the case of no constant term immediately carries over, such that for $\bar{ρ} = ρ_{0}$ , $| ρ_{0} | < 1$ and $ε_{t}$ i.i.d. $(0, σ_{0}^{2})$ , as $n \to \infty$ ,

τ_{n} = 2 (l_{n} ({\hat{θ}}_{n}) - l_{n} ({\tilde{θ}}_{n})) \overset{d}{\to} χ_{1}^{2} .

In the implementations of the asymptotic test, we use the p-value, $p_{n}$ , calculated as the tail probability of $τ_{n}$ in the limiting $χ_{1}^{2} -$ distribution, and reject if $p_{n}$ is smaller than the nominal level $α$ .

Column A in Table 1 illustrates this for $x_{t}$ in (21) generated with $δ_{0} = 0,$ $ρ_{0} = {0.5,0.9},$ and $σ_{0}^{2} = 1$ . Moreover, $ε_{t}$ is simulated as an i.i.d. $N (0, σ_{0}^{2})$ sequence, and $x_{0} = 0$ . The empirical rejection frequencies are reported based on $N = 10,000$ repetitions, with nominal level $α = 0.05$ . Results for $n \in {15,25,...,1000}$ are given in column A of Table 1. Observe, as noted, that quite a large sample is required for the limiting $χ_{1}^{2}$ distribution to be a good approximation, in particular with $ρ_{0} = 0.9$ (Duffee & Stanton, 2008, and references therein).

Remark 20 (Empirical rejection probabilities). At the chosen (nominal) level $α$ , with $q_{α}$ the corresponding $1 - α$ quantile of the limiting distribution, the true rejection probability at sample length $n$ is $α_{n} = P (τ_{n} > q_{α})$ . The Monte Carlo estimator is the empirical rejection frequency computed as $α_{n, N} = \frac{1}{N} \sum_{i = 1}^{N} I (p_{n, i} < α)$ , where $p_{n, i}$ is the p-value in Monte Carlo replication $i$ , $i = 1,2,..., N$ . It follows that the simulation uncertainty of $α_{n, N}$ is given by

V (α_{n, N}) = α_{n} (1 - α_{n}) / N

(Hendry, 1984), and for a correctly sized test, with $α_{n} = 0.05$ and $N {= 10}^{4}$ , the 95% confidence bound for $α_{n, N}$ is $[0.0456,0.0544]$ . Similar considerations hold for the bootstrap simulations of the test, with $p_{n, i}$ replaced by $p_{n, B, i}^{*}$ (see also Remark 14).

Table 1. Empirical Rejection Frequencies of Asymptotic and Bootstrap Tests in the Homoskedastic and Heteroskedastic Case

	Homoskedastic Case		Heteroskedastic Case
	(A)	(B)	(C)	(D)	(E)	(F)
$n$	Asymp.	iid boot.	Asymp.	iid boot.	Wild boot.	Wild boot.
					$N(0,1)$	Rademacher
	$ρ_{0} = \bar{ρ} = 0.5$
15	$0.0825$	$0.0512$	$0.1319$	$0.1015$	$0.0720$	$0.0462$
25	$0.0733$	$0.0541$	$0.1329$	$0.1121$	$0.0685$	$0.0496$
50	$0.0623$	$0.0519$	$0.1370$	$0.1265$	$0.0625$	$0.0486$
100	$0.0538$	$0.0511$	$0.1425$	$0.1383$	$0.0586$	$0.0488$
250	$0.0521$	$0.0508$	$0.1428$	$0.1419$	$0.0593$	$0.0557$
500	$0.0498$	$0.0478$	$0.1398$	$0.1375$	$0.0526$	$0.0491$
1000	$0.0495$	$0.0493$	$0.1434$	$0.1463$	$0.0515$	$0.0502$
	$ρ_{0} = \bar{ρ} = 0.9$
15	$0.1971$	$0.0558$	$0.1797$	$0.0597$	$0.0688$	$0.0534$
25	$0.1651$	$0.0552$	$0.1727$	$0.0688$	$0.0670$	$0.0563$
50	$0.1218$	$0.0526$	$0.1603$	$0.0875$	$0.0628$	$0.0530$
100	$0.0866$	$0.0507$	$0.1518$	$0.1016$	$0.0565$	$0.0501$
250	$0.0664$	$0.0511$	$0.1444$	$0.1226$	$0.0538$	$0.0494$
500	$0.0559$	$0.0480$	$0.1452$	$0.1323$	$0.0500$	$0.0474$
1000	$0.0548$	$0.0516$	$0.1463$	$0.1376$	$0.0515$	$0.0503$

Note. The data generating process is given by (21) with $ρ_{0} \in {0.5,0.9}$ and $δ_{0} = 0$ and the bootstrap process defined in (22). In panels (A) and (B), the innovations $ε_{t}$ are i.i.d. $N (0, σ_{0}^{2})$ distributed with $σ_{0}^{2} = 1$ , while in panels (C)–(F) $ε_{t}$ are independently $N (0, σ_{t}^{2})$ distributed, with $σ_{t}^{2}$ given in (25). The number of bootstrap replications is $B = 399$ and $N = 10,000$ replications.

The iid Bootstrap Test

To illustrate bootstrap-based testing, we apply a restricted recursive bootstrap in terms of residuals estimated under $H_{0}$ (see “Bootstrap Validity Under $H_{0}$ ”).

Specifically, for steps $A$ and $B$ in the section “Bootstrap in the Autoregressive Model,” the bootstrap samples ${x_{t}^{*}}_{t = 0}^{n}$ are sampled from

x_{t}^{*} = {\tilde{δ}}_{n} + \bar{ρ} x_{t - 1}^{*} + ε_{t}^{*}, t = 1,2,..., n,

(22)

with $x_{0}^{*} = x_{0}$ and $ε_{t}^{*}$ drawn with replacement (for wild, see the section “The Wild bootstrap test” below) from ${{\tilde{ε}}_{t}^{c}}_{t = 1}^{n},$ where ${\tilde{ε}}_{t}^{c}$ are defined as in³ (8) in terms of

{\tilde{ε}}_{t} = ε_{t} (\bar{ρ}, {\tilde{δ}}_{n}) = x_{t} - \bar{ρ} x_{t - 1} - {\tilde{δ}}_{n} .

(23)

For the bootstrap sample, ${x_{t}^{*}}_{t = 0}^{n}$ , we estimate the unrestricted and restricted models and calculate the bootstrap statistic $τ_{n}^{*}$ , given by

τ_{n}^{*} = 2 (l_{n}^{*} ({\hat{θ}}_{n}^{*}) - l_{n}^{*} ({\tilde{θ}}_{n}^{*})) .

Here, ${\hat{θ}}_{n}^{*} = ({\hat{δ}}_{n}^{*}, {\hat{ρ}}_{n}^{*}, {\hat{σ}}_{n}^{* 2})^{'}$ and ${\tilde{θ}}_{n}^{*} = ({\tilde{δ}}_{n}^{*}, \bar{ρ}, {\tilde{σ}}_{n}^{* 2})^{'}$ denote the unrestricted and restricted bootstrap estimators, respectively, in terms of the bootstrap log-likelihood function

l_{n}^{*} (θ) = - \frac{n}{2} (\log σ^{2} + n^{- 1} \sum_{t = 1}^{n} ε_{t}^{* 2} (ρ, δ) / σ^{2}), ε_{t}^{*} (ρ, δ) = x_{t}^{*} - ρ x_{t - 1}^{*} - δ .

For step $C$ , the bootstrap test is based on replicating just given arguments to obtain ${τ_{n, b}^{*}}_{b = 1}^{B}$ with $B$ denoting the number of bootstrap repetitions. As discussed in Remark 9, the empirical bootstrap p-value is computed as the tail probability,

p_{n, B}^{*} = \frac{1}{B} \sum_{b = 1}^{B} I (τ_{n, b}^{*} \geq τ_{n}) .

(24)

With $B = 399$ bootstrap repetitions, the empirical rejection frequencies for the $N = 10,000$ Monte Carlo repetitions are presented in column B in Table 1.

The Wild Bootstrap Test

The wild bootstrap design is as given in the section “iid Bootstrap Test,” except that $ε_{t}^{*}$ for the wild bootstrap is resampled by

ε_{t}^{*} = w_{t}^{*} {\tilde{ε}}_{t},

with $w_{t}^{*}$ i.i.d. $(0,1)$ distributed and ${\tilde{ε}}_{t}$ defined in (23). In the simulations, $w_{t}^{*}$ is chosen as $* N (0,1)$ and Rademacher distributed, respectively (Remark 10).

For the simulations reported in Table 1, $ε_{t}$ are assumed not to be i.i.d. $N (0, σ_{0}^{2})$ distributed in order to illustrate the impacts of heteroskedasticity. Specifically, we set $ε_{t} \overset{d}{=} N (0, σ_{t}^{2})$ with $σ_{t}^{2} = 1$ for $t = 1,2,..., [n / 2]$ and $σ_{t}^{2} = 15$ for $t = [n / 2] + 1,..., n$ . That is,

σ_{t}^{2} = σ_{0,1}^{2} + σ_{0,2}^{2} I (t > [n / 2]),

(25)

with $σ_{0,1}^{2} = 1$ and $σ_{0,2}^{2} = 15$ . In this case the asymptotic test is not consistent as demonstrated in column A of Table 1, which reports the empirical rejection probabilities for the asymptotic test. The asymptotic test is severely oversized—and even for $n = 500$ the empirical size is not close to $α = 0.05$ . Also the iid bootstrap is not asymptotically valid. With the iid bootstrap, as in the section “iid Bootstrap Test,” the results are reported in column D, and we observe that the results are similar to the asymptotic test. Intuitively, this reflects that the bootstrap series, ${x_{t}^{*}}_{t = 0}^{n}$ , does not mimic the properties of the original data series ${x_{t}}_{t = 0}^{n}$ . This can be illustrated by Figure 1, where panel A shows pronounced heteroskedasticity of the estimated residuals, ${\tilde{ε}}_{t}$ , for one sample. Panel B shows a single iid resampled sample ${ε_{t}^{*}}_{t = 1}^{n}$ from ${{\tilde{ε}}_{t}}_{t = 1}^{n}$ , and as, by definition, in particular the ordering change, the $ε_{t}^{*}$ series does not mimic the heteroskedasticity as seen in the estimated residuals in panel A.

For the wild bootstrap, columns E and F in Table 1 report the empirical rejection frequencies for the wild bootstrap test with, as mentioned, $w_{t}^{*}$ distributed as $N (0,1)$ and Rademacher, respectively. The empirical size for the wild bootstrap is quite close to the nominal level, with the Rademacher distribution performing slightly better. Likewise, panels C and D in Figure 1 illustrate that the wild bootstrap $ε_{t}^{*}$ series more closely mimisc the properties of the original ${\tilde{ε}}_{t}$ series.

Open in new tab

Figure 1. Residuals from a typical simulation with sample length $n = 500$ . Panel (A) shows a sample of restricted residuals ${\tilde{ε}}_{t}^{c},$ while panels (B)–(D) show corresponding bootstrap innovations ${ε_{t}^{*}}_{t = 1}^{n}$ for the iid bootstrap and the wild bootstrap, with the auxiliary $w_{t}^{*}$ Gaussian and Rademacher distributed, respectively.

Bootstrap Under the Alternative

To establish asymptotic validity, it was shown that $τ_{n}^{*}$ is also asymptotically $χ^{2}$ -distributed under the alternative, leading to a consistent bootstrap test. To illustrate this, we consider the empirical probability of rejecting a false hypothesis.

Specifically for the iid bootstrap test in the section “iid Bootstrap Test,” let the data $x_{t}$ be generated with true value $θ_{0} = (ρ_{0}, σ_{0}^{2}, δ_{0})^{'}$ as before. The hypothesis of interest is $H_{0} : ρ = \bar{ρ}$ and we let $\bar{ρ} \in {0.875,...,0.70}$ to illustrate the empirical power of the bootstrap test and asymptotic test. From Table 2 it follows that the bootstrap is comparable to the asymptotic test in terms of empirical power.

Table 2. Empirical Rejection Frequencies of Asymptotic and Bootstrap Tests Under the Alternative

$\bar{ρ}$	Asymptotic	iid bootstrap
$0.9$	$0.0651$	$0.0485$
$0.875$	$0.0980$	$0.0810$
$0.85$	$0.2911$	$0.2658$
$0.825$	$0.5533$	$0.5283$
$0.8$	$0.7730$	$0.7567$
$0.75$	$0.9617$	$0.9578$
$0.7$	$0.9958$	$0.9954$

Note. The data-generating process is given by (21) with $n = 250$ and $ρ_{0} = 0.9$ , such that $ρ_{0} \neq \bar{ρ}$ except for the first row entry. The number of bootstrap replications is $B = 399$ and $N = 10,000$ replications.

Moment Condition, $E | ε_{t} |^{k} < \infty$ for $k \geq 2$

When establishing asymptotic validity of the bootstrap the moment condition, $E (ε_{t}^{4}) < \infty$ was discussed, and it was mentioned that while it is a sufficient condition, it may not be necessary. On the other hand, $E (ε_{t}^{2}) < \infty$ seems necessary unless the bootstrap algorithm is based on permutation, see also Table 3 and the text before this.

To illustrate this, we consider the data $x_{t}$ as generated with true value $θ_{0} = (ρ_{0}, σ_{0}^{2}, δ_{0})^{'}$ as before for samples of size $n$ , $n \in {15,...,1000}$ . The i.i.d. innovations $ε_{t}$ are simulated from the Student’s $t_{v}$ -distribution, where the degrees of freedom $v$ , $v \in {3 / 2,3,5}$ . Specifically, for $v = 3 / 2$ , $E | ε_{t} |^{k}$ is finite, only for $k < 3 / 2$ , thus allowing first order, but not second order, moments of $ε_{t}$ . For $v = 3$ and $v = 5$ , the second and finite fourth-order moments are finite, respectively. Also note that the Gaussian case is included as a reference. Table 3 shows that the asymptotic test (based on the $χ_{1}^{2}$ approximation) for all the four cases has empirical rejection rates far from the nominal level of $α = 0.05$ , even for large samples $n$ , where $n \in {15,...,1000}$ .

As to the bootstrap design, Table 3 reports bootstrap simulations based on iid sampling and the wild (Rademacher). For $v = 3 / 2$ , as expected, neither the wild nor the iid bootstraps have empirical rejection rates close to the nominal level. For $v \geq 3$ , both the wild and iid bootstrap work surprisingly well, even when $v = 3$ .

Additionally, Table 3 reports bootstrap-based testing where $ε_{t}^{*}$ are iid sampled, but without replacement. This, which is referred to as the permutation bootstrap, works well in terms of empirical rejection frequencies, even for $v = 3 / 2$ . In general, the permutation bootstrap works well in the context of heavy-tailed i.i.d. $ε_{t}$ (see Cavaliere, Nielsen, & Rahbek, 2020, and also the discussion in “Heavy-Tailed Autoregressive Models” for AR models with heavy-tailed innovations).

Table 3. Empirical Rejection Frequencies With Heavy Tailed Innovations

	Asymptotic	Iid	Wild	Permutation
$n$	Student’s $t_{ν}, ν = 3 / 2$
15	0.1434	0.0395	0.0248	0.0499
25	0.0959	0.0382	0.0256	0.0520
50	0.0563	0.0350	0.0268	0.0488
100	0.0363	0.0340	0.0262	0.0443
250	0.0290	0.0350	0.0326	0.0480
500	0.0249	0.0333	0.0322	0.0465
1000	0.0278	0.0351	0.0348	0.0513
$n$	Student’s $t_{ν}, ν = 3$
15	0.1809	0.0508	0.0494	0.0508
25	0.1512	0.0491	0.0475	0.0522
50	0.1129	0.0494	0.0496	0.0521
100	0.0806	0.0491	0.0486	0.0505
250	0.0596	0.0487	0.0514	0.0490
500	0.0578	0.0528	0.0515	0.0534
1000	0.0499	0.0465	0.0486	0.0481
$n$	Student’s $t_{ν}, ν = 5$
	0.1894	0.0519	0.0531	0.0511
25	0.1571	0.0486	0.0515	0.0499
50	0.1119	0.0523	0.0495	0.0507
100	0.0807	0.0503	0.0497	0.0503
250	0.0639	0.0486	0.0477	0.0486
500	0.0542	0.0484	0.0478	0.0485
1000	0.0540	0.0498	0.0501	0.0503

Note. Empirical rejection frequencies of asymptotic and bootstrap tests with $ε_{t}$ i.i.d. Student’s $t_{ν}$ for $ν \in {3 / 2,3,5}$ . Results are reported for the iid bootstrap, the wild boostrap with $w_{t}$ Rademacher distributed, and the permutation bootstrap. Simulations are reported for sample lengths $n \in {15,...,1000}$ . The number of bootstrap replications is $B = 399$ and $N = 10,000$ replications.

A Selection of Further Topics

In the previous sections the simple AR model of order one was used to introduce key ideas and challenges of the iid and wild bootstrap schemes when applied to testing the hypothesis $H_{0} : ρ = \bar{ρ}$ . In this section, we provide an overview of recent selected results for the bootstrap when applied to different testing problems in econometric time series models. The overview is not meant to be exhaustive (see, e.g., Cavaliere & Rahbek, 2020, for a review of the bootstrap with more technical details, as well the references therein).

Nonstationary (Vector) Autoregressive Models

The iid Bootstrap

Consider initially the AR model in (1) again. The hypothesis of nonstationarity is given by $H_{0} : ρ = 1,$ or equivalently, with $π = ρ - 1$ , $H_{0} : π = 0$ , in the AR model restated as

Δ x_{t} = π x_{t - 1} + ε_{t},

(26)

where $Δ x_{t} = x_{t} - x_{t - 1}$ . It follows that the test-statistic $τ_{n}$ in (5) can be written as

τ_{n} = W_{n} (π = 0) = {\hat{π}}_{n}^{2} \sum_{t = 1}^{n} x_{t - 1}^{2} / σ_{0}^{2}, with {\hat{π}}_{n} = \sum_{t = 1}^{n} Δ x_{t} x_{t - 1} / \sum_{t = 1}^{n} x_{t - 1}^{2}

and under $H_{0}$ ,

τ_{n} \overset{d}{\to} τ = (\int_{0}^{1} B (u) d B (u))^{2} / \int_{0}^{1} B^{2} (u) d u,

(27)

where $B (u)$ is a standard Brownian motion, $u \in (0,1)$ , and $τ$ is the (squared) Dickey-Fuller distribution (Hamilton, 1994). In this case, in terms of the bootstrap in “Bootstrap in the Autoregressive Model,” the restricted recursive bootstrap, given by $Δ x_{t}^{*} = ε_{t}^{*}$ with $ε_{t}^{*}$ iid resampled, is asymptotically valid. This holds as in this case $τ_{n}^{*} {\overset{d^{*}}{\to}}_{p} τ$ , under both $H_{0}$ and the alternative. In contrast, and as discussed in Basawa, Mallik, McCormick, Reeves, and Taylor (1991), the unrestricted recursive bootstrap based on the recursion $Δ x_{t}^{*} = {\hat{π}}_{n} x_{t - 1}^{*} + ε_{t}^{*}$ , with $ε_{t}^{*}$ iid resampled, is invalid. This follows as the corresponding bootstrap statistic, $τ_{n}^{*}$ , under $H_{0}$ converges in distribution to $\tilde{τ}$ , $\tilde{τ} \neq τ$ . Precisely, the bootstrap conditional distribution function converges weakly rather than in probability (Basawa et al., 1991; Cavaliere & Georgiev, 2020; Cavaliere et al., 2015).

The univariate case of testing for nonstationarity is a special case of the more general hypothesis of nonstationarity in vector AR models for $X_{t} \in ℝ^{p}$ with general lag-structure, as given by

Δ X_{t} = π X_{t - 1} + \sum_{i = 1}^{k} γ_{i} Δ X_{t - i} + ε_{t},

(28)

where $ε_{t}$ are i.i.d. $N_{p} (0, Ω)$ distributed and the initial values $(X_{0}, Δ X_{0},..., Δ X_{1 - k})$ are fixed in the statistical analysis. Moreover, $π$ and ${(γ_{i})}_{i = 1}^{k}$ are $p \times p$ matrices. The hypothesis of nonstationarity of $X_{t}$ is given by the hypothesis of reduced rank $r, 0 \leq r < p$ , of $π$ (Johansen, 1996). Specifically, with $H_{r} : rank (π) \leq r$ , it follows that this may be written as

H_{r} : π = α β^{'},

where $α$ and $β$ are $(p \times r)$ dimensional matrices. Under the nonstationarity conditions in Johansen (1996, Theorem 4.2), it follows that $X_{t}$ is a nonstationary process, with $r$ stationary, or co-integrating, relations $β^{'} X_{t}$ , and $(p - r)$ common trends given by $δ^{'} \sum_{i = 1}^{t} ε_{i}$ , with $δ$ $(p \times (p - r))$ dimensional of full rank, and such that $δ^{'} α = 0$ . The likelihood-ratio statistic $τ_{n} (r)$ for co-integration rank $r$ satisfies, under $H_{r}$ and the mentioned nonstationarity conditions, that

τ_{n} (r) \overset{d}{\to} τ (r) = t r {(\int_{0}^{1} B (u) d B^{'} (u))^{'} {(\int_{0}^{1} B (u) B {(u)}^{'} d u)}^{- 1} (\int_{0}^{1} B (u) d B^{'} (u))},

which is a multivariate version of (27) in terms of the $(p - r)$ -dimensional standard Brownian motion, $B (\cdot)$ . Cavaliere, Rahbek, and Taylor (2012) consider the recursive restricted bootstrap based on

Δ X_{t}^{*} = π_{n}^{*} X_{t - 1}^{*} + \sum_{i = 1}^{k} γ_{n, i}^{*} Δ X_{t - i}^{*} + ε_{t}^{*},

with $π_{n}^{*} = {\tilde{α}}_{n} {\tilde{β}}^{'}_{n}$ and $γ_{n, i}^{*} = {\tilde{γ}}_{n, i}$ ; that is, the bootstrap true values are given by the estimators under $H_{r}$ . Asymptotic validity of the iid bootstrap is established in Cavaliere et al. (2012), by showing that $τ_{n}^{*} (r) {\overset{d^{*}}{\to}}_{p} τ (r)$ under $H_{r}$ and the alternative.

Cavaliere et al. (2015) extend the analysis to hypothesis testing on the co-integration (matrix) parameter $β$ . Specifically, Cavaliere et al. (2015, Proposition 1 and Theorem 1) establish that under the hypothesis $H_{\bar{r}} : β = \bar{β}$ , the bootstrap likelihood ratio statistic, $τ_{n}^{*}$ , satisfies $τ_{n}^{*} {\overset{d^{*}}{\to}}_{p} χ_{(p - r) r}^{2}$ . Importantly, it is also established that under the alternative, $τ_{n}^{*}$ has a limiting distribution (in distribution) in terms of a diffusion process with a stochastic diffusion coefficient, and hence it is bounded in probability such that the bootstrap-based test is asymptotically valid.

The Wild Bootstrap

In order to allow for possible heteroskedasticity in the $ε_{t}$ sequence in (28), the application of the wild bootstrap has also been studied. Results for application of the wild bootstrap in general lag univariate AR models in Gonçalves and Kilian (2004), with $ε_{t}$ allowed to have general time-varying volatility structures, such as ARCH and stochastic volatility, have been generalized to the testing the hypothesis of co-integration $H_{r}$ in Cavaliere, Rahbek, and Taylor (2010a, 2010b, 2014) and Boswijk, Cavaliere, Rahbek, and Taylor (2016).

Moreover, Boswijk et al. (2016) and Boswijk, Cavaliere, Georgiev, and Rahbek (2019) consider general hypothesis testing on the co-integration parameters $α$ and $β$ , with $π = α β^{'}$ in (28). They consider the case of stochastic volatility, where $ε_{t} = Ω_{t}^{1 / 2} z_{t}$ , with the $p$ -dimensional $z_{t}$ i.i.d. $(0,1)$ and the time-varying $(p \times p)$ -dimensional $Ω_{t} = Ω (t / n)$ . Moreover, with “ $\overset{w}{\to}$ “ denoting weak convergence, it is assumed that for $u \in (0,1)$ ,

n^{- 1 / 2} \sum_{t = 1}^{[n u]} ε_{t} \overset{w}{\to} \int_{0}^{u} Ω^{1 / 2} (s) d B (s),

(29)

where $B$ is a $p$ -dimensional standard Brownian motion, which generalizes the i.i.d. $(0, Ω)$ assumption, where $n^{- 1 / 2} \sum_{t = 1}^{[n u]} ε_{t} \overset{w}{\to} Ω^{1 / 2} B (u)$ . Specifically, the limiting process in (29) is a continuous-time martingale, with in general an unknown covariance (kernel). This implies that the limiting distribution of the test statistic(s) $τ_{n}$ , for example for the mentioned hypotheses $H_{r}$ and $H_{\bar{r}}$ , will depend on unknown nuisance parameters, which again means asymptotic inference is infeasible in practice. In contrast, for the wild bootstrap, it is established that $n^{- 1 / 2} \sum_{t = 1}^{[n u]} ε_{t}^{*}$ has the same limiting distribution (in probability), and, as a result, the wild bootstrap is asymptotically valid as shown in Boswijk et al. (2016, 2019), under some additional regularity conditions to be verified.

Time-Varying Conditional Volatility Models

As discussed in Andrews (2000), applying bootstrap-based testing in ARCH models is in general difficult, and may be invalid in certain cases, due to general problems arising when testing hypotheses in time series models when one or more parameters under the null may be “on the boundary of the parameter space” (p. 399).

To illustrate, consider here $x_{t}$ given by a linear ARCH model of order $q$

x_{t} = σ_{t} (θ) z_{t}, t = 1,..., n,

with $z_{t}$ i.i.d. $(0,1)$ and

σ_{t}^{2} (θ) = ω + \sum_{i = 1}^{q} α_{i} x_{t - i}^{2} .

In the statistical analysis, the initial values $(x_{0},..., x_{- q + 1})$ are fixed, and the parameter $θ = {(ω, α_{1},..., α_{q})}^{'} \in Θ$ , where

Θ = {θ \in ℝ^{q + 1} : ω^{2} > 0, and α_{i} \geq 0 for i = 1,..., q} .

Thus by definition of the parameter space $Θ$ , if for the true value $θ_{0} = {(ω_{0}, α_{0,1},..., α_{0, q})}^{'}$ it holds that $α_{0, j} = 0$ for some $j$ , the true value $θ_{0}$ is on the boundary of $Θ$ .

The fact that it is unknown a priori which of the ARCH coefficients may or may not be zero leads to nonpivotal limiting distributions of test statistics and estimators. Consider here the likelihood ratio statistic $τ_{n}$ for the nullity of the $q$ -th order ARCH coefficient; that is, the hypothesis $H_{q} : α_{q} = 0$ . With the Gaussian likelihood function given by

l_{n} (θ) = - \frac{1}{2} \sum_{t = 1}^{n} (\log σ_{t}^{2} (θ) + x_{t}^{2} / σ_{t}^{2} (θ)),

by definition, $τ_{n} = 2 (l_{n} ({\hat{θ}}_{n}) - l_{n} ({\tilde{θ}}_{n}))$ , where the unrestricted⁴ Gaussian MLE is given by ${\hat{θ}}_{n} = \arg \max_{θ \in Θ} l_{n} (θ)$ , while ${\tilde{θ}}_{n}$ is the Gaussian MLE under $H_{q}$ . By Andrews (1999, 2001), it follows that $τ_{n}$ has a limiting distribution, which is non-standard. In addition, the limiting distribution of $τ_{n}$ is nonpivotal as, crucially, it depends on whether $α_{0, i} > 0$ , or $α_{0, i} = 0$ for $i = 1,..., q - 1$ under $H_{q}$ .

While this implies that the unrestricted bootstrap is invalid (Andrews, 2000), it follows by Cavaliere, Nielsen, and Rahbek (2017) that the iid restricted bootstrap is asymptotically valid under mild conditions for the simple case of the first order ARCH with $q = 1$ . Moreover, Cavaliere, Nielsen, Pedersen, and Rahbek (2020) demonstrate validity of a modified restricted bootstrap, which can be applied for general testing problems in parametric models with parameters on the boundary under the null. Specifically, for the case of ARCH of order $q$ , consider the bootstrap process

x_{t}^{*} = σ_{t}^{*} (θ_{n}^{*}) z_{t}^{*},

with $z_{t}^{*}$ iid resampled from ${\hat{z}}_{t} = x_{t} / σ_{t} ({\hat{θ}}_{n})$ , after recentering and rescaling these. The bootstrap conditional volatility process $σ_{t}^{* 2} (θ_{n}^{*})$ is given by

σ_{t}^{* 2} (θ_{n}^{*}) = ω_{n}^{*} + \sum_{i = 1}^{n} α_{n, i}^{*} x_{t - 1}^{2},

(30)

with $ω_{n}^{*} = {\tilde{ω}}_{n}$ and $α_{n, i}^{*} = {\tilde{α}}_{n, i} I ({\tilde{α}}_{n, i} > c_{n})$ , with $c_{n}$ a deterministic sequence that satisfies (a) $c_{n} \to 0$ , and (b) $n^{1 / 2} c_{n} \to \infty$ , as $n \to \infty$ . The bootstrap scheme is referred to as modified since in (30) the bootstrap true values $α_{n, i}^{*}$ —by “ shrinking”—are set to zero for $i = 1,..., q - 1$ , provided ${\tilde{α}}_{n, i}$ is small relative to $c_{n}$ . Note that with the time-varying bootstrap volatility defined by (30), this is a case of a fixed design (or rather fixed volatility) bootstrap (see step $A$ in “Bootstrap in the Autoregressive Model”). The general fixed volatility bootstrap for ARCH models is considered in Cavaliere et al. (2018), and the modified, by shrinking, fixed volatility bootstrap is shown in Cavaliere, Nielsen, Pedersen, and Rahbek (2020, Proposition 1) to be asymptotically valid. Simulations there show that both the fixed volatility and the recursive with $x_{t - i}^{2} = x_{t - i}^{* 2}$ in (30) bootstrap based tests have empirical rejection frequencis that are close to the nominal level for small and moderate sample sizes $n$ . Moreover, as for the discussion of the moment requirements for the AR bootstrap, simulations indicate that while sufficient, the moment constraints on the original ARCH process imposed to establish validity are also not necessary.

Double Autoregressive Models

The double autoregressive (DAR) model combines the AR and ARCH models, as both the conditional mean and conditional variance depend on lagged levels of the process (see Ling, 2004, 2007; and for a multivariate “co-integrated” version, Nielsen & Rahbek, 2014).

Consider here the first order DAR model as given by

Δ x_{t} = π x_{t - 1} + σ_{t} (θ) z_{t}, σ_{t}^{2} (θ) = σ^{2} + α x_{t - 1}^{2}, t = 1,2,..., n,

(31)

with $z_{t}$ i.i.d. $N (0,1)$ , $x_{0}$ is fixed in the statistical analysis and the parameter given by $θ = {(π, σ^{2}, α)}^{'} \in Θ,$ with $Θ = {θ \in ℝ^{3} : σ^{2} > 0 and α \geq 0}$ .

A notable special feature of the DAR process is that for $π = 0$ the process is strictly stationary for any $0 < α < 2.42$ (Borkovec & Klüppelberg, 2001; Ling, 2004, 2007), while the process is nonstationary when $π = α = 0$ . With $π = 0$ and $α \in (0,2.42)$ , while being strictly stationary the DAR process $x_{t}$ has infinite variance (and only finite fractional moments).

From the specification of the parameter space $Θ$ evidently for $α_{0} = 0$ , the true value $θ_{0}$ is on the boundary, raising the issues discussed in the section “Time-Varying Conditional Volatility Models” in relation to the ARCH model. Asymptotic theory for the Gaussian likelihood-based MLE with $α_{0} > 0$ is given by Ling (2004), while Cavaliere and Rahbek (2020) extend the results to allow for the boundary case. Klüppelberg, Maller, van de Vyver, and Wee (2002) derive the asymptotic distribution of the likelihood ratio statistic $τ_{n}$ for the hypothesis of nonstationarity as given by $H_{0} : π = α = 0$ (see also Chen, Li, & Ling, 2013). Different versions of a bootstrap-based test are discussed in Cavaliere and Rahbek (2020). In particular, validity is established for a restricted bootstrap given by $Δ x_{t}^{*} = {\tilde{σ}}_{n} z_{t}^{*}$ , where ${\tilde{σ}}_{n}^{2}$ is the MLE of $σ^{2}$ under $H_{0}$ , while $z_{t}^{*}$ is obtained by iid resampling of unrestricted residuals, ${\hat{z}}_{t}$ (recentered and rescaled). That is, the unrestricted residuals ${\hat{z}}_{t}$ are given by

{\hat{z}}_{t} = (Δ x_{t} - {\hat{π}}_{n} x_{t - 1}) / σ_{t} ({\hat{θ}}_{n}),

with ${\hat{θ}}_{n} = {({\hat{π}}_{n}, {\hat{σ}}_{n}, {\hat{α}}_{n})}^{'}$ the unrestricted MLE. In line with the discussion in the section “Bootstrap in the Autoregressive Model” regarding restricted and unrestricted residuals, this choice ensures that ${\hat{z}}_{t}$ for large $n$ is “close” to the true $z_{t}$ , irrespective of whether the null $H_{0}$ is true or not. While the validity result is shown for this choice, simulations indicate that in practice the difference between choosing to resample from ${\hat{z}}_{t}$ or from ${\tilde{z}}_{t} = Δ x_{t} / {\tilde{σ}}_{n}$ is negligible. (See Cavaliere & Rahbek, 2020, for a detailed discussion of this as well as asymptotic theory for different bootstraps.)

Heavy-Tailed Autoregressive Models

So far results for the (vector) AR models have been derived under the assumption that the innovations $ε_{t}$ have mean zero and a finite variance $σ^{2}$ , or some time-varying, possibly conditional, variance $σ_{t}^{2}$ when discussing heteroskedasticity. To allow for more extreme events, and phenomena such as “ bubble” periods with local explosive behavior, this assumption was relaxed in Davis and Resnick (1985a, 1985b, 1986) and Davis and Song (2020), where the i.i.d. innovations $ε_{t}$ are allowed to have infinite variance. Specifically, they consider the case of $ε_{t}$ i.i.d. with a stable distribution such as the Cauchy; that is, “heavy-tailed” as the tails of the distribution of $ε_{t}$ are assumed to decay at a rate that is slower than the Gaussian (exponential) rate.

Two key examples are given by the classic AR model and the so-called noncausal AR model of order one in terms of i.i.d. stable distributed $ε_{t}$ ,

AR : x_{t} = ρ x_{t - 1} + ε_{t}, and {AR}^{+} : x_{t} = ρ^{+} x_{t + 1} + ε_{t} .

(32)

For the standard, and hence causal, $AR$ , recall that with $t = 1,..., n$ , $x_{0}$ is the initial value that is fixed in the statistical analysis, while for the noncausal ${AR}^{+}$ $x_{n}$ is the “initial value” due to the forward recursion. Noncausal ${AR}^{+}$ type models have become popular as they seem to capture well the dynamics of phenomena such as bubbles where, after period of exponential type growth, the process “collapses.” Interestingly, and linking the heavy-tail ${AR}^{+}$ models with the DAR model in “Double Autoregressive Models,” the ${AR}^{+}$ process in (32) can be shown to have a causal “semi-strong” representation as the DAR process in (31) (see Gourieroux & Zakoian, 2017).

Consider testing the hypothesis $H_{0}^{+} : ρ^{+} = \bar{ρ}$ using the Gaussian likelihood-based statistic $τ_{n}^{+}$ , given by

τ_{n}^{+} = ({\hat{ρ}}_{n}^{+} - \bar{ρ}) (\sum_{t = 1}^{n - 1} x_{t + 1})^{1 / 2} / {\hat{σ}}_{n},

where ${\hat{ρ}}_{n}^{+} = \sum_{t = 1}^{n - 1} x_{t} x_{t + 1} / \sum_{t = 1}^{n} x_{t + 1}^{2}$ and ${\hat{σ}}_{n}^{2} = n^{- 1} \sum_{t = 1}^{n - 1} {(x_{t} - {\hat{ρ}}_{n}^{+} x_{t - 1})}^{2}$ . While the test statistic $τ_{n}^{+}$ is analogous to the (square root of the) previously studied statistic $τ_{n}$ in (5), the limiting distribution is nonstandard, as the $ε_{t}$ are assumed to be i.i.d. stable distributed. For example, with $ε_{t}$ Cauchy distributed, $(n / \log n) ({\hat{ρ}}_{n}^{+} - \bar{ρ})$ is asymptotically distributed as $(1 + \bar{ρ}) C χ_{1}^{2}$ , where $C$ is standard Cauchy distributed and $τ_{n}^{+} = O_{P} (n^{- 1 / 2} \log n)$ . In general for stable distributions asymptotic testing is infeasible as the limiting distributions depend on the “tail index” (which is one for the Cauchy) of the innovations $ε_{t}$ , and moreover the normalization (which is $n^{- 1 / 2} \log n$ for the Cauchy case) depend on further, and in practice, unknown quantities. In the case of stable distributed innovations, Cavaliere, Nielsen, and Rahbek (2020) discuss validity of the recursive bootstrap scheme similar to (10),

x_{t}^{*} = \bar{ρ} x_{t + 1}^{*} + ε_{t}^{*},

initialized with $x_{n}^{*} = x_{n}$ , and with $ε_{t}^{*}$ resampled from the restricted residuals ${{\tilde{ε}}_{t}^{+}}_{t = 1}^{n - 1}$ , where ${\tilde{ε}}_{t}^{+} = x_{t} - \bar{ρ} x_{t + 1}$ . Crucially the $ε_{t}^{*}$ are not sampled by iid resampling with replacement, as this would lead to an invalid bootstrap test (Athreya, 1987; Knight, 1989). Instead, $ε_{t}^{*}$ are resampled without replacement; that is, by permuting ${{\tilde{ε}}_{t}^{+}}_{t = 1}^{n - 1}$ , or, in combination with the wild, by permuting ${{\tilde{ε}}_{t}^{+} w_{t}^{*}}_{t = 1}^{n - 1}$ , where $w_{t}^{*}$ are i.i.d. Rademacher distributed (Remark 10). With $τ_{n}^{+ *}$ the bootstrap statistic based on the permutation, or the combined permutation-wild bootstrap, Theorem 1 in Cavaliere, Nielsen, and Rahbek (2020) establishes validity of the bootstrap-based test under the null hypothesis for general ${AR}^{+}$ models.

Similarly, Cavaliere, Georgiev, and Taylor (2016) establish bootstrap validity for the so-called sieve bootstrap in Bühlmann (1997) for general causal AR models with heavy tails. Also note that in terms of testing for the presence of bubbles based on one-sided testing using the supremum of recursively computed Dickey-Fuller type statistics as in Phillips, Wu, and Yu (2011), Harvey, Leybourne, Sollis, and Taylor (2016) establish validity of a wild bootstrap based test to allow for heteroskedasticity.

Conclusions and Further Readings

This article has provided an introduction to key steps required for a successful implementation of bootstrap hypothesis testing to time series models. In the framework of a simple autoregressive model, we have discussed the (large-sample) validity of recursive bootstrap algorithms, where the bootstrap sample is constructed by mimicking the dependence structure of the original data, as implied by the econometric model at hand. We have discussed the main requirements for bootstrap inference to be valid; that is, to mimic the asymptotic distribution of the original test statistic under the null hypothesis and to be bounded (in probability) when the null hypothesis is false. To do so, we have introduced the needed probability tools (which involve dealing with the conditional—hence random—nature of the bootstrap measure) and shown how to apply them in order to assess the large sample behavior of the bootstrap estimators and test statistics.

Needless to say, this article is not meant to be exhaustive or to provide a comprehensive treatment of bootstrap inference. We have focused on bootstrap hypothesis testing which, although being one of the most common applications of the bootstrap, is not the only one. Further common implementations of the bootstrap involve computation of standard errors, the construction of confidence intervals for the parameters of interest, and bias correction of the estimators (see, inter alia, Horowitz, 2001, or MacKinnon, 2006, for reviews).

This article is also not exhaustive in terms of bootstrap algorithms. For instance we have not discussed bootstrap algorithms based on resampling a block of observations (or residuals) rather than resampling single observations (or residuals). As introduced by Künsch (1989), block bootstrap methods are quite flexible and powerful in cases where one does not have strong a priori beliefs or knowledge about the dependence structure of the data (see also Lahiri, 2003; Politis & Romano, 1994; Shao, 2010). Subsampling methods represent a valid alternative (or complement) to recursive, model-based bootstrap (Politis, Romano, & Wolf, 1999). We have focused on a case (the first order AR model) where the data-generating process is described by a finite dimensional vector of parameters. Extensions to infinite-dimensional parameter spaces, as for the case of general linear processes, are available in the literature; a classic example is the $AR (\infty)$ sieve bootstrap of Kreiss (1988, 1992), Bühlmann (1997), and Gonçalves and Kilian (2007).

It is important to emphasize that there is a rich literature discussing finite-sample improvements of the bootstrap in various models, including time series models, based on Edgeworth expansions of the $τ_{n}$ and $τ_{n}^{*}$ statistics, as briefly mentioned in Remark 10. An introduction to Edgeworth expansions as applied for the bootstrap can be found in van der Vaart (2000, chapter 23.3; see also Hall, 1992; Horowitz, 2001).

Finally, it is worth noticing that throughout this article bootstrap validity under the null is defined as the fact that under the null the bootstrap test statistic converges to the asymptotic (null) distribution of the original statistic. This implies that, under rather mild regularity conditions (such as continuity of the limiting distribution), if the null hypothesis holds then the bootstrap $p$ -value $p_{n}^{*}$ is asymptotically uniformly distributed (Remark 15). Combined with establishing that the bootstrap statistic is bounded in probability under the alternative such that $p_{n}^{*} \overset{p}{\to} 0$ , this was used to establish validity of the bootstrap (see “Bootstrap Validity Under the Alternative”). Cavaliere and Rahbek (2020) discuss in detail different definitions of asymptotic validity and their verification under different assumptions. An example is Cavaliere and Georgiev (2020), where validity of the bootstrap test is defined directly in terms of asymptotic uniformity of the bootstrap $p$ -values, rather than the properties of (consistent) estimation of the limiting null distribution of the original test statistic. By doing so, the bootstrap can be employed in cases where the limiting distribution of the original statistic may not be well defined, or in cases where the boostrap distribution does not converge in probability but, rather, converges in distribution (see, e.g., Boswijk, Cavaliere, Georgiev, & Rahbek, 2019, and the discussions in sections “Nonstationary [Vector] Autoregressive Models” and “Heavy-Tailed Autoregressive Models” where this was applied).

Browse All

Share Link

Article contents

An Introduction to Bootstrap Theory in Time Series Econometrics

Summary

Keywords

Bootstrap in Time Series Econometrics

The Autoregressive Model

Asymptotics for the Autoregressive Model

Bootstrap in the Autoregressive Model

Asymptotic Theory for the Recursive Bootstrap

Bootstrap Probability, Expectation, and Convergence

Bootstrap Validity Under $H_{0}$

Bootstrap Validity Under the Alternative

Finite Sample Behavior

Asymptotic Test

The iid Bootstrap Test

The Wild Bootstrap Test

Bootstrap Under the Alternative

Moment Condition, $E | ε_{t} |^{k} < \infty$ for $k \geq 2$

A Selection of Further Topics

Nonstationary (Vector) Autoregressive Models

The iid Bootstrap

The Wild Bootstrap

Time-Varying Conditional Volatility Models

Double Autoregressive Models

Heavy-Tailed Autoregressive Models

Conclusions and Further Readings

Acknowledgments

References

Appendix

Proof of Lemma 3

Notes

Related Articles

Browse All

Share Link

Article contents

An Introduction to Bootstrap Theory in Time Series Econometrics

Summary

Keywords

Bootstrap in Time Series Econometrics

The Autoregressive Model

Asymptotics for the Autoregressive Model

Bootstrap in the Autoregressive Model

Asymptotic Theory for the Recursive Bootstrap

Bootstrap Probability, Expectation, and Convergence

Bootstrap Validity Under H0

Bootstrap Validity Under the Alternative

Finite Sample Behavior

Asymptotic Test

Table 1. Empirical Rejection Frequencies of Asymptotic and Bootstrap Tests in the Homoskedastic and Heteroskedastic Case

The iid Bootstrap Test

The Wild Bootstrap Test

Bootstrap Under the Alternative

Table 2. Empirical Rejection Frequencies of Asymptotic and Bootstrap Tests Under the Alternative

Moment Condition, E|εt|k<∞ for k≥2

Table 3. Empirical Rejection Frequencies With Heavy Tailed Innovations

A Selection of Further Topics

Nonstationary (Vector) Autoregressive Models

The iid Bootstrap

The Wild Bootstrap

Time-Varying Conditional Volatility Models

Double Autoregressive Models

Heavy-Tailed Autoregressive Models

Conclusions and Further Readings

Acknowledgments

References

Appendix

Proof of Lemma 3

Notes

Related Articles

Bootstrap Validity Under $H_{0}$

Moment Condition, $E | ε_{t} |^{k} < \infty$ for $k \geq 2$