An Introduction to Bootstrap Theory in Time Series Econometrics
 Giuseppe Cavaliere, Giuseppe CavaliereDepartment of Economics, University of Bologna; Department of Economics, University of Exeter
 Heino Bohn NielsenHeino Bohn NielsenDepartment of Economics, University of Copenhagen
 and Anders RahbekAnders RahbekDepartment of Economics, University of Copenhagen
Summary
While often simple to implement in practice, application of the bootstrap in econometric modeling of economic and financial time series requires establishing validity of the bootstrap. Establishing bootstrap asymptotic validity relies on verifying often nonstandard regularity conditions. In particular, bootstrap versions of classic convergence in probability and distribution, and hence of laws of large numbers and central limit theorems, are critical ingredients. Crucially, these depend on the type of bootstrap applied (e.g., wild or independently and identically distributed (i.i.d.) bootstrap) and on the underlying econometric model and data. Regularity conditions and their implications for possible improvements in terms of (empirical) size and power for bootstrapbased testing differ from standard asymptotic testing, which can be illustrated by simulations.
Bootstrap in Time Series Econometrics
Bootstrap in econometrics is frequently applied in the context of estimation and testing (Berkowitz & Kilian, 2000; Cavaliere & Rahbek, 2020; Davidson & MacKinnon, 2006; Horowitz, 2001, 2003; MacKinnon, 2009). As an example, consider the case where some test statistic, ${\tau}_{n\phantom{\rule{0.2em}{0ex}}}$ say, is of interest given a sample of (timeseries) data ${x}_{1}\mathrm{,...,}{x}_{n}$ with initial values ${x}_{0},{x}_{1}\mathrm{,..,}{x}_{p}$ for $p\ge 0$, ${\left\{{x}_{t}\right\}}_{t=p}^{n}\phantom{\rule{0.1em}{0ex}}$ with ${x}_{t}\in \mathbb{R}$. Under suitable regularity conditions, including typically (a) stationarity and ergodicity of the ${x}_{t}$ process and (b) finite moments conditions on the form $E{x}_{t}^{2}{}^{k}<\infty $ for some $k\ge 1$, it holds that under the null hypothesis of interest, ${\mathcal{H}}_{0}$ say,
where $q$ denotes the degrees of freedom and “$\stackrel{d}{\to}$” denotes convergence in distribution. Moreover, under the alternative, or when ${\mathcal{H}}_{0}$ is not true, the statistic ${\tau}_{n}$ diverges. In standard asymptotic testing, the ${\chi}_{q}^{2}$ distributional approximation of the test statistic ${\tau}_{n}$ is frequently applied using the $1\alpha $ quantile of the ${\chi}_{q}^{2}$ distribution or, equivalently, by calculating the pvalue.
A bootstrapbased test is often motivated by noting that the asymptotic approximation may not be good in finite samples, which may lead, for a finite number of observations $n$, to an actual size larger, or smaller, than the nominal level $\alpha $. This is often corrected by applying even simple bootstrap schemes. An additional motivation for the bootstrap is that the underlying model used for estimation—and on which the derivation of ${\tau}_{n}$ is based—may be misspecified. A typical example is the assumption of homoskedasticity of model innovations, which often in practice is challenged in the modeling of macro and financial data, where (conditional and unconditional) heteroskedasticity is typically present. In this case, the socalled wild bootstrap may correct for such heteroskedasticity. Likewise, for other types of misspecification and model departures, different bootstrap schemes may be applied depending on which type of misspecification is of concern. In addition, in some cases the limiting distribution of ${\tau}_{n}$ cannot be tabulated, for example, because it is a function of unknown nuisance parameters. The bootstrap may resolve such issues. However, it should be emphasized that any bootstrapbased test is—as for the original asymptotic test—only valid under certain regularity conditions. These regularity conditions are important to check and understand as in many cases application of what may be thought of as a “ standard bootstrap” may be invalid and misleading, despite its popularity in many empirical applications. Typical situations in which a standard bootstrap does not work can be found in the context of nonstationary variables and when data exhibit heavytailed distributions, as is typical in financial data. When application of existing, or standard, bootstrap schemes fails to work, these may be corrected by more elaborate bootstrap schemes, as is richly documented in recent research in econometrics (see inter alia Cavaliere & Rahbek, 2020, and the references therein).
The bootstrap is a simulationbased approach that is typically simple to apply in the context of inference and testing. Based on some bootstrap scheme, the bootstrap is based on generating new bootstrap samples of data and the test statistic of interest, denoted by ${\left\{{x}_{t}^{\ast}\right\}}_{t=p}^{n}$ and ${\tau}_{n}^{\ast}$, respectively. The (re)generation of such bootstrap data is based on keeping the original sample ${\left\{{x}_{t}\right\}}_{t=p}^{n}$ fixed.
If the limiting distribution (in the bootstrap sense) of the bootstrap test statistic ${\tau}_{n}^{\ast}$ is identical to the limiting distribution of the original statistic ${\tau}_{n}$ under the null of ${\mathcal{H}}_{0}$, such as the ${\chi}_{q}^{2}$ distribution noted earlier, this implies validity under the null of the bootstrap. If, in addition, under the alternative when ${\mathcal{H}}_{0}$ is not true, the bootstrap statistic ${\tau}_{n}^{\ast}$ does not diverge as the sample size $n$ increases (or, at most it diverges at a slower rate than the original statistic), the bootstrap is asymptotically valid. This is the case when under the alternative the bootstraplimiting distribution of ${\tau}_{n}^{\ast}$ is identical to the limiting distribution of the original statistic or when ${\tau}_{n}^{\ast}$ is bounded in (bootstrap) probability.
As a leading example throughout, and to fix ideas, we consider the firstorder autoregressive (AR) model. The section “Autoregressive Model” provides a summary of standard econometric nonbootstrap analysis of the AR model before turning to the discussion of the implementation and (asymptotic) theory of the bootstrap in the section “Bootstrap in the Autoregressive Model.” Next, a Monte Carlo study is used to discuss finite sample behavior of different bootstrap schemes (see “Finite Sample Behavior”) and recent advances in econometric time series bootstrap analysis are provided (see “Selection of Further Topics”). The conclusion provides an overview of some important further topics and approaches to bootstrap inference.
The Autoregressive Model
In order to present bootstrap theory and arguments, a brief summary of results and arguments for the (nonbootstrap) AR model of order one is provided. The results and asymptotic arguments presented here are well known (see, e.g., Hamilton, 1994; Hayashi, 2000).
Consider the AR model of order one
with the initial value ${x}_{0}$ fixed in the statistical analysis and the innovations${\epsilon}_{t}$ are assumed to be independently and identically distributed (i.i.d). $\text{N}\left(\mathrm{0,}{\sigma}^{2}\right)$ (see also Remark 2 regarding relaxing the assumption of Gaussianity). The parameter of the model is $\theta ={\left(\rho ,{\sigma}^{2}\right)}^{\prime}$ with ${\theta}_{0}={\left({\rho}_{0},{\sigma}_{0}^{2}\right)}^{\prime}$ denoting the true value. For estimation, the parameter space is given by
For the true parameter ${\theta}_{0}$, we assume ${\theta}_{0}\in {\Theta}_{0}\subset \Theta $, with ${\Theta}_{0}=\left\{\theta \in \Theta {\rho}_{0}<1\right\}$, such that ${x}_{t}$ in (1) for ${\theta}_{0}\in {\Theta}_{0}$ has a stationary and geometrically ergodic solution given by
With $\overline{\rho}$ some fixed value, consider testing the hypothesis ${\mathcal{H}}_{0}$ given by
The test is based on the likelihood ratio test statistic, ${\tau}_{n}$, defined in terms of the Gaussian loglikelihood function
with ${x}_{0}$ fixed and ${\epsilon}_{t}\left(\rho \right)={x}_{t}\rho {x}_{t1}$. Standard optimization gives the unrestricted maximum likelihood estimator (MLE), ${\widehat{\theta}}_{n}=({\widehat{\rho}}_{n},{\widehat{\sigma}}_{n}^{2}{)}^{\prime}$, where
The restricted estimator ${\tilde{\theta}}_{n}={\left({\tilde{\rho}}_{n},{\tilde{\sigma}}_{n}^{2}\right)}^{\prime}$ is obtained by maximization under ${\mathcal{H}}_{0}$, and is given by
It follows that the likelihood ratio (LR) test statistic is given by
Moreover, under ${\mathcal{H}}_{0}$ and regularity conditions detailed in the section “Asymptotics for the Autoregressive Model,” with $\overline{\rho}={\rho}_{0}$ and ${\theta}_{0}\in {\Theta}_{0}$ it holds that ${\tau}_{n}\stackrel{d}{\to}{\chi}_{1}^{2}$. Note in this respect that as long as the true ${\rho}_{0}$ is not “ too large” the asymptotic ${\chi}_{1}^{2}$ approximation is a good approximation even for small samples. In contrast, and as exemplified in Table 1, if either ${\epsilon}_{t}$ is not i.i.d. $\text{N}\left(\mathrm{0,}{\sigma}^{2}\right)$ or if ${\rho}_{0}=0.9$ the ${\chi}_{1}^{2}$ distribution is not a good approximation of the actual distribution of ${\tau}_{n}$ for small, or even moderate, finite samples of size $n$.
Remark 1 (Wald and t statistics). Although focus is on the likelihood ratio test statistics, other statistics could be considered as well. For instance, the classic Wald statistic (${W}_{n}$) is given by
and, under the same regularity conditions required for the likelihood ratio test, when $\overline{\rho}={\rho}_{0}$ and ${\theta}_{0}\in {\Theta}_{0}$, it holds that ${W}_{n}\left(\rho =\overline{\rho}\right)\stackrel{d}{\to}{\chi}_{1}^{2}$. It is also customary to consider the $t$ratio statistic, defined as
where the socalled standard error $\text{se}\left({\widehat{\rho}}_{n}\right)$ is defined as ${({\widehat{V}}_{n}/n)}^{1/2}$, with
an unrestricted (i.e., obtained without imposing the null hypothesis) estimator of the asymptotic variance of ${n}^{1/2}({\widehat{\rho}}_{n}{\rho}_{0})$. Notice that ${t}_{n}\left(\rho =\overline{\rho}\right)=\text{sign}\left({\widehat{\rho}}_{n}\overline{\rho}\right){\left({W}_{n}\left(\rho =\overline{\rho}\right)\right)}^{1/2}$ and, when $\overline{\rho}={\rho}_{0}$ and ${\theta}_{0}\in {\Theta}_{0}$, ${t}_{n}\left(\rho =\overline{\rho}\right)\stackrel{d}{\to}N\left(\mathrm{0,1}\right)$.
Remark 2 (Quasi likelihood). Often if the ${\epsilon}_{t}$ sequence is assumed to be i.i.d., Gaussian is relaxed to i.i.d. $(0,{\sigma}^{2})$ for some unknown distribution. When the Gaussian loglikelihood function in (2) in this case is used to obtain estimators and test statistics, these are referred to as (Gaussian) quasi MLE and quasi LR statistics, respectively. This is also the case when ${\epsilon}_{t}$ is assumed to be mean zero and (conditionally or unconditionally) heteroskedastic, as for example with some autoregressive conditional heteroskedastic (ARCH) specification or with a structural break in the variance (see the section “Finite Sample Behavior”).
Asymptotics for the Autoregressive Model
To ease the presentation of the key arguments for the standard nonbootstrap asymptotic analysis, assume here without loss of generality that ${\sigma}^{2}$ is fixed at the true value—that is, ${\sigma}^{2}={\sigma}_{0}^{2}$. Accordingly, the parameter is $\theta =\rho $ and the likelihood function in (2) simplifies as
The MLE ${\widehat{\rho}}_{n}$ is given by (3), while the $L{R}_{n}\left(\rho =\overline{\rho}\right)$ statistic in (4) in this case is identical to the Wald statistic $({W}_{n})$
By construction, ${\tau}_{n}$ is a simple function of ${n}^{1/2}\left({\widehat{\rho}}_{n}\overline{\rho}\right)$ and ${n}^{1}{\displaystyle {\sum}_{t=1}^{n}}{x}_{t1}^{2}$, and the limiting distribution is found by applying a law of large numbers (LLN) to the average ${n}^{1}{\displaystyle {\sum}_{t=1}^{n}}{x}_{t1}^{2}$, as well as a central limit theorem (CLT) to ${n}^{1/2}\left({\widehat{\rho}}_{n}\overline{\rho}\right)$.
With $\overline{\rho}={\rho}_{0}\in {\Theta}_{0}$, the AR process ${x}_{t}$ is stationary and ergodic, and has finite variance, $V\left({x}_{t}\right)={\omega}_{0}={\sigma}_{0}^{2}/\left(1{\rho}_{0}^{2}\right)$. In particular, it follows by the LLN for stationary and ergodic processes that
Next, by definition of the MLE,
Here ${m}_{t}={\epsilon}_{t}{x}_{t1}$ is a martingale difference sequence (mds) with respect to the filtration ${\mathcal{F}}_{t}$, where ${\mathcal{F}}_{t}=\sigma \left({x}_{t},{x}_{t1}\mathrm{...}\right)$, as $E{m}_{t}<\infty $, and $E\left({m}_{t}{\mathcal{F}}_{t1}\right)=0$. Moreover, the conditional second order moment converges in probability,
This implies by the CLT for mds that ${n}^{1/2}{\displaystyle {\sum}_{t=1}^{n}}{m}_{t}\stackrel{d}{\to}\text{N}\left(\mathrm{0,}{\sigma}_{0}^{2}{\omega}_{0}\right)$ (Hall & Heyde, 1980; Hamilton, 1994) and such that
Remark 3 (Lindeberg condition). Note that for a CLT for mds in general to hold (Hall & Heyde, 1980), a Lindebergtype condition of the form ${\gamma}_{n}={n}^{1}{\displaystyle {\sum}_{t=1}^{n}}E\left({m}_{t}^{2}\mathbb{I}\left({m}_{t}>\delta {n}^{1/2}\right)\right)\to 0$ for any $\delta >0$ must hold, where $\mathbb{I}(\cdot )$ is the indicator function. This holds here as, by stationarity,
which tends to zero as $n\to \infty $ by dominated convergence under the moment condition $\text{E}\left({m}_{t}^{2}\right)<\infty $ as implied by $\text{E}\left({\epsilon}_{t}^{2}\right)<\infty $ and independence of ${\epsilon}_{t}$ and ${x}_{t1}$.
Collecting terms, it follows that we have the following result.
Lemma 1. For the AR(1) model in (1) with ${\theta}_{0}\in {\Theta}_{0}$ and ${\epsilon}_{t}$ i.i.d. $\left(\mathrm{0,}{\sigma}_{0}^{2}\right)$, it follows that with ${\tau}_{n}$ given by (5), under the null ${\tau}_{n}\stackrel{d}{\to}{\chi}_{1}^{2}$ as $n\to \infty $.
Note that in Lemma 1, it is not assumed that the ${\epsilon}_{t}$ sequence is Gaussian, but instead the lemma is formulated in terms of the milder sufficient condition that ${\epsilon}_{t}$ is an i.i.d. $\left(\mathrm{0,}{\sigma}^{2}\right)$ sequence (Remark 2). Thus the essential regularity conditions for the lemma to hold are that ${\rho}_{0}<1$ and ${\epsilon}_{t}$ i.i.d. $\left(\mathrm{0,}{\sigma}^{2}\right)$.
Remark 4 (Heteroskedasticity). Note that if ${\epsilon}_{t}={\sigma}_{t}{z}_{t}$, with ${z}_{t}$ i.i.d. $\text{N}\left(\mathrm{0,1}\right)$ and ${\sigma}_{t}^{2}$ given by an ARCH process with $\text{E}\left({\sigma}_{t}^{2}\right)={\sigma}_{\epsilon}^{2}<\infty $, it follows that ${\tau}_{n}\stackrel{d}{\to}c{\chi}_{1}^{2}$, with $c={\sigma}_{\epsilon}^{2}/{\sigma}_{0}^{2}$. This reflects in general that if ${\epsilon}_{t}$ are (conditionally, or unconditionally) heteroskedastic, the limiting distribution of the test statistic ${\tau}_{n}$ is not ${\chi}_{1}^{2}$. This is wellknown in the context of regression models, and corrections of the test statistic (White, 1980) are typically applied to ensure valid asymptotic inference; the bootstrap, or more precisely, the wild bootstrap discussed later, may correct for such misspecification without the need to correct the original test statistic (cf. Gonçalves & Kilian, 2004).
Remark 5 (Test statistic). For ${\sigma}^{2}$ an unknown parameter to be estimated, such that $\theta ={\left({\sigma}^{2},\rho \right)}^{\prime}$, note that the LR statistic in (4) can be written as
with ${\tilde{\sigma}}_{n}^{2}={n}^{1}{\displaystyle {\sum}_{t=1}^{n}}{\epsilon}_{t}^{2}\left(\overline{\rho}\right)$. By Lemma 1, ${w}_{n}={n}^{1}{W}_{n}\left(\rho =\overline{\rho}\right)\stackrel{p}{\to}0$, and by a Taylor expansion, $\mathrm{log}\left(1w\right)=w+o\left(w\right)$, with $o\left(w\right)$ a term that tends to zero as $w\to 0$. Hence,
where ${o}_{p}\left(1\right)$ denotes a term that converges to zero in probability as $n\to \infty $ (see, e.g., van der Vaart, 2000, Lemma 2.12, for details on ${o}_{p}(\cdot )$ (and ${O}_{p}(\cdot )$ notation and the stochastic [Taylor] expansion as applied here).
Bootstrap in the Autoregressive Model
The general idea behind a bootstrap algorithm as implemented in the context of likelihoodbased testing in the first order AR model can be summarized as follows:
Step A. With the original data ${\left\{{x}_{t}\right\}}_{t=0}^{n}$ fixed, generate a sample of bootstrap data ${\left\{{x}_{t}^{\ast}\right\}}_{t=0}^{n}$ using some bootstrap scheme with bootstrap true parameter, ${\theta}_{n}^{\ast}={\left({\rho}_{n}^{\ast},{\sigma}_{n}^{\ast 2}\right)}^{\prime}$. Specifically, for the AR model, set ${x}_{0}^{\ast}={x}_{0}$ and generate ${x}_{t}^{\ast}$ recursively by
with ${z}_{t}={x}_{t1}^{\ast}$ for a recursive bootstrap scheme, while ${z}_{t}={x}_{t1}$ for a fixed design bootstrap scheme. As to the choice of bootstrap innovations ${\epsilon}_{t}^{\ast}$ in terms of the original data ${\left\{{x}_{t}\right\}}_{t=0}^{n}$ and a bootstrap sampling distribution as detailed after step C.
Step B. Compute the bootstrap (quasi) MLE$\phantom{\rule{0.2em}{0ex}}{\widehat{\rho}}_{n}^{\ast}$ and the bootstrap LR statistic ${\tau}_{n}^{\ast}=L{R}_{n}^{\ast}\left(\rho ={\rho}_{n}^{\ast}\right)$, where the bootstrap loglikelihood function ${\mathcal{l}}_{n}^{\ast}\left(\theta \right)$ as a function of $\theta $ is given by
Step C. Generate ${\left\{{x}_{t,b}^{\ast}\right\}}_{t=0}^{n}$, ${\widehat{\theta}}_{n,b}^{\ast}$ and ${\tau}_{n,b}^{\ast}$ for $b=\mathrm{1,2,...,}B$ by repeating Steps $\mathcal{A}$ and $\mathcal{B}$, and use the empirical distribution of ${\left\{{\tau}_{n,b}^{\ast}\right\}}_{b=1}^{B}$ for testing based on the original statistic ${\tau}_{n}=L{R}_{n}\left(\rho ={\rho}_{0}\right)$. Precisely, with the bootstrap pvalue set to ${p}_{n,B}^{\ast}={B}^{1}{\displaystyle {\sum}_{b=1}^{B}}\mathbb{I}({\tau}_{n}\ge {\tau}_{n,b}^{\ast})$, ${\mathcal{H}}_{0}$ is rejected when ${p}_{n,B}^{\ast}<\alpha $, with $\alpha $ the nominal level. (For the choice of bootstrap repetitions $B$, see Remark 9).
As to the definition in step $\mathcal{A}$ of the bootstrap innovations ${\epsilon}_{t}^{\ast}$, usually these are obtained by i.i.d. draws with replacement from recentered estimated model residuals; henceforth referred to as iid bootstrap, or iid resampling. The model residuals can be either estimated under the null (${\mathcal{H}}_{0}$ imposed) or without imposing the null. That is, the ${\epsilon}_{t}^{\ast}$ are in the first case resampled from centered residuals, ${\left\{{\tilde{\epsilon}}_{t}^{c}\right\}}_{t=1}^{n}$, where
Using unrestricted residuals, the bootstrap ${\epsilon}_{t}^{\ast}$ innovations are resampled from ${\left\{{\widehat{\epsilon}}_{t}^{c}\right\}}_{t=1}^{n}$, where
Centering is required because by doing so both of the residual series ${\widehat{\epsilon}}_{t}^{c}\phantom{\rule{0.1em}{0ex}}$ and ${\tilde{\epsilon}}_{t}^{c}$ have empirical mean zero, ${n}^{1}{\displaystyle {\sum}_{t=1}^{n}}{\widehat{\epsilon}}_{t}^{c}=0$ and ${n}^{1}{\displaystyle {\sum}_{t=1}^{n}}{\tilde{\epsilon}}_{t}^{c}=0$, and ideally “ mimic” the true ${\epsilon}_{t}$. In particular, under ${\mathcal{H}}_{0}$, ${\epsilon}_{t}\left(\overline{\rho}\right)={\epsilon}_{t}\left({\rho}_{0}\right)={\epsilon}_{t}$, and, moreover, ${\widehat{\rho}}_{n}\stackrel{p}{\to}{\rho}_{0}$. In contrast, under the alternative, when ${\mathcal{H}}_{0}$ does not hold, while ${\epsilon}_{t}\left(\overline{\rho}\right)\ne {\epsilon}_{t}$, it still holds that ${\widehat{\rho}}_{n}\stackrel{p}{\to}{\rho}_{0}$. Hence, while one may expect that the unrestricted residuals ${\widehat{\epsilon}}_{t}^{c}$ perform better under the alternative, at least asymptotically, in practice little difference is found between the two.
An alternative to iid resampling is the socalled wild bootstrap, where either ${\epsilon}_{t}^{\ast}={\tilde{\epsilon}}_{t}^{c}{w}_{t}^{\ast},$ or ${\epsilon}_{t}^{\ast}={\widehat{\epsilon}}_{t}^{c}{w}_{t}^{\ast}$, with ${w}_{t}^{\ast}$ an auxiliary i.i.d. sequence, independent of the original data, with $\text{E}\left({w}_{t}^{\ast}\right)=0$ and $\text{V}\left({w}_{t}^{\ast}\right)=1$. A simple example is ${w}_{t}^{\ast}$ i.i.d. $\text{N}\left(\mathrm{0,1}\right)$ (see Remark 10 for alternative specifications). The wild bootstrap is typically motivated by its potential ability to allow for possible model misspecification in the sense that it allows for (conditional and unconditional) heteroskedasticity in the innovations ${\epsilon}_{t}$. To see this (e.g., ${\epsilon}_{t}^{\ast}={\tilde{\epsilon}}_{t}^{c}{w}_{t}^{\ast}$), it follows that (conditionally on the data ${\left\{{x}_{t}\right\}}_{t=0}^{n}$), the ${\epsilon}_{t}^{\ast}$ are i.i.d. distributed with mean zero and with timevarying variance,
This way, the variation in ${\epsilon}_{t}^{\ast}$ from the wild bootstrap “in essence reflects the heteroskedasticity of the original data” (Liu, 1988, p. 1704) as is also illustrated in the Monte Carlo simulations in the section on “Finite Sample Behavior.” This differs from the iid bootstrap, where the bootstrap innovations ${\epsilon}_{t}^{\ast}$ (conditionally on the data) are i.i.d., with a discrete distribution given by $P\left({\epsilon}_{t}^{\ast}={\tilde{\epsilon}}_{j}^{c}{\left\{{x}_{t}\right\}}_{t=0}^{n}\right)={n}^{1}$, for $j=\mathrm{1,2,...,}n$. Specifically, as mentioned earlier, the (empirical, or conditional) mean is zero, $\text{E}\left({\epsilon}_{t}^{\ast}{\left\{{x}_{t}\right\}}_{t=0}^{n}\right)=0$, and the variance is given by
That is, for fixed $n$, the variance for the iid bootstrap innovations ${\epsilon}_{t}^{\ast}$ is constant (and equal to the empirical variance of the estimated residuals), while depending on time $t$ for the wild bootstrap.
In short, the bootstrap scheme in (6), ${x}_{t}^{\ast}={\rho}_{n}^{\ast}{z}_{t}+{\epsilon}_{t}^{\ast}$, with ${z}_{t}={x}_{t1}$ or ${z}_{t}={x}_{t1}^{\ast}$, depends on two types of randomness: (a) the variation of the original data ${\left\{{x}_{t}\right\}}_{t=0}^{n}$; and (b) the bootstrap resampling for ${\left\{{\epsilon}_{t}^{\ast}\right\}}_{t=1}^{n}$ (wild or iid resampling). This is important for the application of the LLN and the CLT to establish bootstrap validity as demonstrated in the next.
Some further remarks are in order.
Remark 6 (t statistics and bootstrap standard errors). When interest is in bootstrapping the $t$ ratio statistic (or Waldtype statistics), then steps $\mathcal{A}\u2013\mathcal{C}$ can be used with ${\tau}_{n}^{\ast}$, now defined as (taking the case of unknown ${\sigma}^{2}$ to illustrate)
where the socalled bootstrap standard error ${\text{se}}^{\ast}({\widehat{\rho}}_{n}^{\ast})$, the bootstrap analog of $\text{se}({\widehat{\rho}}_{n})$, given by
with ${V}_{n}^{\ast}$ being an estimator of the variance of ${\widehat{\rho}}_{n}^{\ast}{\rho}_{0}$, such as
where ${\widehat{\sigma}}_{n}^{\ast 2}$ is the sample variance of the bootstrap residuals, ${\widehat{\sigma}}_{n}^{\ast 2}={n}^{1}{\displaystyle {\sum}_{t=1}^{n}}{\epsilon}_{t}^{\ast}({\widehat{\rho}}_{n}^{\ast})$, ${\epsilon}_{t}^{\ast}({\widehat{\rho}}_{n}^{\ast})={x}_{t}^{\ast}{\widehat{\rho}}_{n}^{\ast}{x}_{t1}^{\ast}$. It is worth noticing that the bootstrap standard error could also be computed without exploiting the autoregressive structure of the model; this can be done by simply resorting to the $B$ bootstrap realizations of ${\widehat{\rho}}_{n}^{\ast}$, say ${\widehat{\rho}}_{n,b}^{\ast}$, $b=\mathrm{1,...,}B$. Specifically, one could consider the following bootstrap standard error
which is straightforward to compute once the $B$ realizations of ${\widehat{\rho}}_{n}^{\ast}$ have been obtained.
Remark 7 ([Un]restricted bootstrap). As to the choice in step $\mathcal{A}$ of the bootstrap true value ${\theta}_{n}^{\ast}={\left({\rho}_{n}^{\ast},{\sigma}_{n}^{\ast 2}\right)}^{\prime}$ (or simply, ${\rho}_{n}^{\ast}$ for the AR model), one may set ${\theta}_{n}^{\ast}$ to the value of the unrestricted estimator ${\widehat{\theta}}_{n}$, ${\theta}_{n}^{\ast}={\widehat{\theta}}_{n}^{\ast}$, which is referred to as unrestricted bootstrap. If ${\theta}_{n}^{\ast}={\tilde{\theta}}_{n}$, the restricted estimator, the bootstrap is referred to as restricted. It should be emphasized that for the unrestricted bootstrap, the bootstrap likelihood ratio statistic ${\tau}_{n}^{\ast}$ is derived for the hypothesis $\rho ={\widehat{\rho}}_{n}$, while the original hypothesis ${\mathcal{H}}_{0}:\rho =\overline{\rho}$ is considered for the restricted bootstrap. While both choices are widely applied in existing literature, the restricted bootstrap in the context of testing is more popular in econometrics (see, e.g., Davidson & MacKinnon, 2000).
Remark 8 (Recursive and fixed design bootstrap). In step $\mathcal{B}$ of the algorithm in equation (6), with ${z}_{t}={x}_{t1}^{\ast}$, this is an example of a recursive bootstrap. That is, the original autoregressive structure for ${x}_{t}$ is replicated for the bootstrap process ${x}_{t}^{\ast}$. However, with ${z}_{t}={x}_{t1}$, the original data ${x}_{t1}$ are used as lagged value of ${x}_{t}^{\ast}$, such that ${x}_{t}^{\ast}$ is not an autoregressive process even conditionally on the data. The fixed design bootstrap typically simplifies (some of) the asymptotic arguments, and is often found to behave as well as the recursive bootstrap (see, e.g., Gonçalves & Kilian, 2004, 2007, for general AR models, and Cavaliere, Pedersen, & Rahbek, 2018, for ARCH models).
Remark 9 (Bootstrap pvalue). In step $\mathcal{C}$, the bootstrap pvalue ${p}_{n,B}^{\ast}$ is defined as ${p}_{n,B}^{\ast}={B}^{1}{\displaystyle {\sum}_{b=1}^{B}}\mathbb{I}({\tau}_{n}\ge {\tau}_{n,b}^{\ast})$, with $B$ the number of bootstrap repetitions (see also Remark 14). Typical choices are $B=199$, $399$, or $999$ (see also Andrews & Buchinsky, 2000, and Davidson & MacKinnon, 2000, for details on the choice of $B$).
Remark 10 (Choice of ${w}_{t}^{*}$). With respect to the choice of distribution of the i.i.d. sequence ${w}_{t}^{\ast}$ for the wild bootstrap, Liu (1988) provides a detailed discussion of various choices based on socalled Edgeworth expansions (see, e.g., Hall, 1992; van der Vaart, 2000, for an introduction) of test statistics similar to ${\tau}_{n}$. In particular, Liu (1988), with ${\xi}_{k}=\text{E}({w}_{t}^{k}),\phantom{\rule{0.2em}{0ex}}k\ge 1$, emphasizes ${\xi}_{1}=0$, ${\xi}_{2}=1$ as well as ${\xi}_{3}=1$ as important. For the case of ${\xi}_{3}=1$, emphasis is on possible skewness, while ${\xi}_{3}=0$ works well in the case of symmetry. In applications, standard choices for ${w}_{t}^{\ast}$, all with ${\xi}_{1}=0$ and ${\xi}_{2}=1$, include the Gaussian, Rademacher, and Mammen distributions.^{1} It follows that ${\xi}_{3}=0$ for the first two (with ${\xi}_{4}=3$ and $1$, respectively), while ${\xi}_{3}=1$ (and ${\xi}_{4}=2$) for the Mammen distribution.
Remark 11 (Parametric bootstrap). In step $\mathcal{A}$ of the bootstrap scheme, one may also use a socalled parametric bootstrap, where the bootstrap innovations ${\epsilon}_{t}^{\ast}$ are generated as i.i.d. $\text{N}\left(\mathrm{0,}{\sigma}_{n}^{\ast 2}\right)$. While this parametric bootstrap performs well in the case where the true innovations ${\epsilon}_{t}$ are Gaussian, this may not be the case when the distribution of ${\epsilon}_{t}$ is nonGaussian (Horowitz, 2001).
Asymptotic Theory for the Recursive Bootstrap
In order to discuss regularity conditions under which bootstrapbased testing holds, consider here the details of verification of the asymptotic validity of the recursive bootstrap for the AR model.
Thus, with the AR model given in (1), consider here the recursive restricted bootstrap scheme as defined by setting ${z}_{t}={x}_{t1}^{\ast}$ and ${\rho}_{n}^{\ast}=\overline{\rho}$ in step $\mathcal{A}$ of the bootstrap algorithm. In short, the ${x}_{t}^{\ast}$ bootstrap sequence is here generated as
with ${x}_{0}^{\ast}={x}_{0}$ and $\overline{\rho}={\rho}^{\ast}$ as the bootstrap true value. Moreover, we consider the classic case of iid resampling from the autoregressive residuals obtained under ${\mathcal{H}}_{0}$, that is, from ${\left\{{\tilde{\epsilon}}_{t}^{c}\right\}}_{t=1}^{n}$ defined in (8). The statistic of interest is ${\tau}_{n}={W}_{n}\left(\rho =\overline{\rho}\right)$ in (5), which is computed using the bootstrap sample ${\left\{{x}_{t}^{\ast}\right\}}_{t=0}^{n}$ as
While ${x}_{t}^{\ast}$ clearly has some features similar to ${x}_{t}$, one cannot apply standard concepts such as stationarity and ergodicity when analyzing the asymptotic behavior of ${\tau}_{n}^{\ast}$, due to two types of randomness: the bootstrap resampling distribution and the distribution of the original data, ${\left\{{x}_{t}\right\}}_{t=0}^{n}\phantom{\rule{0.1em}{0ex}}$. Introduce therefore the bootstrap equivalent concepts of convergence in probability and distribution, which reflect the fact that inference is based on conditioning on the original data that are themselves random.
Bootstrap Probability, Expectation, and Convergence
With ${\text{P}}^{\ast}(\cdot )$ denoting the bootstrap probability—that is, the probability conditional on the data—the iid bootstrap innovations ${\epsilon}_{t}^{\ast}$ are by definition i.i.d. distributed with
Similarly, the expectation ${\text{E}}^{\ast}(\cdot )$ is defined by ${\text{E}}^{\ast}(\cdot )=\text{E}(\cdot {\left\{{x}_{t}\right\}}_{t=0}^{n})$. As an example, consider the expectation of ${\epsilon}_{t}^{\ast}\phantom{\rule{0.1em}{0ex}}$conditionally on the data. It follows that, as already discussed, ${\text{E}}^{\ast}({\epsilon}_{t}^{\ast})=0$ as
by the definition in (8). Next, consider the variance of ${\epsilon}_{t}^{\ast}$ conditionally on the data, ${\text{V}}^{\ast}\left({\epsilon}_{t}^{\ast}\right)$. Again, by definition, ${\text{V}}^{\ast}\left({\epsilon}_{t}^{\ast}\right)={\text{E}}^{\ast}({\epsilon}_{t}^{\ast 2}){\left({\text{E}}^{\ast}\left({\epsilon}_{t}^{\ast}\right)\right)}^{2}$ and hence as ${\text{E}}^{\ast}\left({\epsilon}_{t}^{\ast}\right)=0$, it follows that
That is, the variance conditional on the data is equal to the sample variance of the original estimated residuals under ${\mathcal{H}}_{0}$. In particular, ${\text{V}}^{\ast}\left({\epsilon}_{t}^{\ast}\right)$ is a random variable (in terms of the original probability measure) and moreover, by the LLN for i.i.d. variables, under ${\mathcal{H}}_{0}$ with $\overline{\rho}={\rho}_{0}$,
Note that for the wild bootstrap, ${\text{V}}^{\ast}\left({\epsilon}_{t}^{\ast}\right)={\text{V}}^{\ast}\left({\tilde{\epsilon}}_{t}^{c}{w}_{t}^{\ast}\right)=({\tilde{\epsilon}}_{t}^{c}{)}^{2}$, emphasizing that the wild bootstrap indeed “mimics” heteroskedasticity and the iid bootstrap does not.
Similar to the definition of ${\text{P}}^{\ast}(\cdot )$ and ${\text{E}}^{\ast}(\cdot )$ this motivates the definition of the bootstrap equivalent of convergence in probability, denoted “${\stackrel{{p}^{\ast}}{\to}}_{p}$.” Formally, a sequence of stochastic variables ${X}_{n}^{\ast}$ is said to converge in probability conditional on the data (or, to converge in ${\text{P}}^{\ast}$probability, in probability) to $c$ (possibly random), if ${\text{P}}^{\ast}\left({X}_{n}^{\ast}c>\delta \right)$ converge in probability to zero. This can be stated as
As an example, it follows that ${X}_{n}^{\ast}={n}^{1}{\displaystyle {\sum}_{t=1}^{n}}{\epsilon}_{t}^{\ast}{\stackrel{{p}^{\ast}}{\to}}_{p}0$ as by the bootstrap equivalent of Markov’s inequality^{2} one has,
By definition,
with ${\text{E}}^{\ast}\left({\epsilon}_{t}^{\ast 2}\right)={\text{V}}^{\ast}\left({\epsilon}_{t}^{\ast}\right)$, and for $s\ne t$,
Hence, since ${\text{E}}^{\ast}{\left({X}_{n}^{\ast}\right)}^{2}={n}^{1}{\text{V}}^{\ast}\left({\epsilon}_{t}^{\ast}\right)$, we conclude that
A key ingredient in the asymptotic analysis of the nonbootstrap AR model is the CLT, and we need a bootstrap equivalent of the CLT and a bootstrap equivalent of convergence in distribution. By definition, ${X}_{n}\stackrel{d}{\to}X$ if ${F}_{{X}_{n}}\left(x\right)=P\left({X}_{n}\le x\right)\to {F}_{X}\left(x\right)=P\left(X\le x\right)$ at all continuity points of ${F}_{X}(\cdot )$. Likewise, ${X}_{n}^{\ast}$ converge in distribution to $X$ conditional on the data (or, as sometimes used, ${X}_{n}^{\ast}$ converges “weakly in probability”); that is, ${X}_{n}^{\ast}{\stackrel{{d}^{\ast}}{\to}}_{p}X$, if the bootstrap cumulative distribution function converges in probability. Specifically, ${X}_{n}^{\ast}{\stackrel{{d}^{\ast}}{\to}}_{p}X$ if
at all continuity points of ${F}_{X}(\cdot )$. Alternatively, weak convergence in probability may be defined in terms of convergence in probability of the bootstrap characteristic function as in (15).
The next lemma illustrates that, as one might expect, the sum of bootstrap innovations ${\epsilon}_{t}^{\ast}$ is asymptotically Gaussian (in probability).
Lemma 2 (van der Vaart, 2000, Theorem 23.4). With ${X}_{n}^{\ast}={n}^{1/2}{\displaystyle {\sum}_{t=1}^{n}}{\epsilon}_{t}^{\ast}$, then
The proof, as for most bootstrap CLTs, is based on applying a CLT for triangular arrays, as ${\left\{{\epsilon}_{t}^{\ast}\right\}}_{t=1}^{n}$ are sampled from ${\tilde{\epsilon}}_{t}^{c}$, which depends on $n$.
To give an idea of the underlying theory, consider here verifying (14) using a classic approach based on the characteristic function (Durret, 2019, proof of Theorem 3.4.10). For a random variable $X$, the characteristic function defines uniquely the distribution of $X$ and is defined by $\varphi \left(s\right)=\text{E}\left(\mathrm{exp}\left(is\phantom{\rule{0.2em}{0ex}}X\right)\right)$. Here, $i$ is the complex (unit imaginary) number that satisfies ${i}^{2}=1$ and $s\in \mathbb{R}$, and for $X\stackrel{d}{=}\text{N}\left(\mathrm{0,}{\sigma}_{0}^{2}\right)$ it holds that $\varphi \left(s\right)=\mathrm{exp}(\frac{{s}^{2}}{2}{\sigma}_{0}^{2})$.
With the bootstrap characteristic function of ${X}_{n}^{\ast}$ defined by ${\varphi}_{n}^{\ast}\left(s\right)={\text{E}}^{\ast}\left(\mathrm{exp}\left(is{X}_{n}^{\ast}\right)\right)$, it follows that (14) holds if
Note first, as ${\epsilon}_{t}^{\ast}\phantom{\rule{0.1em}{0ex}}$ are i.i.d. conditionally on the data,
Next, a Taylor expansion of $\mathrm{exp}(\cdot )$ at $s=0$ gives
using ${\text{E}}^{\ast}\left({\epsilon}_{t}^{\ast}\right)=0$ and ${\text{E}}^{\ast}\left({\epsilon}_{t}^{\ast 2}\right)\stackrel{p}{\to}{\sigma}_{0}^{2}$ (see [12]; Durret, 2019, Lemma 3.3.19). It therefore follows as desired that
as for any sequence ${c}_{n}$, with ${c}_{n}\stackrel{p}{\to}c\in \u2102$ as $n\to \infty $, then similar to Durret (2019, Theorem 3.4.2), ${\left(1{n}^{1}{c}_{n}\right)}^{n}\stackrel{p}{\to}\mathrm{exp}\left(c\right)$.
Remark 12 (Lindeberg condition). The CLTs in Durret (2019, Theorem 3.4.10) and van der Vaart (2000, Theorem 23.4) for triangular arrays follow by verifying
in addition to the bootstrap Lindeberg condition,
A simple way to see that ${\gamma}_{n}^{\ast}\stackrel{p}{\to}0$ is for example to note that if (the rather strong moment condition) $\text{E}({\epsilon}_{t}^{4})<\infty $ holds, the LLN applies to ${n}^{1}{\displaystyle {\sum}_{t=1}^{n}}{\left({\tilde{\epsilon}}_{t}^{c}\right)}^{4}$ and hence,
Bootstrap Validity Under ${\mathcal{H}}_{0}$
As briefly mentioned in the introduction, it is important for the application of the bootstrap that the limiting distribution (in probability) of the bootstrap test statistic has the same limiting distribution as the original test statistic when the null is true. Stated differently, we wish here to establish that under ${\mathcal{H}}_{0}$ with $\overline{\rho}={\rho}_{0},$
By definition, the bootstrap estimator ${\widehat{\rho}}_{n}^{\ast}$ is given by
such that by the bootstrap scheme employed, that is ${x}_{t}^{\ast}={\rho}_{0}{x}_{t1}^{\ast}+{\epsilon}_{t}^{\ast}$, it follows that
Here a bootstrap CLT should be used for the first term (Should be (i) as refers to equation just above!)
a), and a bootstrap LLN for the second term (b) (Should be (ii) as refers to equation) to find the limiting behavior of the bootstrap estimator and, hence, of the test statistic ${\tau}_{n}^{\ast}$. Consider first (b), which is an average of lagged ${x}_{t}^{\ast}$ squared, with
As ${\epsilon}_{t}^{\ast}$ depends on $n$, and the data ${\left\{{x}_{t}\right\}}_{t=0}^{n}$, the concepts of stationarity and ergodicity—while applying to ${x}_{t}$—do not apply to ${x}_{t}^{\ast}$. However, the following lemma holds, which establishes that the LLN holds for the average of ${x}_{t}^{\ast}$ and ${x}_{t}^{\ast 2}$.
Lemma 3. Suppose that ${\left\{{x}_{t}\right\}}_{t=0}^{n}$ is given by (1) with ${\rho}_{0}<1$ and ${\epsilon}_{t}$ i.i.d. $\left(\mathrm{0,}{\sigma}_{0}^{2}\right)$. Assume furthermore, with ${\epsilon}_{t}^{\ast}$ iid sampled with replacement from ${\left\{{\tilde{\epsilon}}_{t}^{c}\right\}}_{t=1}^{n}$ and ${x}_{t}^{\ast}$ given by (10). Then, as $n\to \infty $,
The proof of Lemma 3 is given in the appendix. Note that for the case of ${\rho}_{0}=0$ the arguments are similar to the arguments used to establish ${n}^{1}{\displaystyle {\sum}_{t=1}^{n}}{\epsilon}_{t}^{\ast}{\stackrel{{p}^{\ast}}{\to}}_{p}0$.
Remark 13 (LLN triangular arrays). Lemma 3 is the bootstrap equivalent of the weak law of large numbers for triangular arrays (see also Durret, 2019, Theorem 2.2.6).
Next, consider the CLT candidate term (b),
As for the nonbootstrap case, with ${\mathcal{F}}_{t}^{\ast}=\sigma \left({x}_{t}^{\ast},{x}_{t1}^{\ast}\mathrm{,...,}{x}_{0}^{\ast}\right)$, then (conditionally on the data) a bootstrap CLT for martingale difference arrays (mda) can be applied. In particular, ${\text{E}}^{\ast}\left({\epsilon}_{t}^{\ast}{x}_{t1}^{\ast}{\mathcal{F}}_{t1}^{\ast}\right)={x}_{t1}^{\ast}{\text{E}}^{\ast}\left({\epsilon}_{t}^{\ast}\right)=0$, while for the conditional second order moment (conditional on the data), it follows by application of Lemma 3 that
It remains to establish the bootstrap Lindeberg condition, ${\gamma}_{n}^{\ast}{\stackrel{{p}^{\ast}}{\to}}_{p}0$, where
Similar to Remark 12, this follows by using that for some (arbitrarily small) $\eta >0$,
which holds provided $\text{E}{\epsilon}_{t}{}^{2+\eta}<\infty $, using arguments as in Lemma 3. (Note that in Remark 12 the same argument is used for $\eta =2$.)
Hence, with ${\epsilon}_{t}^{\ast}$ iid resampled from ${\left\{{\tilde{\epsilon}}_{t}^{c}\right\}}_{t=1}^{n}$, ${\epsilon}_{t}$ i.i.d. $\left(\mathrm{0,}{\sigma}_{0}^{2}\right)$ with $\text{E}{\epsilon}_{t}{}^{2+\eta}<\infty $, it follows that the limiting distribution (in probability) of the bootstrap MLE is given by
This immediately leads to the desired result:
Theorem 1. Under ${\mathcal{H}}_{0}$ with ${\rho}_{0}<1$, and with ${\epsilon}_{t}^{\ast}$ iid resampled from ${\left\{{\tilde{\epsilon}}_{t}^{c}\right\}}_{t=1}^{n}$, with ${\epsilon}_{t}$ being i.i.d. $\left(\mathrm{0,}{\sigma}_{0}^{2}\right)$ with $\text{E}{\epsilon}_{t}{}^{2+\eta}<\infty $ for some $\eta >0$, it holds that
Remark 14 (Bootstrap pvalue). The bootstrap pvalue ${p}_{n,B}^{\ast}$ in step $\mathcal{C}$ of the bootstrap algorithm is an approximation to the “true” bootstrap pvalue ${p}_{n}^{\ast}$, where ${p}_{n}^{\ast}={\text{P}}^{\ast}\left({\tau}_{n}^{\ast}>\mathrm{}{\tau}_{n}\right)$, in the sense that ${p}_{n,B}^{\ast}\stackrel{p}{\to}{p}_{n}^{\ast}$ as $B$ tends to infinity. (For details in terms of the stronger concept of “almost sure” convergence, see, e.g., Cavaliere, Nielsen, & Rahbek, 2015, Remark 2.)
Remark 15 (Validity under ${\mathcal{H}}_{0}$). Note that by Cavaliere et al. (2015, Corollary 1) for the bootstrap pvalue ${p}_{n}^{\ast}$ (see Remark 14), it follows that as the limiting ${\chi}_{1}^{2}$ distribution has a continuous distribution function, under the conditions of Theorem 1, ${p}_{n}^{\ast}\stackrel{d}{\to}\text{U}$, with $\text{U}$ uniformly distributed on $\left[\mathrm{0,1}\right]$ (Hansen, 1996, 2000).
Remark 16 (Moments of ${\epsilon}_{t}$). Note that $\text{E}{\epsilon}_{t}{}^{2+\eta}<\infty $ for some $\eta $, or $\text{E}\left({\epsilon}_{t}^{4}\right)<\infty $ as is often used, is required. This reflects that, as typically found for the bootstrap, further moment restrictions are used to prove bootstrap validity than in the nonbootstrap case. This is because complexities arise when applying bootstrap LLNs and arguments in connection to establishing bootstrap Lindebergtype conditions (Cavaliere & Rahbek, 2020). However, notably, while the higher order moment conditions are sufficient for the mathematical arguments, their necessity is often not reflected in bootstrap simulations (see “Finite Sample Behavior”).
Remark 17 (Introducing ${\sigma}^{2}$). As for the nonbootstrap case, the result in Theorem 1 also holds for the case where ${\sigma}^{2}$ is treated as a parameter.
Remark 18 (Consistency of bootstrap standard errors). It is important to notice that the fact that ${n}^{1/2}({\widehat{\rho}}_{n}^{\ast}{\rho}_{0})$ converges in conditional distribution (in probability) to the Gaussian distribution does not imply that the bootstrap standard error is consistent. Intuitively, this happens because convergence in distribution alone does not imply convergence of moments. Hence, the econometrician must evaluate on a casebycase basis whether the bootstrap standard errors are consistent. An interesting result is provided by Hahn and Liao (2019), who report that the bootstrap standard error computed by simulation using $B$ bootstrap repetitions does not underestimate the population standard error (see Remark 6), and hence bootstrap inference based on the $t$ statistic coupled with the numerically computed standard errors is conservative (i.e., the typeone error associated to the bootstrap test does not exceed the userchosen significance level in large samples).
Bootstrap Validity Under the Alternative
Consider here the convergence of the bootstrap statistic ${\tau}_{n}^{\ast}$ in (11) when the alternative holds. That is, assume here that the original data are generated with true value ${\theta}_{0}=({\rho}_{0},{\sigma}_{0}^{2}{)}^{\prime}$, but the hypothesis tested is as before ${\mathcal{H}}_{0}:\rho =\overline{\rho}$ with $\overline{\rho}\ne {\rho}_{0}$. As argued below, Theorem 1 holds under the alternative as well, such that
For the application of bootstrapbased testing, this implies that under the alternative, as ${W}_{n}\left(\rho =\overline{\rho}\right)$ diverges while ${W}_{n}^{\ast}\left(\rho =\overline{\rho}\right)$ converges in distribution, the bootstrapbased test will reject with probability tending to one. That is, asymptotic bootstrap validity holds since by Cavaliere et al. (2015, Corollary 1) the bootstrap pvalue ${p}_{n}^{\ast}$, defined in Remark 15, tends to zero in probability under the alternative, ${p}_{n}^{\ast}\stackrel{p}{\to}0$.
A key argument for (20) to hold is to note that the identity,
holds independently of whether $\overline{\rho}$ is the data true value ${\rho}_{0}$ or not. That is, $\overline{\rho}$ is by construction the bootstrap true value, such that under the null and also under the alternative, the bootstrap estimator can be rewritten as
What differs is that ${\epsilon}_{t}^{\ast}$ under the alternative is resampled from recentered residuals ${\tilde{\epsilon}}_{t}^{c}$, with
That is, while the identity ${\tilde{\epsilon}}_{t}={\epsilon}_{t}$ holds under the null hypothesis when $\overline{\rho}={\rho}_{0}$, this is not the case under the alternative. Hence, to establish (20), a repeated application of the bootstrap LLN (applied to ${\sum}_{t=1}^{n}}{x}_{t1}^{\ast 2$) and CLT (applied to ${\sum}_{t=1}^{n}}{\epsilon}_{t}^{\ast}{x}_{t1}^{\ast$) is needed under the alternative. For the AR process of order one considered here, the arguments are based on simple modifications of the theory under ${\mathcal{H}}_{0}$.
Remark 19 (Theory for the wild bootstrap). The same results can be shown to apply for the wild bootstrap in the case of conditional heteroskedasticity (Gonçalves & Kilian, 2004, 2007).
Finite Sample Behavior
Throughout, the focus has been on establishing asymptotic validity. This was done by verifying that the bootstrap statistic ${\tau}_{n}^{\ast}$ has the same limiting distribution (in probability) as the original statistic ${\tau}_{n}$ under the null hypothesis. Moreover, the same was argued to hold under the alternative. To illustrate the finite sample performance of the iid and wild bootstraps for the AR model, this section highlights some selected typical findings for the bootstrap based on a small and simple (to replicate) Monte Carlo study. Thus the Monte Carlo study here is not meant to be elaborate; exhaustive and detailed bootstrap Monte Carlobased investigations are given in several papers (see, e.g., Gonçalves & Kilian, 2004, with special attention to higher order AR models, as well as the references in Cavaliere & Rahbek, 2020).
The Monte Carlo results reported here highlight the importance of the assumptions for the established validity of the bootstrapbased test of ${\mathcal{H}}_{0}:\rho =\overline{\rho}$ in the autoregressive model. Specifically, it was argued that the true value of the autoregressive root ${\rho}_{0}$ for ${x}_{t}$ should satisfy ${\rho}_{0}<1$, and it was emphasized that ${\epsilon}_{t}$ is an i.i.d. $\left(\mathrm{0,}{\sigma}_{0}^{2}\right)$ sequence, such that $\text{E}{\epsilon}_{t}{}^{2+\eta}<\infty $, or rather, $\text{E}\left({\epsilon}_{t}^{4}\right)<\infty $.
With details of the Monte Carlo designs and consideration given below, we initially mention the following findings for bootstrap simulations in the AR model with a constant term. The findings are typical for existing applications of the bootstrap and are standard in time series contexts.
With ${\epsilon}_{t}$ i.i.d. $\text{N}\left(\mathrm{0,}{\sigma}_{0}^{2}\right)$ the ${\chi}_{1}^{2}$based asymptotic test performs well for even small samples of size $n$ in terms of empirical rejection frequencies, or empirical size, for ${\rho}_{0}=0.5$, while for ${\rho}_{0}=0.9$ the asymptotic test fails as its empirical size is not close to the nominal level $\alpha $. In comparison, the iid (and wild) bootstrapbased test has empirical size close to the nominal level in both cases, see Table 1 columns A and B for ${\rho}_{0}\in \left\{\mathrm{0.5,0.9}\right\}$, ${\sigma}_{0}^{2}=1$, and $n\in \left\{\mathrm{15,25,...,1000}\right\}$ with $\alpha =0.05$.
With ${\epsilon}_{t}$ independently $\text{N}\left(\mathrm{0,}{\sigma}_{t}^{2}\right)$ distributed with ${\sigma}_{t}^{2}$ timevarying (heteroskedasticity), neither the ${\chi}_{1}^{2}$based asymptotic nor the iid bootstrapbased tests have empirical size close to the nominal level $\alpha $, which contrasts with the wild bootstrapbased test. This is illustrated in Table 1 columns C–F for a timechanged volatility, ${\sigma}_{t}^{2}={\sigma}_{\mathrm{0,1}}^{2}+{\sigma}_{\mathrm{0,2}}^{2}\mathbb{I}\left(t\ge n/2]\right)$, with ${\sigma}_{\mathrm{0,1}}^{2}=1<{\sigma}_{\mathrm{0,2}}^{2}=15$, and as before ${\rho}_{0}=\left\{\mathrm{0.5,0.9}\right\}$ and $n\in \left\{\mathrm{15,25,...,1000}\right\}$ with $\alpha =0.05$.
In terms of empirical rejection frequencies under the alternative, bootstrapbased tests and the asymptotic test are comparable. This is illustrated in Table 1, with ${\epsilon}_{t}$ i.i.d. $N\left(\mathrm{0,}{\sigma}_{0}^{2}\right)\phantom{\rule{0.1em}{0ex}}\phantom{\rule{0.2em}{0ex}}$ for $n=250$ and ${\rho}_{0}=0.9$, and the test of ${\mathcal{H}}_{0}:\rho =\overline{\rho}$ is considered for values of $\overline{\rho}$ ranging from $0.70$ to $0.875$ (with $\overline{\rho}={\rho}_{0}$ included as a benchmark).
The wild bootstrap is often motivated by its ability to replicate underlying heteroskedasticity (see the section “Bootstrap in the Autoregressive Model” and Remark 10). This is illustrated in Figure 1, where panel A shows the empirical residuals ${\tilde{\epsilon}}_{t}$ from one of the draws in Table 1 with ${\sigma}_{t}^{2}={\sigma}_{\mathrm{0,1}}^{2}+{\sigma}_{\mathrm{0,2}}^{2}\mathbb{I}\left(t\ge n/2]\right)$. Figure 1, in panels B, C, and D, illustrates that, while ${\epsilon}_{t}^{\ast}$ replicates the heteroskedasticity for the wild bootstrap, this is not the case for the iid bootstrap.
In terms of requirements for finite moments of the i.i.d. sequence ${\epsilon}_{t}$, the section “Bootstrap Validity Under ${\mathcal{H}}_{0}$” discussed sufficiency and necessity of the condition $\text{E}{\epsilon}_{t}{}^{2+\eta}<\infty $ for some $\eta >0$, and it was conjectured that $\text{E}\left({\epsilon}_{t}^{2}\right)<\infty $ was sufficient. This is illustrated in Table 2, which shows that when ${\epsilon}_{t}$ does not have a finite variance, then the asymptotic test, as well as the wild and iid bootstrapbased tests, fail to have correct empirical size. However, when ${\epsilon}_{t}$ has a finite variance, while the asymptotic test has empirical size far from the nominal, the bootstraps work despite the lack of fourth order moments, for example. Also, Table 2 shows results from a socalled permutation bootstrap (see the section “Moment Condition, $\text{E}{\epsilon}_{t}{}^{k}<\infty $ for $k\ge 2$”).
Asymptotic Test
We consider ${x}_{t}$ as given by the AR model of order one, with a constant term $\delta $ included,
with ${\epsilon}_{t}$ i.i.d. $\text{N}(\mathrm{0,}{\sigma}^{2})$ and ${x}_{0}$ fixed. The parameters are given by $\theta =(\delta ,\rho ,{\sigma}^{2}{)}^{\prime}\in \Theta ={\mathbb{R}}^{2}\times \left(\mathrm{0,}\infty \right)$, with ${\theta}_{0}=({\delta}_{0},{\rho}_{0},{\sigma}_{0}^{2}{)}^{\prime}$ the true value, where ${\theta}_{0}\in {\Theta}_{0}=\left\{\theta \in \Theta \phantom{\rule{0.1em}{0ex}}\phantom{\rule{0.2em}{0ex}}{\rho}_{0}<1\right\}$ and the hypothesis of interest is given by ${\mathcal{H}}_{0}:\rho =\overline{\rho}$.
The unrestricted and restricted (Gaussian likelihoodbased) estimators that maximize
with ${x}_{0}$ fixed and ${\epsilon}_{t}\left(\rho ,\delta \right)={x}_{t}\rho {x}_{t1}\delta $, are given by
respectively. The theory from the case of no constant term immediately carries over, such that for $\overline{\rho}={\rho}_{0}$, ${\rho}_{0}<1$ and ${\epsilon}_{t}$ i.i.d. $\left(\mathrm{0,}{\sigma}_{0}^{2}\right)$, as $n\to \infty $,
In the implementations of the asymptotic test, we use the pvalue, ${p}_{n}$, calculated as the tail probability of ${\tau}_{n}$ in the limiting ${\chi}_{1}^{2}$ distribution, and reject if ${p}_{n}$ is smaller than the nominal level $\alpha $.
Column A in Table 1 illustrates this for ${x}_{t}$ in (21) generated with ${\delta}_{0}=\mathrm{0,}$${\rho}_{0}=\left\{\mathrm{0.5,0.9}\right\},$ and ${\sigma}_{0}^{2}=1$. Moreover, ${\epsilon}_{t}$ is simulated as an i.i.d. $\text{N}\left(\mathrm{0,}{\sigma}_{0}^{2}\right)$ sequence, and ${x}_{0}=0$. The empirical rejection frequencies are reported based on $N=\mathrm{10,000}$ repetitions, with nominal level $\alpha =0.05$. Results for $n\in \left\{\mathrm{15,25,...,1000}\right\}$ are given in column A of Table 1. Observe, as noted, that quite a large sample is required for the limiting ${\chi}_{1}^{2}$ distribution to be a good approximation, in particular with ${\rho}_{0}=0.9$ (Duffee & Stanton, 2008, and references therein).
Remark 20 (Empirical rejection probabilities). At the chosen (nominal) level $\alpha $, with ${q}_{\alpha}$ the corresponding $1\alpha $ quantile of the limiting distribution, the true rejection probability at sample length $n$ is ${\alpha}_{n}=P({\tau}_{n}>\mathrm{}{q}_{\alpha})$. The Monte Carlo estimator is the empirical rejection frequency computed as ${\alpha}_{n,N}=\frac{1}{N}{\displaystyle {\sum}_{i=1}^{N}}\mathbb{I}({p}_{n,i}<\alpha )$, where ${p}_{n,i}$ is the pvalue in Monte Carlo replication $i$, $i=\mathrm{1,2,...,}N$. It follows that the simulation uncertainty of ${\alpha}_{n,N}$ is given by
(Hendry, 1984), and for a correctly sized test, with ${\alpha}_{n}=0.05$ and $N{=10}^{4}$, the 95% confidence bound for ${\alpha}_{n,N}$ is $[\mathrm{0.0456,0.0544}]$. Similar considerations hold for the bootstrap simulations of the test, with ${p}_{n,i}$ replaced by ${p}_{n,B,i}^{\ast}$ (see also Remark 14).
Table 1. Empirical Rejection Frequencies of Asymptotic and Bootstrap Tests in the Homoskedastic and Heteroskedastic Case
Homoskedastic Case 
Heteroskedastic Case 


(A) 
(B) 
(C) 
(D) 
(E) 
(F) 

$n$ 
Asymp. 
iid boot. 
Asymp. 
iid boot. 
Wild boot. 
Wild boot. 

$\text{N(0,1)}$ 
Rademacher 

${\rho}_{0}=\overline{\rho}=0.5$ 

15 
$0.0825$ 
$0.0512$ 
$0.1319$ 
$0.1015$ 
$0.0720$ 
$0.0462$ 

25 
$0.0733$ 
$0.0541$ 
$0.1329$ 
$0.1121$ 
$0.0685$ 
$0.0496$ 

50 
$0.0623$ 
$0.0519$ 
$0.1370$ 
$0.1265$ 
$0.0625$ 
$0.0486$ 

100 
$0.0538$ 
$0.0511$ 
$0.1425$ 
$0.1383$ 
$0.0586$ 
$0.0488$ 

250 
$0.0521$ 
$0.0508$ 
$0.1428$ 
$0.1419$ 
$0.0593$ 
$0.0557$ 

500 
$0.0498$ 
$0.0478$ 
$0.1398$ 
$0.1375$ 
$0.0526$ 
$0.0491$ 

1000 
$0.0495$ 
$0.0493$ 
$0.1434$ 
$0.1463$ 
$0.0515$ 
$0.0502$ 

${\rho}_{0}=\overline{\rho}=0.9$ 

15 
$0.1971$ 
$0.0558$ 
$0.1797$ 
$0.0597$ 
$0.0688$ 
$0.0534$ 

25 
$0.1651$ 
$0.0552$ 
$0.1727$ 
$0.0688$ 
$0.0670$ 
$0.0563$ 

50 
$0.1218$ 
$0.0526$ 
$0.1603$ 
$0.0875$ 
$0.0628$ 
$0.0530$ 

100 
$0.0866$ 
$0.0507$ 
$0.1518$ 
$0.1016$ 
$0.0565$ 
$0.0501$ 

250 
$0.0664$ 
$0.0511$ 
$0.1444$ 
$0.1226$ 
$0.0538$ 
$0.0494$ 

500 
$0.0559$ 
$0.0480$ 
$0.1452$ 
$0.1323$ 
$0.0500$ 
$0.0474$ 

1000 
$0.0548$ 
$0.0516$ 
$0.1463$ 
$0.1376$ 
$0.0515$ 
$0.0503$ 
Note. The data generating process is given by (21) with ${\rho}_{0}\in \left\{\mathrm{0.5,0.9}\right\}$ and ${\delta}_{0}=0$ and the bootstrap process defined in (22). In panels (A) and (B), the innovations ${\epsilon}_{t}$ are i.i.d.$\text{N}(\mathrm{0,}{\sigma}_{0}^{2})$ distributed with ${\sigma}_{0}^{2}=1$, while in panels (C)–(F) ${\epsilon}_{t}$ are independently $\text{N}(\mathrm{0,}{\sigma}_{t}^{2})$ distributed, with ${\sigma}_{t}^{2}$ given in (25). The number of bootstrap replications is $B=399$ and $N=\mathrm{10,000}$ replications.
The iid Bootstrap Test
To illustrate bootstrapbased testing, we apply a restricted recursive bootstrap in terms of residuals estimated under ${\mathcal{H}}_{0}$ (see “Bootstrap Validity Under ${\mathcal{H}}_{0}$”).
Specifically, for steps $\mathcal{A}$ and $\mathcal{B}$ in the section “Bootstrap in the Autoregressive Model,” the bootstrap samples ${\left\{{x}_{t}^{\ast}\right\}}_{t=0}^{n}$ are sampled from
with ${x}_{0}^{\ast}={x}_{0}$ and ${\epsilon}_{t}^{\ast}$ drawn with replacement (for wild, see the section “The Wild bootstrap test” below) from ${\left\{{\tilde{\epsilon}}_{t}^{c}\right\}}_{t=1}^{n},$ where ${\tilde{\epsilon}}_{t}^{c}$ are defined as in^{3} (8) in terms of
For the bootstrap sample, ${\left\{{x}_{t}^{\ast}\right\}}_{t=0}^{n}$, we estimate the unrestricted and restricted models and calculate the bootstrap statistic ${\tau}_{n}^{\ast}$, given by
Here, ${\widehat{\theta}}_{n}^{\ast}=({\widehat{\delta}}_{n}^{\ast},{\widehat{\rho}}_{n}^{\ast},{\widehat{\sigma}}_{n}^{\ast 2}{)}^{\prime}$ and ${\tilde{\theta}}_{n}^{\ast}=({\tilde{\delta}}_{n}^{\ast},\overline{\rho},{\tilde{\sigma}}_{n}^{\ast 2}{)}^{\prime}$ denote the unrestricted and restricted bootstrap estimators, respectively, in terms of the bootstrap loglikelihood function
For step $\mathcal{C}$, the bootstrap test is based on replicating just given arguments to obtain ${\left\{{\tau}_{n,b}^{\ast}\right\}}_{b=1}^{B}\phantom{\rule{0.1em}{0ex}}$ with $B$ denoting the number of bootstrap repetitions. As discussed in Remark 9, the empirical bootstrap pvalue is computed as the tail probability,
With $B=399$ bootstrap repetitions, the empirical rejection frequencies for the $N=\mathrm{10,000}$ Monte Carlo repetitions are presented in column B in Table 1.
The Wild Bootstrap Test
The wild bootstrap design is as given in the section “iid Bootstrap Test,” except that ${\epsilon}_{t}^{\ast}$ for the wild bootstrap is resampled by
with ${w}_{t}^{\ast}$ i.i.d. $\left(\mathrm{0,1}\right)$ distributed and ${\tilde{\epsilon}}_{t}$ defined in (23). In the simulations, ${w}_{t}^{\ast}$ is chosen as $*N(\mathrm{0,1})$ and Rademacher distributed, respectively (Remark 10).
For the simulations reported in Table 1, ${\epsilon}_{t}$ are assumed not to be i.i.d. $\text{N}\left(\mathrm{0,}{\sigma}_{0}^{2}\right)$ distributed in order to illustrate the impacts of heteroskedasticity. Specifically, we set ${\epsilon}_{t}\stackrel{d}{=}\text{N}(\mathrm{0,}{\sigma}_{t}^{2})$ with ${\sigma}_{t}^{2}=1$ for $t=\mathrm{1,2,...,}[n/2]$ and ${\sigma}_{t}^{2}=15$ for $t=[n/2]+\mathrm{1,...,}n$. That is,
with ${\sigma}_{\mathrm{0,1}}^{2}=1$ and ${\sigma}_{\mathrm{0,2}}^{2}=15$. In this case the asymptotic test is not consistent as demonstrated in column A of Table 1, which reports the empirical rejection probabilities for the asymptotic test. The asymptotic test is severely oversized—and even for $n=500$ the empirical size is not close to $\alpha =0.05$. Also the iid bootstrap is not asymptotically valid. With the iid bootstrap, as in the section “iid Bootstrap Test,” the results are reported in column D, and we observe that the results are similar to the asymptotic test. Intuitively, this reflects that the bootstrap series, ${\left\{{x}_{t}^{\ast}\right\}}_{t=0}^{n}$, does not mimic the properties of the original data series ${\left\{{x}_{t}\right\}}_{t=0}^{n}$. This can be illustrated by Figure 1, where panel A shows pronounced heteroskedasticity of the estimated residuals, ${\tilde{\epsilon}}_{t}$, for one sample. Panel B shows a single iid resampled sample ${\left\{{\epsilon}_{t}^{\ast}\right\}}_{t=1}^{n}$ from ${\left\{{\tilde{\epsilon}}_{t}\right\}}_{t=1}^{n}$, and as, by definition, in particular the ordering change, the ${\epsilon}_{t}^{\ast}$ series does not mimic the heteroskedasticity as seen in the estimated residuals in panel A.
For the wild bootstrap, columns E and F in Table 1 report the empirical rejection frequencies for the wild bootstrap test with, as mentioned, ${w}_{t}^{\ast}$ distributed as $\text{N}\left(\mathrm{0,1}\right)$ and Rademacher, respectively. The empirical size for the wild bootstrap is quite close to the nominal level, with the Rademacher distribution performing slightly better. Likewise, panels C and D in Figure 1 illustrate that the wild bootstrap ${\epsilon}_{t}^{\ast}$ series more closely mimisc the properties of the original ${\tilde{\epsilon}}_{t}$ series.
Bootstrap Under the Alternative
To establish asymptotic validity, it was shown that ${\tau}_{n}^{\ast}$ is also asymptotically ${\chi}^{2}$distributed under the alternative, leading to a consistent bootstrap test. To illustrate this, we consider the empirical probability of rejecting a false hypothesis.
Specifically for the iid bootstrap test in the section “iid Bootstrap Test,” let the data ${x}_{t}$ be generated with true value ${\theta}_{0}=({\rho}_{0},{\sigma}_{0}^{2},{\delta}_{0}{)}^{\prime}$ as before. The hypothesis of interest is ${\mathcal{H}}_{0}:\rho =\overline{\rho}$ and we let $\overline{\rho}\in \left\{\mathrm{0.875,...,0.70}\right\}$ to illustrate the empirical power of the bootstrap test and asymptotic test. From Table 2 it follows that the bootstrap is comparable to the asymptotic test in terms of empirical power.
Table 2. Empirical Rejection Frequencies of Asymptotic and Bootstrap Tests Under the Alternative
$\overline{\rho}$ 
Asymptotic 
iid bootstrap 

$0.9$ 
$0.0651$ 
$0.0485$ 
$0.875$ 
$0.0980$ 
$0.0810$ 
$0.85$ 
$0.2911$ 
$0.2658$ 
$0.825$ 
$0.5533$ 
$0.5283$ 
$0.8$ 
$0.7730$ 
$0.7567$ 
$0.75$ 
$0.9617$ 
$0.9578$ 
$0.7$ 
$0.9958$ 
$0.9954$ 
Note. The datagenerating process is given by (21) with $n=250$ and ${\rho}_{0}=0.9$, such that ${\rho}_{0}\ne \overline{\rho}$ except for the first row entry. The number of bootstrap replications is $B=399$ and $N=\mathrm{10,000}$ replications.
Moment Condition, $\text{E}{\epsilon}_{t}{}^{k}<\infty $ for $k\ge 2$
When establishing asymptotic validity of the bootstrap the moment condition, $\text{E}({\epsilon}_{t}^{4})<\infty $ was discussed, and it was mentioned that while it is a sufficient condition, it may not be necessary. On the other hand, $\text{E}({\epsilon}_{t}^{2})<\infty $ seems necessary unless the bootstrap algorithm is based on permutation, see also Table 3 and the text before this.
To illustrate this, we consider the data ${x}_{t}$ as generated with true value ${\theta}_{0}=({\rho}_{0},{\sigma}_{0}^{2},{\delta}_{0}{)}^{\prime}$ as before for samples of size $n$, $n\in \left\{\mathrm{15,...,1000}\right\}$. The i.i.d. innovations ${\epsilon}_{t}$ are simulated from the Student’s ${t}_{v}$distribution, where the degrees of freedom $v$, $v\in \{3/\mathrm{2,3,5}\}$. Specifically, for $v=3/2$, $\text{E}{\epsilon}_{t}{}^{k}$ is finite, only for $k<3/2$, thus allowing first order, but not second order, moments of ${\epsilon}_{t}$. For $v=3$ and $v=5$, the second and finite fourthorder moments are finite, respectively. Also note that the Gaussian case is included as a reference. Table 3 shows that the asymptotic test (based on the ${\chi}_{1}^{2}$ approximation) for all the four cases has empirical rejection rates far from the nominal level of $\alpha =0.05$, even for large samples $n$, where $n\in \left\{\mathrm{15,...,1000}\right\}$.
As to the bootstrap design, Table 3 reports bootstrap simulations based on iid sampling and the wild (Rademacher). For $v=3/2$, as expected, neither the wild nor the iid bootstraps have empirical rejection rates close to the nominal level. For $v\ge 3$, both the wild and iid bootstrap work surprisingly well, even when $v=3$.
Additionally, Table 3 reports bootstrapbased testing where ${\epsilon}_{t}^{\ast}$ are iid sampled, but without replacement. This, which is referred to as the permutation bootstrap, works well in terms of empirical rejection frequencies, even for $v=3/2$. In general, the permutation bootstrap works well in the context of heavytailed i.i.d. ${\epsilon}_{t}$ (see Cavaliere, Nielsen, & Rahbek, 2020, and also the discussion in “HeavyTailed Autoregressive Models” for AR models with heavytailed innovations).
Table 3. Empirical Rejection Frequencies With Heavy Tailed Innovations
Asymptotic 
Iid 
Wild 
Permutation 


$n$ 
Student’s ${t}_{\nu},\phantom{\rule{0.3em}{0ex}}\nu =3/2$ 

15 
0.1434 
0.0395 
0.0248 
0.0499 
25 
0.0959 
0.0382 
0.0256 
0.0520 
50 
0.0563 
0.0350 
0.0268 
0.0488 
100 
0.0363 
0.0340 
0.0262 
0.0443 
250 
0.0290 
0.0350 
0.0326 
0.0480 
500 
0.0249 
0.0333 
0.0322 
0.0465 
1000 
0.0278 
0.0351 
0.0348 
0.0513 
$n$ 
Student’s ${t}_{\nu},\phantom{\rule{0.3em}{0ex}}\nu =3$ 

15 
0.1809 
0.0508 
0.0494 
0.0508 
25 
0.1512 
0.0491 
0.0475 
0.0522 
50 
0.1129 
0.0494 
0.0496 
0.0521 
100 
0.0806 
0.0491 
0.0486 
0.0505 
250 
0.0596 
0.0487 
0.0514 
0.0490 
500 
0.0578 
0.0528 
0.0515 
0.0534 
1000 
0.0499 
0.0465 
0.0486 
0.0481 
$n$ 
Student’s ${t}_{\nu},\phantom{\rule{0.3em}{0ex}}\nu =5$ 

0.1894 
0.0519 
0.0531 
0.0511 

25 
0.1571 
0.0486 
0.0515 
0.0499 
50 
0.1119 
0.0523 
0.0495 
0.0507 
100 
0.0807 
0.0503 
0.0497 
0.0503 
250 
0.0639 
0.0486 
0.0477 
0.0486 
500 
0.0542 
0.0484 
0.0478 
0.0485 
1000 
0.0540 
0.0498 
0.0501 
0.0503 
Note. Empirical rejection frequencies of asymptotic and bootstrap tests with ${\epsilon}_{t}$ i.i.d. Student’s ${t}_{\nu}$ for $\nu \in \{3/\mathrm{2,3,5}\}$. Results are reported for the iid bootstrap, the wild boostrap with ${w}_{t}$ Rademacher distributed, and the permutation bootstrap. Simulations are reported for sample lengths $n\in \left\{\mathrm{15,...,1000}\right\}$. The number of bootstrap replications is $B=399$ and $N=\mathrm{10,000}$ replications.
A Selection of Further Topics
In the previous sections the simple AR model of order one was used to introduce key ideas and challenges of the iid and wild bootstrap schemes when applied to testing the hypothesis ${\mathcal{H}}_{0}:\rho =\overline{\rho}$. In this section, we provide an overview of recent selected results for the bootstrap when applied to different testing problems in econometric time series models. The overview is not meant to be exhaustive (see, e.g., Cavaliere & Rahbek, 2020, for a review of the bootstrap with more technical details, as well the references therein).
Nonstationary (Vector) Autoregressive Models
The iid Bootstrap
Consider initially the AR model in (1) again. The hypothesis of nonstationarity is given by ${\mathcal{H}}_{0}:\rho =\mathrm{1,}$ or equivalently, with $\pi =\rho 1$, ${\mathcal{H}}_{0}:\pi =0$, in the AR model restated as
where $\mathrm{\Delta}{x}_{t}={x}_{t}{x}_{t1}$. It follows that the teststatistic ${\tau}_{n}$ in (5) can be written as
and under ${\mathcal{H}}_{0}$,
where $B\left(u\right)$ is a standard Brownian motion, $u\in \left(\mathrm{0,1}\right)$, and $\tau $ is the (squared) DickeyFuller distribution (Hamilton, 1994). In this case, in terms of the bootstrap in “Bootstrap in the Autoregressive Model,” the restricted recursive bootstrap, given by $\mathrm{\Delta}{x}_{t}^{\ast}={\epsilon}_{t}^{\ast}$ with ${\epsilon}_{t}^{\ast}$ iid resampled, is asymptotically valid. This holds as in this case ${\tau}_{n}^{\ast}{\stackrel{{d}^{\ast}}{\to}}_{p}\tau $, under both ${\mathcal{H}}_{0}$ and the alternative. In contrast, and as discussed in Basawa, Mallik, McCormick, Reeves, and Taylor (1991), the unrestricted recursive bootstrap based on the recursion $\mathrm{\Delta}{x}_{t}^{\ast}={\widehat{\pi}}_{n}{x}_{t1}^{\ast}+{\epsilon}_{t}^{\ast}$, with ${\epsilon}_{t}^{\ast}$ iid resampled, is invalid. This follows as the corresponding bootstrap statistic, ${\tau}_{n}^{\ast}$, under ${\mathcal{H}}_{0}$ converges in distribution to $\tilde{\tau}$, $\tilde{\tau}\ne \tau $. Precisely, the bootstrap conditional distribution function converges weakly rather than in probability (Basawa et al., 1991; Cavaliere & Georgiev, 2020; Cavaliere et al., 2015).
The univariate case of testing for nonstationarity is a special case of the more general hypothesis of nonstationarity in vector AR models for ${X}_{t}\in {\mathbb{R}}^{p}$ with general lagstructure, as given by
where ${\epsilon}_{t}$ are i.i.d. ${\text{N}}_{p}\left(\mathrm{0,}\Omega \right)$ distributed and the initial values $\left({X}_{0},\mathrm{\Delta}{X}_{0}\mathrm{,...,}\mathrm{\Delta}{X}_{1k}\right)$ are fixed in the statistical analysis. Moreover, $\pi $ and ${\left({\gamma}_{i}\right)}_{i=1}^{k}$ are $p\times p$ matrices. The hypothesis of nonstationarity of ${X}_{t}$ is given by the hypothesis of reduced rank $r,\phantom{\rule{0.1em}{0ex}}0\le r<p$, of $\pi $ (Johansen, 1996). Specifically, with ${\mathcal{H}}_{r}:\text{rank}\left(\pi \right)\le r$, it follows that this may be written as
where $\alpha $ and $\beta $ are $\left(p\times r\right)$ dimensional matrices. Under the nonstationarity conditions in Johansen (1996, Theorem 4.2), it follows that ${X}_{t}$ is a nonstationary process, with $r$ stationary, or cointegrating, relations ${\beta}^{\prime}{X}_{t}$, and $\left(pr\right)$ common trends given by ${\delta}^{\prime}{\displaystyle {\sum}_{i=1}^{t}}{\epsilon}_{i}$, with $\delta $$\left(p\times \left(pr\right)\right)$ dimensional of full rank, and such that ${\delta}^{\prime}\alpha =0$. The likelihoodratio statistic ${\tau}_{n}\left(r\right)$ for cointegration rank $r$ satisfies, under ${\mathcal{H}}_{r}$ and the mentioned nonstationarity conditions, that
which is a multivariate version of (27) in terms of the $\left(pr\right)$dimensional standard Brownian motion, $B(\cdot )$. Cavaliere, Rahbek, and Taylor (2012) consider the recursive restricted bootstrap based on
with ${\pi}_{n}^{\ast}={\tilde{\alpha}}_{n}{{\tilde{\beta}}^{\prime}}_{n}$ and ${\gamma}_{n,i}^{\ast}={\tilde{\gamma}}_{n,i}$; that is, the bootstrap true values are given by the estimators under ${\mathcal{H}}_{r}$. Asymptotic validity of the iid bootstrap is established in Cavaliere et al. (2012), by showing that ${\tau}_{n}^{\ast}\left(r\right){\stackrel{{d}^{\ast}}{\to}}_{p}\tau \left(r\right)$ under ${\mathcal{H}}_{r}$ and the alternative.
Cavaliere et al. (2015) extend the analysis to hypothesis testing on the cointegration (matrix) parameter $\beta $. Specifically, Cavaliere et al. (2015, Proposition 1 and Theorem 1) establish that under the hypothesis ${\mathcal{H}}_{\overline{r}}:\beta =\overline{\beta}$, the bootstrap likelihood ratio statistic, ${\tau}_{n}^{\ast}$, satisfies ${\tau}_{n}^{\ast}{\stackrel{{d}^{\ast}}{\to}}_{p}{\chi}_{\left(pr\right)r}^{2}$. Importantly, it is also established that under the alternative, ${\tau}_{n}^{\ast}$ has a limiting distribution (in distribution) in terms of a diffusion process with a stochastic diffusion coefficient, and hence it is bounded in probability such that the bootstrapbased test is asymptotically valid.
The Wild Bootstrap
In order to allow for possible heteroskedasticity in the ${\epsilon}_{t}$ sequence in (28), the application of the wild bootstrap has also been studied. Results for application of the wild bootstrap in general lag univariate AR models in Gonçalves and Kilian (2004), with ${\epsilon}_{t}$ allowed to have general timevarying volatility structures, such as ARCH and stochastic volatility, have been generalized to the testing the hypothesis of cointegration ${\mathcal{H}}_{r}$ in Cavaliere, Rahbek, and Taylor (2010a, 2010b, 2014) and Boswijk, Cavaliere, Rahbek, and Taylor (2016).
Moreover, Boswijk et al. (2016) and Boswijk, Cavaliere, Georgiev, and Rahbek (2019) consider general hypothesis testing on the cointegration parameters $\alpha $ and $\beta $, with $\pi =\alpha {\beta}^{\prime}$ in (28). They consider the case of stochastic volatility, where ${\epsilon}_{t}={\Omega}_{t}^{1/2}{z}_{t}$, with the $p$dimensional ${z}_{t}$ i.i.d. $(\mathrm{0,1})$ and the timevarying $\left(p\times p\right)$dimensional ${\Omega}_{t}=\Omega \left(t/n\right)$. Moreover, with “$\stackrel{w}{\to}$“ denoting weak convergence, it is assumed that for $u\in \left(\mathrm{0,1}\right)$,
where $B$ is a $p$dimensional standard Brownian motion, which generalizes the i.i.d. $\left(\mathrm{0,}\Omega \right)$ assumption, where ${n}^{1/2}{\displaystyle {\sum}_{t=1}^{[nu]}}{\epsilon}_{t}\stackrel{w}{\to}{\Omega}^{1/2}B\left(u\right)$. Specifically, the limiting process in (29) is a continuoustime martingale, with in general an unknown covariance (kernel). This implies that the limiting distribution of the test statistic(s) ${\tau}_{n}$, for example for the mentioned hypotheses ${\mathcal{H}}_{r}$ and ${\mathcal{H}}_{\overline{r}}$, will depend on unknown nuisance parameters, which again means asymptotic inference is infeasible in practice. In contrast, for the wild bootstrap, it is established that ${n}^{1/2}{\displaystyle {\sum}_{t=1}^{[nu]}}{\epsilon}_{t}^{\ast}$ has the same limiting distribution (in probability), and, as a result, the wild bootstrap is asymptotically valid as shown in Boswijk et al. (2016, 2019), under some additional regularity conditions to be verified.
TimeVarying Conditional Volatility Models
As discussed in Andrews (2000), applying bootstrapbased testing in ARCH models is in general difficult, and may be invalid in certain cases, due to general problems arising when testing hypotheses in time series models when one or more parameters under the null may be “on the boundary of the parameter space” (p. 399).
To illustrate, consider here ${x}_{t}$ given by a linear ARCH model of order $q$
with ${z}_{t}$ i.i.d. $\left(\mathrm{0,1}\right)$ and
In the statistical analysis, the initial values $\left({x}_{0}\mathrm{,...,}{x}_{q+1}\right)$ are fixed, and the parameter $\theta ={\left(\omega ,{\alpha}_{1}\mathrm{,...,}{\alpha}_{q}\right)}^{\prime}\in \Theta $, where
Thus by definition of the parameter space $\Theta $, if for the true value ${\theta}_{0}={\left({\omega}_{0},{\alpha}_{\mathrm{0,1}}\mathrm{,...,}{\alpha}_{\mathrm{0,}q}\right)}^{\prime}$ it holds that ${\alpha}_{\mathrm{0,}j}=0$ for some $j$, the true value ${\theta}_{0}$ is on the boundary of $\Theta $.
The fact that it is unknown a priori which of the ARCH coefficients may or may not be zero leads to nonpivotal limiting distributions of test statistics and estimators. Consider here the likelihood ratio statistic ${\tau}_{n}$ for the nullity of the $q$th order ARCH coefficient; that is, the hypothesis ${\mathcal{H}}_{q}:{\alpha}_{q}=0$. With the Gaussian likelihood function given by
by definition, ${\tau}_{n}=2({\mathcal{l}}_{n}({\widehat{\theta}}_{n}){\mathcal{l}}_{n}({\tilde{\theta}}_{n}))\mathrm{}$, where the unrestricted^{4} Gaussian MLE is given by ${\widehat{\theta}}_{n}=\mathrm{arg}{\mathrm{max}}_{\theta \in \Theta}{\mathcal{l}}_{n}\left(\theta \right)$, while ${\tilde{\theta}}_{n}$ is the Gaussian MLE under ${\mathcal{H}}_{q}$. By Andrews (1999, 2001), it follows that ${\tau}_{n}$ has a limiting distribution, which is nonstandard. In addition, the limiting distribution of ${\tau}_{n}$ is nonpivotal as, crucially, it depends on whether ${\alpha}_{\mathrm{0,}i}>0$, or ${\alpha}_{\mathrm{0,}i}=0\phantom{\rule{0.1em}{0ex}}$ for $i=\mathrm{1,...,}q1$ under ${\mathcal{H}}_{q}$.
While this implies that the unrestricted bootstrap is invalid (Andrews, 2000), it follows by Cavaliere, Nielsen, and Rahbek (2017) that the iid restricted bootstrap is asymptotically valid under mild conditions for the simple case of the first order ARCH with $q=1$. Moreover, Cavaliere, Nielsen, Pedersen, and Rahbek (2020) demonstrate validity of a modified restricted bootstrap, which can be applied for general testing problems in parametric models with parameters on the boundary under the null. Specifically, for the case of ARCH of order $q$, consider the bootstrap process
with ${z}_{t}^{\ast}$ iid resampled from ${\widehat{z}}_{t}={x}_{t}/{\sigma}_{t}({\widehat{\theta}}_{n})$, after recentering and rescaling these. The bootstrap conditional volatility process ${\sigma}_{t}^{\ast 2}\left({\theta}_{n}^{\ast}\right)$ is given by
with ${\omega}_{n}^{\ast}={\tilde{\omega}}_{n}$ and ${\alpha}_{n,i}^{\ast}={\tilde{\alpha}}_{n,i}\mathbb{I}\left({\tilde{\alpha}}_{n,i}>\mathrm{}{c}_{n}\right)$, with ${c}_{n}$ a deterministic sequence that satisfies (a) ${c}_{n}\to 0$, and (b) ${n}^{1/2}{c}_{n}\to \infty $, as $n\to \infty $. The bootstrap scheme is referred to as modified since in (30) the bootstrap true values ${\alpha}_{n,i}^{\ast}$—by “ shrinking”—are set to zero for $i=\mathrm{1,...,}q1$, provided ${\tilde{\alpha}}_{n,i}$ is small relative to ${c}_{n}$. Note that with the timevarying bootstrap volatility defined by (30), this is a case of a fixed design (or rather fixed volatility) bootstrap (see step $\mathcal{A}$ in “Bootstrap in the Autoregressive Model”). The general fixed volatility bootstrap for ARCH models is considered in Cavaliere et al. (2018), and the modified, by shrinking, fixed volatility bootstrap is shown in Cavaliere, Nielsen, Pedersen, and Rahbek (2020, Proposition 1) to be asymptotically valid. Simulations there show that both the fixed volatility and the recursive with ${x}_{ti}^{2}={x}_{ti}^{\ast 2}$ in (30) bootstrap based tests have empirical rejection frequencis that are close to the nominal level for small and moderate sample sizes $n$. Moreover, as for the discussion of the moment requirements for the AR bootstrap, simulations indicate that while sufficient, the moment constraints on the original ARCH process imposed to establish validity are also not necessary.
Double Autoregressive Models
The double autoregressive (DAR) model combines the AR and ARCH models, as both the conditional mean and conditional variance depend on lagged levels of the process (see Ling, 2004, 2007; and for a multivariate “cointegrated” version, Nielsen & Rahbek, 2014).
Consider here the first order DAR model as given by
with ${z}_{t}$ i.i.d. $\text{N}\left(\mathrm{0,1}\right)$, ${x}_{0}$ is fixed in the statistical analysis and the parameter given by $\theta ={\left(\pi ,{\sigma}^{2},\alpha \right)}^{\prime}\in \Theta ,$ with $\Theta =\left\{\theta \in {\mathbb{R}}^{3}:{\sigma}^{2}>0\text{and}\alpha \ge 0\right\}$.
A notable special feature of the DAR process is that for $\pi =0$ the process is strictly stationary for any $0<\alpha <2.42$ (Borkovec & Klüppelberg, 2001; Ling, 2004, 2007), while the process is nonstationary when $\pi =\alpha =0$. With $\pi =0$ and $\alpha \in \left(\mathrm{0,2.42}\right)$, while being strictly stationary the DAR process ${x}_{t}$ has infinite variance (and only finite fractional moments).
From the specification of the parameter space $\Theta $ evidently for ${\alpha}_{0}=0$, the true value ${\theta}_{0}$ is on the boundary, raising the issues discussed in the section “TimeVarying Conditional Volatility Models” in relation to the ARCH model. Asymptotic theory for the Gaussian likelihoodbased MLE with ${\alpha}_{0}>0$ is given by Ling (2004), while Cavaliere and Rahbek (2020) extend the results to allow for the boundary case. Klüppelberg, Maller, van de Vyver, and Wee (2002) derive the asymptotic distribution of the likelihood ratio statistic ${\tau}_{n}$ for the hypothesis of nonstationarity as given by ${\mathcal{H}}_{0}:\pi =\alpha =0$ (see also Chen, Li, & Ling, 2013). Different versions of a bootstrapbased test are discussed in Cavaliere and Rahbek (2020). In particular, validity is established for a restricted bootstrap given by $\mathrm{\Delta}{x}_{t}^{\ast}={\tilde{\sigma}}_{n}{z}_{t}^{\ast}$, where ${\tilde{\sigma}}_{n}^{2}$ is the MLE of ${\sigma}^{2}$ under ${\mathcal{H}}_{0}$, while ${z}_{t}^{\ast}$ is obtained by iid resampling of unrestricted residuals, ${\widehat{z}}_{t}$ (recentered and rescaled). That is, the unrestricted residuals ${\widehat{z}}_{t}$ are given by
with ${\widehat{\theta}}_{n}={\left({\widehat{\pi}}_{n},{\widehat{\sigma}}_{n},{\widehat{\alpha}}_{n}\right)}^{\prime}$ the unrestricted MLE. In line with the discussion in the section “Bootstrap in the Autoregressive Model” regarding restricted and unrestricted residuals, this choice ensures that ${\widehat{z}}_{t}$ for large $n$ is “close” to the true ${z}_{t}$, irrespective of whether the null ${\mathcal{H}}_{0}$ is true or not. While the validity result is shown for this choice, simulations indicate that in practice the difference between choosing to resample from ${\widehat{z}}_{t}$ or from ${\tilde{z}}_{t}=\mathrm{\Delta}{x}_{t}/{\tilde{\sigma}}_{n}$ is negligible. (See Cavaliere & Rahbek, 2020, for a detailed discussion of this as well as asymptotic theory for different bootstraps.)
HeavyTailed Autoregressive Models
So far results for the (vector) AR models have been derived under the assumption that the innovations ${\epsilon}_{t}$ have mean zero and a finite variance ${\sigma}^{2}$, or some timevarying, possibly conditional, variance ${\sigma}_{t}^{2}$ when discussing heteroskedasticity. To allow for more extreme events, and phenomena such as “ bubble” periods with local explosive behavior, this assumption was relaxed in Davis and Resnick (1985a, 1985b, 1986) and Davis and Song (2020), where the i.i.d. innovations ${\epsilon}_{t}$ are allowed to have infinite variance. Specifically, they consider the case of ${\epsilon}_{t}$ i.i.d. with a stable distribution such as the Cauchy; that is, “heavytailed” as the tails of the distribution of ${\epsilon}_{t}$ are assumed to decay at a rate that is slower than the Gaussian (exponential) rate.
Two key examples are given by the classic AR model and the socalled noncausal AR model of order one in terms of i.i.d. stable distributed ${\epsilon}_{t}$,
For the standard, and hence causal, $\text{AR}$, recall that with $t=\mathrm{1,...,}n$, ${x}_{0}$ is the initial value that is fixed in the statistical analysis, while for the noncausal ${\text{AR}}^{+}$${x}_{n}$ is the “initial value” due to the forward recursion. Noncausal ${\text{AR}}^{+}$ type models have become popular as they seem to capture well the dynamics of phenomena such as bubbles where, after period of exponential type growth, the process “collapses.” Interestingly, and linking the heavytail ${\text{AR}}^{+}$ models with the DAR model in “Double Autoregressive Models,” the ${\text{AR}}^{+}$ process in (32) can be shown to have a causal “semistrong” representation as the DAR process in (31) (see Gourieroux & Zakoian, 2017).
Consider testing the hypothesis ${\mathcal{H}}_{0}^{+}:{\rho}^{+}=\overline{\rho}$ using the Gaussian likelihoodbased statistic ${\tau}_{n}^{+}$, given by
where ${\widehat{\rho}}_{n}^{+}={\displaystyle {\sum}_{t=1}^{n1}}{x}_{t}{x}_{t+1}/{\displaystyle {\sum}_{t=1}^{n}}{x}_{t+1}^{2}$ and ${\widehat{\sigma}}_{n}^{2}={n}^{1}{\displaystyle {\sum}_{t=1}^{n1}}{\left({x}_{t}{\widehat{\rho}}_{n}^{+}{x}_{t1}\right)}^{2}$. While the test statistic ${\tau}_{n}^{+}$ is analogous to the (square root of the) previously studied statistic ${\tau}_{n}$ in (5), the limiting distribution is nonstandard, as the ${\epsilon}_{t}$ are assumed to be i.i.d. stable distributed. For example, with ${\epsilon}_{t}$ Cauchy distributed, $(n/\mathrm{log}n)(\mathrm{}{\widehat{\rho}}_{n}^{+}\overline{\rho})$ is asymptotically distributed as $\left(1+\overline{\rho}\right)\mathcal{C}{\chi}_{1}^{2}$, where $\mathcal{C}$ is standard Cauchy distributed and ${\tau}_{n}^{+}={O}_{P}\left({n}^{1/2}\mathrm{log}n\right)$. In general for stable distributions asymptotic testing is infeasible as the limiting distributions depend on the “tail index” (which is one for the Cauchy) of the innovations ${\epsilon}_{t}$, and moreover the normalization (which is ${n}^{1/2}\mathrm{log}n$ for the Cauchy case) depend on further, and in practice, unknown quantities. In the case of stable distributed innovations, Cavaliere, Nielsen, and Rahbek (2020) discuss validity of the recursive bootstrap scheme similar to (10),
initialized with ${x}_{n}^{\ast}={x}_{n}$, and with ${\epsilon}_{t}^{\ast}$ resampled from the restricted residuals ${\left\{{\tilde{\epsilon}}_{t}^{+}\right\}}_{t=1}^{n1}$, where ${\tilde{\epsilon}}_{t}^{+}={x}_{t}\overline{\rho}{x}_{t+1}$. Crucially the ${\epsilon}_{t}^{\ast}$ are not sampled by iid resampling with replacement, as this would lead to an invalid bootstrap test (Athreya, 1987; Knight, 1989). Instead, ${\epsilon}_{t}^{\ast}$ are resampled without replacement; that is, by permuting ${\left\{{\tilde{\epsilon}}_{t}^{+}\right\}}_{t=1}^{n1}$, or, in combination with the wild, by permuting ${\left\{{\tilde{\epsilon}}_{t}^{+}{w}_{t}^{\ast}\right\}}_{t=1}^{n1}$, where ${w}_{t}^{\ast}$ are i.i.d. Rademacher distributed (Remark 10). With ${\tau}_{n}^{+\ast}$ the bootstrap statistic based on the permutation, or the combined permutationwild bootstrap, Theorem 1 in Cavaliere, Nielsen, and Rahbek (2020) establishes validity of the bootstrapbased test under the null hypothesis for general ${\text{AR}}^{+}$ models.
Similarly, Cavaliere, Georgiev, and Taylor (2016) establish bootstrap validity for the socalled sieve bootstrap in Bühlmann (1997) for general causal AR models with heavy tails. Also note that in terms of testing for the presence of bubbles based on onesided testing using the supremum of recursively computed DickeyFuller type statistics as in Phillips, Wu, and Yu (2011), Harvey, Leybourne, Sollis, and Taylor (2016) establish validity of a wild bootstrap based test to allow for heteroskedasticity.
Conclusions and Further Readings
This article has provided an introduction to key steps required for a successful implementation of bootstrap hypothesis testing to time series models. In the framework of a simple autoregressive model, we have discussed the (largesample) validity of recursive bootstrap algorithms, where the bootstrap sample is constructed by mimicking the dependence structure of the original data, as implied by the econometric model at hand. We have discussed the main requirements for bootstrap inference to be valid; that is, to mimic the asymptotic distribution of the original test statistic under the null hypothesis and to be bounded (in probability) when the null hypothesis is false. To do so, we have introduced the needed probability tools (which involve dealing with the conditional—hence random—nature of the bootstrap measure) and shown how to apply them in order to assess the large sample behavior of the bootstrap estimators and test statistics.
Needless to say, this article is not meant to be exhaustive or to provide a comprehensive treatment of bootstrap inference. We have focused on bootstrap hypothesis testing which, although being one of the most common applications of the bootstrap, is not the only one. Further common implementations of the bootstrap involve computation of standard errors, the construction of confidence intervals for the parameters of interest, and bias correction of the estimators (see, inter alia, Horowitz, 2001, or MacKinnon, 2006, for reviews).
This article is also not exhaustive in terms of bootstrap algorithms. For instance we have not discussed bootstrap algorithms based on resampling a block of observations (or residuals) rather than resampling single observations (or residuals). As introduced by Künsch (1989), block bootstrap methods are quite flexible and powerful in cases where one does not have strong a priori beliefs or knowledge about the dependence structure of the data (see also Lahiri, 2003; Politis & Romano, 1994; Shao, 2010). Subsampling methods represent a valid alternative (or complement) to recursive, modelbased bootstrap (Politis, Romano, & Wolf, 1999). We have focused on a case (the first order AR model) where the datagenerating process is described by a finite dimensional vector of parameters. Extensions to infinitedimensional parameter spaces, as for the case of general linear processes, are available in the literature; a classic example is the $\text{AR}(\infty \text{)}$ sieve bootstrap of Kreiss (1988, 1992), Bühlmann (1997), and Gonçalves and Kilian (2007).
It is important to emphasize that there is a rich literature discussing finitesample improvements of the bootstrap in various models, including time series models, based on Edgeworth expansions of the ${\tau}_{n}$ and ${\tau}_{n}^{\ast}$ statistics, as briefly mentioned in Remark 10. An introduction to Edgeworth expansions as applied for the bootstrap can be found in van der Vaart (2000, chapter 23.3; see also Hall, 1992; Horowitz, 2001).
Finally, it is worth noticing that throughout this article bootstrap validity under the null is defined as the fact that under the null the bootstrap test statistic converges to the asymptotic (null) distribution of the original statistic. This implies that, under rather mild regularity conditions (such as continuity of the limiting distribution), if the null hypothesis holds then the bootstrap $p$value ${p}_{n}^{\ast}$ is asymptotically uniformly distributed (Remark 15). Combined with establishing that the bootstrap statistic is bounded in probability under the alternative such that ${p}_{n}^{\ast}\stackrel{p}{\to}0$, this was used to establish validity of the bootstrap (see “Bootstrap Validity Under the Alternative”). Cavaliere and Rahbek (2020) discuss in detail different definitions of asymptotic validity and their verification under different assumptions. An example is Cavaliere and Georgiev (2020), where validity of the bootstrap test is defined directly in terms of asymptotic uniformity of the bootstrap $p$values, rather than the properties of (consistent) estimation of the limiting null distribution of the original test statistic. By doing so, the bootstrap can be employed in cases where the limiting distribution of the original statistic may not be well defined, or in cases where the boostrap distribution does not converge in probability but, rather, converges in distribution (see, e.g., Boswijk, Cavaliere, Georgiev, & Rahbek, 2019, and the discussions in sections “Nonstationary [Vector] Autoregressive Models” and “HeavyTailed Autoregressive Models” where this was applied).
Acknowledgments
This research was supported by Danish Council for Independent Research (DSF Grant 701500028B).
References
 Andrews, D. W. K. (1999). Estimation when a parameter is on a boundary. Econometrica, 67(6), 1341–1383.
 Andrews, D. W. K. (2000). Inconsistency of the bootstrap when a parameter is on the boundary of the parameter space. Econometrica, 68(2), 399–405.
 Andrews, D. W. K. (2001). Testing when a parameter is on the boundary of the maintained hypothesis. Econometrica, 69(3), 683–734.
 Andrews, D. W. K., & Buchinsky, M. (2000). A threestep method for choosing the number of bootstrap repetitions. Econometrica, 68, 23–51.
 Athreya, K. B. (1987). Bootstrap of the mean in the infinite variance case. Annals of Statistics, 15, 724–731.
 Basawa, I. V., Mallik, A. K., McCormick, W. P., Reeves, J. H., & Taylor, R. L. (1991). Bootstrapping unstable firstorder autoregressive processes. Annals of Statistics, 19, 1098–1101.
 Berkowitz, J., & Kilian, L. (2000). Recent developments in bootstrapping time series. Econometric Reviews, 19(1), 1–48.
 Borkovec, M., & Klüppelberg, C. (2001). The tail of the stationary distribution of an autoregressive process with ARCH(1) errors. Annals of Applied Probability, 11(4), 1220–1241.
 Boswijk, H. P., Cavaliere, G., Georgiev, I., & Rahbek, A. (2019). Bootstrapping nonstationary stochastic volatility. Working Paper TI 2019–083/III. Tinbergen Institute.
 Boswijk, H. P., Cavaliere, G., Rahbek, A., & Taylor, A. M. R. (2016). Inference on cointegration parameters in heteroskedastic vector autoregressions. Journal of Econometrics, 192, 64–85.
 Bühlmann, P. (1997). Sieve bootstrap for time series. Bernoulli, 3, 123–148.
 Cavaliere, G., & Georgiev, I. (2020). Inference under random limit bootstrap measures. Econometrica.
 Cavaliere, G., Georgiev, I., & Taylor, A. M. R. (2016). Sievebased inference for infinitevariance linear processes. Annals of Statistics, 44(4), 1467–1494.
 Cavaliere, G., Nielsen, H. B., Pedersen, R. S., & Rahbek, A. (2020). Bootstrap inference on the boundary of the parameter space with application to conditional volatility models. Journal of Econometrics.
 Cavaliere, G., Nielsen, H. B., & Rahbek, A. (2015). Bootstrap testing of hypotheses on cointegration relations in vector autoregressive models. Econometrica, 83(2), 813–831.
 Cavaliere, G., Nielsen, H. B., & Rahbek, A. (2017). On the consistency of bootstrap testing for a parameter on the boundary of the parameter space. Journal of Time Series Analysis, 38(4), 513–534.
 Cavaliere, G., Nielsen, H. B., & Rahbek, A. (2020). Bootstrapping noncausal autoregressions. with applications to explosive bubble modeling. Journal of Business and Economic Statistics, 38(1), 55–67.
 Cavaliere, G., Pedersen, R. S., & Rahbek, A. (2018). The fixed volatility bootstrap for a class of ARCH(q) models. Journal of Time Series Analysis, 39(6), 920–941.
 Cavaliere, G., & Rahbek, A. (2020). A primer on bootstrap testing of hypotheses in time series models: With an application to double autoregressive models. Econometric Theory, 1–48.
 Cavaliere, G., Rahbek, A., & Taylor, A. M. R. (2010a). Cointegration rank testing under conditional heteroskedasticity. Econometric Theory, 26, 1719–1760.
 Cavaliere, G., Rahbek, A., & Taylor, A. M. R. (2010b). Testing for cointegration in vector autoregressions with nonstationary volatility. Journal of Econometrics, 158, 7–24.
 Cavaliere, G., Rahbek, A., & Taylor, A. M. R. (2012). Bootstrap determination of the cointegration rank in vector autoregressive models. Econometrica, 80, 1721–1740.
 Cavaliere, G., Rahbek, A., & Taylor, A. M. R. (2014). Bootstrap determination of the cointegration rank in vector autoregressive heteroskedastic VAR models. Econometric Reviews, 33, 606–650.
 Chen, M., Li, D., & Ling, S. (2013). Nonstationarity and quasimaximum likelihood estimation on a double autoregressive model. Journal of Time Series Analysis, 35, 189–202.
 Davidson, R., & MacKinnon, J. G. (2000). Bootstrap tests: How many bootstraps? Econometric Reviews, 19, 55–68.
 Davidson, R., & MacKinnon, J. G. (2006). Bootstrap methods in econometrics. In T. C. Mills & K. D. Patterson (Eds.), Palgrave handbook of econometrics: Vol. 1. Econometric theory (pp. 812–838). Basingstoke, UK: Palgrave Macmillan.
 Davis, R., & Resnick, S. (1985a). Limit theory for moving averages of random variables with regularly varying tail probabilities. Annals of Probability, 13, 179–195.
 Davis, R., & Resnick, S. (1985b). More limit theory for the sample correlation function of moving averages. Stochastic Processes and Their Applications, 20, 257–279.
 Davis, R., & Resnick, S. (1986). Limit theory for the sample covariance and correlation functions of moving averages. Annals of Statistics, 14, 533–558.
 Davis, R., & Song, L. (2020). Noncausal vector AR processes with application to economic time series. Journal of Econometrics, 216(1), 246–267.
 Duffee, G. R., & Stanton, R. H. (2008). Evidence on simulation inference for near unitroot processes with implications for term structure estimation. Journal of Financial Econometrics, 6(1), 108–142.
 Durret, R. (2019). Probability: Theory and examples (5th ed.). Cambridge, UK: Cambridge University Press.
 Gonçalves, S., & Kilian, L. (2004). Bootstrapping autoregressions with conditional heteroskedasticity of unknown form. Journal of Econometrics, 123, 89–120.
 Gonçalves, S., & Kilian, L. (2007). Asymptotic and bootstrap inference for AR(∞) processes with conditional heteroskedasticity. Econometric Reviews, 26(6), 609–641.
 Gourieroux, C., & Zakoan, J.M. (2017). Local explosion modelling by noncausal process. Journal of the Royal Statistical Society, Series B, 79, 737–756.
 Hahn, J., & Liao, Z. (2019). Bootstrap standard error estimates and inference. Working Paper. Los Angeles, CA: UCLA.
 Hall, P. (1992). The bootstrap and Edgeworth expansions. New York, NY: Springer.
 Hall, P., & Heyde, C. C. (1980). Martingale limit theory and its application. New York, NY: Academic Press.
 Hamilton, J. D. (1994). Time series analysis. Princeton, NJ: Princeton University Press.
 Hansen, B. (1996). Inference when a nuisance parameter is not identified under the null hypothesis. Econometrica, 64(2), 413–430.
 Hansen, B. (2000). Testing for structural change in conditional models. Journal of Econometrics, 97, 93–115.
 Harvey, D., Leybourne, S. J., Sollis, R., & Taylor, A. M. R. (2016). Tests for explosive financial bubbles in the presence of nonstationary volatility. Journal of Empirical Finance, 38(B), 548–574.
 Hayashi, F. (2000). Econometrics. Princeton, NJ: Princeton University Press.
 Hendry, D. F. (1984). Monte Carlo experimentation in econometrics. In Z. Grilishes & M. D. Intriligator (Eds.), Handbook of econometrics (Vol. 2, pp. 937–976). Amsterdam, The Netherlands: North Holand.
 Horowitz, J. L. (2001). The bootstrap. In Z. Grilishes & M. D. Intriligator (Eds.), Handbook of econometrics (Vol. 5, pp. 3159–3228). Amsterdam, The Netherlands: North Holand.
 Horowitz, J. L. (2003). The bootstrap in econometrics. Statistical Science, 18(2), 211–218.
 Johansen, S. (1996). Likelihoodbased inference in cointegrated vector autoregressive models. Oxford, UK: Oxford University Press.
 Klüppelberg, C., Maller, R. A., Van de Vyver, M., & Wee, D. (2002). Testing for reduction to random walk in autoregressive conditional heteroskedasticity models. Econometrics Journal, 5, 387–416.
 Knight, K. (1989). On the bootstrap of the mean in the infinite variance case. Annals of Statistics, 17, 1168–1175.
 Kreiss, J. P. (1988). Asymptotic statistical inference for a class of stochastic processes. Habilitationsschrift (Unpublished thesis). Universität Hamburg.
 Kreiss, J. P. (1992). Bootstrap procedures for AR(∞)processes. In K. H. Jöckel, G. Rothe, & W. Sendler (Eds.), Bootstrapping and related techniques. Lecture Notes in Economics and Mathematical Systems (Vol. 376). Berlin, Germany: Springer.
 Künsch, H. (1989). The jackknife and the bootstrap for general stationary observations. Annals of Statistics, 17, 1217–1241.
 Lahiri, S. N. (2003). Resampling methods for dependent data. New York, NY: Springer.
 Ling, S. (2004). Estimation and testing stationarity for doubleautoregressive models. Journal of the Royal Statistical Society B, 66, 63–78.
 Ling, S. (2007). A double AR(p) model: Structure and estimation. Statistica Sinica, 17, 161–175.
 Liu, R. Y. (1988). Bootstrap procedures under some non i.i.d. models. Annals of Statistics, 16, 1696–1708.
 MacKinnon, J. G. (2006). Bootstrap methods in econometrics. Economic Record, 82, 2–18.
 MacKinnon, J. G. (2009). Bootstrap hypothesis testing. In D. A. Belsley & J. Kontoghiorghes (Eds.), Handbook of computational econometrics (pp. 183–213). Chichester, UK: Wiley.
 Nielsen, H. B., & Rahbek, A. (2014). Unit root vector autoregression with volatility induced stationarity. Journal of Empirical Finance, 29, 144–167.
 Phillips, P. C. B., Wu, Y., & Yu, J. (2011). Explosive behavior in the 1990s Nasdaq: When did exuberance escalate stock values? International Economic Review, 52, 201–226.
 Politis, D., & Romano, J. (1994). The stationary bootstrap. Journal of the American Statistical Association, 89(428), 1303–1313.
 Politis, D. N., Romano, J. P., & Wolf, M. (1999). Subsampling. New York, NY: Springer.
 Shao, X. (2010). The dependent wild bootstrap. Journal of the American Statistical Association, 105, 218–235.
 van der Vaart, A. W. (2000). Asymptotic statistics. Cambridge, UK: Cambridge University Press.
 White, H. (1980). A heteroskedasticityconsistent covariance matrix estimator and a direct test for heteroskedasticity. Econometrica, 48, 817–838.
Appendix
Proof of Lemma 3
Consider the bootstrap process ${\left\{{x}_{t}^{\ast}\right\}}_{t=0}^{n}$ as given by (10). Without loss of generality set ${x}_{0}^{\ast}=0$, such that by simple recursion,
with ${\epsilon}_{j}^{\ast}=0$ for $j\le 0$, $L$ the lag operator, $L{\epsilon}_{t}^{\ast}={\epsilon}_{t1}^{\ast}$, and $\rho \left(z\right)={\displaystyle {\sum}_{i=0}^{\infty}}{\rho}_{0}^{i}{z}^{i}$, for $z\in \u2102$. The summation to infinity means one can use standard manipulations of the lag polynomial $\rho (\cdot )$ as well known from the time series literature (see, e.g., Hamilton, 1994; Johansen, 1996, and references therein). Note first that as ${\rho}_{0}<1$, the coefficients ${\rho}_{0}^{i}$ in $\rho \left(z\right)$ are exponentially decreasing and $\rho \left(z\right)$ is convergent for $z<1+\delta \phantom{\rule{0.1em}{0ex}}$for some $\delta >0$ and $\rho \left(1\right)={\left(1{\rho}_{0}\right)}^{1}$.
When considering the average of ${x}_{t}^{\ast}$, ${n}^{1}{\displaystyle {\sum}_{t=1}^{n}}{x}_{t}^{\ast}$, one may for example use an expansion of $\rho \left(z\right)$ (see Remark 21) around $z=1$, such that
with ${\rho}^{\ast}\left(z\right)={\displaystyle {\sum}_{i=1}^{\infty}}{\theta}_{i}{z}^{i}$ with ${\theta}_{i}=i{\rho}_{0}^{i}$ exponentially decreasing. This immediately gives
The first term tends to zero by (13). Next, for the second term use that ${\epsilon}_{t}^{\ast}$ are i.i.d. (conditionally on the data) such that
with $c={\rho}_{0}{\left(1{\rho}_{0}\right)}^{2}$ and where ${n}^{1}{\displaystyle {\sum}_{t=1}^{n}}{\tilde{\epsilon}}_{t}^{c}\stackrel{p}{\to}\kappa <\infty $ for $\kappa $ a constant by standard application of the LLN.
Turning to the average of ${x}_{t}^{\ast 2}$, note that
where
With ${\rho}_{2}\left(z\right)={\displaystyle {\sum}_{i=0}^{\infty}}{\rho}_{0}^{2i}{z}^{i}$ and ${\omega}_{0}={\rho}_{2}\left(1\right){\sigma}_{0}^{2}={\sigma}_{0}^{2}/\left(1{\rho}_{0}^{2}\right)$ the first term can be written as
where ${\eta}_{t}^{\ast}={\epsilon}_{t}^{\ast 2}{\sigma}_{0}^{2}$. Thus, using identical arguments, ${n}^{1}{\displaystyle {\sum}_{t=1}^{n}}{\rho}_{2}\left(L\right){\eta}_{t}^{\ast}{\stackrel{{p}^{\ast}}{\to}}_{p}0$.
It remains to show that ${\delta}_{n}^{\ast}{\stackrel{{p}^{\ast}}{\to}}_{p}0$. First, observe that by definition of the double summation,
As ${\text{E}}^{\ast}\left({\epsilon}_{ti}^{\ast}{\epsilon}_{tj}^{\ast}\right)=0$ for $j\ne i$, it follows that
Here
with $\tilde{c}={\rho}_{0}^{2}{\sigma}_{0}^{4}/\left(1{\rho}_{0}^{2}\right)$.
Remark 21. The equality
with ${\rho}^{\ast}\left(z\right)={\displaystyle {\sum}_{i=1}^{\infty}}{\theta}_{i}{z}^{i1}$ and ${\theta}_{i}=i{\rho}_{0}^{i}$ follows by the identity
Also note that the polynomial ${\rho}^{\ast}\left(z\right)$ is convergent for $z<1+\delta \phantom{\rule{0.1em}{0ex}}$ as the coefficients ${\theta}_{i}=i{\rho}_{0}^{i}$ of ${\rho}^{\ast}\left(z\right)$ are exponentially decreasing.
Notes
1. The Rademacher distribution is a twopoint distribution on $\pm 1$, each with probability a half, whereas the Mammen distribution is a twopoint distribution on $\frac{1+\sqrt{5}}{2}$
and $\frac{1\sqrt{5}}{2}$ with probabilities given by ${\scriptscriptstyle \frac{\sqrt{5}1}{2\sqrt{5}}}$ and ${\scriptscriptstyle \frac{\sqrt{5}+1}{2\sqrt{5}}}$, respectively.
2. Markov’s inequality:$\phantom{\rule{0.2em}{0ex}}P\left(X>\delta \right)\le EX{}^{k}/{\delta}^{k}$ for any $\delta >0$ and $\kappa \ge 1$.
3. Note that as the model here includes a constant term, strictly speaking the recentering of ${\tilde{\epsilon}}_{t}$ is not needed.
4. A complete discussion of specifications and properties of the parameter space $\Theta $ is given in Cavaliere, Nielsen, Pedersen, and Rahbek (2020).