Show Summary Details

Page of

Printed from Oxford Research Encyclopedias, Economics and Finance. Under the terms of the licence agreement, an individual user may print out a single article for personal use (for details see Privacy Policy and Legal Notice).

date: 05 March 2021

# An Introduction to Bootstrap Theory in Time Series Econometrics

• Giuseppe Cavaliere, Giuseppe CavaliereDepartment of Economics, University of Bologna; Department of Economics, University of Exeter
• Heino Bohn NielsenHeino Bohn NielsenDepartment of Economics, University of Copenhagen
•  and Anders RahbekAnders RahbekDepartment of Economics, University of Copenhagen

### Summary

While often simple to implement in practice, application of the bootstrap in econometric modeling of economic and financial time series requires establishing validity of the bootstrap. Establishing bootstrap asymptotic validity relies on verifying often nonstandard regularity conditions. In particular, bootstrap versions of classic convergence in probability and distribution, and hence of laws of large numbers and central limit theorems, are critical ingredients. Crucially, these depend on the type of bootstrap applied (e.g., wild or independently and identically distributed (i.i.d.) bootstrap) and on the underlying econometric model and data. Regularity conditions and their implications for possible improvements in terms of (empirical) size and power for bootstrap-based testing differ from standard asymptotic testing, which can be illustrated by simulations.

### Bootstrap in Time Series Econometrics

Bootstrap in econometrics is frequently applied in the context of estimation and testing (Berkowitz & Kilian, 2000; Cavaliere & Rahbek, 2020; Davidson & MacKinnon, 2006; Horowitz, 2001, 2003; MacKinnon, 2009). As an example, consider the case where some test statistic, $τn$ say, is of interest given a sample of (time-series) data $x1,...,xn$ with initial values $x0,x−1,..,x−p$ for $p≥0$, ${xt}t=−pn$ with $xt∈ℝ$. Under suitable regularity conditions, including typically (a) stationarity and ergodicity of the $xt$ process and (b) finite moments conditions on the form $E|xt2|k<∞$ for some $k≥1$, it holds that under the null hypothesis of interest, $H0$ say,

$Display mathematics$

where $q$ denotes the degrees of freedom and “$→d$” denotes convergence in distribution. Moreover, under the alternative, or when $H0$ is not true, the statistic $τn$ diverges. In standard asymptotic testing, the $χq2$ distributional approximation of the test statistic $τn$ is frequently applied using the $1−α$ quantile of the $χq2$ distribution or, equivalently, by calculating the p-value.

A bootstrap-based test is often motivated by noting that the asymptotic approximation may not be good in finite samples, which may lead, for a finite number of observations $n$, to an actual size larger, or smaller, than the nominal level $α$. This is often corrected by applying even simple bootstrap schemes. An additional motivation for the bootstrap is that the underlying model used for estimation—and on which the derivation of $τn$ is based—may be misspecified. A typical example is the assumption of homoskedasticity of model innovations, which often in practice is challenged in the modeling of macro and financial data, where (conditional and unconditional) heteroskedasticity is typically present. In this case, the so-called wild bootstrap may correct for such heteroskedasticity. Likewise, for other types of misspecification and model departures, different bootstrap schemes may be applied depending on which type of misspecification is of concern. In addition, in some cases the limiting distribution of $τn$ cannot be tabulated, for example, because it is a function of unknown nuisance parameters. The bootstrap may resolve such issues. However, it should be emphasized that any bootstrap-based test is—as for the original asymptotic test—only valid under certain regularity conditions. These regularity conditions are important to check and understand as in many cases application of what may be thought of as a “ standard bootstrap” may be invalid and misleading, despite its popularity in many empirical applications. Typical situations in which a standard bootstrap does not work can be found in the context of nonstationary variables and when data exhibit heavy-tailed distributions, as is typical in financial data. When application of existing, or standard, bootstrap schemes fails to work, these may be corrected by more elaborate bootstrap schemes, as is richly documented in recent research in econometrics (see inter alia Cavaliere & Rahbek, 2020, and the references therein).

The bootstrap is a simulation-based approach that is typically simple to apply in the context of inference and testing. Based on some bootstrap scheme, the bootstrap is based on generating new bootstrap samples of data and the test statistic of interest, denoted by ${xt∗}t=−pn$ and $τn∗$, respectively. The (re-)generation of such bootstrap data is based on keeping the original sample ${xt}t=−pn$ fixed.

If the limiting distribution (in the bootstrap sense) of the bootstrap test statistic $τn∗$ is identical to the limiting distribution of the original statistic $τn$ under the null of $H0$, such as the $χq2$ distribution noted earlier, this implies validity under the null of the bootstrap. If, in addition, under the alternative when $H0$ is not true, the bootstrap statistic $τn∗$ does not diverge as the sample size $n$ increases (or, at most it diverges at a slower rate than the original statistic), the bootstrap is asymptotically valid. This is the case when under the alternative the bootstrap-limiting distribution of $τn∗$ is identical to the limiting distribution of the original statistic or when $τn∗$ is bounded in (bootstrap) probability.

As a leading example throughout, and to fix ideas, we consider the first-order autoregressive (AR) model. The section “Autoregressive Model” provides a summary of standard econometric nonbootstrap analysis of the AR model before turning to the discussion of the implementation and (asymptotic) theory of the bootstrap in the section “Bootstrap in the Autoregressive Model.” Next, a Monte Carlo study is used to discuss finite sample behavior of different bootstrap schemes (see “Finite Sample Behavior”) and recent advances in econometric time series bootstrap analysis are provided (see “Selection of Further Topics”). The conclusion provides an overview of some important further topics and approaches to bootstrap inference.

### The Autoregressive Model

In order to present bootstrap theory and arguments, a brief summary of results and arguments for the (nonbootstrap) AR model of order one is provided. The results and asymptotic arguments presented here are well known (see, e.g., Hamilton, 1994; Hayashi, 2000).

Consider the AR model of order one

$Display mathematics$(1)

with the initial value $x0$ fixed in the statistical analysis and the innovations$εt$ are assumed to be independently and identically distributed (i.i.d). $N(0,σ2)$ (see also Remark 2 regarding relaxing the assumption of Gaussianity). The parameter of the model is $θ=(ρ,σ2)′$ with $θ0=(ρ0,σ02)′$ denoting the true value. For estimation, the parameter space is given by

$Display mathematics$

For the true parameter $θ0$, we assume $θ0∈Θ0⊂Θ$, with $Θ0={θ∈Θ||ρ0|<1}$, such that $xt$ in (1) for $θ0∈Θ0$ has a stationary and geometrically ergodic solution given by

$Display mathematics$

With $ρ¯$ some fixed value, consider testing the hypothesis $H0$ given by

$Display mathematics$

The test is based on the likelihood ratio test statistic, $τn$, defined in terms of the Gaussian log-likelihood function

$Display mathematics$(2)

with $x0$ fixed and $εt(ρ)=xt−ρxt−1$. Standard optimization gives the unrestricted maximum likelihood estimator (MLE), $θ^n=(ρ^n,σ^n2)′$, where

$Display mathematics$(3)

The restricted estimator $θ˜n=(ρ˜n,σ˜n2)′$ is obtained by maximization under $H0$, and is given by

$Display mathematics$

It follows that the likelihood ratio (LR) test statistic is given by

$Display mathematics$(4)

Moreover, under $H0$ and regularity conditions detailed in the section “Asymptotics for the Autoregressive Model,” with $ρ¯=ρ0$ and $θ0∈Θ0$ it holds that $τn→dχ12$. Note in this respect that as long as the true $ρ0$ is not “ too large” the asymptotic $χ12$ approximation is a good approximation even for small samples. In contrast, and as exemplified in Table 1, if either $εt$ is not i.i.d. $N(0,σ2)$ or if $ρ0=0.9$ the $χ12$ distribution is not a good approximation of the actual distribution of $τn$ for small, or even moderate, finite samples of size $n$.

Remark 1 (Wald and t statistics). Although focus is on the likelihood ratio test statistics, other statistics could be considered as well. For instance, the classic Wald statistic ($Wn$) is given by

$Display mathematics$

and, under the same regularity conditions required for the likelihood ratio test, when $ρ¯=ρ0$ and $θ0∈Θ0$, it holds that $Wn(ρ=ρ¯)→dχ12$. It is also customary to consider the $t$-ratio statistic, defined as

$Display mathematics$

where the so-called standard error $se(ρ^n)$ is defined as $(V^n/n)1/2$, with

$Display mathematics$

an unrestricted (i.e., obtained without imposing the null hypothesis) estimator of the asymptotic variance of $n1/2(ρ^n−ρ0)$. Notice that $tn(ρ=ρ¯)=sign(ρ^n−ρ¯)(Wn(ρ=ρ¯))1/2$ and, when $ρ¯=ρ0$ and $θ0∈Θ0$, $tn(ρ=ρ¯)→dN(0,1)$.

Remark 2 (Quasi likelihood). Often if the $εt$ sequence is assumed to be i.i.d., Gaussian is relaxed to i.i.d. $(0,σ2)$ for some unknown distribution. When the Gaussian log-likelihood function in (2) in this case is used to obtain estimators and test statistics, these are referred to as (Gaussian) quasi MLE and quasi LR statistics, respectively. This is also the case when $εt$ is assumed to be mean zero and (conditionally or unconditionally) heteroskedastic, as for example with some autoregressive conditional heteroskedastic (ARCH) specification or with a structural break in the variance (see the section “Finite Sample Behavior”).

#### Asymptotics for the Autoregressive Model

To ease the presentation of the key arguments for the standard nonbootstrap asymptotic analysis, assume here without loss of generality that $σ2$ is fixed at the true value—that is, $σ2=σ02$. Accordingly, the parameter is $θ=ρ$ and the likelihood function in (2) simplifies as

$Display mathematics$

The MLE $ρ^n$ is given by (3), while the $LRn(ρ=ρ¯)$ statistic in (4) in this case is identical to the Wald statistic $(Wn)$

$Display mathematics$(5)

By construction, $τn$ is a simple function of $n1/2(ρ^n−ρ¯)$ and $n−1∑t=1nxt−12$, and the limiting distribution is found by applying a law of large numbers (LLN) to the average $n−1∑t=1nxt−12$, as well as a central limit theorem (CLT) to $n1/2(ρ^n−ρ¯)$.

With $ρ¯=ρ0∈Θ0$, the AR process $xt$ is stationary and ergodic, and has finite variance, $V(xt)=ω0=σ02/(1−ρ02)$. In particular, it follows by the LLN for stationary and ergodic processes that

$Display mathematics$

Next, by definition of the MLE,

$Display mathematics$

Here $mt=εtxt−1$ is a martingale difference sequence (mds) with respect to the filtration $Ft$, where $Ft=σ(xt,xt−1...)$, as $E|mt|<∞$, and $E(mt|Ft−1)=0$. Moreover, the conditional second order moment converges in probability,

$Display mathematics$

This implies by the CLT for mds that $n−1/2∑t=1nmt→dN(0,σ02ω0)$ (Hall & Heyde, 1980; Hamilton, 1994) and such that

$Display mathematics$

Remark 3 (Lindeberg condition). Note that for a CLT for mds in general to hold (Hall & Heyde, 1980), a Lindeberg-type condition of the form $γn=n−1∑t=1nE(mt2I(|mt|>δn1/2))→0$ for any $δ>0$ must hold, where $I(⋅)$ is the indicator function. This holds here as, by stationarity,

$Display mathematics$

which tends to zero as $n→∞$ by dominated convergence under the moment condition $E(mt2)<∞$ as implied by $E(εt2)<∞$ and independence of $εt$ and $xt−1$.

Collecting terms, it follows that we have the following result.

Lemma 1. For the AR(1) model in (1) with $θ0∈Θ0$ and $εt$ i.i.d. $(0,σ02)$, it follows that with $τn$ given by (5), under the null $τn→dχ12$ as $n→∞$.

Note that in Lemma 1, it is not assumed that the $εt$ sequence is Gaussian, but instead the lemma is formulated in terms of the milder sufficient condition that $εt$ is an i.i.d. $(0,σ2)$ sequence (Remark 2). Thus the essential regularity conditions for the lemma to hold are that $|ρ0|<1$ and $εt$ i.i.d. $(0,σ2)$.

Remark 4 (Heteroskedasticity). Note that if $εt=σtzt$, with $zt$ i.i.d. $N(0,1)$ and $σt2$ given by an ARCH process with $E(σt2)=σε2<∞$, it follows that $τn→dcχ12$, with $c=σε2/σ02$. This reflects in general that if $εt$ are (conditionally, or unconditionally) heteroskedastic, the limiting distribution of the test statistic $τn$ is not $χ12$. This is wellknown in the context of regression models, and corrections of the test statistic (White, 1980) are typically applied to ensure valid asymptotic inference; the bootstrap, or more precisely, the wild bootstrap discussed later, may correct for such misspecification without the need to correct the original test statistic (cf. Gonçalves & Kilian, 2004).

Remark 5 (Test statistic). For $σ2$ an unknown parameter to be estimated, such that $θ=(σ2,ρ)′$, note that the LR statistic in (4) can be written as

$Display mathematics$

with $σ˜n2=n−1∑t=1nεt2(ρ¯)$. By Lemma 1, $wn=n−1Wn(ρ=ρ¯)→p0$, and by a Taylor expansion, $−log(1−w)=w+o(w)$, with $o(w)$ a term that tends to zero as $w→0$. Hence,

$Display mathematics$

where $op(1)$ denotes a term that converges to zero in probability as $n→∞$ (see, e.g., van der Vaart, 2000, Lemma 2.12, for details on $op(⋅)$ (and $Op(⋅)$ notation and the stochastic [Taylor] expansion as applied here).

### Bootstrap in the Autoregressive Model

The general idea behind a bootstrap algorithm as implemented in the context of likelihood-based testing in the first order AR model can be summarized as follows:

Step A. With the original data ${xt}t=0n$ fixed, generate a sample of bootstrap data ${xt∗}t=0n$ using some bootstrap scheme with bootstrap true parameter, $θn∗=(ρn∗,σn∗2)′$. Specifically, for the AR model, set $x0∗=x0$ and generate $xt∗$ recursively by

$Display mathematics$

with $zt=xt−1∗$ for a recursive bootstrap scheme, while $zt=xt−1$ for a fixed design bootstrap scheme. As to the choice of bootstrap innovations $εt∗$ in terms of the original data ${xt}t=0n$ and a bootstrap sampling distribution as detailed after step C.

Step B. Compute the bootstrap (quasi) MLE$ρ^n∗$ and the bootstrap LR statistic $τn∗=LRn∗(ρ=ρn∗)$, where the bootstrap log-likelihood function $ln∗(θ)$ as a function of $θ$ is given by

$Display mathematics$

Step C. Generate ${xt,b∗}t=0n$, $θ^n,b∗$ and $τn,b∗$ for $b=1,2,...,B$ by repeating Steps $A$ and $B$, and use the empirical distribution of ${τn,b∗}b=1B$ for testing based on the original statistic $τn=LRn(ρ=ρ0)$. Precisely, with the bootstrap p-value set to $pn,B∗=B−1∑b=1BI(τn≥τn,b∗)$, $H0$ is rejected when $pn,B∗<α$, with $α$ the nominal level. (For the choice of bootstrap repetitions $B$, see Remark 9).

As to the definition in step $A$ of the bootstrap innovations $εt∗$, usually these are obtained by i.i.d. draws with replacement from re-centered estimated model residuals; henceforth referred to as iid bootstrap, or iid resampling. The model residuals can be either estimated under the null ($H0$ imposed) or without imposing the null. That is, the $εt∗$ are in the first case resampled from centered residuals, ${ε˜tc}t=1n$, where

$Display mathematics$(8)

Using unrestricted residuals, the bootstrap $εt∗$ innovations are resampled from ${ε^tc}t=1n$, where

$Display mathematics$(9)

Centering is required because by doing so both of the residual series $ε^tc$ and $ε˜tc$ have empirical mean zero, $n−1∑t=1nε^tc=0$ and $n−1∑t=1nε˜tc=0$, and ideally “ mimic” the true $εt$. In particular, under $H0$, $εt(ρ¯)=εt(ρ0)=εt$, and, moreover, $ρ^n→pρ0$. In contrast, under the alternative, when $H0$ does not hold, while $εt(ρ¯)≠εt$, it still holds that $ρ^n→pρ0$. Hence, while one may expect that the unrestricted residuals $ε^tc$ perform better under the alternative, at least asymptotically, in practice little difference is found between the two.

An alternative to iid resampling is the so-called wild bootstrap, where either $εt∗=ε˜tcwt∗,$ or $εt∗=ε^tcwt∗$, with $wt∗$ an auxiliary i.i.d. sequence, independent of the original data, with $E(wt∗)=0$ and $V(wt∗)=1$. A simple example is $wt∗$ i.i.d. $N(0,1)$ (see Remark 10 for alternative specifications). The wild bootstrap is typically motivated by its potential ability to allow for possible model misspecification in the sense that it allows for (conditional and unconditional) heteroskedasticity in the innovations $εt$. To see this (e.g., $εt∗=ε˜tcwt∗$), it follows that (conditionally on the data ${xt}t=0n$), the $εt∗$ are i.i.d. distributed with mean zero and with time-varying variance,

$Display mathematics$

This way, the variation in $εt∗$ from the wild bootstrap “in essence reflects the heteroskedasticity of the original data” (Liu, 1988, p. 1704) as is also illustrated in the Monte Carlo simulations in the section on “Finite Sample Behavior.” This differs from the iid bootstrap, where the bootstrap innovations $εt∗$ (conditionally on the data) are i.i.d., with a discrete distribution given by $P(εt∗=ε˜jc|{xt}t=0n)=n−1$, for $j=1,2,...,n$. Specifically, as mentioned earlier, the (empirical, or conditional) mean is zero, $E(εt∗|{xt}t=0n)=0$, and the variance is given by

$Display mathematics$

That is, for fixed $n$, the variance for the iid bootstrap innovations $εt∗$ is constant (and equal to the empirical variance of the estimated residuals), while depending on time $t$ for the wild bootstrap.

In short, the bootstrap scheme in (6), $xt∗=ρn∗zt+εt∗$, with $zt=xt−1$ or $zt=xt−1∗$, depends on two types of randomness: (a) the variation of the original data ${xt}t=0n$; and (b) the bootstrap resampling for ${εt∗}t=1n$ (wild or iid resampling). This is important for the application of the LLN and the CLT to establish bootstrap validity as demonstrated in the next.

Some further remarks are in order.

Remark 6 (t statistics and bootstrap standard errors). When interest is in bootstrapping the $t$ ratio statistic (or Wald-type statistics), then steps $A–C$ can be used with $τn∗$, now defined as (taking the case of unknown $σ2$ to illustrate)

$Display mathematics$

where the so-called bootstrap standard error $se∗(ρ^n∗)$, the bootstrap analog of $se(ρ^n)$, given by

$Display mathematics$

with $Vn∗$ being an estimator of the variance of $ρ^n∗−ρ0$, such as

$Display mathematics$

where $σ^n∗2$ is the sample variance of the bootstrap residuals, $σ^n∗2=n−1∑t=1nεt∗(ρ^n∗)$, $εt∗(ρ^n∗)=xt∗−ρ^n∗xt−1∗$. It is worth noticing that the bootstrap standard error could also be computed without exploiting the autoregressive structure of the model; this can be done by simply resorting to the $B$ bootstrap realizations of $ρ^n∗$, say $ρ^n,b∗$, $b=1,...,B$. Specifically, one could consider the following bootstrap standard error

$Display mathematics$

which is straightforward to compute once the $B$ realizations of $ρ^n∗$ have been obtained.

Remark 7 ([Un-]restricted bootstrap). As to the choice in step $A$ of the bootstrap true value $θn∗=(ρn∗,σn∗2)′$ (or simply, $ρn∗$ for the AR model), one may set $θn∗$ to the value of the unrestricted estimator $θ^n$, $θn∗=θ^n∗$, which is referred to as unrestricted bootstrap. If $θn∗=θ˜n$, the restricted estimator, the bootstrap is referred to as restricted. It should be emphasized that for the unrestricted bootstrap, the bootstrap likelihood ratio statistic $τn∗$ is derived for the hypothesis $ρ=ρ^n$, while the original hypothesis $H0:ρ=ρ¯$ is considered for the restricted bootstrap. While both choices are widely applied in existing literature, the restricted bootstrap in the context of testing is more popular in econometrics (see, e.g., Davidson & MacKinnon, 2000).

Remark 8 (Recursive and fixed design bootstrap). In step $B$ of the algorithm in equation (6), with $zt=xt−1∗$, this is an example of a recursive bootstrap. That is, the original autoregressive structure for $xt$ is replicated for the bootstrap process $xt∗$. However, with $zt=xt−1$, the original data $xt−1$ are used as lagged value of $xt∗$, such that $xt∗$ is not an autoregressive process even conditionally on the data. The fixed design bootstrap typically simplifies (some of) the asymptotic arguments, and is often found to behave as well as the recursive bootstrap (see, e.g., Gonçalves & Kilian, 2004, 2007, for general AR models, and Cavaliere, Pedersen, & Rahbek, 2018, for ARCH models).

Remark 9 (Bootstrap p-value). In step $C$, the bootstrap p-value $pn,B∗$ is defined as $pn,B∗=B−1∑b=1BI(τn≥τn,b∗)$, with $B$ the number of bootstrap repetitions (see also Remark 14). Typical choices are $B=199$, $399$, or $999$ (see also Andrews & Buchinsky, 2000, and Davidson & MacKinnon, 2000, for details on the choice of $B$).

Remark 10 (Choice of $wt*$). With respect to the choice of distribution of the i.i.d. sequence $wt∗$ for the wild bootstrap, Liu (1988) provides a detailed discussion of various choices based on so-called Edgeworth expansions (see, e.g., Hall, 1992; van der Vaart, 2000, for an introduction) of test statistics similar to $τn$. In particular, Liu (1988), with $ξk=E(wtk),k≥1$, emphasizes $ξ1=0$, $ξ2=1$ as well as $ξ3=1$ as important. For the case of $ξ3=1$, emphasis is on possible skewness, while $ξ3=0$ works well in the case of symmetry. In applications, standard choices for $wt∗$, all with $ξ1=0$ and $ξ2=1$, include the Gaussian, Rademacher, and Mammen distributions.1 It follows that $ξ3=0$ for the first two (with $ξ4=3$ and $1$, respectively), while $ξ3=1$ (and $ξ4=2$) for the Mammen distribution.

Remark 11 (Parametric bootstrap). In step $A$ of the bootstrap scheme, one may also use a so-called parametric bootstrap, where the bootstrap innovations $εt∗$ are generated as i.i.d. $N(0,σn∗2)$. While this parametric bootstrap performs well in the case where the true innovations $εt$ are Gaussian, this may not be the case when the distribution of $εt$ is non-Gaussian (Horowitz, 2001).

#### Asymptotic Theory for the Recursive Bootstrap

In order to discuss regularity conditions under which bootstrap-based testing holds, consider here the details of verification of the asymptotic validity of the recursive bootstrap for the AR model.

Thus, with the AR model given in (1), consider here the recursive restricted bootstrap scheme as defined by setting $zt=xt−1∗$ and $ρn∗=ρ¯$ in step $A$ of the bootstrap algorithm. In short, the $xt∗$ bootstrap sequence is here generated as

$Display mathematics$(10)

with $x0∗=x0$ and $ρ¯=ρ∗$ as the bootstrap true value. Moreover, we consider the classic case of iid resampling from the autoregressive residuals obtained under $H0$, that is, from ${ε˜tc}t=1n$ defined in (8). The statistic of interest is $τn=Wn(ρ=ρ¯)$ in (5), which is computed using the bootstrap sample ${xt∗}t=0n$ as

$Display mathematics$(11)

While $xt∗$ clearly has some features similar to $xt$, one cannot apply standard concepts such as stationarity and ergodicity when analyzing the asymptotic behavior of $τn∗$, due to two types of randomness: the bootstrap resampling distribution and the distribution of the original data, ${xt}t=0n$. Introduce therefore the bootstrap equivalent concepts of convergence in probability and distribution, which reflect the fact that inference is based on conditioning on the original data that are themselves random.

#### Bootstrap Probability, Expectation, and Convergence

With $P∗(⋅)$ denoting the bootstrap probability—that is, the probability conditional on the data—the iid bootstrap innovations $εt∗$ are by definition i.i.d. distributed with

$Display mathematics$

Similarly, the expectation $E∗(⋅)$ is defined by $E∗(⋅)=E(⋅|{xt}t=0n)$. As an example, consider the expectation of $εt∗$conditionally on the data. It follows that, as already discussed, $E∗(εt∗)=0$ as

$Display mathematics$

by the definition in (8). Next, consider the variance of $εt∗$ conditionally on the data, $V∗(εt∗)$. Again, by definition, $V∗(εt∗)=E∗(εt∗2)−(E∗(εt∗))2$ and hence as $E∗(εt∗)=0$, it follows that

$Display mathematics$

That is, the variance conditional on the data is equal to the sample variance of the original estimated residuals under $H0$. In particular, $V∗(εt∗)$ is a random variable (in terms of the original probability measure) and moreover, by the LLN for i.i.d. variables, under $H0$ with $ρ¯=ρ0$,

$Display mathematics$(12)

Note that for the wild bootstrap, $V∗(εt∗)=V∗(ε˜tcwt∗)=(ε˜tc)2$, emphasizing that the wild bootstrap indeed “mimics” heteroskedasticity and the iid bootstrap does not.

Similar to the definition of $P∗(⋅)$ and $E∗(⋅)$ this motivates the definition of the bootstrap equivalent of convergence in probability, denoted “$→p∗p$.” Formally, a sequence of stochastic variables $Xn∗$ is said to converge in probability conditional on the data (or, to converge in $P∗$-probability, in probability) to $c$ (possibly random), if $P∗(|Xn∗−c|>δ)$ converge in probability to zero. This can be stated as

$Display mathematics$

As an example, it follows that $Xn∗=n−1∑t=1nεt∗→p∗p0$ as by the bootstrap equivalent of Markov’s inequality2 one has,

$Display mathematics$

By definition,

$Display mathematics$

with $E∗(εt∗2)=V∗(εt∗)$, and for $s≠t$,

$Display mathematics$

Hence, since $E∗(Xn∗)2=n−1V∗(εt∗)$, we conclude that

$Display mathematics$(13)

A key ingredient in the asymptotic analysis of the nonbootstrap AR model is the CLT, and we need a bootstrap equivalent of the CLT and a bootstrap equivalent of convergence in distribution. By definition, $Xn→dX$ if $FXn(x)=P(Xn≤x)→FX(x)=P(X≤x)$ at all continuity points of $FX(⋅)$. Likewise, $Xn∗$ converge in distribution to $X$ conditional on the data (or, as sometimes used, $Xn∗$ converges “weakly in probability”); that is, $Xn∗→d∗pX$, if the bootstrap cumulative distribution function converges in probability. Specifically, $Xn∗→d∗pX$ if

$Display mathematics$

at all continuity points of $FX(⋅)$. Alternatively, weak convergence in probability may be defined in terms of convergence in probability of the bootstrap characteristic function as in (15).

The next lemma illustrates that, as one might expect, the sum of bootstrap innovations $εt∗$ is asymptotically Gaussian (in probability).

Lemma 2 (van der Vaart, 2000, Theorem 23.4). With $Xn∗=n−1/2∑t=1nεt∗$, then

$Display mathematics$(14)

The proof, as for most bootstrap CLTs, is based on applying a CLT for triangular arrays, as ${εt∗}t=1n$ are sampled from $ε˜tc$, which depends on $n$.

To give an idea of the underlying theory, consider here verifying (14) using a classic approach based on the characteristic function (Durret, 2019, proof of Theorem 3.4.10). For a random variable $X$, the characteristic function defines uniquely the distribution of $X$ and is defined by $ϕ(s)=E(exp(isX))$. Here, $i$ is the complex (unit imaginary) number that satisfies $i2=−1$ and $s∈ℝ$, and for $X=dN(0,σ02)$ it holds that $ϕ(s)=exp(−s22σ02)$.

With the bootstrap characteristic function of $Xn∗$ defined by $ϕn∗(s)=E∗(exp(isXn∗))$, it follows that (14) holds if

$Display mathematics$(15)

Note first, as $εt∗$ are i.i.d. conditionally on the data,

$Display mathematics$

Next, a Taylor expansion of $exp(⋅)$ at $s=0$ gives

$Display mathematics$

using $E∗(εt∗)=0$ and $E∗(εt∗2)→pσ02$ (see [12]; Durret, 2019, Lemma 3.3.19). It therefore follows as desired that

$Display mathematics$

as for any sequence $cn$, with $cn→pc∈ℂ$ as $n→∞$, then similar to Durret (2019, Theorem 3.4.2), $(1−n−1cn)n→pexp(−c)$.

Remark 12 (Lindeberg condition). The CLTs in Durret (2019, Theorem 3.4.10) and van der Vaart (2000, Theorem 23.4) for triangular arrays follow by verifying

$Display mathematics$

in addition to the bootstrap Lindeberg condition,

$Display mathematics$

A simple way to see that $γn∗→p0$ is for example to note that if (the rather strong moment condition) $E(εt4)<∞$ holds, the LLN applies to $n−1∑t=1n(ε˜tc)4$ and hence,

$Display mathematics$

#### Bootstrap Validity Under $H0$

As briefly mentioned in the introduction, it is important for the application of the bootstrap that the limiting distribution (in probability) of the bootstrap test statistic has the same limiting distribution as the original test statistic when the null is true. Stated differently, we wish here to establish that under $H0$ with $ρ¯=ρ0,$

$Display mathematics$

By definition, the bootstrap estimator $ρ^n∗$ is given by

$Display mathematics$

such that by the bootstrap scheme employed, that is $xt∗=ρ0xt−1∗+εt∗$, it follows that

$Display mathematics$

Here a bootstrap CLT should be used for the first term (Should be (i) as refers to equation just above!)

a), and a bootstrap LLN for the second term (b) (Should be (ii) as refers to equation) to find the limiting behavior of the bootstrap estimator and, hence, of the test statistic $τn∗$. Consider first (b), which is an average of lagged $xt∗$ squared, with

$Display mathematics$(16)

As $εt∗$ depends on $n$, and the data ${xt}t=0n$, the concepts of stationarity and ergodicity—while applying to $xt$—do not apply to $xt∗$. However, the following lemma holds, which establishes that the LLN holds for the average of $xt∗$ and $xt∗2$.

Lemma 3. Suppose that ${xt}t=0n$ is given by (1) with $|ρ0|<1$ and $εt$ i.i.d. $(0,σ02)$. Assume furthermore, with $εt∗$ iid sampled with replacement from ${ε˜tc}t=1n$ and $xt∗$ given by (10). Then, as $n→∞$,

$Display mathematics$

The proof of Lemma 3 is given in the appendix. Note that for the case of $ρ0=0$ the arguments are similar to the arguments used to establish $n−1∑t=1nεt∗→p∗p0$.

Remark 13 (LLN triangular arrays). Lemma 3 is the bootstrap equivalent of the weak law of large numbers for triangular arrays (see also Durret, 2019, Theorem 2.2.6).

Next, consider the CLT candidate term (b),

$Display mathematics$

As for the nonbootstrap case, with $Ft∗=σ(xt∗,xt−1∗,...,x0∗)$, then (conditionally on the data) a bootstrap CLT for martingale difference arrays (mda) can be applied. In particular, $E∗(εt∗xt−1∗|Ft−1∗)=xt−1∗E∗(εt∗)=0$, while for the conditional second order moment (conditional on the data), it follows by application of Lemma 3 that

$Display mathematics$

It remains to establish the bootstrap Lindeberg condition, $γn∗→p∗p0$, where

$Display mathematics$(17)

Similar to Remark 12, this follows by using that for some (arbitrarily small) $η>0$,

$Display mathematics$

which holds provided $E|εt|2+η<∞$, using arguments as in Lemma 3. (Note that in Remark 12 the same argument is used for $η=2$.)

Hence, with $εt∗$ iid resampled from ${ε˜tc}t=1n$, $εt$ i.i.d. $(0,σ02)$ with $E|εt|2+η<∞$, it follows that the limiting distribution (in probability) of the bootstrap MLE is given by

$Display mathematics$(18)

This immediately leads to the desired result:

Theorem 1. Under $H0$ with $|ρ0|<1$, and with $εt∗$ iid resampled from ${ε˜tc}t=1n$, with $εt$ being i.i.d. $(0,σ02)$ with $E|εt|2+η<∞$ for some $η>0$, it holds that

$Display mathematics$(19)

Remark 14 (Bootstrap p-value). The bootstrap p-value $pn,B∗$ in step $C$ of the bootstrap algorithm is an approximation to the “true” bootstrap p-value $pn∗$, where $pn∗=P∗(τn∗>τn)$, in the sense that $pn,B∗→ppn∗$ as $B$ tends to infinity. (For details in terms of the stronger concept of “almost sure” convergence, see, e.g., Cavaliere, Nielsen, & Rahbek, 2015, Remark 2.)

Remark 15 (Validity under $H0$). Note that by Cavaliere et al. (2015, Corollary 1) for the bootstrap p-value $pn∗$ (see Remark 14), it follows that as the limiting $χ12$ distribution has a continuous distribution function, under the conditions of Theorem 1, $pn∗→dU$, with $U$ uniformly distributed on $[0,1]$ (Hansen, 1996, 2000).

Remark 16 (Moments of $εt$). Note that $E|εt|2+η<∞$ for some $η$, or $E(εt4)<∞$ as is often used, is required. This reflects that, as typically found for the bootstrap, further moment restrictions are used to prove bootstrap validity than in the nonbootstrap case. This is because complexities arise when applying bootstrap LLNs and arguments in connection to establishing bootstrap Lindeberg-type conditions (Cavaliere & Rahbek, 2020). However, notably, while the higher order moment conditions are sufficient for the mathematical arguments, their necessity is often not reflected in bootstrap simulations (see “Finite Sample Behavior”).

Remark 17 (Introducing $σ2$). As for the nonbootstrap case, the result in Theorem 1 also holds for the case where $σ2$ is treated as a parameter.

Remark 18 (Consistency of bootstrap standard errors). It is important to notice that the fact that $n1/2(ρ^n∗−ρ0)$ converges in conditional distribution (in probability) to the Gaussian distribution does not imply that the bootstrap standard error is consistent. Intuitively, this happens because convergence in distribution alone does not imply convergence of moments. Hence, the econometrician must evaluate on a case-by-case basis whether the bootstrap standard errors are consistent. An interesting result is provided by Hahn and Liao (2019), who report that the bootstrap standard error computed by simulation using $B$ bootstrap repetitions does not underestimate the population standard error (see Remark 6), and hence bootstrap inference based on the $t$ statistic coupled with the numerically computed standard errors is conservative (i.e., the type-one error associated to the bootstrap test does not exceed the user-chosen significance level in large samples).

#### Bootstrap Validity Under the Alternative

Consider here the convergence of the bootstrap statistic $τn∗$ in (11) when the alternative holds. That is, assume here that the original data are generated with true value $θ0=(ρ0,σ02)′$, but the hypothesis tested is as before $H0:ρ=ρ¯$ with $ρ¯≠ρ0$. As argued below, Theorem 1 holds under the alternative as well, such that

$Display mathematics$(20)

For the application of bootstrap-based testing, this implies that under the alternative, as $Wn(ρ=ρ¯)$ diverges while $Wn∗(ρ=ρ¯)$ converges in distribution, the bootstrap-based test will reject with probability tending to one. That is, asymptotic bootstrap validity holds since by Cavaliere et al. (2015, Corollary 1) the bootstrap p-value $pn∗$, defined in Remark 15, tends to zero in probability under the alternative, $pn∗→p0$.

A key argument for (20) to hold is to note that the identity,

$Display mathematics$

holds independently of whether $ρ¯$ is the data true value $ρ0$ or not. That is, $ρ¯$ is by construction the bootstrap true value, such that under the null and also under the alternative, the bootstrap estimator can be rewritten as

$Display mathematics$

What differs is that $εt∗$ under the alternative is resampled from recentered residuals $ε˜tc$, with

$Display mathematics$

That is, while the identity $ε˜t=εt$ holds under the null hypothesis when $ρ¯=ρ0$, this is not the case under the alternative. Hence, to establish (20), a repeated application of the bootstrap LLN (applied to $∑t=1nxt−1∗2$) and CLT (applied to $∑t=1nεt∗xt−1∗$) is needed under the alternative. For the AR process of order one considered here, the arguments are based on simple modifications of the theory under $H0$.

Remark 19 (Theory for the wild bootstrap). The same results can be shown to apply for the wild bootstrap in the case of conditional heteroskedasticity (Gonçalves & Kilian, 2004, 2007).

### Finite Sample Behavior

Throughout, the focus has been on establishing asymptotic validity. This was done by verifying that the bootstrap statistic $τn∗$ has the same limiting distribution (in probability) as the original statistic $τn$ under the null hypothesis. Moreover, the same was argued to hold under the alternative. To illustrate the finite sample performance of the iid and wild bootstraps for the AR model, this section highlights some selected typical findings for the bootstrap based on a small and simple (to replicate) Monte Carlo study. Thus the Monte Carlo study here is not meant to be elaborate; exhaustive and detailed bootstrap Monte Carlo-based investigations are given in several papers (see, e.g., Gonçalves & Kilian, 2004, with special attention to higher order AR models, as well as the references in Cavaliere & Rahbek, 2020).

The Monte Carlo results reported here highlight the importance of the assumptions for the established validity of the bootstrap-based test of $H0:ρ=ρ¯$ in the autoregressive model. Specifically, it was argued that the true value of the autoregressive root $ρ0$ for $xt$ should satisfy $|ρ0|<1$, and it was emphasized that $εt$ is an i.i.d. $(0,σ02)$ sequence, such that $E|εt|2+η<∞$, or rather, $E(εt4)<∞$.

With details of the Monte Carlo designs and consideration given below, we initially mention the following findings for bootstrap simulations in the AR model with a constant term. The findings are typical for existing applications of the bootstrap and are standard in time series contexts.

(1)

With $εt$ i.i.d. $N(0,σ02)$ the $χ12$-based asymptotic test performs well for even small samples of size $n$ in terms of empirical rejection frequencies, or empirical size, for $ρ0=0.5$, while for $ρ0=0.9$ the asymptotic test fails as its empirical size is not close to the nominal level $α$. In comparison, the iid (and wild) bootstrap-based test has empirical size close to the nominal level in both cases, see Table 1 columns A and B for $ρ0∈{0.5,0.9}$, $σ02=1$, and $n∈{15,25,...,1000}$ with $α=0.05$.

(2)

With $εt$ independently $N(0,σt2)$ distributed with $σt2$ time-varying (heteroskedasticity), neither the $χ12$-based asymptotic nor the iid bootstrap-based tests have empirical size close to the nominal level $α$, which contrasts with the wild bootstrap-based test. This is illustrated in Table 1 columns C–F for a time-changed volatility, $σt2=σ0,12+σ0,22I(t≥n/2])$, with $σ0,12=1<σ0,22=15$, and as before $ρ0={0.5,0.9}$ and $n∈{15,25,...,1000}$ with $α=0.05$.

(3)

In terms of empirical rejection frequencies under the alternative, bootstrap-based tests and the asymptotic test are comparable. This is illustrated in Table 1, with $εt$ i.i.d. $N(0,σ02)$ for $n=250$ and $ρ0=0.9$, and the test of $H0:ρ=ρ¯$ is considered for values of $ρ¯$ ranging from $0.70$ to $0.875$ (with $ρ¯=ρ0$ included as a benchmark).

(4)

The wild bootstrap is often motivated by its ability to replicate underlying heteroskedasticity (see the section “Bootstrap in the Autoregressive Model” and Remark 10). This is illustrated in Figure 1, where panel A shows the empirical residuals $ε˜t$ from one of the draws in Table 1 with $σt2=σ0,12+σ0,22I(t≥n/2])$. Figure 1, in panels B, C, and D, illustrates that, while $εt∗$ replicates the heteroskedasticity for the wild bootstrap, this is not the case for the iid bootstrap.

(5)

In terms of requirements for finite moments of the i.i.d. sequence $εt$, the section “Bootstrap Validity Under $H0$” discussed sufficiency and necessity of the condition $E|εt|2+η<∞$ for some $η>0$, and it was conjectured that $E(εt2)<∞$ was sufficient. This is illustrated in Table 2, which shows that when $εt$ does not have a finite variance, then the asymptotic test, as well as the wild and iid bootstrap-based tests, fail to have correct empirical size. However, when $εt$ has a finite variance, while the asymptotic test has empirical size far from the nominal, the bootstraps work despite the lack of fourth order moments, for example. Also, Table 2 shows results from a so-called permutation bootstrap (see the section “Moment Condition, $E|εt|k<∞$ for $k≥2$”).

#### Asymptotic Test

We consider $xt$ as given by the AR model of order one, with a constant term $δ$ included,

$Display mathematics$(21)

with $εt$ i.i.d. $N(0,σ2)$ and $x0$ fixed. The parameters are given by $θ=(δ,ρ,σ2)′∈Θ=ℝ2×(0,∞)$, with $θ0=(δ0,ρ0,σ02)′$ the true value, where $θ0∈Θ0={θ∈Θ||ρ0|<1}$ and the hypothesis of interest is given by $H0:ρ=ρ¯$.

The unrestricted and restricted (Gaussian likelihood-based) estimators that maximize

$Display mathematics$

with $x0$ fixed and $εt(ρ,δ)=xt−ρxt−1−δ$, are given by

$Display mathematics$

respectively. The theory from the case of no constant term immediately carries over, such that for $ρ¯=ρ0$, $|ρ0|<1$ and $εt$ i.i.d. $(0,σ02)$, as $n→∞$,

$Display mathematics$

In the implementations of the asymptotic test, we use the p-value, $pn$, calculated as the tail probability of $τn$ in the limiting $χ12−$ distribution, and reject if $pn$ is smaller than the nominal level $α$.

Column A in Table 1 illustrates this for $xt$ in (21) generated with $δ0=0,$$ρ0={0.5,0.9},$ and $σ02=1$. Moreover, $εt$ is simulated as an i.i.d. $N(0,σ02)$ sequence, and $x0=0$. The empirical rejection frequencies are reported based on $N=10,000$ repetitions, with nominal level $α=0.05$. Results for $n∈{15,25,...,1000}$ are given in column A of Table 1. Observe, as noted, that quite a large sample is required for the limiting $χ12$ distribution to be a good approximation, in particular with $ρ0=0.9$ (Duffee & Stanton, 2008, and references therein).

Remark 20 (Empirical rejection probabilities). At the chosen (nominal) level $α$, with $qα$ the corresponding $1−α$ quantile of the limiting distribution, the true rejection probability at sample length $n$ is $αn=P(τn>qα)$. The Monte Carlo estimator is the empirical rejection frequency computed as $αn,N=1N∑i=1NI(pn,i<α)$, where $pn,i$ is the p-value in Monte Carlo replication $i$, $i=1,2,...,N$. It follows that the simulation uncertainty of $αn,N$ is given by

$Display mathematics$

(Hendry, 1984), and for a correctly sized test, with $αn=0.05$ and $N=104$, the 95% confidence bound for $αn,N$ is $[0.0456,0.0544]$. Similar considerations hold for the bootstrap simulations of the test, with $pn,i$ replaced by $pn,B,i∗$ (see also Remark 14).

#### Table 1. Empirical Rejection Frequencies of Asymptotic and Bootstrap Tests in the Homoskedastic and Heteroskedastic Case

Homoskedastic Case

Heteroskedastic Case

(A)

(B)

(C)

(D)

(E)

(F)

$n$

Asymp.

iid boot.

Asymp.

iid boot.

Wild boot.

Wild boot.

$N(0,1)$

$ρ0=ρ¯=0.5$

15

$0.0825$

$0.0512$

$0.1319$

$0.1015$

$0.0720$

$0.0462$

25

$0.0733$

$0.0541$

$0.1329$

$0.1121$

$0.0685$

$0.0496$

50

$0.0623$

$0.0519$

$0.1370$

$0.1265$

$0.0625$

$0.0486$

100

$0.0538$

$0.0511$

$0.1425$

$0.1383$

$0.0586$

$0.0488$

250

$0.0521$

$0.0508$

$0.1428$

$0.1419$

$0.0593$

$0.0557$

500

$0.0498$

$0.0478$

$0.1398$

$0.1375$

$0.0526$

$0.0491$

1000

$0.0495$

$0.0493$

$0.1434$

$0.1463$

$0.0515$

$0.0502$

$ρ0=ρ¯=0.9$

15

$0.1971$

$0.0558$

$0.1797$

$0.0597$

$0.0688$

$0.0534$

25

$0.1651$

$0.0552$

$0.1727$

$0.0688$

$0.0670$

$0.0563$

50

$0.1218$

$0.0526$

$0.1603$

$0.0875$

$0.0628$

$0.0530$

100

$0.0866$

$0.0507$

$0.1518$

$0.1016$

$0.0565$

$0.0501$

250

$0.0664$

$0.0511$

$0.1444$

$0.1226$

$0.0538$

$0.0494$

500

$0.0559$

$0.0480$

$0.1452$

$0.1323$

$0.0500$

$0.0474$

1000

$0.0548$

$0.0516$

$0.1463$

$0.1376$

$0.0515$

$0.0503$

Note. The data generating process is given by (21) with $ρ0∈{0.5,0.9}$ and $δ0=0$ and the bootstrap process defined in (22). In panels (A) and (B), the innovations $εt$ are i.i.d.$N(0,σ02)$ distributed with $σ02=1$, while in panels (C)–(F) $εt$ are independently $N(0,σt2)$ distributed, with $σt2$ given in (25). The number of bootstrap replications is $B=399$ and $N=10,000$ replications.

#### The iid Bootstrap Test

To illustrate bootstrap-based testing, we apply a restricted recursive bootstrap in terms of residuals estimated under $H0$ (see “Bootstrap Validity Under $H0$”).

Specifically, for steps $A$ and $B$ in the section “Bootstrap in the Autoregressive Model,” the bootstrap samples ${xt∗}t=0n$ are sampled from

$Display mathematics$(22)

with $x0∗=x0$ and $εt∗$ drawn with replacement (for wild, see the section “The Wild bootstrap test” below) from ${ε˜tc}t=1n,$ where $ε˜tc$ are defined as in3 (8) in terms of

$Display mathematics$(23)

For the bootstrap sample, ${xt∗}t=0n$, we estimate the unrestricted and restricted models and calculate the bootstrap statistic $τn∗$, given by

$Display mathematics$

Here, $θ^n∗=(δ^n∗,ρ^n∗,σ^n∗2)′$ and $θ˜n∗=(δ˜n∗,ρ¯,σ˜n∗2)′$ denote the unrestricted and restricted bootstrap estimators, respectively, in terms of the bootstrap log-likelihood function

$Display mathematics$

For step $C$, the bootstrap test is based on replicating just given arguments to obtain ${τn,b∗}b=1B$ with $B$ denoting the number of bootstrap repetitions. As discussed in Remark 9, the empirical bootstrap p-value is computed as the tail probability,

$Display mathematics$(24)

With $B=399$ bootstrap repetitions, the empirical rejection frequencies for the $N=10,000$ Monte Carlo repetitions are presented in column B in Table 1.

#### The Wild Bootstrap Test

The wild bootstrap design is as given in the section “iid Bootstrap Test,” except that $εt∗$ for the wild bootstrap is resampled by

$Display mathematics$

with $wt∗$ i.i.d. $(0,1)$ distributed and $ε˜t$ defined in (23). In the simulations, $wt∗$ is chosen as $*N(0,1)$ and Rademacher distributed, respectively (Remark 10).

For the simulations reported in Table 1, $εt$ are assumed not to be i.i.d. $N(0,σ02)$ distributed in order to illustrate the impacts of heteroskedasticity. Specifically, we set $εt=dN(0,σt2)$ with $σt2=1$ for $t=1,2,...,[n/2]$ and $σt2=15$ for $t=[n/2]+1,...,n$. That is,

$Display mathematics$(25)

with $σ0,12=1$ and $σ0,22=15$. In this case the asymptotic test is not consistent as demonstrated in column A of Table 1, which reports the empirical rejection probabilities for the asymptotic test. The asymptotic test is severely oversized—and even for $n=500$ the empirical size is not close to $α=0.05$. Also the iid bootstrap is not asymptotically valid. With the iid bootstrap, as in the section “iid Bootstrap Test,” the results are reported in column D, and we observe that the results are similar to the asymptotic test. Intuitively, this reflects that the bootstrap series, ${xt∗}t=0n$, does not mimic the properties of the original data series ${xt}t=0n$. This can be illustrated by Figure 1, where panel A shows pronounced heteroskedasticity of the estimated residuals, $ε˜t$, for one sample. Panel B shows a single iid resampled sample ${εt∗}t=1n$ from ${ε˜t}t=1n$, and as, by definition, in particular the ordering change, the $εt∗$ series does not mimic the heteroskedasticity as seen in the estimated residuals in panel A.

For the wild bootstrap, columns E and F in Table 1 report the empirical rejection frequencies for the wild bootstrap test with, as mentioned, $wt∗$ distributed as $N(0,1)$ and Rademacher, respectively. The empirical size for the wild bootstrap is quite close to the nominal level, with the Rademacher distribution performing slightly better. Likewise, panels C and D in Figure 1 illustrate that the wild bootstrap $εt∗$ series more closely mimisc the properties of the original $ε˜t$ series.

#### Bootstrap Under the Alternative

To establish asymptotic validity, it was shown that $τn∗$ is also asymptotically $χ2$-distributed under the alternative, leading to a consistent bootstrap test. To illustrate this, we consider the empirical probability of rejecting a false hypothesis.

Specifically for the iid bootstrap test in the section “iid Bootstrap Test,” let the data $xt$ be generated with true value $θ0=(ρ0,σ02,δ0)′$ as before. The hypothesis of interest is $H0:ρ=ρ¯$ and we let $ρ¯∈{0.875,...,0.70}$ to illustrate the empirical power of the bootstrap test and asymptotic test. From Table 2 it follows that the bootstrap is comparable to the asymptotic test in terms of empirical power.

#### Table 2. Empirical Rejection Frequencies of Asymptotic and Bootstrap Tests Under the Alternative

$ρ¯$

Asymptotic

iid bootstrap

$0.9$

$0.0651$

$0.0485$

$0.875$

$0.0980$

$0.0810$

$0.85$

$0.2911$

$0.2658$

$0.825$

$0.5533$

$0.5283$

$0.8$

$0.7730$

$0.7567$

$0.75$

$0.9617$

$0.9578$

$0.7$

$0.9958$

$0.9954$

Note. The data-generating process is given by (21) with $n=250$ and $ρ0=0.9$, such that $ρ0≠ρ¯$ except for the first row entry. The number of bootstrap replications is $B=399$ and $N=10,000$ replications.

#### Moment Condition, $E|εt|k<∞$ for $k≥2$

When establishing asymptotic validity of the bootstrap the moment condition, $E(εt4)<∞$ was discussed, and it was mentioned that while it is a sufficient condition, it may not be necessary. On the other hand, $E(εt2)<∞$ seems necessary unless the bootstrap algorithm is based on permutation, see also Table 3 and the text before this.

To illustrate this, we consider the data $xt$ as generated with true value $θ0=(ρ0,σ02,δ0)′$ as before for samples of size $n$, $n∈{15,...,1000}$. The i.i.d. innovations $εt$ are simulated from the Student’s $tv$-distribution, where the degrees of freedom $v$, $v∈{3/2,3,5}$. Specifically, for $v=3/2$, $E|εt|k$ is finite, only for $k<3/2$, thus allowing first order, but not second order, moments of $εt$. For $v=3$ and $v=5$, the second and finite fourth-order moments are finite, respectively. Also note that the Gaussian case is included as a reference. Table 3 shows that the asymptotic test (based on the $χ12$ approximation) for all the four cases has empirical rejection rates far from the nominal level of $α=0.05$, even for large samples $n$, where $n∈{15,...,1000}$.

As to the bootstrap design, Table 3 reports bootstrap simulations based on iid sampling and the wild (Rademacher). For $v=3/2$, as expected, neither the wild nor the iid bootstraps have empirical rejection rates close to the nominal level. For $v≥3$, both the wild and iid bootstrap work surprisingly well, even when $v=3$.

Additionally, Table 3 reports bootstrap-based testing where $εt∗$ are iid sampled, but without replacement. This, which is referred to as the permutation bootstrap, works well in terms of empirical rejection frequencies, even for $v=3/2$. In general, the permutation bootstrap works well in the context of heavy-tailed i.i.d. $εt$ (see Cavaliere, Nielsen, & Rahbek, 2020, and also the discussion in “Heavy-Tailed Autoregressive Models” for AR models with heavy-tailed innovations).

#### Table 3. Empirical Rejection Frequencies With Heavy Tailed Innovations

Asymptotic

Iid

Wild

Permutation

$n$

Student’s $tν,ν=3/2$

15

0.1434

0.0395

0.0248

0.0499

25

0.0959

0.0382

0.0256

0.0520

50

0.0563

0.0350

0.0268

0.0488

100

0.0363

0.0340

0.0262

0.0443

250

0.0290

0.0350

0.0326

0.0480

500

0.0249

0.0333

0.0322

0.0465

1000

0.0278

0.0351

0.0348

0.0513

$n$

Student’s $tν,ν=3$

15

0.1809

0.0508

0.0494

0.0508

25

0.1512

0.0491

0.0475

0.0522

50

0.1129

0.0494

0.0496

0.0521

100

0.0806

0.0491

0.0486

0.0505

250

0.0596

0.0487

0.0514

0.0490

500

0.0578

0.0528

0.0515

0.0534

1000

0.0499

0.0465

0.0486

0.0481

$n$

Student’s $tν,ν=5$

0.1894

0.0519

0.0531

0.0511

25

0.1571

0.0486

0.0515

0.0499

50

0.1119

0.0523

0.0495

0.0507

100

0.0807

0.0503

0.0497

0.0503

250

0.0639

0.0486

0.0477

0.0486

500

0.0542

0.0484

0.0478

0.0485

1000

0.0540

0.0498

0.0501

0.0503

Note. Empirical rejection frequencies of asymptotic and bootstrap tests with $εt$ i.i.d. Student’s $tν$ for $ν∈{3/2,3,5}$. Results are reported for the iid bootstrap, the wild boostrap with $wt$ Rademacher distributed, and the permutation bootstrap. Simulations are reported for sample lengths $n∈{15,...,1000}$. The number of bootstrap replications is $B=399$ and $N=10,000$ replications.

### A Selection of Further Topics

In the previous sections the simple AR model of order one was used to introduce key ideas and challenges of the iid and wild bootstrap schemes when applied to testing the hypothesis $H0:ρ=ρ¯$. In this section, we provide an overview of recent selected results for the bootstrap when applied to different testing problems in econometric time series models. The overview is not meant to be exhaustive (see, e.g., Cavaliere & Rahbek, 2020, for a review of the bootstrap with more technical details, as well the references therein).

#### Nonstationary (Vector) Autoregressive Models

##### The iid Bootstrap

Consider initially the AR model in (1) again. The hypothesis of nonstationarity is given by $H0:ρ=1,$ or equivalently, with $π=ρ−1$, $H0:π=0$, in the AR model restated as

$Display mathematics$(26)

where $Δxt=xt−xt−1$. It follows that the test-statistic $τn$ in (5) can be written as

$Display mathematics$

and under $H0$,

$Display mathematics$(27)

where $B(u)$ is a standard Brownian motion, $u∈(0,1)$, and $τ$ is the (squared) Dickey-Fuller distribution (Hamilton, 1994). In this case, in terms of the bootstrap in “Bootstrap in the Autoregressive Model,” the restricted recursive bootstrap, given by $Δxt∗=εt∗$ with $εt∗$ iid resampled, is asymptotically valid. This holds as in this case $τn∗→d∗pτ$, under both $H0$ and the alternative. In contrast, and as discussed in Basawa, Mallik, McCormick, Reeves, and Taylor (1991), the unrestricted recursive bootstrap based on the recursion $Δxt∗=π^nxt−1∗+εt∗$, with $εt∗$ iid resampled, is invalid. This follows as the corresponding bootstrap statistic, $τn∗$, under $H0$ converges in distribution to $τ˜$, $τ˜≠τ$. Precisely, the bootstrap conditional distribution function converges weakly rather than in probability (Basawa et al., 1991; Cavaliere & Georgiev, 2020; Cavaliere et al., 2015).

The univariate case of testing for nonstationarity is a special case of the more general hypothesis of nonstationarity in vector AR models for $Xt∈ℝp$ with general lag-structure, as given by

$Display mathematics$(28)

where $εt$ are i.i.d. $Np(0,Ω)$ distributed and the initial values $(X0,ΔX0,...,ΔX1−k)$ are fixed in the statistical analysis. Moreover, $π$ and $(γi)i=1k$ are $p×p$ matrices. The hypothesis of nonstationarity of $Xt$ is given by the hypothesis of reduced rank $r,0≤r, of $π$ (Johansen, 1996). Specifically, with $Hr:rank(π)≤r$, it follows that this may be written as

$Display mathematics$

where $α$ and $β$ are $(p×r)$ dimensional matrices. Under the nonstationarity conditions in Johansen (1996, Theorem 4.2), it follows that $Xt$ is a nonstationary process, with $r$ stationary, or co-integrating, relations $β′Xt$, and $(p−r)$ common trends given by $δ′∑i=1tεi$, with $δ$$(p×(p−r))$ dimensional of full rank, and such that $δ′α=0$. The likelihood-ratio statistic $τn(r)$ for co-integration rank $r$ satisfies, under $Hr$ and the mentioned nonstationarity conditions, that

$Display mathematics$

which is a multivariate version of (27) in terms of the $(p−r)$-dimensional standard Brownian motion, $B(⋅)$. Cavaliere, Rahbek, and Taylor (2012) consider the recursive restricted bootstrap based on

$Display mathematics$

with $πn∗=α˜nβ˜′n$ and $γn,i∗=γ˜n,i$; that is, the bootstrap true values are given by the estimators under $Hr$. Asymptotic validity of the iid bootstrap is established in Cavaliere et al. (2012), by showing that $τn∗(r)→d∗pτ(r)$ under $Hr$ and the alternative.

Cavaliere et al. (2015) extend the analysis to hypothesis testing on the co-integration (matrix) parameter $β$. Specifically, Cavaliere et al. (2015, Proposition 1 and Theorem 1) establish that under the hypothesis $Hr¯:β=β¯$, the bootstrap likelihood ratio statistic, $τn∗$, satisfies $τn∗→d∗pχ(p−r)r2$. Importantly, it is also established that under the alternative, $τn∗$ has a limiting distribution (in distribution) in terms of a diffusion process with a stochastic diffusion coefficient, and hence it is bounded in probability such that the bootstrap-based test is asymptotically valid.

##### The Wild Bootstrap

In order to allow for possible heteroskedasticity in the $εt$ sequence in (28), the application of the wild bootstrap has also been studied. Results for application of the wild bootstrap in general lag univariate AR models in Gonçalves and Kilian (2004), with $εt$ allowed to have general time-varying volatility structures, such as ARCH and stochastic volatility, have been generalized to the testing the hypothesis of co-integration $Hr$ in Cavaliere, Rahbek, and Taylor (2010a, 2010b, 2014) and Boswijk, Cavaliere, Rahbek, and Taylor (2016).

Moreover, Boswijk et al. (2016) and Boswijk, Cavaliere, Georgiev, and Rahbek (2019) consider general hypothesis testing on the co-integration parameters $α$ and $β$, with $π=αβ′$ in (28). They consider the case of stochastic volatility, where $εt=Ωt1/2zt$, with the $p$-dimensional $zt$ i.i.d. $(0,1)$ and the time-varying $(p×p)$-dimensional $Ωt=Ω(t/n)$. Moreover, with “$→w$“ denoting weak convergence, it is assumed that for $u∈(0,1)$,

$Display mathematics$(29)

where $B$ is a $p$-dimensional standard Brownian motion, which generalizes the i.i.d. $(0,Ω)$ assumption, where $n−1/2∑t=1[nu]εt→wΩ1/2B(u)$. Specifically, the limiting process in (29) is a continuous-time martingale, with in general an unknown covariance (kernel). This implies that the limiting distribution of the test statistic(s) $τn$, for example for the mentioned hypotheses $Hr$ and $Hr¯$, will depend on unknown nuisance parameters, which again means asymptotic inference is infeasible in practice. In contrast, for the wild bootstrap, it is established that $n−1/2∑t=1[nu]εt∗$ has the same limiting distribution (in probability), and, as a result, the wild bootstrap is asymptotically valid as shown in Boswijk et al. (2016, 2019), under some additional regularity conditions to be verified.

##### Time-Varying Conditional Volatility Models

As discussed in Andrews (2000), applying bootstrap-based testing in ARCH models is in general difficult, and may be invalid in certain cases, due to general problems arising when testing hypotheses in time series models when one or more parameters under the null may be “on the boundary of the parameter space” (p. 399).

To illustrate, consider here $xt$ given by a linear ARCH model of order $q$

$Display mathematics$

with $zt$ i.i.d. $(0,1)$ and

$Display mathematics$

In the statistical analysis, the initial values $(x0,...,x−q+1)$ are fixed, and the parameter $θ=(ω,α1,...,αq)′∈Θ$, where

$Display mathematics$

Thus by definition of the parameter space $Θ$, if for the true value $θ0=(ω0,α0,1,...,α0,q)′$ it holds that $α0,j=0$ for some $j$, the true value $θ0$ is on the boundary of $Θ$.

The fact that it is unknown a priori which of the ARCH coefficients may or may not be zero leads to nonpivotal limiting distributions of test statistics and estimators. Consider here the likelihood ratio statistic $τn$ for the nullity of the $q$-th order ARCH coefficient; that is, the hypothesis $Hq:αq=0$. With the Gaussian likelihood function given by

$Display mathematics$

by definition, $τn=2(ln(θ^n)−ln(θ˜n))$, where the unrestricted4 Gaussian MLE is given by $θ^n=argmaxθ∈Θln(θ)$, while $θ˜n$ is the Gaussian MLE under $Hq$. By Andrews (1999, 2001), it follows that $τn$ has a limiting distribution, which is non-standard. In addition, the limiting distribution of $τn$ is nonpivotal as, crucially, it depends on whether $α0,i>0$, or $α0,i=0$ for $i=1,...,q−1$ under $Hq$.

While this implies that the unrestricted bootstrap is invalid (Andrews, 2000), it follows by Cavaliere, Nielsen, and Rahbek (2017) that the iid restricted bootstrap is asymptotically valid under mild conditions for the simple case of the first order ARCH with $q=1$. Moreover, Cavaliere, Nielsen, Pedersen, and Rahbek (2020) demonstrate validity of a modified restricted bootstrap, which can be applied for general testing problems in parametric models with parameters on the boundary under the null. Specifically, for the case of ARCH of order $q$, consider the bootstrap process

$Display mathematics$

with $zt∗$ iid resampled from $z^t=xt/σt(θ^n)$, after recentering and rescaling these. The bootstrap conditional volatility process $σt∗2(θn∗)$ is given by

$Display mathematics$(30)

with $ωn∗=ω˜n$ and $αn,i∗=α˜n,iI(α˜n,i>cn)$, with $cn$ a deterministic sequence that satisfies (a) $cn→0$, and (b) $n1/2cn→∞$, as $n→∞$. The bootstrap scheme is referred to as modified since in (30) the bootstrap true values $αn,i∗$—by “ shrinking”—are set to zero for $i=1,...,q−1$, provided $α˜n,i$ is small relative to $cn$. Note that with the time-varying bootstrap volatility defined by (30), this is a case of a fixed design (or rather fixed volatility) bootstrap (see step $A$ in “Bootstrap in the Autoregressive Model”). The general fixed volatility bootstrap for ARCH models is considered in Cavaliere et al. (2018), and the modified, by shrinking, fixed volatility bootstrap is shown in Cavaliere, Nielsen, Pedersen, and Rahbek (2020, Proposition 1) to be asymptotically valid. Simulations there show that both the fixed volatility and the recursive with $xt−i2=xt−i∗2$ in (30) bootstrap based tests have empirical rejection frequencis that are close to the nominal level for small and moderate sample sizes $n$. Moreover, as for the discussion of the moment requirements for the AR bootstrap, simulations indicate that while sufficient, the moment constraints on the original ARCH process imposed to establish validity are also not necessary.

##### Double Autoregressive Models

The double autoregressive (DAR) model combines the AR and ARCH models, as both the conditional mean and conditional variance depend on lagged levels of the process (see Ling, 2004, 2007; and for a multivariate “co-integrated” version, Nielsen & Rahbek, 2014).

Consider here the first order DAR model as given by

$Display mathematics$(31)

with $zt$ i.i.d. $N(0,1)$, $x0$ is fixed in the statistical analysis and the parameter given by $θ=(π,σ2,α)′∈Θ,$ with $Θ={θ∈ℝ3:σ2>0andα≥0}$.

A notable special feature of the DAR process is that for $π=0$ the process is strictly stationary for any $0<α<2.42$ (Borkovec & Klüppelberg, 2001; Ling, 2004, 2007), while the process is nonstationary when $π=α=0$. With $π=0$ and $α∈(0,2.42)$, while being strictly stationary the DAR process $xt$ has infinite variance (and only finite fractional moments).

From the specification of the parameter space $Θ$ evidently for $α0=0$, the true value $θ0$ is on the boundary, raising the issues discussed in the section “Time-Varying Conditional Volatility Models” in relation to the ARCH model. Asymptotic theory for the Gaussian likelihood-based MLE with $α0>0$ is given by Ling (2004), while Cavaliere and Rahbek (2020) extend the results to allow for the boundary case. Klüppelberg, Maller, van de Vyver, and Wee (2002) derive the asymptotic distribution of the likelihood ratio statistic $τn$ for the hypothesis of nonstationarity as given by $H0:π=α=0$ (see also Chen, Li, & Ling, 2013). Different versions of a bootstrap-based test are discussed in Cavaliere and Rahbek (2020). In particular, validity is established for a restricted bootstrap given by $Δxt∗=σ˜nzt∗$, where $σ˜n2$ is the MLE of $σ2$ under $H0$, while $zt∗$ is obtained by iid resampling of unrestricted residuals, $z^t$ (recentered and rescaled). That is, the unrestricted residuals $z^t$ are given by

$Display mathematics$

with $θ^n=(π^n,σ^n,α^n)′$ the unrestricted MLE. In line with the discussion in the section “Bootstrap in the Autoregressive Model” regarding restricted and unrestricted residuals, this choice ensures that $z^t$ for large $n$ is “close” to the true $zt$, irrespective of whether the null $H0$ is true or not. While the validity result is shown for this choice, simulations indicate that in practice the difference between choosing to resample from $z^t$ or from $z˜t=Δxt/σ˜n$ is negligible. (See Cavaliere & Rahbek, 2020, for a detailed discussion of this as well as asymptotic theory for different bootstraps.)

##### Heavy-Tailed Autoregressive Models

So far results for the (vector) AR models have been derived under the assumption that the innovations $εt$ have mean zero and a finite variance $σ2$, or some time-varying, possibly conditional, variance $σt2$ when discussing heteroskedasticity. To allow for more extreme events, and phenomena such as “ bubble” periods with local explosive behavior, this assumption was relaxed in Davis and Resnick (1985a, 1985b, 1986) and Davis and Song (2020), where the i.i.d. innovations $εt$ are allowed to have infinite variance. Specifically, they consider the case of $εt$ i.i.d. with a stable distribution such as the Cauchy; that is, “heavy-tailed” as the tails of the distribution of $εt$ are assumed to decay at a rate that is slower than the Gaussian (exponential) rate.

Two key examples are given by the classic AR model and the so-called noncausal AR model of order one in terms of i.i.d. stable distributed $εt$,

$Display mathematics$(32)

For the standard, and hence causal, $AR$, recall that with $t=1,...,n$, $x0$ is the initial value that is fixed in the statistical analysis, while for the noncausal $AR+$$xn$ is the “initial value” due to the forward recursion. Noncausal $AR+$ type models have become popular as they seem to capture well the dynamics of phenomena such as bubbles where, after period of exponential type growth, the process “collapses.” Interestingly, and linking the heavy-tail $AR+$ models with the DAR model in “Double Autoregressive Models,” the $AR+$ process in (32) can be shown to have a causal “semi-strong” representation as the DAR process in (31) (see Gourieroux & Zakoian, 2017).

Consider testing the hypothesis $H0+:ρ+=ρ¯$ using the Gaussian likelihood-based statistic $τn+$, given by

$Display mathematics$

where $ρ^n+=∑t=1n−1xtxt+1/∑t=1nxt+12$ and $σ^n2=n−1∑t=1n−1(xt−ρ^n+xt−1)2$. While the test statistic $τn+$ is analogous to the (square root of the) previously studied statistic $τn$ in (5), the limiting distribution is nonstandard, as the $εt$ are assumed to be i.i.d. stable distributed. For example, with $εt$ Cauchy distributed, $(n/logn)(ρ^n+−ρ¯)$ is asymptotically distributed as $(1+ρ¯)Cχ12$, where $C$ is standard Cauchy distributed and $τn+=OP(n−1/2logn)$. In general for stable distributions asymptotic testing is infeasible as the limiting distributions depend on the “tail index” (which is one for the Cauchy) of the innovations $εt$, and moreover the normalization (which is $n−1/2logn$ for the Cauchy case) depend on further, and in practice, unknown quantities. In the case of stable distributed innovations, Cavaliere, Nielsen, and Rahbek (2020) discuss validity of the recursive bootstrap scheme similar to (10),

$Display mathematics$

initialized with $xn∗=xn$, and with $εt∗$ resampled from the restricted residuals ${ε˜t+}t=1n−1$, where $ε˜t+=xt−ρ¯xt+1$. Crucially the $εt∗$ are not sampled by iid resampling with replacement, as this would lead to an invalid bootstrap test (Athreya, 1987; Knight, 1989). Instead, $εt∗$ are resampled without replacement; that is, by permuting ${ε˜t+}t=1n−1$, or, in combination with the wild, by permuting ${ε˜t+wt∗}t=1n−1$, where $wt∗$ are i.i.d. Rademacher distributed (Remark 10). With $τn+∗$ the bootstrap statistic based on the permutation, or the combined permutation-wild bootstrap, Theorem 1 in Cavaliere, Nielsen, and Rahbek (2020) establishes validity of the bootstrap-based test under the null hypothesis for general $AR+$ models.

Similarly, Cavaliere, Georgiev, and Taylor (2016) establish bootstrap validity for the so-called sieve bootstrap in Bühlmann (1997) for general causal AR models with heavy tails. Also note that in terms of testing for the presence of bubbles based on one-sided testing using the supremum of recursively computed Dickey-Fuller type statistics as in Phillips, Wu, and Yu (2011), Harvey, Leybourne, Sollis, and Taylor (2016) establish validity of a wild bootstrap based test to allow for heteroskedasticity.

This article has provided an introduction to key steps required for a successful implementation of bootstrap hypothesis testing to time series models. In the framework of a simple autoregressive model, we have discussed the (large-sample) validity of recursive bootstrap algorithms, where the bootstrap sample is constructed by mimicking the dependence structure of the original data, as implied by the econometric model at hand. We have discussed the main requirements for bootstrap inference to be valid; that is, to mimic the asymptotic distribution of the original test statistic under the null hypothesis and to be bounded (in probability) when the null hypothesis is false. To do so, we have introduced the needed probability tools (which involve dealing with the conditional—hence random—nature of the bootstrap measure) and shown how to apply them in order to assess the large sample behavior of the bootstrap estimators and test statistics.

Needless to say, this article is not meant to be exhaustive or to provide a comprehensive treatment of bootstrap inference. We have focused on bootstrap hypothesis testing which, although being one of the most common applications of the bootstrap, is not the only one. Further common implementations of the bootstrap involve computation of standard errors, the construction of confidence intervals for the parameters of interest, and bias correction of the estimators (see, inter alia, Horowitz, 2001, or MacKinnon, 2006, for reviews).

This article is also not exhaustive in terms of bootstrap algorithms. For instance we have not discussed bootstrap algorithms based on resampling a block of observations (or residuals) rather than resampling single observations (or residuals). As introduced by Künsch (1989), block bootstrap methods are quite flexible and powerful in cases where one does not have strong a priori beliefs or knowledge about the dependence structure of the data (see also Lahiri, 2003; Politis & Romano, 1994; Shao, 2010). Subsampling methods represent a valid alternative (or complement) to recursive, model-based bootstrap (Politis, Romano, & Wolf, 1999). We have focused on a case (the first order AR model) where the data-generating process is described by a finite dimensional vector of parameters. Extensions to infinite-dimensional parameter spaces, as for the case of general linear processes, are available in the literature; a classic example is the $AR(∞)$ sieve bootstrap of Kreiss (1988, 1992), Bühlmann (1997), and Gonçalves and Kilian (2007).

It is important to emphasize that there is a rich literature discussing finite-sample improvements of the bootstrap in various models, including time series models, based on Edgeworth expansions of the $τn$ and $τn∗$ statistics, as briefly mentioned in Remark 10. An introduction to Edgeworth expansions as applied for the bootstrap can be found in van der Vaart (2000, chapter 23.3; see also Hall, 1992; Horowitz, 2001).

Finally, it is worth noticing that throughout this article bootstrap validity under the null is defined as the fact that under the null the bootstrap test statistic converges to the asymptotic (null) distribution of the original statistic. This implies that, under rather mild regularity conditions (such as continuity of the limiting distribution), if the null hypothesis holds then the bootstrap $p$-value $pn∗$ is asymptotically uniformly distributed (Remark 15). Combined with establishing that the bootstrap statistic is bounded in probability under the alternative such that $pn∗→p0$, this was used to establish validity of the bootstrap (see “Bootstrap Validity Under the Alternative”). Cavaliere and Rahbek (2020) discuss in detail different definitions of asymptotic validity and their verification under different assumptions. An example is Cavaliere and Georgiev (2020), where validity of the bootstrap test is defined directly in terms of asymptotic uniformity of the bootstrap $p$-values, rather than the properties of (consistent) estimation of the limiting null distribution of the original test statistic. By doing so, the bootstrap can be employed in cases where the limiting distribution of the original statistic may not be well defined, or in cases where the boostrap distribution does not converge in probability but, rather, converges in distribution (see, e.g., Boswijk, Cavaliere, Georgiev, & Rahbek, 2019, and the discussions in sections “Nonstationary [Vector] Autoregressive Models” and “Heavy-Tailed Autoregressive Models” where this was applied).

### Acknowledgments

This research was supported by Danish Council for Independent Research (DSF Grant 7015-00028B).

#### References

• Andrews, D. W. K. (1999). Estimation when a parameter is on a boundary. Econometrica, 67(6), 1341–1383.
• Andrews, D. W. K. (2000). Inconsistency of the bootstrap when a parameter is on the boundary of the parameter space. Econometrica, 68(2), 399–405.
• Andrews, D. W. K. (2001). Testing when a parameter is on the boundary of the maintained hypothesis. Econometrica, 69(3), 683–734.
• Andrews, D. W. K., & Buchinsky, M. (2000). A three-step method for choosing the number of bootstrap repetitions. Econometrica, 68, 23–51.
• Athreya, K. B. (1987). Bootstrap of the mean in the infinite variance case. Annals of Statistics, 15, 724–731.
• Basawa, I. V., Mallik, A. K., McCormick, W. P., Reeves, J. H., & Taylor, R. L. (1991). Bootstrapping unstable first-order autoregressive processes. Annals of Statistics, 19, 1098–1101.
• Berkowitz, J., & Kilian, L. (2000). Recent developments in bootstrapping time series. Econometric Reviews, 19(1), 1–48.
• Borkovec, M., & Klüppelberg, C. (2001). The tail of the stationary distribution of an autoregressive process with ARCH(1) errors. Annals of Applied Probability, 11(4), 1220–1241.
• Boswijk, H. P., Cavaliere, G., Georgiev, I., & Rahbek, A. (2019). Bootstrapping non-stationary stochastic volatility. Working Paper TI 2019–083/III. Tinbergen Institute.
• Boswijk, H. P., Cavaliere, G., Rahbek, A., & Taylor, A. M. R. (2016). Inference on co-integration parameters in heteroskedastic vector autoregressions. Journal of Econometrics, 192, 64–85.
• Bühlmann, P. (1997). Sieve bootstrap for time series. Bernoulli, 3, 123–148.
• Cavaliere, G., & Georgiev, I. (2020). Inference under random limit bootstrap measures. Econometrica.
• Cavaliere, G., Georgiev, I., & Taylor, A. M. R. (2016). Sieve-based inference for infinite-variance linear processes. Annals of Statistics, 44(4), 1467–1494.
• Cavaliere, G., Nielsen, H. B., Pedersen, R. S., & Rahbek, A. (2020). Bootstrap inference on the boundary of the parameter space with application to conditional volatility models. Journal of Econometrics.
• Cavaliere, G., Nielsen, H. B., & Rahbek, A. (2015). Bootstrap testing of hypotheses on co-integration relations in vector autoregressive models. Econometrica, 83(2), 813–831.
• Cavaliere, G., Nielsen, H. B., & Rahbek, A. (2017). On the consistency of bootstrap testing for a parameter on the boundary of the parameter space. Journal of Time Series Analysis, 38(4), 513–534.
• Cavaliere, G., Nielsen, H. B., & Rahbek, A. (2020). Bootstrapping noncausal autoregressions. with applications to explosive bubble modeling. Journal of Business and Economic Statistics, 38(1), 55–67.
• Cavaliere, G., Pedersen, R. S., & Rahbek, A. (2018). The fixed volatility bootstrap for a class of ARCH(q) models. Journal of Time Series Analysis, 39(6), 920–941.
• Cavaliere, G., Rahbek, A., & Taylor, A. M. R. (2010a). Co-integration rank testing under conditional heteroskedasticity. Econometric Theory, 26, 1719–1760.
• Cavaliere, G., Rahbek, A., & Taylor, A. M. R. (2010b). Testing for co-integration in vector autoregressions with non-stationary volatility. Journal of Econometrics, 158, 7–24.
• Cavaliere, G., Rahbek, A., & Taylor, A. M. R. (2012). Bootstrap determination of the co-integration rank in vector autoregressive models. Econometrica, 80, 1721–1740.
• Cavaliere, G., Rahbek, A., & Taylor, A. M. R. (2014). Bootstrap determination of the co-integration rank in vector autoregressive heteroskedastic VAR models. Econometric Reviews, 33, 606–650.
• Chen, M., Li, D., & Ling, S. (2013). Non-stationarity and quasi-maximum likelihood estimation on a double autoregressive model. Journal of Time Series Analysis, 35, 189–202.
• Davidson, R., & MacKinnon, J. G. (2000). Bootstrap tests: How many bootstraps? Econometric Reviews, 19, 55–68.
• Davidson, R., & MacKinnon, J. G. (2006). Bootstrap methods in econometrics. In T. C. Mills & K. D. Patterson (Eds.), Palgrave handbook of econometrics: Vol. 1. Econometric theory (pp. 812–838). Basingstoke, UK: Palgrave Macmillan.
• Davis, R., & Resnick, S. (1985a). Limit theory for moving averages of random variables with regularly varying tail probabilities. Annals of Probability, 13, 179–195.
• Davis, R., & Resnick, S. (1985b). More limit theory for the sample correlation function of moving averages. Stochastic Processes and Their Applications, 20, 257–279.
• Davis, R., & Resnick, S. (1986). Limit theory for the sample covariance and correlation functions of moving averages. Annals of Statistics, 14, 533–558.
• Davis, R., & Song, L. (2020). Noncausal vector AR processes with application to economic time series. Journal of Econometrics, 216(1), 246–267.
• Duffee, G. R., & Stanton, R. H. (2008). Evidence on simulation inference for near unit-root processes with implications for term structure estimation. Journal of Financial Econometrics, 6(1), 108–142.
• Durret, R. (2019). Probability: Theory and examples (5th ed.). Cambridge, UK: Cambridge University Press.
• Gonçalves, S., & Kilian, L. (2004). Bootstrapping autoregressions with conditional heteroskedasticity of unknown form. Journal of Econometrics, 123, 89–120.
• Gonçalves, S., & Kilian, L. (2007). Asymptotic and bootstrap inference for AR(∞) processes with conditional heteroskedasticity. Econometric Reviews, 26(6), 609–641.
• Gourieroux, C., & Zakoan, J.-M. (2017). Local explosion modelling by non-causal process. Journal of the Royal Statistical Society, Series B, 79, 737–756.
• Hahn, J., & Liao, Z. (2019). Bootstrap standard error estimates and inference. Working Paper. Los Angeles, CA: UCLA.
• Hall, P. (1992). The bootstrap and Edgeworth expansions. New York, NY: Springer.
• Hall, P., & Heyde, C. C. (1980). Martingale limit theory and its application. New York, NY: Academic Press.
• Hamilton, J. D. (1994). Time series analysis. Princeton, NJ: Princeton University Press.
• Hansen, B. (1996). Inference when a nuisance parameter is not identified under the null hypothesis. Econometrica, 64(2), 413–430.
• Hansen, B. (2000). Testing for structural change in conditional models. Journal of Econometrics, 97, 93–115.
• Harvey, D., Leybourne, S. J., Sollis, R., & Taylor, A. M. R. (2016). Tests for explosive financial bubbles in the presence of non-stationary volatility. Journal of Empirical Finance, 38(B), 548–574.
• Hayashi, F. (2000). Econometrics. Princeton, NJ: Princeton University Press.
• Hendry, D. F. (1984). Monte Carlo experimentation in econometrics. In Z. Grilishes & M. D. Intriligator (Eds.), Handbook of econometrics (Vol. 2, pp. 937–976). Amsterdam, The Netherlands: North Holand.
• Horowitz, J. L. (2001). The bootstrap. In Z. Grilishes & M. D. Intriligator (Eds.), Handbook of econometrics (Vol. 5, pp. 3159–3228). Amsterdam, The Netherlands: North Holand.
• Horowitz, J. L. (2003). The bootstrap in econometrics. Statistical Science, 18(2), 211–218.
• Johansen, S. (1996). Likelihood-based inference in cointegrated vector autoregressive models. Oxford, UK: Oxford University Press.
• Klüppelberg, C., Maller, R. A., Van de Vyver, M., & Wee, D. (2002). Testing for reduction to random walk in autoregressive conditional heteroskedasticity models. Econometrics Journal, 5, 387–416.
• Knight, K. (1989). On the bootstrap of the mean in the infinite variance case. Annals of Statistics, 17, 1168–1175.
• Kreiss, J. P. (1988). Asymptotic statistical inference for a class of stochastic processes. Habilitationsschrift (Unpublished thesis). Universität Hamburg.
• Kreiss, J. P. (1992). Bootstrap procedures for AR(∞)-processes. In K. H. Jöckel, G. Rothe, & W. Sendler (Eds.), Bootstrapping and related techniques. Lecture Notes in Economics and Mathematical Systems (Vol. 376). Berlin, Germany: Springer.
• Künsch, H. (1989). The jackknife and the bootstrap for general stationary observations. Annals of Statistics, 17, 1217–1241.
• Lahiri, S. N. (2003). Resampling methods for dependent data. New York, NY: Springer.
• Ling, S. (2004). Estimation and testing stationarity for double-autoregressive models. Journal of the Royal Statistical Society B, 66, 63–78.
• Ling, S. (2007). A double AR(p) model: Structure and estimation. Statistica Sinica, 17, 161–175.
• Liu, R. Y. (1988). Bootstrap procedures under some non i.i.d. models. Annals of Statistics, 16, 1696–1708.
• MacKinnon, J. G. (2006). Bootstrap methods in econometrics. Economic Record, 82, 2–18.
• MacKinnon, J. G. (2009). Bootstrap hypothesis testing. In D. A. Belsley & J. Kontoghiorghes (Eds.), Handbook of computational econometrics (pp. 183–213). Chichester, UK: Wiley.
• Nielsen, H. B., & Rahbek, A. (2014). Unit root vector autoregression with volatility induced stationarity. Journal of Empirical Finance, 29, 144–167.
• Phillips, P. C. B., Wu, Y., & Yu, J. (2011). Explosive behavior in the 1990s Nasdaq: When did exuberance escalate stock values? International Economic Review, 52, 201–226.
• Politis, D., & Romano, J. (1994). The stationary bootstrap. Journal of the American Statistical Association, 89(428), 1303–1313.
• Politis, D. N., Romano, J. P., & Wolf, M. (1999). Subsampling. New York, NY: Springer.
• Shao, X. (2010). The dependent wild bootstrap. Journal of the American Statistical Association, 105, 218–235.
• van der Vaart, A. W. (2000). Asymptotic statistics. Cambridge, UK: Cambridge University Press.
• White, H. (1980). A heteroskedasticity-consistent covariance matrix estimator and a direct test for heteroskedasticity. Econometrica, 48, 817–838.

### Proof of Lemma 3

Consider the bootstrap process ${xt∗}t=0n$ as given by (10). Without loss of generality set $x0∗=0$, such that by simple recursion,

$Display mathematics$

with $εj∗=0$ for $j≤0$, $L$ the lag operator, $Lεt∗=εt−1∗$, and $ρ(z)=∑i=0∞ρ0izi$, for $z∈ℂ$. The summation to infinity means one can use standard manipulations of the lag polynomial $ρ(⋅)$ as well known from the time series literature (see, e.g., Hamilton, 1994; Johansen, 1996, and references therein). Note first that as $|ρ0|<1$, the coefficients $ρ0i$ in $ρ(z)$ are exponentially decreasing and $ρ(z)$ is convergent for $|z|<1+δ$for some $δ>0$ and $ρ(1)=(1−ρ0)−1$.

When considering the average of $xt∗$, $n−1∑t=1nxt∗$, one may for example use an expansion of $ρ(z)$ (see Remark 21) around $z=1$, such that

$Display mathematics$

with $ρ∗(z)=∑i=1∞θizi$ with $θi=−iρ0i$ exponentially decreasing. This immediately gives

$Display mathematics$

The first term tends to zero by (13). Next, for the second term use that $εt∗$ are i.i.d. (conditionally on the data) such that

$Display mathematics$

with $c=|ρ0|(1−|ρ0|)−2$ and where $n−1∑t=1n|ε˜tc|→pκ<∞$ for $κ$ a constant by standard application of the LLN.

Turning to the average of $xt∗2$, note that

$Display mathematics$

where

$Display mathematics$

With $ρ2(z)=∑i=0∞ρ02izi$ and $ω0=ρ2(1)σ02=σ02/(1−ρ02)$ the first term can be written as

$Display mathematics$

where $ηt∗=εt∗2−σ02$. Thus, using identical arguments, $n−1∑t=1nρ2(L)ηt∗→p∗p0$.

It remains to show that $δn∗→p∗p0$. First, observe that by definition of the double summation,

$Display mathematics$

As $E∗(εt−i∗εt−j∗)=0$ for $j≠i$, it follows that

$Display mathematics$

Here

$Display mathematics$

with $c˜=ρ02σ04/(1−ρ02)$.

Remark 21. The equality

$Display mathematics$

with $ρ∗(z)=∑i=1∞θizi−1$ and $θi=−iρ0i$ follows by the identity

$Display mathematics$

Also note that the polynomial $ρ∗(z)$ is convergent for $|z|<1+δ$ as the coefficients $θi=−iρ0i$ of $ρ∗(z)$ are exponentially decreasing.

### Notes

• 1. The Rademacher distribution is a two-point distribution on $±1$, each with probability a half, whereas the Mammen distribution is a two-point distribution on $1+52$

and $1−52$ with probabilities given by $5−125$ and $5+125$, respectively.

• 2. Markov’s inequality:$P(|X|>δ)≤E|X|k/δk$ for any $δ>0$ and $κ≥1$.

• 3. Note that as the model here includes a constant term, strictly speaking the recentering of $ε˜t$ is not needed.

• 4. A complete discussion of specifications and properties of the parameter space $Θ$ is given in Cavaliere, Nielsen, Pedersen, and Rahbek (2020).