Show Summary Details

Page of

Printed from Oxford Research Encyclopedias, Economics and Finance. Under the terms of the licence agreement, an individual user may print out a single article for personal use (for details see Privacy Policy and Legal Notice).

date: 05 March 2021

An Introduction to Bootstrap Theory in Time Series Econometricsfree

  • Giuseppe Cavaliere, Giuseppe CavaliereDepartment of Economics, University of Bologna; Department of Economics, University of Exeter
  • Heino Bohn NielsenHeino Bohn NielsenDepartment of Economics, University of Copenhagen
  •  and Anders RahbekAnders RahbekDepartment of Economics, University of Copenhagen

Summary

While often simple to implement in practice, application of the bootstrap in econometric modeling of economic and financial time series requires establishing validity of the bootstrap. Establishing bootstrap asymptotic validity relies on verifying often nonstandard regularity conditions. In particular, bootstrap versions of classic convergence in probability and distribution, and hence of laws of large numbers and central limit theorems, are critical ingredients. Crucially, these depend on the type of bootstrap applied (e.g., wild or independently and identically distributed (i.i.d.) bootstrap) and on the underlying econometric model and data. Regularity conditions and their implications for possible improvements in terms of (empirical) size and power for bootstrap-based testing differ from standard asymptotic testing, which can be illustrated by simulations.

Bootstrap in Time Series Econometrics

Bootstrap in econometrics is frequently applied in the context of estimation and testing (Berkowitz & Kilian, 2000; Cavaliere & Rahbek, 2020; Davidson & MacKinnon, 2006; Horowitz, 2001, 2003; MacKinnon, 2009). As an example, consider the case where some test statistic, τn say, is of interest given a sample of (time-series) data x1,...,xn with initial values x0,x1,..,xp for p0, {xt}t=pn with xt. Under suitable regularity conditions, including typically (a) stationarity and ergodicity of the xt process and (b) finite moments conditions on the form E|xt2|k< for some k1, it holds that under the null hypothesis of interest, H0 say,

τndχq2asn,

where q denotes the degrees of freedom and “d” denotes convergence in distribution. Moreover, under the alternative, or when H0 is not true, the statistic τn diverges. In standard asymptotic testing, the χq2 distributional approximation of the test statistic τn is frequently applied using the 1α quantile of the χq2 distribution or, equivalently, by calculating the p-value.

A bootstrap-based test is often motivated by noting that the asymptotic approximation may not be good in finite samples, which may lead, for a finite number of observations n, to an actual size larger, or smaller, than the nominal level α. This is often corrected by applying even simple bootstrap schemes. An additional motivation for the bootstrap is that the underlying model used for estimation—and on which the derivation of τn is based—may be misspecified. A typical example is the assumption of homoskedasticity of model innovations, which often in practice is challenged in the modeling of macro and financial data, where (conditional and unconditional) heteroskedasticity is typically present. In this case, the so-called wild bootstrap may correct for such heteroskedasticity. Likewise, for other types of misspecification and model departures, different bootstrap schemes may be applied depending on which type of misspecification is of concern. In addition, in some cases the limiting distribution of τn cannot be tabulated, for example, because it is a function of unknown nuisance parameters. The bootstrap may resolve such issues. However, it should be emphasized that any bootstrap-based test is—as for the original asymptotic test—only valid under certain regularity conditions. These regularity conditions are important to check and understand as in many cases application of what may be thought of as a “ standard bootstrap” may be invalid and misleading, despite its popularity in many empirical applications. Typical situations in which a standard bootstrap does not work can be found in the context of nonstationary variables and when data exhibit heavy-tailed distributions, as is typical in financial data. When application of existing, or standard, bootstrap schemes fails to work, these may be corrected by more elaborate bootstrap schemes, as is richly documented in recent research in econometrics (see inter alia Cavaliere & Rahbek, 2020, and the references therein).

The bootstrap is a simulation-based approach that is typically simple to apply in the context of inference and testing. Based on some bootstrap scheme, the bootstrap is based on generating new bootstrap samples of data and the test statistic of interest, denoted by {xt}t=pn and τn, respectively. The (re-)generation of such bootstrap data is based on keeping the original sample {xt}t=pn fixed.

If the limiting distribution (in the bootstrap sense) of the bootstrap test statistic τn is identical to the limiting distribution of the original statistic τn under the null of H0, such as the χq2 distribution noted earlier, this implies validity under the null of the bootstrap. If, in addition, under the alternative when H0 is not true, the bootstrap statistic τn does not diverge as the sample size n increases (or, at most it diverges at a slower rate than the original statistic), the bootstrap is asymptotically valid. This is the case when under the alternative the bootstrap-limiting distribution of τn is identical to the limiting distribution of the original statistic or when τn is bounded in (bootstrap) probability.

As a leading example throughout, and to fix ideas, we consider the first-order autoregressive (AR) model. The section “Autoregressive Model” provides a summary of standard econometric nonbootstrap analysis of the AR model before turning to the discussion of the implementation and (asymptotic) theory of the bootstrap in the section “Bootstrap in the Autoregressive Model.” Next, a Monte Carlo study is used to discuss finite sample behavior of different bootstrap schemes (see “Finite Sample Behavior”) and recent advances in econometric time series bootstrap analysis are provided (see “Selection of Further Topics”). The conclusion provides an overview of some important further topics and approaches to bootstrap inference.

The Autoregressive Model

In order to present bootstrap theory and arguments, a brief summary of results and arguments for the (nonbootstrap) AR model of order one is provided. The results and asymptotic arguments presented here are well known (see, e.g., Hamilton, 1994; Hayashi, 2000).

Consider the AR model of order one

xt=ρxt1+εt,t=1,2,...,n,(1)

with the initial value x0 fixed in the statistical analysis and the innovationsεt are assumed to be independently and identically distributed (i.i.d). N(0,σ2) (see also Remark 2 regarding relaxing the assumption of Gaussianity). The parameter of the model is θ=(ρ,σ2) with θ0=(ρ0,σ02) denoting the true value. For estimation, the parameter space is given by

θΘ=×(0,).

For the true parameter θ0, we assume θ0Θ0Θ, with Θ0={θΘ||ρ0|<1}, such that xt in (1) for θ0Θ0 has a stationary and geometrically ergodic solution given by

xt=i=0ρ0iεti.

With ρ¯ some fixed value, consider testing the hypothesis H0 given by

H0:ρ=ρ¯.

The test is based on the likelihood ratio test statistic, τn, defined in terms of the Gaussian log-likelihood function

ln(θ)=n2(logσ2+n1t=1nεt2(ρ)/σ2),(2)

with x0 fixed and εt(ρ)=xtρxt1. Standard optimization gives the unrestricted maximum likelihood estimator (MLE), θ^n=(ρ^n,σ^n2), where

ρ^n=n1t=1nxtxt1(n1t=1nxt12)1andσ^n2=n1t=1nεt2(ρ^n).(3)

The restricted estimator θ˜n=(ρ˜n,σ˜n2) is obtained by maximization under H0, and is given by

ρ˜n=ρ¯andσ˜n2=n1t=1nεt2(ρ¯).

It follows that the likelihood ratio (LR) test statistic is given by

τn=LRn(ρ=ρ¯)=2(ln(θ^n)ln(θ˜n))=nlog(σ˜n2/σ^n2).(4)

Moreover, under H0 and regularity conditions detailed in the section “Asymptotics for the Autoregressive Model,” with ρ¯=ρ0 and θ0Θ0 it holds that τndχ12. Note in this respect that as long as the true ρ0 is not “ too large” the asymptotic χ12 approximation is a good approximation even for small samples. In contrast, and as exemplified in Table 1, if either εt is not i.i.d. N(0,σ2) or if ρ0=0.9 the χ12 distribution is not a good approximation of the actual distribution of τn for small, or even moderate, finite samples of size n.

Remark 1 (Wald and t statistics). Although focus is on the likelihood ratio test statistics, other statistics could be considered as well. For instance, the classic Wald statistic (Wn) is given by

Wn(ρ=ρ¯)=(ρ^nρ¯)2t=1nxt12/σ^n2

and, under the same regularity conditions required for the likelihood ratio test, when ρ¯=ρ0 and θ0Θ0, it holds that Wn(ρ=ρ¯)dχ12. It is also customary to consider the t-ratio statistic, defined as

tn(ρ=ρ¯)=ρ^nρ¯se(ρ^n)

where the so-called standard error se(ρ^n) is defined as (V^n/n)1/2, with

V^n=σ^n2(n1t=1nxt12)1

an unrestricted (i.e., obtained without imposing the null hypothesis) estimator of the asymptotic variance of n1/2(ρ^nρ0). Notice that tn(ρ=ρ¯)=sign(ρ^nρ¯)(Wn(ρ=ρ¯))1/2 and, when ρ¯=ρ0 and θ0Θ0, tn(ρ=ρ¯)dN(0,1).

Remark 2 (Quasi likelihood). Often if the εt sequence is assumed to be i.i.d., Gaussian is relaxed to i.i.d. (0,σ2) for some unknown distribution. When the Gaussian log-likelihood function in (2) in this case is used to obtain estimators and test statistics, these are referred to as (Gaussian) quasi MLE and quasi LR statistics, respectively. This is also the case when εt is assumed to be mean zero and (conditionally or unconditionally) heteroskedastic, as for example with some autoregressive conditional heteroskedastic (ARCH) specification or with a structural break in the variance (see the section “Finite Sample Behavior”).

Asymptotics for the Autoregressive Model

To ease the presentation of the key arguments for the standard nonbootstrap asymptotic analysis, assume here without loss of generality that σ2 is fixed at the true value—that is, σ2=σ02. Accordingly, the parameter is θ=ρ and the likelihood function in (2) simplifies as

ln(θ)=ln(ρ)=n2(logσ02+n1t=1nεt2(ρ)/σ02).

The MLE ρ^n is given by (3), while the LRn(ρ=ρ¯) statistic in (4) in this case is identical to the Wald statistic (Wn)

τn=Wn(ρ=ρ¯)=(ρ^nρ¯)2t=1nxt12/σ02.(5)

By construction, τn is a simple function of n1/2(ρ^nρ¯) and n1t=1nxt12, and the limiting distribution is found by applying a law of large numbers (LLN) to the average n1t=1nxt12, as well as a central limit theorem (CLT) to n1/2(ρ^nρ¯).

With ρ¯=ρ0Θ0, the AR process xt is stationary and ergodic, and has finite variance, V(xt)=ω0=σ02/(1ρ02). In particular, it follows by the LLN for stationary and ergodic processes that

n1t=1nxt12pV(xt)=ω0.

Next, by definition of the MLE,

n1/2(ρ^nρ0)=n1/2t=1nεtxt1(n1t=1nxt12)1.

Here mt=εtxt1 is a martingale difference sequence (mds) with respect to the filtration Ft, where Ft=σ(xt,xt1...), as E|mt|<, and E(mt|Ft1)=0. Moreover, the conditional second order moment converges in probability,

n1t=1nE(mt2|Ft1)=σ02n1t=1nxt12pσ02ω0.

This implies by the CLT for mds that n1/2t=1nmtdN(0,σ02ω0) (Hall & Heyde, 1980; Hamilton, 1994) and such that

n1/2(ρ^nρ0)dN(0,σ02/ω0)=dN(0,1ρ02).

Remark 3 (Lindeberg condition). Note that for a CLT for mds in general to hold (Hall & Heyde, 1980), a Lindeberg-type condition of the form γn=n1t=1nE(mt2I(|mt|>δn1/2))0 for any δ>0 must hold, where I() is the indicator function. This holds here as, by stationarity,

γn=n1t=1nE(mt2I(|mt|>δn1/2))=E(mt2I(|mt|>δn1/2)),

which tends to zero as n by dominated convergence under the moment condition E(mt2)< as implied by E(εt2)< and independence of εt and xt1.

Collecting terms, it follows that we have the following result.

Lemma 1. For the AR(1) model in (1) with θ0Θ0 and εt i.i.d. (0,σ02), it follows that with τn given by (5), under the null τndχ12 as n.

Note that in Lemma 1, it is not assumed that the εt sequence is Gaussian, but instead the lemma is formulated in terms of the milder sufficient condition that εt is an i.i.d. (0,σ2) sequence (Remark 2). Thus the essential regularity conditions for the lemma to hold are that |ρ0|<1 and εt i.i.d. (0,σ2).

Remark 4 (Heteroskedasticity). Note that if εt=σtzt, with zt i.i.d. N(0,1) and σt2 given by an ARCH process with E(σt2)=σε2<, it follows that τndcχ12, with c=σε2/σ02. This reflects in general that if εt are (conditionally, or unconditionally) heteroskedastic, the limiting distribution of the test statistic τn is not χ12. This is wellknown in the context of regression models, and corrections of the test statistic (White, 1980) are typically applied to ensure valid asymptotic inference; the bootstrap, or more precisely, the wild bootstrap discussed later, may correct for such misspecification without the need to correct the original test statistic (cf. Gonçalves & Kilian, 2004).

Remark 5 (Test statistic). For σ2 an unknown parameter to be estimated, such that θ=(σ2,ρ), note that the LR statistic in (4) can be written as

τn=LRn(ρ=ρ¯)=nlog(1n1Wn(ρ=ρ¯)σ02σ˜n2),

with σ˜n2=n1t=1nεt2(ρ¯). By Lemma 1, wn=n1Wn(ρ=ρ¯)p0, and by a Taylor expansion, log(1w)=w+o(w), with o(w) a term that tends to zero as w0. Hence,

τn=Wn(ρ=ρ¯)+op(1),

where op(1) denotes a term that converges to zero in probability as n (see, e.g., van der Vaart, 2000, Lemma 2.12, for details on op() (and Op() notation and the stochastic [Taylor] expansion as applied here).

Bootstrap in the Autoregressive Model

The general idea behind a bootstrap algorithm as implemented in the context of likelihood-based testing in the first order AR model can be summarized as follows:

Step A. With the original data {xt}t=0n fixed, generate a sample of bootstrap data {xt}t=0n using some bootstrap scheme with bootstrap true parameter, θn=(ρn,σn2). Specifically, for the AR model, set x0=x0 and generate xt recursively by

xt=ρnzt+εt,

with zt=xt1 for a recursive bootstrap scheme, while zt=xt1 for a fixed design bootstrap scheme. As to the choice of bootstrap innovations εt in terms of the original data {xt}t=0n and a bootstrap sampling distribution as detailed after step C.

Step B. Compute the bootstrap (quasi) MLEρ^n and the bootstrap LR statistic τn=LRn(ρ=ρn), where the bootstrap log-likelihood function ln(θ) as a function of θ is given by

ln(θ)=n2(logσ2+n1t=1nεt2(ρ)/σ2),withεt(ρ)=xtρzt.

Step C. Generate {xt,b}t=0n, θ^n,b and τn,b for b=1,2,...,B by repeating Steps A and B, and use the empirical distribution of {τn,b}b=1B for testing based on the original statistic τn=LRn(ρ=ρ0). Precisely, with the bootstrap p-value set to pn,B=B1b=1BI(τnτn,b), H0 is rejected when pn,B<α, with α the nominal level. (For the choice of bootstrap repetitions B, see Remark 9).

As to the definition in step A of the bootstrap innovations εt, usually these are obtained by i.i.d. draws with replacement from re-centered estimated model residuals; henceforth referred to as iid bootstrap, or iid resampling. The model residuals can be either estimated under the null (H0 imposed) or without imposing the null. That is, the εt are in the first case resampled from centered residuals, {ε˜tc}t=1n, where

ε˜tc=εt(ρ¯)n1t=1nεt(ρ¯),withεt(ρ¯)=xtρ¯xt1.(8)

Using unrestricted residuals, the bootstrap εt innovations are resampled from {ε^tc}t=1n, where

ε^tc=εt(ρ^n)n1t=1nεt(ρ^n),withεt(ρ^n)=xtρ^nxt1.(9)

Centering is required because by doing so both of the residual series ε^tc and ε˜tc have empirical mean zero, n1t=1nε^tc=0 and n1t=1nε˜tc=0, and ideally “ mimic” the true εt. In particular, under H0, εt(ρ¯)=εt(ρ0)=εt, and, moreover, ρ^npρ0. In contrast, under the alternative, when H0 does not hold, while εt(ρ¯)εt, it still holds that ρ^npρ0. Hence, while one may expect that the unrestricted residuals ε^tc perform better under the alternative, at least asymptotically, in practice little difference is found between the two.

An alternative to iid resampling is the so-called wild bootstrap, where either εt=ε˜tcwt, or εt=ε^tcwt, with wt an auxiliary i.i.d. sequence, independent of the original data, with E(wt)=0 and V(wt)=1. A simple example is wt i.i.d. N(0,1) (see Remark 10 for alternative specifications). The wild bootstrap is typically motivated by its potential ability to allow for possible model misspecification in the sense that it allows for (conditional and unconditional) heteroskedasticity in the innovations εt. To see this (e.g., εt=ε˜tcwt), it follows that (conditionally on the data {xt}t=0n), the εt are i.i.d. distributed with mean zero and with time-varying variance,

V(εt|{xt}t=0n)=(ε˜tc)2.

This way, the variation in εt from the wild bootstrap “in essence reflects the heteroskedasticity of the original data” (Liu, 1988, p. 1704) as is also illustrated in the Monte Carlo simulations in the section on “Finite Sample Behavior.” This differs from the iid bootstrap, where the bootstrap innovations εt (conditionally on the data) are i.i.d., with a discrete distribution given by P(εt=ε˜jc|{xt}t=0n)=n1, for j=1,2,...,n. Specifically, as mentioned earlier, the (empirical, or conditional) mean is zero, E(εt|{xt}t=0n)=0, and the variance is given by

V(εt|{xt}t=0n)=n1t=1n(ε˜tc)2.

That is, for fixed n, the variance for the iid bootstrap innovations εt is constant (and equal to the empirical variance of the estimated residuals), while depending on time t for the wild bootstrap.

In short, the bootstrap scheme in (6), xt=ρnzt+εt, with zt=xt1 or zt=xt1, depends on two types of randomness: (a) the variation of the original data {xt}t=0n; and (b) the bootstrap resampling for {εt}t=1n (wild or iid resampling). This is important for the application of the LLN and the CLT to establish bootstrap validity as demonstrated in the next.

Some further remarks are in order.

Remark 6 (t statistics and bootstrap standard errors). When interest is in bootstrapping the t ratio statistic (or Wald-type statistics), then steps AC can be used with τn, now defined as (taking the case of unknown σ2 to illustrate)

τn=tn(ρ=ρ¯)=ρ^nρ¯se(ρ^n)

where the so-called bootstrap standard error se(ρ^n), the bootstrap analog of se(ρ^n), given by

se(ρ^n)=(n1V^n)1/2

with Vn being an estimator of the variance of ρ^nρ0, such as

V^n=σ^n2(n1t=1nxt12)1

where σ^n2 is the sample variance of the bootstrap residuals, σ^n2=n1t=1nεt(ρ^n), εt(ρ^n)=xtρ^nxt1. It is worth noticing that the bootstrap standard error could also be computed without exploiting the autoregressive structure of the model; this can be done by simply resorting to the B bootstrap realizations of ρ^n, say ρ^n,b, b=1,...,B. Specifically, one could consider the following bootstrap standard error

seB(ρ^n)=(1B1b=1B(ρ^n,bρ^n,b¯)2)1/2,ρ^n,b¯=1Bb=1Bρ^n,b,

which is straightforward to compute once the B realizations of ρ^n have been obtained.

Remark 7 ([Un-]restricted bootstrap). As to the choice in step A of the bootstrap true value θn=(ρn,σn2) (or simply, ρn for the AR model), one may set θn to the value of the unrestricted estimator θ^n, θn=θ^n, which is referred to as unrestricted bootstrap. If θn=θ˜n, the restricted estimator, the bootstrap is referred to as restricted. It should be emphasized that for the unrestricted bootstrap, the bootstrap likelihood ratio statistic τn is derived for the hypothesis ρ=ρ^n, while the original hypothesis H0:ρ=ρ¯ is considered for the restricted bootstrap. While both choices are widely applied in existing literature, the restricted bootstrap in the context of testing is more popular in econometrics (see, e.g., Davidson & MacKinnon, 2000).

Remark 8 (Recursive and fixed design bootstrap). In step B of the algorithm in equation (6), with zt=xt1, this is an example of a recursive bootstrap. That is, the original autoregressive structure for xt is replicated for the bootstrap process xt. However, with zt=xt1, the original data xt1 are used as lagged value of xt, such that xt is not an autoregressive process even conditionally on the data. The fixed design bootstrap typically simplifies (some of) the asymptotic arguments, and is often found to behave as well as the recursive bootstrap (see, e.g., Gonçalves & Kilian, 2004, 2007, for general AR models, and Cavaliere, Pedersen, & Rahbek, 2018, for ARCH models).

Remark 9 (Bootstrap p-value). In step C, the bootstrap p-value pn,B is defined as pn,B=B1b=1BI(τnτn,b), with B the number of bootstrap repetitions (see also Remark 14). Typical choices are B=199, 399, or 999 (see also Andrews & Buchinsky, 2000, and Davidson & MacKinnon, 2000, for details on the choice of B).

Remark 10 (Choice of wt*). With respect to the choice of distribution of the i.i.d. sequence wt for the wild bootstrap, Liu (1988) provides a detailed discussion of various choices based on so-called Edgeworth expansions (see, e.g., Hall, 1992; van der Vaart, 2000, for an introduction) of test statistics similar to τn. In particular, Liu (1988), with ξk=E(wtk),k1, emphasizes ξ1=0, ξ2=1 as well as ξ3=1 as important. For the case of ξ3=1, emphasis is on possible skewness, while ξ3=0 works well in the case of symmetry. In applications, standard choices for wt, all with ξ1=0 and ξ2=1, include the Gaussian, Rademacher, and Mammen distributions.1 It follows that ξ3=0 for the first two (with ξ4=3 and 1, respectively), while ξ3=1 (and ξ4=2) for the Mammen distribution.

Remark 11 (Parametric bootstrap). In step A of the bootstrap scheme, one may also use a so-called parametric bootstrap, where the bootstrap innovations εt are generated as i.i.d. N(0,σn2). While this parametric bootstrap performs well in the case where the true innovations εt are Gaussian, this may not be the case when the distribution of εt is non-Gaussian (Horowitz, 2001).

Asymptotic Theory for the Recursive Bootstrap

In order to discuss regularity conditions under which bootstrap-based testing holds, consider here the details of verification of the asymptotic validity of the recursive bootstrap for the AR model.

Thus, with the AR model given in (1), consider here the recursive restricted bootstrap scheme as defined by setting zt=xt1 and ρn=ρ¯ in step A of the bootstrap algorithm. In short, the xt bootstrap sequence is here generated as

xt=ρ¯xt1+εt,(10)

with x0=x0 and ρ¯=ρ as the bootstrap true value. Moreover, we consider the classic case of iid resampling from the autoregressive residuals obtained under H0, that is, from {ε˜tc}t=1n defined in (8). The statistic of interest is τn=Wn(ρ=ρ¯) in (5), which is computed using the bootstrap sample {xt}t=0n as

τn=Wn(ρ=ρ¯)=(ρ^nρ¯)2t=1n(xt1)2/σ02andρ^n=t=1nxtxt1(t=1nxt12)1.(11)

While xt clearly has some features similar to xt, one cannot apply standard concepts such as stationarity and ergodicity when analyzing the asymptotic behavior of τn, due to two types of randomness: the bootstrap resampling distribution and the distribution of the original data, {xt}t=0n. Introduce therefore the bootstrap equivalent concepts of convergence in probability and distribution, which reflect the fact that inference is based on conditioning on the original data that are themselves random.

Bootstrap Probability, Expectation, and Convergence

With P() denoting the bootstrap probability—that is, the probability conditional on the data—the iid bootstrap innovations εt are by definition i.i.d. distributed with

P(εt=ε˜jc)=P(εt=ε˜jc|{xt}t=0n)=n1forj=1,2,...,n.

Similarly, the expectation E() is defined by E()=E(|{xt}t=0n). As an example, consider the expectation of εtconditionally on the data. It follows that, as already discussed, E(εt)=0 as

E(εt)=j=1nP(εt=ε˜jc)ε˜jc=n1j=1nε˜jc=0,

by the definition in (8). Next, consider the variance of εt conditionally on the data, V(εt). Again, by definition, V(εt)=E(εt2)(E(εt))2 and hence as E(εt)=0, it follows that

V(εt)=j=1nP(εt=ε˜jc)(ε˜jc)2=n1j=1n(ε˜jc)2.

That is, the variance conditional on the data is equal to the sample variance of the original estimated residuals under H0. In particular, V(εt) is a random variable (in terms of the original probability measure) and moreover, by the LLN for i.i.d. variables, under H0 with ρ¯=ρ0,

V(εt)=E(εt2)pV(εt)=σ02.(12)

Note that for the wild bootstrap, V(εt)=V(ε˜tcwt)=(ε˜tc)2, emphasizing that the wild bootstrap indeed “mimics” heteroskedasticity and the iid bootstrap does not.

Similar to the definition of P() and E() this motivates the definition of the bootstrap equivalent of convergence in probability, denoted “pp.” Formally, a sequence of stochastic variables Xn is said to converge in probability conditional on the data (or, to converge in P-probability, in probability) to c (possibly random), if P(|Xnc|>δ) converge in probability to zero. This can be stated as

Xncp*p0ifP(|Xnc|>δ)p0foranyδ>0.

As an example, it follows that Xn=n1t=1nεtpp0 as by the bootstrap equivalent of Markov’s inequality2 one has,

P(|Xn|>δ)E(Xn)2/δ2.

By definition,

E(Xn)2=n2E(t=1nεt2+2t=1ns=t+1nεtεs),

with E(εt2)=V(εt), and for st,

E(εsεt)=E(εs)E(εt)=0.

Hence, since E(Xn)2=n1V(εt), we conclude that

Xn=n1t=1nεtpp0.(13)

A key ingredient in the asymptotic analysis of the nonbootstrap AR model is the CLT, and we need a bootstrap equivalent of the CLT and a bootstrap equivalent of convergence in distribution. By definition, XndX if FXn(x)=P(Xnx)FX(x)=P(Xx) at all continuity points of FX(). Likewise, Xn converge in distribution to X conditional on the data (or, as sometimes used, Xn converges “weakly in probability”); that is, XndpX, if the bootstrap cumulative distribution function converges in probability. Specifically, XndpX if

FXn(x)=P(Xnx)pFX(x),

at all continuity points of FX(). Alternatively, weak convergence in probability may be defined in terms of convergence in probability of the bootstrap characteristic function as in (15).

The next lemma illustrates that, as one might expect, the sum of bootstrap innovations εt is asymptotically Gaussian (in probability).

Lemma 2 (van der Vaart, 2000, Theorem 23.4). With Xn=n1/2t=1nεt, then

XndpX=dN(0,σ02).(14)

The proof, as for most bootstrap CLTs, is based on applying a CLT for triangular arrays, as {εt}t=1n are sampled from ε˜tc, which depends on n.

To give an idea of the underlying theory, consider here verifying (14) using a classic approach based on the characteristic function (Durret, 2019, proof of Theorem 3.4.10). For a random variable X, the characteristic function defines uniquely the distribution of X and is defined by ϕ(s)=E(exp(isX)). Here, i is the complex (unit imaginary) number that satisfies i2=1 and s, and for X=dN(0,σ02) it holds that ϕ(s)=exp(s22σ02).

With the bootstrap characteristic function of Xn defined by ϕn(s)=E(exp(isXn)), it follows that (14) holds if

ϕn(s)pexp(s22σ02)=ϕ(s).(15)

Note first, as εt are i.i.d. conditionally on the data,

E(exp(isXn))=E(t=1nexp(isXn))=t=1nE(exp(isn1/2εt))=(E(exp(isn1/2εt)))n.

Next, a Taylor expansion of exp() at s=0 gives

E(exp(isn1/2εt))=1+isn1/2E(εt)12s2n1E(εt2)+op(n1)=1n1(12s2σ02)+op(n1),

using E(εt)=0 and E(εt2)pσ02 (see [12]; Durret, 2019, Lemma 3.3.19). It therefore follows as desired that

E(exp(isXn))=(1n1(12s2σ02)+op(n1))npϕ(s),

as for any sequence cn, with cnpc as n, then similar to Durret (2019, Theorem 3.4.2), (1n1cn)npexp(c).

Remark 12 (Lindeberg condition). The CLTs in Durret (2019, Theorem 3.4.10) and van der Vaart (2000, Theorem 23.4) for triangular arrays follow by verifying

E(Xn2)=n1t=1nE(εt2)pσ02,

in addition to the bootstrap Lindeberg condition,

γn=n1t=1n*E(εt2I(|εt|>δn1/2))=n1t=1n(ε˜tc)2I(|ε˜tc|>δn1/2)p0.

A simple way to see that γnp0 is for example to note that if (the rather strong moment condition) E(εt4)< holds, the LLN applies to n1t=1n(ε˜tc)4 and hence,

γn=n1t=1n(ε˜tc)2I(|ε˜tc|>δn1/2)1nδ2n1t=1n(ε˜tc)4p0.

Bootstrap Validity Under H0

As briefly mentioned in the introduction, it is important for the application of the bootstrap that the limiting distribution (in probability) of the bootstrap test statistic has the same limiting distribution as the original test statistic when the null is true. Stated differently, we wish here to establish that under H0 with ρ¯=ρ0,

τn=Wn(ρ=ρ0)=(ρ^nρ0)2t=1nxt12/σ02dpχ12.

By definition, the bootstrap estimator ρ^n is given by

ρ^n=n1t=1nxtxt1(n1t=1nxt12)1,

such that by the bootstrap scheme employed, that is xt=ρ0xt1+εt, it follows that

n1/2(ρ^nρ0)=n1/2t=1nεtxt1(i)(n1t=1nxt12)1(ii).

Here a bootstrap CLT should be used for the first term (Should be (i) as refers to equation just above!)

a), and a bootstrap LLN for the second term (b) (Should be (ii) as refers to equation) to find the limiting behavior of the bootstrap estimator and, hence, of the test statistic τn. Consider first (b), which is an average of lagged xt squared, with

xt=ρ0xt1+εt=i=0t1ρ0iεti+ρ0tx0.(16)

As εt depends on n, and the data {xt}t=0n, the concepts of stationarity and ergodicity—while applying to xt—do not apply to xt. However, the following lemma holds, which establishes that the LLN holds for the average of xt and xt2.

Lemma 3. Suppose that {xt}t=0n is given by (1) with |ρ0|<1 and εt i.i.d. (0,σ02). Assume furthermore, with εt iid sampled with replacement from {ε˜tc}t=1n and xt given by (10). Then, as n,

n1t=1nxt1pp0andn1t=1nxt12ppω0=σ02(1ρ02)1.

The proof of Lemma 3 is given in the appendix. Note that for the case of ρ0=0 the arguments are similar to the arguments used to establish n1t=1nεtpp0.

Remark 13 (LLN triangular arrays). Lemma 3 is the bootstrap equivalent of the weak law of large numbers for triangular arrays (see also Durret, 2019, Theorem 2.2.6).

Next, consider the CLT candidate term (b),

n1/2t=1nεtxt1.

As for the nonbootstrap case, with Ft=σ(xt,xt1,...,x0), then (conditionally on the data) a bootstrap CLT for martingale difference arrays (mda) can be applied. In particular, E(εtxt1|Ft1)=xt1E(εt)=0, while for the conditional second order moment (conditional on the data), it follows by application of Lemma 3 that

n1t=1nE((εtxt1)2|Ft1)=n1t=1nxt12E(εt2)ppσ02ω0.

It remains to establish the bootstrap Lindeberg condition, γnpp0, where

γn=n1t=1nE((εtxt1)2I(|εtxt1|>δn)|Ft1).(17)

Similar to Remark 12, this follows by using that for some (arbitrarily small) η>0,

γn1n1+η/2δηt=1nE(|εtxt1|2+η|Ft1)=1nη/2δηn1t=1n|xt1|2+ηE|εt|2+ηpp0,

which holds provided E|εt|2+η<, using arguments as in Lemma 3. (Note that in Remark 12 the same argument is used for η=2.)

Hence, with εt iid resampled from {ε˜tc}t=1n, εt i.i.d. (0,σ02) with E|εt|2+η<, it follows that the limiting distribution (in probability) of the bootstrap MLE is given by

n1/2(ρ^nρ0)dpN(0,σ02ω01).(18)

This immediately leads to the desired result:

Theorem 1. Under H0 with |ρ0|<1, and with εt iid resampled from {ε˜tc}t=1n, with εt being i.i.d. (0,σ02) with E|εt|2+η< for some η>0, it holds that

τn=Wn(ρ=ρ0)dpχ12.(19)

Remark 14 (Bootstrap p-value). The bootstrap p-value pn,B in step C of the bootstrap algorithm is an approximation to the “true” bootstrap p-value pn, where pn=P(τn>τn), in the sense that pn,Bppn as B tends to infinity. (For details in terms of the stronger concept of “almost sure” convergence, see, e.g., Cavaliere, Nielsen, & Rahbek, 2015, Remark 2.)

Remark 15 (Validity under H0). Note that by Cavaliere et al. (2015, Corollary 1) for the bootstrap p-value pn (see Remark 14), it follows that as the limiting χ12 distribution has a continuous distribution function, under the conditions of Theorem 1, pndU, with U uniformly distributed on [0,1] (Hansen, 1996, 2000).

Remark 16 (Moments of εt). Note that E|εt|2+η< for some η, or E(εt4)< as is often used, is required. This reflects that, as typically found for the bootstrap, further moment restrictions are used to prove bootstrap validity than in the nonbootstrap case. This is because complexities arise when applying bootstrap LLNs and arguments in connection to establishing bootstrap Lindeberg-type conditions (Cavaliere & Rahbek, 2020). However, notably, while the higher order moment conditions are sufficient for the mathematical arguments, their necessity is often not reflected in bootstrap simulations (see “Finite Sample Behavior”).

Remark 17 (Introducing σ2). As for the nonbootstrap case, the result in Theorem 1 also holds for the case where σ2 is treated as a parameter.

Remark 18 (Consistency of bootstrap standard errors). It is important to notice that the fact that n1/2(ρ^nρ0) converges in conditional distribution (in probability) to the Gaussian distribution does not imply that the bootstrap standard error is consistent. Intuitively, this happens because convergence in distribution alone does not imply convergence of moments. Hence, the econometrician must evaluate on a case-by-case basis whether the bootstrap standard errors are consistent. An interesting result is provided by Hahn and Liao (2019), who report that the bootstrap standard error computed by simulation using B bootstrap repetitions does not underestimate the population standard error (see Remark 6), and hence bootstrap inference based on the t statistic coupled with the numerically computed standard errors is conservative (i.e., the type-one error associated to the bootstrap test does not exceed the user-chosen significance level in large samples).

Bootstrap Validity Under the Alternative

Consider here the convergence of the bootstrap statistic τn in (11) when the alternative holds. That is, assume here that the original data are generated with true value θ0=(ρ0,σ02), but the hypothesis tested is as before H0:ρ=ρ¯ with ρ¯ρ0. As argued below, Theorem 1 holds under the alternative as well, such that

τndpχ12.(20)

For the application of bootstrap-based testing, this implies that under the alternative, as Wn(ρ=ρ¯) diverges while Wn(ρ=ρ¯) converges in distribution, the bootstrap-based test will reject with probability tending to one. That is, asymptotic bootstrap validity holds since by Cavaliere et al. (2015, Corollary 1) the bootstrap p-value pn, defined in Remark 15, tends to zero in probability under the alternative, pnp0.

A key argument for (20) to hold is to note that the identity,

εt=xtρ¯xt1,

holds independently of whether ρ¯ is the data true value ρ0 or not. That is, ρ¯ is by construction the bootstrap true value, such that under the null and also under the alternative, the bootstrap estimator can be rewritten as

n1/2(ρ^nρ¯)=n1/2t=1nεtxt1(n1t=1nxt12)1.

What differs is that εt under the alternative is resampled from recentered residuals ε˜tc, with

ε˜t=εt(ρ¯)=xtρ¯xt1=εt+(ρ¯ρ0)xt1εt.

That is, while the identity ε˜t=εt holds under the null hypothesis when ρ¯=ρ0, this is not the case under the alternative. Hence, to establish (20), a repeated application of the bootstrap LLN (applied to t=1nxt12) and CLT (applied to t=1nεtxt1) is needed under the alternative. For the AR process of order one considered here, the arguments are based on simple modifications of the theory under H0.

Remark 19 (Theory for the wild bootstrap). The same results can be shown to apply for the wild bootstrap in the case of conditional heteroskedasticity (Gonçalves & Kilian, 2004, 2007).

Finite Sample Behavior

Throughout, the focus has been on establishing asymptotic validity. This was done by verifying that the bootstrap statistic τn has the same limiting distribution (in probability) as the original statistic τn under the null hypothesis. Moreover, the same was argued to hold under the alternative. To illustrate the finite sample performance of the iid and wild bootstraps for the AR model, this section highlights some selected typical findings for the bootstrap based on a small and simple (to replicate) Monte Carlo study. Thus the Monte Carlo study here is not meant to be elaborate; exhaustive and detailed bootstrap Monte Carlo-based investigations are given in several papers (see, e.g., Gonçalves & Kilian, 2004, with special attention to higher order AR models, as well as the references in Cavaliere & Rahbek, 2020).

The Monte Carlo results reported here highlight the importance of the assumptions for the established validity of the bootstrap-based test of H0:ρ=ρ¯ in the autoregressive model. Specifically, it was argued that the true value of the autoregressive root ρ0 for xt should satisfy |ρ0|<1, and it was emphasized that εt is an i.i.d. (0,σ02) sequence, such that E|εt|2+η<, or rather, E(εt4)<.

With details of the Monte Carlo designs and consideration given below, we initially mention the following findings for bootstrap simulations in the AR model with a constant term. The findings are typical for existing applications of the bootstrap and are standard in time series contexts.

(1)

With εt i.i.d. N(0,σ02) the χ12-based asymptotic test performs well for even small samples of size n in terms of empirical rejection frequencies, or empirical size, for ρ0=0.5, while for ρ0=0.9 the asymptotic test fails as its empirical size is not close to the nominal level α. In comparison, the iid (and wild) bootstrap-based test has empirical size close to the nominal level in both cases, see Table 1 columns A and B for ρ0{0.5,0.9}, σ02=1, and n{15,25,...,1000} with α=0.05.

(2)

With εt independently N(0,σt2) distributed with σt2 time-varying (heteroskedasticity), neither the χ12-based asymptotic nor the iid bootstrap-based tests have empirical size close to the nominal level α, which contrasts with the wild bootstrap-based test. This is illustrated in Table 1 columns C–F for a time-changed volatility, σt2=σ0,12+σ0,22I(tn/2]), with σ0,12=1<σ0,22=15, and as before ρ0={0.5,0.9} and n{15,25,...,1000} with α=0.05.

(3)

In terms of empirical rejection frequencies under the alternative, bootstrap-based tests and the asymptotic test are comparable. This is illustrated in Table 1, with εt i.i.d. N(0,σ02) for n=250 and ρ0=0.9, and the test of H0:ρ=ρ¯ is considered for values of ρ¯ ranging from 0.70 to 0.875 (with ρ¯=ρ0 included as a benchmark).

(4)

The wild bootstrap is often motivated by its ability to replicate underlying heteroskedasticity (see the section “Bootstrap in the Autoregressive Model” and Remark 10). This is illustrated in Figure 1, where panel A shows the empirical residuals ε˜t from one of the draws in Table 1 with σt2=σ0,12+σ0,22I(tn/2]). Figure 1, in panels B, C, and D, illustrates that, while εt replicates the heteroskedasticity for the wild bootstrap, this is not the case for the iid bootstrap.

(5)

In terms of requirements for finite moments of the i.i.d. sequence εt, the section “Bootstrap Validity Under H0” discussed sufficiency and necessity of the condition E|εt|2+η< for some η>0, and it was conjectured that E(εt2)< was sufficient. This is illustrated in Table 2, which shows that when εt does not have a finite variance, then the asymptotic test, as well as the wild and iid bootstrap-based tests, fail to have correct empirical size. However, when εt has a finite variance, while the asymptotic test has empirical size far from the nominal, the bootstraps work despite the lack of fourth order moments, for example. Also, Table 2 shows results from a so-called permutation bootstrap (see the section “Moment Condition, E|εt|k< for k2”).

Asymptotic Test

We consider xt as given by the AR model of order one, with a constant term δ included,

xt=δ+ρxt1+εt,t=1,2,...,n,(21)

with εt i.i.d. N(0,σ2) and x0 fixed. The parameters are given by θ=(δ,ρ,σ2)Θ=2×(0,), with θ0=(δ0,ρ0,σ02) the true value, where θ0Θ0={θΘ||ρ0|<1} and the hypothesis of interest is given by H0:ρ=ρ¯.

The unrestricted and restricted (Gaussian likelihood-based) estimators that maximize

ln(θ)=n2(logσ2+n1t=1nεt2(ρ,δ)/σ2),

with x0 fixed and εt(ρ,δ)=xtρxt1δ, are given by

θ^n=(δ^n,ρ^n,σ^n2)andθ˜n=(δ˜n,ρ0,σ˜n2),

respectively. The theory from the case of no constant term immediately carries over, such that for ρ¯=ρ0, |ρ0|<1 and εt i.i.d. (0,σ02), as n,

τn=2(ln(θ^n)ln(θ˜n))dχ12.

In the implementations of the asymptotic test, we use the p-value, pn, calculated as the tail probability of τn in the limiting χ12 distribution, and reject if pn is smaller than the nominal level α.

Column A in Table 1 illustrates this for xt in (21) generated with δ0=0,ρ0={0.5,0.9}, and σ02=1. Moreover, εt is simulated as an i.i.d. N(0,σ02) sequence, and x0=0. The empirical rejection frequencies are reported based on N=10,000 repetitions, with nominal level α=0.05. Results for n{15,25,...,1000} are given in column A of Table 1. Observe, as noted, that quite a large sample is required for the limiting χ12 distribution to be a good approximation, in particular with ρ0=0.9 (Duffee & Stanton, 2008, and references therein).

Remark 20 (Empirical rejection probabilities). At the chosen (nominal) level α, with qα the corresponding 1α quantile of the limiting distribution, the true rejection probability at sample length n is αn=P(τn>qα). The Monte Carlo estimator is the empirical rejection frequency computed as αn,N=1Ni=1NI(pn,i<α), where pn,i is the p-value in Monte Carlo replication i, i=1,2,...,N. It follows that the simulation uncertainty of αn,N is given by

V(αn,N)=αn(1αn)/N

(Hendry, 1984), and for a correctly sized test, with αn=0.05 and N=104, the 95% confidence bound for αn,N is [0.0456,0.0544]. Similar considerations hold for the bootstrap simulations of the test, with pn,i replaced by pn,B,i (see also Remark 14).

Table 1. Empirical Rejection Frequencies of Asymptotic and Bootstrap Tests in the Homoskedastic and Heteroskedastic Case

Homoskedastic Case

Heteroskedastic Case

(A)

(B)

(C)

(D)

(E)

(F)

n

Asymp.

iid boot.

Asymp.

iid boot.

Wild boot.

Wild boot.

N(0,1)

Rademacher

ρ0=ρ¯=0.5

15

0.0825

0.0512

0.1319

0.1015

0.0720

0.0462

25

0.0733

0.0541

0.1329

0.1121

0.0685

0.0496

50

0.0623

0.0519

0.1370

0.1265

0.0625

0.0486

100

0.0538

0.0511

0.1425

0.1383

0.0586

0.0488

250

0.0521

0.0508

0.1428

0.1419

0.0593

0.0557

500

0.0498

0.0478

0.1398

0.1375

0.0526

0.0491

1000

0.0495

0.0493

0.1434

0.1463

0.0515

0.0502

ρ0=ρ¯=0.9

15

0.1971

0.0558

0.1797

0.0597

0.0688

0.0534

25

0.1651

0.0552

0.1727

0.0688

0.0670

0.0563

50

0.1218

0.0526

0.1603

0.0875

0.0628

0.0530

100

0.0866

0.0507

0.1518

0.1016

0.0565

0.0501

250

0.0664

0.0511

0.1444

0.1226

0.0538

0.0494

500

0.0559

0.0480

0.1452

0.1323

0.0500

0.0474

1000

0.0548

0.0516

0.1463

0.1376

0.0515

0.0503

Note. The data generating process is given by (21) with ρ0{0.5,0.9} and δ0=0 and the bootstrap process defined in (22). In panels (A) and (B), the innovations εt are i.i.d.N(0,σ02) distributed with σ02=1, while in panels (C)–(F) εt are independently N(0,σt2) distributed, with σt2 given in (25). The number of bootstrap replications is B=399 and N=10,000 replications.

The iid Bootstrap Test

To illustrate bootstrap-based testing, we apply a restricted recursive bootstrap in terms of residuals estimated under H0 (see “Bootstrap Validity Under H0”).

Specifically, for steps A and B in the section “Bootstrap in the Autoregressive Model,” the bootstrap samples {xt}t=0n are sampled from

xt=δ˜n+ρ¯xt1+εt,t=1,2,...,n,(22)

with x0=x0 and εt drawn with replacement (for wild, see the section “The Wild bootstrap test” below) from {ε˜tc}t=1n, where ε˜tc are defined as in3 (8) in terms of

ε˜t=εt(ρ¯,δ˜n)=xtρ¯xt1δ˜n.(23)

For the bootstrap sample, {xt}t=0n, we estimate the unrestricted and restricted models and calculate the bootstrap statistic τn, given by

τn=2(ln(θ^n)ln(θ˜n)).

Here, θ^n=(δ^n,ρ^n,σ^n2) and θ˜n=(δ˜n,ρ¯,σ˜n2) denote the unrestricted and restricted bootstrap estimators, respectively, in terms of the bootstrap log-likelihood function

ln(θ)=n2(logσ2+n1t=1nεt2(ρ,δ)/σ2),εt(ρ,δ)=xtρxt1δ.

For step C, the bootstrap test is based on replicating just given arguments to obtain {τn,b}b=1B with B denoting the number of bootstrap repetitions. As discussed in Remark 9, the empirical bootstrap p-value is computed as the tail probability,

pn,B=1Bb=1BI(τn,bτn).(24)

With B=399 bootstrap repetitions, the empirical rejection frequencies for the N=10,000 Monte Carlo repetitions are presented in column B in Table 1.

The Wild Bootstrap Test

The wild bootstrap design is as given in the section “iid Bootstrap Test,” except that εt for the wild bootstrap is resampled by

εt=wtε˜t,

with wt i.i.d. (0,1) distributed and ε˜t defined in (23). In the simulations, wt is chosen as *N(0,1) and Rademacher distributed, respectively (Remark 10).

For the simulations reported in Table 1, εt are assumed not to be i.i.d. N(0,σ02) distributed in order to illustrate the impacts of heteroskedasticity. Specifically, we set εt=dN(0,σt2) with σt2=1 for t=1,2,...,[n/2] and σt2=15 for t=[n/2]+1,...,n. That is,

σt2=σ0,12+σ0,22I(t>[n/2]),(25)

with σ0,12=1 and σ0,22=15. In this case the asymptotic test is not consistent as demonstrated in column A of Table 1, which reports the empirical rejection probabilities for the asymptotic test. The asymptotic test is severely oversized—and even for n=500 the empirical size is not close to α=0.05. Also the iid bootstrap is not asymptotically valid. With the iid bootstrap, as in the section “iid Bootstrap Test,” the results are reported in column D, and we observe that the results are similar to the asymptotic test. Intuitively, this reflects that the bootstrap series, {xt}t=0n, does not mimic the properties of the original data series {xt}t=0n. This can be illustrated by Figure 1, where panel A shows pronounced heteroskedasticity of the estimated residuals, ε˜t, for one sample. Panel B shows a single iid resampled sample {εt}t=1n from {ε˜t}t=1n, and as, by definition, in particular the ordering change, the εt series does not mimic the heteroskedasticity as seen in the estimated residuals in panel A.

For the wild bootstrap, columns E and F in Table 1 report the empirical rejection frequencies for the wild bootstrap test with, as mentioned, wt distributed as N(0,1) and Rademacher, respectively. The empirical size for the wild bootstrap is quite close to the nominal level, with the Rademacher distribution performing slightly better. Likewise, panels C and D in Figure 1 illustrate that the wild bootstrap εt series more closely mimisc the properties of the original ε˜t series.

Figure 1. Residuals from a typical simulation with sample length n=500. Panel (A) shows a sample of restricted residuals ε˜tc, while panels (B)–(D) show corresponding bootstrap innovations {εt}t=1n for the iid bootstrap and the wild bootstrap, with the auxiliary wt Gaussian and Rademacher distributed, respectively.

Bootstrap Under the Alternative

To establish asymptotic validity, it was shown that τn is also asymptotically χ2-distributed under the alternative, leading to a consistent bootstrap test. To illustrate this, we consider the empirical probability of rejecting a false hypothesis.

Specifically for the iid bootstrap test in the section “iid Bootstrap Test,” let the data xt be generated with true value θ0=(ρ0,σ02,δ0) as before. The hypothesis of interest is H0:ρ=ρ¯ and we let ρ¯{0.875,...,0.70} to illustrate the empirical power of the bootstrap test and asymptotic test. From Table 2 it follows that the bootstrap is comparable to the asymptotic test in terms of empirical power.

Table 2. Empirical Rejection Frequencies of Asymptotic and Bootstrap Tests Under the Alternative

ρ¯

Asymptotic

iid bootstrap

0.9

0.0651

0.0485

0.875

0.0980

0.0810

0.85

0.2911

0.2658

0.825

0.5533

0.5283

0.8

0.7730

0.7567

0.75

0.9617

0.9578

0.7

0.9958

0.9954

Note. The data-generating process is given by (21) with n=250 and ρ0=0.9, such that ρ0ρ¯ except for the first row entry. The number of bootstrap replications is B=399 and N=10,000 replications.

Moment Condition, E|εt|k< for k2

When establishing asymptotic validity of the bootstrap the moment condition, E(εt4)< was discussed, and it was mentioned that while it is a sufficient condition, it may not be necessary. On the other hand, E(εt2)< seems necessary unless the bootstrap algorithm is based on permutation, see also Table 3 and the text before this.

To illustrate this, we consider the data xt as generated with true value θ0=(ρ0,σ02,δ0) as before for samples of size n, n{15,...,1000}. The i.i.d. innovations εt are simulated from the Student’s tv-distribution, where the degrees of freedom v, v{3/2,3,5}. Specifically, for v=3/2, E|εt|k is finite, only for k<3/2, thus allowing first order, but not second order, moments of εt. For v=3 and v=5, the second and finite fourth-order moments are finite, respectively. Also note that the Gaussian case is included as a reference. Table 3 shows that the asymptotic test (based on the χ12 approximation) for all the four cases has empirical rejection rates far from the nominal level of α=0.05, even for large samples n, where n{15,...,1000}.

As to the bootstrap design, Table 3 reports bootstrap simulations based on iid sampling and the wild (Rademacher). For v=3/2, as expected, neither the wild nor the iid bootstraps have empirical rejection rates close to the nominal level. For v3, both the wild and iid bootstrap work surprisingly well, even when v=3.

Additionally, Table 3 reports bootstrap-based testing where εt are iid sampled, but without replacement. This, which is referred to as the permutation bootstrap, works well in terms of empirical rejection frequencies, even for v=3/2. In general, the permutation bootstrap works well in the context of heavy-tailed i.i.d. εt (see Cavaliere, Nielsen, & Rahbek, 2020, and also the discussion in “Heavy-Tailed Autoregressive Models” for AR models with heavy-tailed innovations).

Table 3. Empirical Rejection Frequencies With Heavy Tailed Innovations

Asymptotic

Iid

Wild

Permutation

n

Student’s tν,ν=3/2

15

0.1434

0.0395

0.0248

0.0499

25

0.0959

0.0382

0.0256

0.0520

50

0.0563

0.0350

0.0268

0.0488

100

0.0363

0.0340

0.0262

0.0443

250

0.0290

0.0350

0.0326

0.0480

500

0.0249

0.0333

0.0322

0.0465

1000

0.0278

0.0351

0.0348

0.0513

n

Student’s tν,ν=3

15

0.1809

0.0508

0.0494

0.0508

25

0.1512

0.0491

0.0475

0.0522

50

0.1129

0.0494

0.0496

0.0521

100

0.0806

0.0491

0.0486

0.0505

250

0.0596

0.0487

0.0514

0.0490

500

0.0578

0.0528

0.0515

0.0534

1000

0.0499

0.0465

0.0486

0.0481

n

Student’s tν,ν=5

0.1894

0.0519

0.0531

0.0511

25

0.1571

0.0486

0.0515

0.0499

50

0.1119

0.0523

0.0495

0.0507

100

0.0807

0.0503

0.0497

0.0503

250

0.0639

0.0486

0.0477

0.0486

500

0.0542

0.0484

0.0478

0.0485

1000

0.0540

0.0498

0.0501

0.0503

Note. Empirical rejection frequencies of asymptotic and bootstrap tests with εt i.i.d. Student’s tν for ν{3/2,3,5}. Results are reported for the iid bootstrap, the wild boostrap with wt Rademacher distributed, and the permutation bootstrap. Simulations are reported for sample lengths n{15,...,1000}. The number of bootstrap replications is B=399 and N=10,000 replications.

A Selection of Further Topics

In the previous sections the simple AR model of order one was used to introduce key ideas and challenges of the iid and wild bootstrap schemes when applied to testing the hypothesis H0:ρ=ρ¯. In this section, we provide an overview of recent selected results for the bootstrap when applied to different testing problems in econometric time series models. The overview is not meant to be exhaustive (see, e.g., Cavaliere & Rahbek, 2020, for a review of the bootstrap with more technical details, as well the references therein).

Nonstationary (Vector) Autoregressive Models

The iid Bootstrap

Consider initially the AR model in (1) again. The hypothesis of nonstationarity is given by H0:ρ=1, or equivalently, with π=ρ1, H0:π=0, in the AR model restated as

Δxt=πxt1+εt,(26)

where Δxt=xtxt1. It follows that the test-statistic τn in (5) can be written as

τn=Wn(π=0)=π^n2t=1nxt12/σ02,withπ^n=t=1nΔxtxt1/t=1nxt12

and under H0,

τndτ=(01B(u)dB(u))2/01B2(u)du,(27)

where B(u) is a standard Brownian motion, u(0,1), and τ is the (squared) Dickey-Fuller distribution (Hamilton, 1994). In this case, in terms of the bootstrap in “Bootstrap in the Autoregressive Model,” the restricted recursive bootstrap, given by Δxt=εt with εt iid resampled, is asymptotically valid. This holds as in this case τndpτ, under both H0 and the alternative. In contrast, and as discussed in Basawa, Mallik, McCormick, Reeves, and Taylor (1991), the unrestricted recursive bootstrap based on the recursion Δxt=π^nxt1+εt, with εt iid resampled, is invalid. This follows as the corresponding bootstrap statistic, τn, under H0 converges in distribution to τ˜, τ˜τ. Precisely, the bootstrap conditional distribution function converges weakly rather than in probability (Basawa et al., 1991; Cavaliere & Georgiev, 2020; Cavaliere et al., 2015).

The univariate case of testing for nonstationarity is a special case of the more general hypothesis of nonstationarity in vector AR models for Xtp with general lag-structure, as given by

ΔXt=πXt1+i=1kγiΔXti+εt,(28)

where εt are i.i.d. Np(0,Ω) distributed and the initial values (X0,ΔX0,...,ΔX1k) are fixed in the statistical analysis. Moreover, π and (γi)i=1k are p×p matrices. The hypothesis of nonstationarity of Xt is given by the hypothesis of reduced rank r,0r<p, of π (Johansen, 1996). Specifically, with Hr:rank(π)r, it follows that this may be written as

Hr:π=αβ,

where α and β are (p×r) dimensional matrices. Under the nonstationarity conditions in Johansen (1996, Theorem 4.2), it follows that Xt is a nonstationary process, with r stationary, or co-integrating, relations βXt, and (pr) common trends given by δi=1tεi, with δ(p×(pr)) dimensional of full rank, and such that δα=0. The likelihood-ratio statistic τn(r) for co-integration rank r satisfies, under Hr and the mentioned nonstationarity conditions, that

τn(r)dτ(r)=tr{(01B(u)dB(u))(01B(u)B(u)du)1(01B(u)dB(u))},

which is a multivariate version of (27) in terms of the (pr)-dimensional standard Brownian motion, B(). Cavaliere, Rahbek, and Taylor (2012) consider the recursive restricted bootstrap based on

ΔXt=πnXt1+i=1kγn,iΔXti+εt,

with πn=α˜nβ˜n and γn,i=γ˜n,i; that is, the bootstrap true values are given by the estimators under Hr. Asymptotic validity of the iid bootstrap is established in Cavaliere et al. (2012), by showing that τn(r)dpτ(r) under Hr and the alternative.

Cavaliere et al. (2015) extend the analysis to hypothesis testing on the co-integration (matrix) parameter β. Specifically, Cavaliere et al. (2015, Proposition 1 and Theorem 1) establish that under the hypothesis Hr¯:β=β¯, the bootstrap likelihood ratio statistic, τn, satisfies τndpχ(pr)r2. Importantly, it is also established that under the alternative, τn has a limiting distribution (in distribution) in terms of a diffusion process with a stochastic diffusion coefficient, and hence it is bounded in probability such that the bootstrap-based test is asymptotically valid.

The Wild Bootstrap

In order to allow for possible heteroskedasticity in the εt sequence in (28), the application of the wild bootstrap has also been studied. Results for application of the wild bootstrap in general lag univariate AR models in Gonçalves and Kilian (2004), with εt allowed to have general time-varying volatility structures, such as ARCH and stochastic volatility, have been generalized to the testing the hypothesis of co-integration Hr in Cavaliere, Rahbek, and Taylor (2010a, 2010b, 2014) and Boswijk, Cavaliere, Rahbek, and Taylor (2016).

Moreover, Boswijk et al. (2016) and Boswijk, Cavaliere, Georgiev, and Rahbek (2019) consider general hypothesis testing on the co-integration parameters α and β, with π=αβ in (28). They consider the case of stochastic volatility, where εt=Ωt1/2zt, with the p-dimensional zt i.i.d. (0,1) and the time-varying (p×p)-dimensional Ωt=Ω(t/n). Moreover, with “w“ denoting weak convergence, it is assumed that for u(0,1),

n1/2t=1[nu]εtw0uΩ1/2(s)dB(s),(29)

where B is a p-dimensional standard Brownian motion, which generalizes the i.i.d. (0,Ω) assumption, where n1/2t=1[nu]εtwΩ1/2B(u). Specifically, the limiting process in (29) is a continuous-time martingale, with in general an unknown covariance (kernel). This implies that the limiting distribution of the test statistic(s) τn, for example for the mentioned hypotheses Hr and Hr¯, will depend on unknown nuisance parameters, which again means asymptotic inference is infeasible in practice. In contrast, for the wild bootstrap, it is established that n1/2t=1[nu]εt has the same limiting distribution (in probability), and, as a result, the wild bootstrap is asymptotically valid as shown in Boswijk et al. (2016, 2019), under some additional regularity conditions to be verified.

Time-Varying Conditional Volatility Models

As discussed in Andrews (2000), applying bootstrap-based testing in ARCH models is in general difficult, and may be invalid in certain cases, due to general problems arising when testing hypotheses in time series models when one or more parameters under the null may be “on the boundary of the parameter space” (p. 399).

To illustrate, consider here xt given by a linear ARCH model of order q

xt=σt(θ)zt,t=1,...,n,

with zt i.i.d. (0,1) and

σt2(θ)=ω+i=1qαixti2.

In the statistical analysis, the initial values (x0,...,xq+1) are fixed, and the parameter θ=(ω,α1,...,αq)Θ, where

Θ={θq+1:ω2>0,andαi0fori=1,...,q}.

Thus by definition of the parameter space Θ, if for the true value θ0=(ω0,α0,1,...,α0,q) it holds that α0,j=0 for some j, the true value θ0 is on the boundary of Θ.

The fact that it is unknown a priori which of the ARCH coefficients may or may not be zero leads to nonpivotal limiting distributions of test statistics and estimators. Consider here the likelihood ratio statistic τn for the nullity of the q-th order ARCH coefficient; that is, the hypothesis Hq:αq=0. With the Gaussian likelihood function given by

ln(θ)=12t=1n(logσt2(θ)+xt2/σt2(θ)),

by definition, τn=2(ln(θ^n)ln(θ˜n)), where the unrestricted4 Gaussian MLE is given by θ^n=argmaxθΘln(θ), while θ˜n is the Gaussian MLE under Hq. By Andrews (1999, 2001), it follows that τn has a limiting distribution, which is non-standard. In addition, the limiting distribution of τn is nonpivotal as, crucially, it depends on whether α0,i>0, or α0,i=0 for i=1,...,q1 under Hq.

While this implies that the unrestricted bootstrap is invalid (Andrews, 2000), it follows by Cavaliere, Nielsen, and Rahbek (2017) that the iid restricted bootstrap is asymptotically valid under mild conditions for the simple case of the first order ARCH with q=1. Moreover, Cavaliere, Nielsen, Pedersen, and Rahbek (2020) demonstrate validity of a modified restricted bootstrap, which can be applied for general testing problems in parametric models with parameters on the boundary under the null. Specifically, for the case of ARCH of order q, consider the bootstrap process

xt=σt(θn)zt,

with zt iid resampled from z^t=xt/σt(θ^n), after recentering and rescaling these. The bootstrap conditional volatility process σt2(θn) is given by

σt2(θn)=ωn+i=1nαn,ixt12,(30)

with ωn=ω˜n and αn,i=α˜n,iI(α˜n,i>cn), with cn a deterministic sequence that satisfies (a) cn0, and (b) n1/2cn, as n. The bootstrap scheme is referred to as modified since in (30) the bootstrap true values αn,i—by “ shrinking”—are set to zero for i=1,...,q1, provided α˜n,i is small relative to cn. Note that with the time-varying bootstrap volatility defined by (30), this is a case of a fixed design (or rather fixed volatility) bootstrap (see step A in “Bootstrap in the Autoregressive Model”). The general fixed volatility bootstrap for ARCH models is considered in Cavaliere et al. (2018), and the modified, by shrinking, fixed volatility bootstrap is shown in Cavaliere, Nielsen, Pedersen, and Rahbek (2020, Proposition 1) to be asymptotically valid. Simulations there show that both the fixed volatility and the recursive with xti2=xti2 in (30) bootstrap based tests have empirical rejection frequencis that are close to the nominal level for small and moderate sample sizes n. Moreover, as for the discussion of the moment requirements for the AR bootstrap, simulations indicate that while sufficient, the moment constraints on the original ARCH process imposed to establish validity are also not necessary.

Double Autoregressive Models

The double autoregressive (DAR) model combines the AR and ARCH models, as both the conditional mean and conditional variance depend on lagged levels of the process (see Ling, 2004, 2007; and for a multivariate “co-integrated” version, Nielsen & Rahbek, 2014).

Consider here the first order DAR model as given by

Δxt=πxt1+σt(θ)zt,σt2(θ)=σ2+αxt12,t=1,2,...,n,(31)

with zt i.i.d. N(0,1), x0 is fixed in the statistical analysis and the parameter given by θ=(π,σ2,α)Θ, with Θ={θ3:σ2>0andα0}.

A notable special feature of the DAR process is that for π=0 the process is strictly stationary for any 0<α<2.42 (Borkovec & Klüppelberg, 2001; Ling, 2004, 2007), while the process is nonstationary when π=α=0. With π=0 and α(0,2.42), while being strictly stationary the DAR process xt has infinite variance (and only finite fractional moments).

From the specification of the parameter space Θ evidently for α0=0, the true value θ0 is on the boundary, raising the issues discussed in the section “Time-Varying Conditional Volatility Models” in relation to the ARCH model. Asymptotic theory for the Gaussian likelihood-based MLE with α0>0 is given by Ling (2004), while Cavaliere and Rahbek (2020) extend the results to allow for the boundary case. Klüppelberg, Maller, van de Vyver, and Wee (2002) derive the asymptotic distribution of the likelihood ratio statistic τn for the hypothesis of nonstationarity as given by H0:π=α=0 (see also Chen, Li, & Ling, 2013). Different versions of a bootstrap-based test are discussed in Cavaliere and Rahbek (2020). In particular, validity is established for a restricted bootstrap given by Δxt=σ˜nzt, where σ˜n2 is the MLE of σ2 under H0, while zt is obtained by iid resampling of unrestricted residuals, z^t (recentered and rescaled). That is, the unrestricted residuals z^t are given by

z^t=(Δxtπ^nxt1)/σt(θ^n),

with θ^n=(π^n,σ^n,α^n) the unrestricted MLE. In line with the discussion in the section “Bootstrap in the Autoregressive Model” regarding restricted and unrestricted residuals, this choice ensures that z^t for large n is “close” to the true zt, irrespective of whether the null H0 is true or not. While the validity result is shown for this choice, simulations indicate that in practice the difference between choosing to resample from z^t or from z˜t=Δxt/σ˜n is negligible. (See Cavaliere & Rahbek, 2020, for a detailed discussion of this as well as asymptotic theory for different bootstraps.)

Heavy-Tailed Autoregressive Models

So far results for the (vector) AR models have been derived under the assumption that the innovations εt have mean zero and a finite variance σ2, or some time-varying, possibly conditional, variance σt2 when discussing heteroskedasticity. To allow for more extreme events, and phenomena such as “ bubble” periods with local explosive behavior, this assumption was relaxed in Davis and Resnick (1985a, 1985b, 1986) and Davis and Song (2020), where the i.i.d. innovations εt are allowed to have infinite variance. Specifically, they consider the case of εt i.i.d. with a stable distribution such as the Cauchy; that is, “heavy-tailed” as the tails of the distribution of εt are assumed to decay at a rate that is slower than the Gaussian (exponential) rate.

Two key examples are given by the classic AR model and the so-called noncausal AR model of order one in terms of i.i.d. stable distributed εt,

AR:xt=ρxt1+εt,andAR+:xt=ρ+xt+1+εt.(32)

For the standard, and hence causal, AR, recall that with t=1,...,n, x0 is the initial value that is fixed in the statistical analysis, while for the noncausal AR+xn is the “initial value” due to the forward recursion. Noncausal AR+ type models have become popular as they seem to capture well the dynamics of phenomena such as bubbles where, after period of exponential type growth, the process “collapses.” Interestingly, and linking the heavy-tail AR+ models with the DAR model in “Double Autoregressive Models,” the AR+ process in (32) can be shown to have a causal “semi-strong” representation as the DAR process in (31) (see Gourieroux & Zakoian, 2017).

Consider testing the hypothesis H0+:ρ+=ρ¯ using the Gaussian likelihood-based statistic τn+, given by

τn+=(ρ^n+ρ¯)(t=1n1xt+1)1/2/σ^n,

where ρ^n+=t=1n1xtxt+1/t=1nxt+12 and σ^n2=n1t=1n1(xtρ^n+xt1)2. While the test statistic τn+ is analogous to the (square root of the) previously studied statistic τn in (5), the limiting distribution is nonstandard, as the εt are assumed to be i.i.d. stable distributed. For example, with εt Cauchy distributed, (n/logn)(ρ^n+ρ¯) is asymptotically distributed as (1+ρ¯)Cχ12, where C is standard Cauchy distributed and τn+=OP(n1/2logn). In general for stable distributions asymptotic testing is infeasible as the limiting distributions depend on the “tail index” (which is one for the Cauchy) of the innovations εt, and moreover the normalization (which is n1/2logn for the Cauchy case) depend on further, and in practice, unknown quantities. In the case of stable distributed innovations, Cavaliere, Nielsen, and Rahbek (2020) discuss validity of the recursive bootstrap scheme similar to (10),

xt=ρ¯xt+1+εt,

initialized with xn=xn, and with εt resampled from the restricted residuals {ε˜t+}t=1n1, where ε˜t+=xtρ¯xt+1. Crucially the εt are not sampled by iid resampling with replacement, as this would lead to an invalid bootstrap test (Athreya, 1987; Knight, 1989). Instead, εt are resampled without replacement; that is, by permuting {ε˜t+}t=1n1, or, in combination with the wild, by permuting {ε˜t+wt}t=1n1, where wt are i.i.d. Rademacher distributed (Remark 10). With τn+ the bootstrap statistic based on the permutation, or the combined permutation-wild bootstrap, Theorem 1 in Cavaliere, Nielsen, and Rahbek (2020) establishes validity of the bootstrap-based test under the null hypothesis for general AR+ models.

Similarly, Cavaliere, Georgiev, and Taylor (2016) establish bootstrap validity for the so-called sieve bootstrap in Bühlmann (1997) for general causal AR models with heavy tails. Also note that in terms of testing for the presence of bubbles based on one-sided testing using the supremum of recursively computed Dickey-Fuller type statistics as in Phillips, Wu, and Yu (2011), Harvey, Leybourne, Sollis, and Taylor (2016) establish validity of a wild bootstrap based test to allow for heteroskedasticity.

Conclusions and Further Readings

This article has provided an introduction to key steps required for a successful implementation of bootstrap hypothesis testing to time series models. In the framework of a simple autoregressive model, we have discussed the (large-sample) validity of recursive bootstrap algorithms, where the bootstrap sample is constructed by mimicking the dependence structure of the original data, as implied by the econometric model at hand. We have discussed the main requirements for bootstrap inference to be valid; that is, to mimic the asymptotic distribution of the original test statistic under the null hypothesis and to be bounded (in probability) when the null hypothesis is false. To do so, we have introduced the needed probability tools (which involve dealing with the conditional—hence random—nature of the bootstrap measure) and shown how to apply them in order to assess the large sample behavior of the bootstrap estimators and test statistics.

Needless to say, this article is not meant to be exhaustive or to provide a comprehensive treatment of bootstrap inference. We have focused on bootstrap hypothesis testing which, although being one of the most common applications of the bootstrap, is not the only one. Further common implementations of the bootstrap involve computation of standard errors, the construction of confidence intervals for the parameters of interest, and bias correction of the estimators (see, inter alia, Horowitz, 2001, or MacKinnon, 2006, for reviews).

This article is also not exhaustive in terms of bootstrap algorithms. For instance we have not discussed bootstrap algorithms based on resampling a block of observations (or residuals) rather than resampling single observations (or residuals). As introduced by Künsch (1989), block bootstrap methods are quite flexible and powerful in cases where one does not have strong a priori beliefs or knowledge about the dependence structure of the data (see also Lahiri, 2003; Politis & Romano, 1994; Shao, 2010). Subsampling methods represent a valid alternative (or complement) to recursive, model-based bootstrap (Politis, Romano, & Wolf, 1999). We have focused on a case (the first order AR model) where the data-generating process is described by a finite dimensional vector of parameters. Extensions to infinite-dimensional parameter spaces, as for the case of general linear processes, are available in the literature; a classic example is the AR() sieve bootstrap of Kreiss (1988, 1992), Bühlmann (1997), and Gonçalves and Kilian (2007).

It is important to emphasize that there is a rich literature discussing finite-sample improvements of the bootstrap in various models, including time series models, based on Edgeworth expansions of the τn and τn statistics, as briefly mentioned in Remark 10. An introduction to Edgeworth expansions as applied for the bootstrap can be found in van der Vaart (2000, chapter 23.3; see also Hall, 1992; Horowitz, 2001).

Finally, it is worth noticing that throughout this article bootstrap validity under the null is defined as the fact that under the null the bootstrap test statistic converges to the asymptotic (null) distribution of the original statistic. This implies that, under rather mild regularity conditions (such as continuity of the limiting distribution), if the null hypothesis holds then the bootstrap p-value pn is asymptotically uniformly distributed (Remark 15). Combined with establishing that the bootstrap statistic is bounded in probability under the alternative such that pnp0, this was used to establish validity of the bootstrap (see “Bootstrap Validity Under the Alternative”). Cavaliere and Rahbek (2020) discuss in detail different definitions of asymptotic validity and their verification under different assumptions. An example is Cavaliere and Georgiev (2020), where validity of the bootstrap test is defined directly in terms of asymptotic uniformity of the bootstrap p-values, rather than the properties of (consistent) estimation of the limiting null distribution of the original test statistic. By doing so, the bootstrap can be employed in cases where the limiting distribution of the original statistic may not be well defined, or in cases where the boostrap distribution does not converge in probability but, rather, converges in distribution (see, e.g., Boswijk, Cavaliere, Georgiev, & Rahbek, 2019, and the discussions in sections “Nonstationary [Vector] Autoregressive Models” and “Heavy-Tailed Autoregressive Models” where this was applied).

Acknowledgments

This research was supported by Danish Council for Independent Research (DSF Grant 7015-00028B).

References

  • Andrews, D. W. K. (1999). Estimation when a parameter is on a boundary. Econometrica, 67(6), 1341–1383.
  • Andrews, D. W. K. (2000). Inconsistency of the bootstrap when a parameter is on the boundary of the parameter space. Econometrica, 68(2), 399–405.
  • Andrews, D. W. K. (2001). Testing when a parameter is on the boundary of the maintained hypothesis. Econometrica, 69(3), 683–734.
  • Andrews, D. W. K., & Buchinsky, M. (2000). A three-step method for choosing the number of bootstrap repetitions. Econometrica, 68, 23–51.
  • Athreya, K. B. (1987). Bootstrap of the mean in the infinite variance case. Annals of Statistics, 15, 724–731.
  • Basawa, I. V., Mallik, A. K., McCormick, W. P., Reeves, J. H., & Taylor, R. L. (1991). Bootstrapping unstable first-order autoregressive processes. Annals of Statistics, 19, 1098–1101.
  • Berkowitz, J., & Kilian, L. (2000). Recent developments in bootstrapping time series. Econometric Reviews, 19(1), 1–48.
  • Borkovec, M., & Klüppelberg, C. (2001). The tail of the stationary distribution of an autoregressive process with ARCH(1) errors. Annals of Applied Probability, 11(4), 1220–1241.
  • Boswijk, H. P., Cavaliere, G., Georgiev, I., & Rahbek, A. (2019). Bootstrapping non-stationary stochastic volatility. Working Paper TI 2019–083/III. Tinbergen Institute.
  • Boswijk, H. P., Cavaliere, G., Rahbek, A., & Taylor, A. M. R. (2016). Inference on co-integration parameters in heteroskedastic vector autoregressions. Journal of Econometrics, 192, 64–85.
  • Bühlmann, P. (1997). Sieve bootstrap for time series. Bernoulli, 3, 123–148.
  • Cavaliere, G., & Georgiev, I. (2020). Inference under random limit bootstrap measures. Econometrica.
  • Cavaliere, G., Georgiev, I., & Taylor, A. M. R. (2016). Sieve-based inference for infinite-variance linear processes. Annals of Statistics, 44(4), 1467–1494.
  • Cavaliere, G., Nielsen, H. B., Pedersen, R. S., & Rahbek, A. (2020). Bootstrap inference on the boundary of the parameter space with application to conditional volatility models. Journal of Econometrics.
  • Cavaliere, G., Nielsen, H. B., & Rahbek, A. (2015). Bootstrap testing of hypotheses on co-integration relations in vector autoregressive models. Econometrica, 83(2), 813–831.
  • Cavaliere, G., Nielsen, H. B., & Rahbek, A. (2017). On the consistency of bootstrap testing for a parameter on the boundary of the parameter space. Journal of Time Series Analysis, 38(4), 513–534.
  • Cavaliere, G., Nielsen, H. B., & Rahbek, A. (2020). Bootstrapping noncausal autoregressions. with applications to explosive bubble modeling. Journal of Business and Economic Statistics, 38(1), 55–67.
  • Cavaliere, G., Pedersen, R. S., & Rahbek, A. (2018). The fixed volatility bootstrap for a class of ARCH(q) models. Journal of Time Series Analysis, 39(6), 920–941.
  • Cavaliere, G., Rahbek, A., & Taylor, A. M. R. (2010a). Co-integration rank testing under conditional heteroskedasticity. Econometric Theory, 26, 1719–1760.
  • Cavaliere, G., Rahbek, A., & Taylor, A. M. R. (2010b). Testing for co-integration in vector autoregressions with non-stationary volatility. Journal of Econometrics, 158, 7–24.
  • Cavaliere, G., Rahbek, A., & Taylor, A. M. R. (2012). Bootstrap determination of the co-integration rank in vector autoregressive models. Econometrica, 80, 1721–1740.
  • Cavaliere, G., Rahbek, A., & Taylor, A. M. R. (2014). Bootstrap determination of the co-integration rank in vector autoregressive heteroskedastic VAR models. Econometric Reviews, 33, 606–650.
  • Chen, M., Li, D., & Ling, S. (2013). Non-stationarity and quasi-maximum likelihood estimation on a double autoregressive model. Journal of Time Series Analysis, 35, 189–202.
  • Davidson, R., & MacKinnon, J. G. (2000). Bootstrap tests: How many bootstraps? Econometric Reviews, 19, 55–68.
  • Davidson, R., & MacKinnon, J. G. (2006). Bootstrap methods in econometrics. In T. C. Mills & K. D. Patterson (Eds.), Palgrave handbook of econometrics: Vol. 1. Econometric theory (pp. 812–838). Basingstoke, UK: Palgrave Macmillan.
  • Davis, R., & Resnick, S. (1985a). Limit theory for moving averages of random variables with regularly varying tail probabilities. Annals of Probability, 13, 179–195.
  • Davis, R., & Resnick, S. (1985b). More limit theory for the sample correlation function of moving averages. Stochastic Processes and Their Applications, 20, 257–279.
  • Davis, R., & Resnick, S. (1986). Limit theory for the sample covariance and correlation functions of moving averages. Annals of Statistics, 14, 533–558.
  • Davis, R., & Song, L. (2020). Noncausal vector AR processes with application to economic time series. Journal of Econometrics, 216(1), 246–267.
  • Duffee, G. R., & Stanton, R. H. (2008). Evidence on simulation inference for near unit-root processes with implications for term structure estimation. Journal of Financial Econometrics, 6(1), 108–142.
  • Durret, R. (2019). Probability: Theory and examples (5th ed.). Cambridge, UK: Cambridge University Press.
  • Gonçalves, S., & Kilian, L. (2004). Bootstrapping autoregressions with conditional heteroskedasticity of unknown form. Journal of Econometrics, 123, 89–120.
  • Gonçalves, S., & Kilian, L. (2007). Asymptotic and bootstrap inference for AR(∞) processes with conditional heteroskedasticity. Econometric Reviews, 26(6), 609–641.
  • Gourieroux, C., & Zakoan, J.-M. (2017). Local explosion modelling by non-causal process. Journal of the Royal Statistical Society, Series B, 79, 737–756.
  • Hahn, J., & Liao, Z. (2019). Bootstrap standard error estimates and inference. Working Paper. Los Angeles, CA: UCLA.
  • Hall, P. (1992). The bootstrap and Edgeworth expansions. New York, NY: Springer.
  • Hall, P., & Heyde, C. C. (1980). Martingale limit theory and its application. New York, NY: Academic Press.
  • Hamilton, J. D. (1994). Time series analysis. Princeton, NJ: Princeton University Press.
  • Hansen, B. (1996). Inference when a nuisance parameter is not identified under the null hypothesis. Econometrica, 64(2), 413–430.
  • Hansen, B. (2000). Testing for structural change in conditional models. Journal of Econometrics, 97, 93–115.
  • Harvey, D., Leybourne, S. J., Sollis, R., & Taylor, A. M. R. (2016). Tests for explosive financial bubbles in the presence of non-stationary volatility. Journal of Empirical Finance, 38(B), 548–574.
  • Hayashi, F. (2000). Econometrics. Princeton, NJ: Princeton University Press.
  • Hendry, D. F. (1984). Monte Carlo experimentation in econometrics. In Z. Grilishes & M. D. Intriligator (Eds.), Handbook of econometrics (Vol. 2, pp. 937–976). Amsterdam, The Netherlands: North Holand.
  • Horowitz, J. L. (2001). The bootstrap. In Z. Grilishes & M. D. Intriligator (Eds.), Handbook of econometrics (Vol. 5, pp. 3159–3228). Amsterdam, The Netherlands: North Holand.
  • Horowitz, J. L. (2003). The bootstrap in econometrics. Statistical Science, 18(2), 211–218.
  • Johansen, S. (1996). Likelihood-based inference in cointegrated vector autoregressive models. Oxford, UK: Oxford University Press.
  • Klüppelberg, C., Maller, R. A., Van de Vyver, M., & Wee, D. (2002). Testing for reduction to random walk in autoregressive conditional heteroskedasticity models. Econometrics Journal, 5, 387–416.
  • Knight, K. (1989). On the bootstrap of the mean in the infinite variance case. Annals of Statistics, 17, 1168–1175.
  • Kreiss, J. P. (1988). Asymptotic statistical inference for a class of stochastic processes. Habilitationsschrift (Unpublished thesis). Universität Hamburg.
  • Kreiss, J. P. (1992). Bootstrap procedures for AR(∞)-processes. In K. H. Jöckel, G. Rothe, & W. Sendler (Eds.), Bootstrapping and related techniques. Lecture Notes in Economics and Mathematical Systems (Vol. 376). Berlin, Germany: Springer.
  • Künsch, H. (1989). The jackknife and the bootstrap for general stationary observations. Annals of Statistics, 17, 1217–1241.
  • Lahiri, S. N. (2003). Resampling methods for dependent data. New York, NY: Springer.
  • Ling, S. (2004). Estimation and testing stationarity for double-autoregressive models. Journal of the Royal Statistical Society B, 66, 63–78.
  • Ling, S. (2007). A double AR(p) model: Structure and estimation. Statistica Sinica, 17, 161–175.
  • Liu, R. Y. (1988). Bootstrap procedures under some non i.i.d. models. Annals of Statistics, 16, 1696–1708.
  • MacKinnon, J. G. (2006). Bootstrap methods in econometrics. Economic Record, 82, 2–18.
  • MacKinnon, J. G. (2009). Bootstrap hypothesis testing. In D. A. Belsley & J. Kontoghiorghes (Eds.), Handbook of computational econometrics (pp. 183–213). Chichester, UK: Wiley.
  • Nielsen, H. B., & Rahbek, A. (2014). Unit root vector autoregression with volatility induced stationarity. Journal of Empirical Finance, 29, 144–167.
  • Phillips, P. C. B., Wu, Y., & Yu, J. (2011). Explosive behavior in the 1990s Nasdaq: When did exuberance escalate stock values? International Economic Review, 52, 201–226.
  • Politis, D., & Romano, J. (1994). The stationary bootstrap. Journal of the American Statistical Association, 89(428), 1303–1313.
  • Politis, D. N., Romano, J. P., & Wolf, M. (1999). Subsampling. New York, NY: Springer.
  • Shao, X. (2010). The dependent wild bootstrap. Journal of the American Statistical Association, 105, 218–235.
  • van der Vaart, A. W. (2000). Asymptotic statistics. Cambridge, UK: Cambridge University Press.
  • White, H. (1980). A heteroskedasticity-consistent covariance matrix estimator and a direct test for heteroskedasticity. Econometrica, 48, 817–838.

Appendix

Proof of Lemma 3

Consider the bootstrap process {xt}t=0n as given by (10). Without loss of generality set x0=0, such that by simple recursion,

xt=i=0t1ρ0iεti=i=0ρ0iεti=ρ(L)εt,

with εj=0 for j0, L the lag operator, Lεt=εt1, and ρ(z)=i=0ρ0izi, for z. The summation to infinity means one can use standard manipulations of the lag polynomial ρ() as well known from the time series literature (see, e.g., Hamilton, 1994; Johansen, 1996, and references therein). Note first that as |ρ0|<1, the coefficients ρ0i in ρ(z) are exponentially decreasing and ρ(z) is convergent for |z|<1+δfor some δ>0 and ρ(1)=(1ρ0)1.

When considering the average of xt, n1t=1nxt, one may for example use an expansion of ρ(z) (see Remark 21) around z=1, such that

xt=ρ(L)εti=ρ(1)εt+ρ(L)Δεt,

with ρ(z)=i=1θizi with θi=iρ0i exponentially decreasing. This immediately gives

n1t=1nxt=(1ρ0)1n1t=1nεt+n1ρ(L)εn.

The first term tends to zero by (13). Next, for the second term use that εt are i.i.d. (conditionally on the data) such that

E|n1ρ(L)εn|=n1E|i=1iρ0iεni|n1i=1i|ρ0|iE|εni|cn2t=1n|ε˜tc|p0,

with c=|ρ0|(1|ρ0|)2 and where n1t=1n|ε˜tc|pκ< for κ a constant by standard application of the LLN.

Turning to the average of xt2, note that

n1t=1nxt2=n1t=1n(i=0ρ0iεti)2=n1t=1ni=0ρ02iεti2+δn,

where

δn=n1t=1nij=1nρ0i+jεtiεtj.

With ρ2(z)=i=0ρ02izi and ω0=ρ2(1)σ02=σ02/(1ρ02) the first term can be written as

n1t=1ni=0ρ02iεti2=n1t=1nρ2(L)ηt+ω0,

where ηt=εt2σ02. Thus, using identical arguments, n1t=1nρ2(L)ηtpp0.

It remains to show that δnpp0. First, observe that by definition of the double summation,

δn=2n1t=1n(i=0ρ0iεtij=1ρ0i+jεtij)=2i=0ρ02i(n1t=1nεtij=1ρ0jεtij).

As E(εtiεtj)=0 for ji, it follows that

E(δn2)=4i=0ρ04iE(n1t=1nεtij=1ρ0jεtij)2=4n2i=0ρ04it=1nE(εtij=1ρ0jεtij)2.

Here

E(εtij=1ρ0jεtij)2=j=1m=1ρ02jE(εti2εtijεtim)=j=1ρ02jE(εti2εtij2)=j=1ρ02j(E(εt2))2=c(n1t=1nε˜t2)2pc˜,

with c˜=ρ02σ04/(1ρ02).

Remark 21. The equality

ρ(z)=ρ(1)+ρ(z)(1z),

with ρ(z)=i=1θizi1 and θi=iρ0i follows by the identity

ρ(z)=ρ(z)ρ(1)1z=i=1iρ0izi1.

Also note that the polynomial ρ(z) is convergent for |z|<1+δ as the coefficients θi=iρ0i of ρ(z) are exponentially decreasing.

Notes

  • 1. The Rademacher distribution is a two-point distribution on ±1, each with probability a half, whereas the Mammen distribution is a two-point distribution on 1+52

    and 152 with probabilities given by 5125 and 5+125, respectively.

  • 2. Markov’s inequality:P(|X|>δ)E|X|k/δk for any δ>0 and κ1.

  • 3. Note that as the model here includes a constant term, strictly speaking the recentering of ε˜t is not needed.

  • 4. A complete discussion of specifications and properties of the parameter space Θ is given in Cavaliere, Nielsen, Pedersen, and Rahbek (2020).