Time-Domain Approach in High-Dimensional Dynamic Factor Models

Example 1

Let $r=1$, only one common factor, and

$$x_{it}=\lambda F_t+{\xi }_{it},$$(4)

where $\lambda \ne 0$. For the covariance among the x’s:

$$\text{cov}(x_{i_{1}t},x_{i_{2}t})=\text{cov}({\chi }_{i_{1}t},{\chi }_{i_{2}t})={\lambda }^2{\sigma }_F^2.$$

By taking the arithmetic mean of the first $n$ processes:$(1/n){\displaystyle {\sum }_{i=1}^n}{x}_{it}=\lambda F_t+(1/n){\displaystyle {\sum }_{i=1}^n}{\xi }_{it}$, that is

$$\frac{1}{n}{\displaystyle \sum _{i=1}^n}{x}_{it}-\lambda F_t=\frac{1}{n}{\displaystyle \sum _{i=1}^n}{\xi }_{it}\mathrm{.}$$

Assuming that the variances of the idiosyncratic components are bounded, ${\sigma }_{{\xi }_i}^2\le {\sigma }_{\xi }^2$, $i\in \mathbb{N}$,

$$\text{var}\left(\frac{1}{n}{\displaystyle \sum _{i=1}^n{\xi }_{it}}\right)\le \frac{1}{n}{\sigma }_{\xi }^2.$$

Thus, in mean square,

$$\underset{n\to \infty }{{\displaystyle \lim }}\frac{1}{n}{\displaystyle \sum _{i=1}^n}{x}_{it}=\lambda F_t.$$(5)

This example is as simple as it is important to grasp the idea leading to estimates for the unobserved components. Here $n$, the size of the cross-section, and $T$, the number of observations for each time series, both tend to infinity. A consequence of $n$ tending to infinity is that the idiosyncratic components are averaged away and the common components are estimated consistently, which is not the case if the model only contains a finite number of processes $x_{it}$.□

Observation 1

The discussion in Example 1, the limit in equation (5) in particular, involves population covariances and mean-square convergence. Examples based on population covariances are used in this article only for heuristic purposes; see also the introduction of principal components in the “Estimation” section, for example.□

The result presented here as Theorem 1 states that the processes $x_{it}$ have a representation like (3), with a slightly different definition of the idiosyncratic components, if and only if their covariances satisfy the condition specified in Assumption 1. Thus the existence of a structure like (3) can be established by a criterion implying only properties of the observable processes $x_{it}$.

Define $x_{nt}=(x_{1t}{x}_{2t}\cdots x_{nt}{)}^{\prime }$. ${\Gamma }_{nk}^x$ is the covariance between $x_{nt}$ and $x_{n,t-k}$:

$${\Gamma }_{nk}^x=\text{E}(x_t{x^{\prime }}_{t-k}).$$(6)

The eigenvalues of the variance-covariance matrix ${\Gamma }_{n0}^x$ are real and non-negative. They are denoted by ${\mu }_{nj}^x,j=1,\dots ,n$, with the assumption that ${\mu }_{nj}^x\ge {\mu }_{n,j+1}^x,j=1,\dots ,n-1$. In the same way ${\chi }_{nt}\mathrm{,}{\xi }_{nt},{\text{Γ}}_{nk}^{\chi },{\text{Γ}}_{nk}^{\xi }$ is defined and the eigenvalues ${\text{μ}}_{nj}^{\chi }$ and ${\mu }_{nj}^{\xi }$. Again, this is assuming that ${\mu }_{nj}^{\chi }\ge {\mu }_{n,j+1}^{\chi }$ and for $j=1,\dots ,n-1$.

Assumption 1

There exists an integer $\text{s}\ge 0$ such that ${\mu }_{n,s+1}^x\le R$ for some positive real $R$ and all $n\in \mathbb{N}$.

If Assumption 1 holds, let $r$ be the minimum of all integers s such that ${\mu }_{n,s+1}^x$ is bounded for $n\in \mathbb{N}$. Of course if $r>0$ then ${{\displaystyle \lim }}_{n\to \infty }{\mu }_{nj}^x=\infty$ for $0<j\le r$.

In Example 1, assuming that ${\sigma }_{{\xi }_i}^2={\sigma }_{\xi }^2$ for all i, it is easily seen that

$${\Gamma }_{n0}^x={\lambda }^2{\sigma }_F^2(1\dots 1{)}^{\prime }(1\dots 1)+{\sigma }_{\xi }^2{I}_n,$$

with eigenvalues

$${\mu }_{n1}^x=n{\lambda }^2{\sigma }_F^2+{\sigma }_{\xi }^2,{\mu }_{n2}^x={\sigma }_{\xi }^2,{\mu }_{n3}^x={\sigma }_{\xi }^2,\dots ,{\mu }_{nn}={\sigma }_{\xi }^2,$$

so that Assumption 1 is fulfilled and $r=1$.

Observation 2

Assumption 1 is a fairly serious restriction. A simple example in which no finite s can be determined can be found in Lippi (2018) and the section “The Finite-Dimension Assumption and the Static Method.”

Theorem 1

Suppose that Assumption 1 holds and $r>0$. Then there exists an r-dimensional weakly stationary vector process $F_t=(F_1{t}_{\dots }{F}_{rt}{)}^{\prime }$ and loadings ${\text{λ}}_{ij}$, $i\in \mathbb{N}$, $j=1,\mathrm{...},r$, such that

$$\begin{array}{l}{x}_{it}={\chi }_{it}+{\xi }_{it},\\ {\chi }_{it}={\lambda }_{i1}{F}_{1t}+\cdots +{\lambda }_{ir}{F}_{rt},\end{array}$$(7)

where:

Conversely, if the processes ${\text{x}}_{\text{i}\text{t}}$ have representation (7) with a $\tilde{r}$-dimensional vector ${\tilde{F}}_t$, and (i), (ii), (iii) hold, then Assumption 1 is fulfilled and $r=\tilde{r}$.

This representation theorem was proved in Chamberlain and Rothschild (1983). It is a basic result in the literature on high-dimensional dynamic factor models taking the time-domain approach. Both the criteria to determine the number of factors and the construction of estimators for the common components and factors rely directly or indirectly on Theorem 1 (see “Estimation”).

Condition (ii) is the definition of an idiosyncratic component in this literature. Note that it is an asymptotic definition. It includes the case in which different idiosyncratic component are orthogonal (see assumption (b) earlier) with a bound for the idiosyncratic variances ${\sigma }_{{\xi }_i}^2$. However, it also includes nondiagonal variance-covariance matrices for the idiosyncratic components. For example, local or sectoral covariances for the idiosyncratic components produce a block-diagonal shape for ${\Gamma }_{n0}^{\xi }$. Imposing a bound for the maximum eigenvalues of the blocks, condition (ii) is fulfilled.

Theorem 1 can be obviously adapted to $r=0$, with (7) holding with ${\chi }_{it}=0$, so that the x’s themselves are idiosyncratic.

Observation 3

Condition (i) in Theorem 1 is crucial. It ensures that no representation with a smaller number of factors is possible. For example, suppose that ${\chi }_{it}={\lambda }_{i1}{F}_{1t}+{\lambda }_{i2}{F}_{2t}$, and that ${\lambda }_{i2}=\alpha {\lambda }_{i1}$ for all $i\in \mathbb{N}$. Then, of course, setting ${\tilde{F}}_t=F_{1t}+\alpha F_{2t}$ produces ${\chi }_{it}={\lambda }_{i1}(F_{1t}+\alpha F_{2t})={\lambda }_{i1}{\tilde{F}}_t$, which is a representation with one factor. It is easy to see that in this case only ${\mu }_{n1}^{\chi }\to \infty$, but also that only ${\mu }_{n1}^{\chi }\to \infty$, that is, that $r$ is equal to one, not two.

Observation 4

It is important to point out that if the infinite-dimensional vector (2) has a factor structure (i.e., if Assumption 1 holds), then $r$, the number of factors, is identified as the number of eigenvalues of ${\Gamma }_{n0}^x$ tending to infinity. Moreover, the components ${\chi }_{it}$ and ${\xi }_{it}$ are identified as well; see Chamberlain (1983), Chamberlain and Rothschild (1983), and Forni and Lippi (2001). In particular, the space spanned by the common components ${\chi }_{it}$ and that spanned by the factors $F_t$ coincide. However, the factors $F_t$ and the corresponding loadings are identified only up to a linear transformation. For, if $F_t$ is a vector of factors and $x_{it}={\lambda }_i{F}_t$, where ${\lambda }_i=\left({\lambda }_{\text{i}}·{\lambda }_{ir}\right)$, then

$$x_{it}=({\lambda }_i{H}^{-1})(H{F}_t)={\lambda }^{*}{F}_t^{*},$$

where $H$ is any non-singular $r\times r$ matrix. As a consequence, if it is convenient, it can be assumed that the factors are orthonormal, that is, mutually orthogonal and unit-variance.

Example 2

Consider again model (8), simplified by assuming $C_t=u_t$, so that $F_{1t}=u_t$, $F_{2t}=u_t{}_{–1}$, and

$$\left(\begin{array}{l}{F}_{1t}\hfill \\ F_{2t}\hfill \end{array}\right)=\left(\begin{array}{l}1\hfill \\ L\hfill \end{array}\right)u_t.$$

This is a singular MA(1) representation. The unusual fact is that the MA matrix (1 L)‘ has a square, stable left inverse of degree 1, namely

$$\left(\begin{array}{ll}1\hfill & 0\hfill \\ -L\hfill & 1\hfill \end{array}\right)\left(\begin{array}{l}1\hfill \\ L\hfill \end{array}\right)=\left(\begin{array}{l}1\hfill \\ 0\hfill \end{array}\right),$$

so that

$$\left(\begin{array}{ll}1\hfill & 0\hfill \\ -L\hfill & 1\hfill \end{array}\right)\left(\begin{array}{l}{F}_{1t}\hfill \\ F_{2t}\hfill \end{array}\right)=\left(\begin{array}{l}1\hfill \\ 0\hfill \end{array}\right)u_t,$$

which is a AR(1) representation.

This unexpected result, that a singular moving average of order one has a finite-degree autoregressive representation, is just an example of the following general result (see Anderson & Deistler, 2008).

Theorem 2

Assume that the r-dimensional vector $F_t$ has the representation

$$F_t=D\left(L\right)u_t,$$

where $u_t$ is a q-dimensional white noise and $q<r$, so that the vector $F_t$ is singular. Moreover, assume that the entries of $D\left(L\right)$ are rational functions of $L:d_{ij}\left(L\right)=h_{ij}\left(L\right)/k_{ij}\left(L\right)$, where $h_{ij}$ and $k_{ij},i=1,\mathrm{\dots },r,j=1,\mathrm{\dots },q$, are polynomials Then, for generic values of the coefficients of ${\text{h}}_{ij}$ and ${\text{k}}_{ij}$, that is, for all values of the coefficients with the exception of a lower-dimensional subset, $F_t$ has an autoregressive representation

$$G\left(L\right)F_t=D\left(0\right)u_t,$$

where (i) $G\left(L\right)$ is an $r\times r$, stable, finite-degree polynomial matrix, (ii) the rank of $D\left(0\right)$ is $q$. (For a more general definition of genericity, see Anderson & Deistler, 2008.)

Using Theorem 2, under the assumption that the entries of $D\left(L\right)$ are rational functions, representation (9) can generically be rewritten as

$$\begin{array}{lll}{x}_{it}\hfill & =\hfill & {\chi }_{it}+{\xi }_{it}\hfill \\ {\chi }_{it}\hfill & =\hfill & {\lambda }_i{F}_t\hfill \\ G(L)F_t\hfill & =\hfill & R{u}_t\hfill \end{array}$$(10)

where $R$ is $r\times q$ and $\text{G}\left(\text{L}\right)$ is $r\times r$, stable and finite-degree. Therefore, if an estimate of $F_t$ is available, the shocks ${\text{u}}_t$ can be estimated by means of a singular VAR for $F_t$.

The matrix $R$ has generically full rank q. As a consequence, generically the vector $u_t$ belongs to the space spanned by $F_t{}_{–s},s\ge 0$; that is, by definition, $u_t$ is generically fundamental for $F_t$.

Example 3

The following elementary example, in which $r=2$ and $q=1$, can illustrate this result:

$$\begin{array}{c}{F}_{1t}=u_t+2u_{t-1}=(1+2L)u_t\\ F_{2t}=u_t+3u_{t-1}=(1+3L)u_t.\end{array}$$(11)

As the polynomials $1+2L$ and $1+3L$ are not invertible, $u_t$ is not fundamental for $F_{1t}$ or for $F_{2t}$. However, multiplying the first equation by 3, the second by 2, and subtracting: $u_t=3F_{1t}–2F_{2t}$, so that $u_t$ belongs to space spanned by current and past values of the vector $F_t$, that is, $u_t$ is fundamental for the vector $F_t$.

This section on the dynamics of the factor model concludes by observing that no special assumption has been imposed on the autocovariance of the idiosyncratic components ${\xi }_{it}$.

Principal Components

In general, in a model with $r$ factors, $r\ge 1$, $r$ averages are needed:

$$x_{nt}^j={\displaystyle \sum _{i=1}^n}{a}_{ij,n}{x}_{it},$$

$j=1,\mathrm{...},r$, such that:

A very convenient solution of this problem consists in setting $x_{nt}^j=({\mu }_{nj}^x{)}^{-1}{P}_{nj,t}^x,j=1,\dots ,r,$ where $P_{nj,t}^x$ is the j-th principal component of the vector $x_{nt}$. Precisely,

$$P_{nj,t}^x=p_{nj}^x{x}_{nt}=p_{nj\mathrm{,1}}^x{x}_{1t}+\cdots +p_{nj,n^{X_{nt}}}^x,$$

where $p_{nj}^x$ is a row eigenvector of ${\Gamma }_{n0}^x$ corresponding to the j-th eigenvalue, such that ${\displaystyle {\sum }_{i=1}^n}{(p_{nj,i}^x)}^2=1$

Observation 5

In Lippi (2018) both eigenvectors of the spectral density of the vector $x_{nt}$, are considered, that is, dynamic eigenvectors and eigenvectors of the variance-covariance matrix of $x_{nt}$ (i.e., static eigenvectors). In that article, to avoid confusion the static eigenvectors are denoted as $p_{nj}^{S,x}$. As no confusion can arise here, $p_{nj}^x$. is used.□

Regarding (i), as is well known, the variance-covariance matrix of the principal-component vector $(P_{n1,t}^x\cdots P_{nr,t}^x{)}^{\prime }$ is the $r\times r$ diagonal matrix whose entry $\left(k,k\right)$ is ${\mu }_{nk}^x$. As a consequence, the variance-covariance matrix of

$$\begin{array}{l}{y}_{nt}=(({\mu }_{n1}^x{)}^{-1}{P}_{n\mathrm{1,}t}^x\cdots ({\mu }_{nr}^x)-1P_{nr,t}^x{)}^{\prime }\\ =(({\mu }_{n1}^x{)}^{-1}{p}_{n1}^x{x}_{nt}\cdots ({\mu }_{nr}^x)-1p_{nr}^x{x}_{nt}{)}^{\prime }\end{array}$$

is the identity matrix $I_r$ and is therefore bounded away from singularity. Using ${\mu }_{nj}^x\to \infty$ as $n\to \infty$ for $j=1,\mathrm{...},r$, and ${\mu }_{n1}^{\xi }\le W$ for all $n$, it is easy to show (ii), that is, that

$$\underset{n\to \infty }{{\displaystyle \lim }}{({\mu }_{nj}^x)}^{-1}{p}_{nj}^x{\xi }_{nj}=0,$$

for $j=1,\dots ,r$.

In conclusion, the first $r$ normalized principal components are an orthonormal basis “converging” in mean square to the space spanned by the factors $F_{jt},j=1,\mathrm{...},r$. (the average of the idiosyncratic components by means of each of the normalized eigenvectors tends to zero in mean square). This is the rationale for expressions like “the normalized principal components converge to the factors.” Though suggestive, such statements should be taken with extreme care. Indeed, as seen in Observation 4, any basis in the factor space provides a vector of static factors.

Estimating the Factors and the Loadings

Principal components, as presented earlier, are unfeasible as estimators of the factors for two reasons. First, an estimate of ${\Gamma }_{n0}^x$, call it ${\widehat{\Gamma }}_{n0}^x$ is observed, which depends on the observed $\left(n,T\right)$ sample of the processes $x_{it}$. Second, the number $r$ of factors is not known.

The number of factors is preliminary to estimation of the principal components. The literature on the determination of $r$ starts with Bai and Ng (2002), who use an information criterion with the variance explained by the factors on one hand, a penalty for adding factors on the other. They show consistency of the criterion as $n$ and $T$ tend to infinity. Improvement and alternatives to Bai and Ng’s criterion are Onatski (2010), Alessi, Barigozzi, and Capasso (2010), and Ahn and Horenstein (2013). A test for the number of factors is proposed in Onatski (2009).

Regarding the number of dynamic factors $q$, consistent criteria are given in Hallin and Liška (2007), Amengual and Watson (2007), and Bai and Ng (2007).

Once $r$ has been determined, using the estimated principal components of the x’s (by means of the eigenvectors of ${\widehat{\Gamma }}_{n0}^x$), estimates of the factors and the loadings are obtained. Standard consistency, that is, convergence in probability, as $n$ and $T$ tend to infinity, is proven in Stock and Watson (2002a, 2002b) and Bai and Ng (2002). In particular, Bai and Ng obtain the rate of convergence $\text{max}\left(T^{–1},n^{–1}\right)$ for the squared distance between the estimated factors and the factor space.

Once the factors have been estimated and $q$ has been determined, estimation of the last equation in (10) provides an estimate of the dynamic shocks $u_t$ and $G\left(L\right)$, thus completing the estimation of the high-dimensional dynamic factor model.

Doz, Giannone, and Reichlin (2011) construct a consistent Gaussian quasi-maximum likelihood estimator as an alternative to principal components. Given $n$, they estimate a factor model by maximum likelihood under the assumption that the idiosyncratic components are mutually orthogonal. They show that the bias resulting by “forcing” this orthogonality into the finite-n model becomes negligible as $n$ tends to infinity.

Example 4

Suppose that $x_t$ is the rate of change of aggregate productivity and that $x_t$ evolves according to the following equation:

$$x_t=p_1{v}_t+p_2{v}_{t-1},$$(17)

where $u_t$ is the shock to technology, $p_i>0,i=1,2$ and $p_1+p_2=1$. Thus $u_t$ determines the increase of productivity $p_1{u}_t$ at time $t$ and the increase $p_2{u}_t$ at time $t+1$. Equation (17) has the interpretation of a learning-by-doing process taking two time periods to complete. Rewriting (17) as

$$x_t=V_t+P{V}_{t-1},$$

where $V_t=p_1{v}_t$ and $P=p_2/p_1$, one has a standard MA(1). If $P<1$ the MA(1) is invertible and estimating an autoregression for $x_t$ would produce an approximation of the structural shock $V_t$. On the other hand, if $P>1$ an autoregression would not produce an approximation of $V_t$. In the first case $V_t$ is fundamental for $x_t$, and in the second case $V_t$ is nonfundamental. On the other hand, the econometrician observes the autocovariances of $x_t$, so that he or she only knows that $x_t$ is an MA(1). Therefore, either additional information is available on $p_1$ and $p_2$, that is on $P$, or the model is not identified. Moreover, the usual practice of estimating autoregressions produces an approximation of the structural shock $V_t$ only if the learning-by-doing process has $p_1>p_2$.□

In conclusion, under the assumption that the idiosyncratic components of the large macroeconomic variables are measurement errors, the dynamic factor model produces estimates of the “true” underlying variables ${\chi }_{it}$, of the structural shocks and impulse-response functions. In particular, singularity of the vector ${\chi }_t$ and Theorem 2 imply that the shocks estimated from representation (10) differ from the structural shocks by a non-singular matrix.

For the comparison of SVAR analysis and structural analysis based on high-dimensional dynamic factor models, see Stock and Watson (2005) and Forni et al. (2009).