Testing for Alpha in Linear Factor Pricing Models with a Large Number of Securities^*

Abstract

This article considers tests of alpha in linear factor pricing models when the number of securities, N, is much larger than the time dimension, T, of the individual return series. We focus on class of tests that are based on Student’s t-tests of individual securities which have a number of advantages over the existing standardized Wald type tests, and propose a test procedure that allows for non-Gaussianity and general forms of weakly cross-correlated errors. It does not require estimation of an invertible error covariance matrix, it is much faster to implement, and is valid even if N is much larger than T. We also show that the proposed test can account for some limited degree of pricing errors allowed under Ross’s arbitrage pricing theory condition. Monte Carlo evidence shows that the proposed test performs remarkably well even when T = 60 and N = 5000. The test is applied to monthly returns on securities in the S&P 500 at the end of each month in real time, using rolling windows of size 60. Statistically significant evidence against Sharpe–Lintner capital asset pricing model and Fama–French three and five factor models are found mainly during the period of Great Recession (2007M12–2009M06).

This article is concerned with testing for the presence of alpha in linear factor pricing models (LFPMs) such as the capital asset pricing model (CAPM) due to Sharpe (1964) and Lintner (1965), or the arbitrage pricing theory (APT) model due to Ross (1976), when factors are observed and the number of securities, N, is quite large relative to the time dimension, T, of the return series under consideration. There exists a large literature in empirical finance that tests various implications of Sharpe–Lintner model. Cross-sectional as well as time-series tests have been proposed and applied in many different contexts. Using time-series regressions, Jensen (1968) was the first to propose using standard t-statistics to test the null hypothesis that the intercept, α_i, in the ordinary least squares (OLS) regression of the excess return of a given security, i, on the excess return of the market portfolio is zero.¹

However, when a large number of securities are under consideration, due to dependence of the errors across securities in the LFPM regressions, the individual t-statistics are correlated which makes controlling the overall size of the test problematic. Gibbons, Ross, and Shanken (1989, GRS) propose an exact multivariate version of the test which deals with this problem if the CAPM regression errors are Gaussian and N < T. This is the standard test used in the literature, but its application has been confined to testing the market efficiency of a relatively small number of portfolios, typically 20−30, using monthly returns observed over relatively long time periods. The use of large T as a way of ensuring that N < T is also likely to increase the possibility of structural breaks in the β′s that could in turn adversely affect the performance of the GRS test.

Recently, there has been a growing body of finance literature which uses individual security returns rather than portfolio returns for the test of pricing errors. Ang, Liu, and Schwarz (2020) show that the smaller variation of beta estimates from creating portfolios may not lead to smaller variation of cross-section regression estimates. Cremers, Halling, and Weinbaum (2015) examine the pricing of both aggregate jump and volatility risk based on individual stocks rather than portfolios. Chordia, Goyal, and Shanken (2017) advocate the use of individual securities to investigate whether the source of expected return variation is from betas or security-specific characteristics.

Out of the two main assumptions that underlie the GRS test, the literature has focused on the implications of non-normal errors for the GRS test, and ways of allowing for non-normal errors when testing $αi=0$⁠. Affleck-Graves and Mcdonald (1989) were among the first to consider the robustness of the GRS test to non-normal errors who, using simulation techniques, find that the size and power of GRS test can be adversely affected if the departure from non-normality of the errors is serious, but conclude that the GRS test is “reasonably robust with respect to typical levels of nonnormality.” (p. 889). More recently, Beaulieu, Dufour, and Khalaf (2007, BDK) and Gungor and Luger (2009) have proposed tests of $αi=0$ that allow for non-normal errors, but retain the restriction N < T. BDK develop an exact test which is applicable to a wide class of non-Gaussian error distributions, and use Monte Carlo simulations to achieve the correct size for their test. GL propose two distribution-free nonparametric sign tests in the case of single factor models that allow the error distribution to be non-normal but require it to be cross-sectionally independent and conditionally symmetrically distributed around zero.

Our primary focus in this article is on multivariate tests of $H0:αi=0,$ for $i=1,2,…,N$⁠, when N > T, while allowing for non-Gaussian and weakly cross-sectionally correlated errors. The latter condition is required for consistent estimation of the error covariance matrix, V, when N is large relative to T. In the case of LFPM regressions with weakly cross-sectionally correlated errors, consistent estimation of V can be achieved by adaptive thresholding which sets to zero elements of the estimator of V that are below a given threshold. Alternatively, feasible estimators of V can be obtained by Bayesian or classical shrinkage procedures that scale down the off-diagonal elements of V relative to its diagonal elements.²Fan, Liao, and Mincheva (2011, 2013) consider consistent estimation of V in the context of an approximate factor model. They assume V is sparse and propose an adaptive thresholding estimator of V, which they show to be positive definite with satisfactory small sample properties. Fan, Liao, and Yao (2015) consider a standardized Wald (SW) test based on the estimator of V proposed by Fan, Liao, and Mincheva (2013) and derive the conditions under which the SW test of H₀ can be asymptotically justified. Gungor and Luger (2016, GL) propose a simulation-based approach for testing pricing errors. They claim that their test procedure is robust against non-normality and cross-sectional dependence in the errors. Gagliardini, Ossola, and Scaillet (2016, GOS) develop two-pass regressions of individual stock returns, allowing time-varying risk premia, and propose a SW test. Lan, Feng, and Luo (2018) use random projection of the N security returns onto a smaller number of portfolios to circumvent the high-dimensional problem when testing for alphas, but require N and T to be of the same order of magnitude. Raponi, Robotti, and Zaffaroni (2019) propose a test of pricing error in cross-section regression for fixed number of time-series observations. They use a bias-corrected estimator of Shanken (1992) to standardize their test statistic. Ma et al. (2020) employ polynomial spline techniques to allow for time variations in factor loadings when testing for alphas. Feng et al. (2022) propose a max-of-square type test of alphas instead of the average used in the literature, and recommend using a combination of the two testing procedures. As noted by He et al. (2021), Bai and Saranadasa (1996, BS) consider yet another SW type test which requires N and T to be of the same order of magnitude.

In this article, we develop a test statistic that initially ignores the off-diagonal elements of V and base the test of H₀ on the average of the squared t-ratios for $αi=0$⁠, over $i=1,2,…,N$⁠. This idea was originally proposed in the working paper version of this article, independently of a similar approach subsequently followed by GOS. Despite the similarity of the two tests, as will be seen, our version of the test performs much better for all combinations of N and T considered in the literature, and delivers excellent size and power even if N is very large (around 5000), in contrast to other tests that tend to over reject as N is increased relative to T. We are also able to establish the asymptotic distribution of proposed test under much weaker conditions and without resorting to high level assumptions.³ We achieve this by making corrections to the numerator of the test statistic to ensure that the test is more accurately centered, and correct the denominator of the test statistic to allow for the effects of non-zero off-diagonal elements of the underlying error covariance matrix. The correction involves consistently estimating $N−1Tr(R2)$⁠, where $R=(ρij)$ is the error correlation matrix. The estimation of $N−1Tr(R2)=N−1∑i=1N ∑j=1Nρij2$ is subject to the curse of dimensionality which we address by using the multiple testing (MT) threshold estimator, $R˜$⁠, recently proposed by Bailey, Pesaran, and Smith (2019, BPS). We show that consistent estimation of $N−1Tr(R2)$ can be achieved under a more general specification of R when compared with tests that require a consistent estimator of the full matrix, R. We are able to establish that the resultant test is applicable more generally and continues to be valid for a wider class of error covariances, and holds even if N rises faster than T. The proposed test is also corrected for small sample effects of non-Gaussian errors, which is of particular importance in finance. We refer to this test as Jensen’s α test of LFPM and denote it by $J^α$⁠. The test can also be viewed as a robust version of a SW test, in cases where the off-diagonal elements of V become relatively less important as $N→∞$⁠. Further, the implementation of the $J^α$ test is computationally less demanding, since it does not involve estimation of an invertible high-dimensional error covariance matrix.

We note that the $J^α$ test is not the first one which is based on the standardized squared t-ratio for $αi=0$⁠. As discussed in He et al. (2021), Srivastava and Du (2008, SD) propose standardized squared t-ratio, using a different standardization from ours. As will be seen below, their standardization results in serious size distortion when N is larger than T (see the SD test discussed in Section 5). Also, Hwang and Satchell (2014) proposed a simulation-based test, using average of the squared t-ratios.

Our assumption regarding the sparsity of V advances on Chamberlain’s (1983) approximate factor model formulation of the asset model, where it is assumed that the largest eigenvalue of V (or R) is uniformly bounded in N (Chamberlain, 1983, p. 1307). We relax this assumption and allow the maximum column sum matrix norm of R to rise with N but at a rate slower than $N$⁠, while controlling the overall sparsity of R by requiring $N−1Tr(R2)$ to be bounded in N. In this way, we are able to allow for two types of cross-sectional error dependence: one due to the presence of weak common factors that are not sufficiently strong to be detectable using standard estimation techniques, such as principal components and another due to the error dependence that arises from interactive and spill-over effects.

We establish that under the null hypothesis $H0: αi=0,$ the $J^α$ test is asymptotically distributed as N(0, 1) for T and $N→∞$ jointly, so long as $N/T2→0, mN=‖R‖1=O(Nδρ)$⁠, $0≤δρ<1/2$⁠, and $N−1Tr(R2)$ is bounded in N. The test is also shown to have power against alternatives that rise in $N1/2T$⁠. We consider the implications of allowing for pricing errors on the asymptotic properties of the $J^α$ test and show that testing H₀ still allows for some very limited degree of non-zero pricing errors. The proofs are quite involved and in some parts rather tedious. For the purpose of clarity, we provide statements of the main theorems with the associated assumptions in the article, but relegate the mathematical details to the Appendix.

Small sample properties of the $J^α$ test are investigated using Monte Carlo experiments designed specifically to match the distributional features of the residuals of Fama–French three factor regressions of individual securities in the Standard & Poor 500 (S&P 500) index. We consider the comparative test results for the following nine sample size combinations, $T∈{60,120,240}$ and $N={50,100,200}$⁠. The $J^α$ test performs well for all sample size combinations with empirical size very close to the chosen nominal value of 5%, and satisfactory power. Comparing the size and power of the $J^α$ test with the GRS test in the case of experiments with N < T for which the GRS statistics can be computed, we find that the $J^α$ test has higher power than the GRS test in most experiments. This could be due to the non-normal errors adversely affecting the GRS test, as reported by Affleck-Graves and Mcdonald (1989, 1990). In addition, the $J^α$ test outperforms the test proposed by GOS as well as the SW test of Fan, Liao, and Yao (2015) and the SD test of Srivastava and Du (2008). The $J^α$ test also outperforms the simulation-based $Fmax$ test of Gungor and Luger (2016) and the BS test of Bai and Saranadasa (1996), which are shown to be substantially undersized across the various designs, and has lower power when compared with the $J^α$ test. Further, we carried out additional experiments using much larger values of N, namely $N=500,1000,$ 2000, and 5000, while keeping T at 60, 120, and 240. We only considered the $J^α$ test for these experiments and found no major evidence of size distortions even for the experiments with T = 60 and N = 5000.

Encouraged by the satisfactory performance of the $J^α$ test even in cases where N is much larger than T, we applied the test to monthly returns on the securities in the S&P 500 index using rolling windows of size T = 60 months. The survivorship bias problem is minimized by considering the sample of securities included in the S&P 500 at the end of each month in real time. We report the $J^α$ test results for CAPM, three and five Fama–French factor models over the period September 1989 to April 2018, and the three sub-periods: (1) the Asian financial crisis (1997M07–1998M12), (2) the Dot-com bubble burst (2000M03–2002M10), and (3) the Great Recession (2007M12–2009M06) periods. We find that the $J^α$ test rejects $H0:αi=0$⁠, mainly during periods of major financial disruptions, particularly the period of Great Recession, with the GOS test rejecting the null for most periods, largely due to its tendency to over-reject when T is short relative to N.

The outline of the rest of the article is as follows. Section 1 sets out the LFPM, formulates the null hypothesis that underlies the tests for alphas which allow for pricing errors and weak latent or missing factors. Section 2 introduces the estimates of alpha and derives the GRS test as a point of departure for dealing with the case where N > T. Section 3 proposes the $J^α$ test for large N panels and derives its asymptotic distribution, and Section 4 summarizes the main theoretical results. Section 5 reports on small sample properties of $J^α$⁠, GRS, GOS, SW, $Fmax$⁠, BS and SD tests, using Monte Carlo techniques. Section 6 presents the empirical application. Section 7 concludes. The proofs of the main theorems are provided in the Appendix, and the lemmas which are used for the proofs, as well as the additional Monte Carlo evidence and the detailed discussion on data sources, are provided in the Supplementary Material.

Notations: We use K and c to denote finite and small positive constants. If ${ft}t=1∞$ is any real sequence and ${gt}t=1∞$ is a sequences of positive real numbers, then $ft=O(gt)$⁠, if there exists a positive finite constant K such that $|ft|/gt≤K$ for all t. $ft=o(gt)$ if $ft/gt→0$ as $t→∞$⁠. If ${ft}t=1∞$ and ${gt}t=1∞$ are both positive sequences of real numbers, then $ft=⊖(gt)$ if there exists $T0≥1$ and positive finite constants C₀ and C₁, such that $inft≥T0(ft/gt)≥C0$ and $supt≥T0(ft/gt)≤C1$⁠. For a N × N matrix $A=(aij)$⁠, the minimum and maximum eigenvalues of matrix A are denoted by $λmin(A)$ and $λmax(A)$⁠, respectively, its trace by $Tr(A)$⁠, its maximum absolute column and row sum matrix norms by $‖A‖∞=supi∑j=1N|aij|$⁠, and, $‖A‖1=supj∑i=1N|aij|$⁠, respectively, its Frobenius and spectral norms by $‖A‖F=Tr(A′A)$⁠, and $‖A‖=λmax1/2(A′A)$⁠, respectively. For a $N×1$ dimensional vector, $α, ‖α‖=(α′α)1/2$⁠.

1 The LFPM and APT Restrictions

The idiosyncratic errors, η_it, are also allowed to be weakly cross-correlated. Specifically, we assume that $ηt=(η1t,η2t,….,ηNt)′=Qηεη,t$⁠, where $εη,t=(εη,1t,εη,2t,….,εη,Nt)′, {εη,it}$ are IID processes over i and t, with zero means , unit variances, $γ2,εη=E(εη,it4)−3$⁠, and $supi,tE(|εη,it|8+c)≤K<∞$⁠, for some c > 0. We denote the correlation matrix of $ηt$ by $Rη=(ρη,ij)$ and note that $Rη=QηQη′$⁠. To ensure that $ut=(u1t,u2t,…,uNt)′$ is weakly cross-correlated, we require that k, the number of weak factors, is finite, and $‖Rη‖1≤‖Qη‖1‖Qη‖∞≤K$⁠. The error specification in Equation (8) is quite general and allows for weak latent common factors as well as network and spatial error cross dependence. We note that common factors cannot substitute for network dependence and allowing for both types of dependence in the errors is important.

2 Preliminaries and the GRS Test

As noted in the introduction, the single most important limiting feature of the GRS and other related tests proposed in the literature is the requirement that T must be larger than N. Due to this, in applications of the GRS test, individual securities are grouped into (sub)portfolios and the GRS test is then typically applied to 20–30 portfolios over relatively long time periods. However, the market efficiency hypothesis implies that $αi=0$ for all individual securities which form the market portfolio, and it is clearly desirable to develop tests which permit N to be much larger than T. This is even more so if we would like to minimize the adverse effects of possible time variations in the $βi$’s.

It is also worth bearing in mind that the GRS test does not impose any restrictions on V, which is possible only because N is taken to be fixed as $T→∞$⁠. Large T is required to take account of non-Gaussian errors. While in the context of the approximate factor models advanced in Chamberlain (1983), the errors are at most weakly correlated, which places restrictions on the off-diagonal elements of V and its inverse. In addition, such restrictions are also statistically important in order to estimate V and its inverse when N > T. The test developed in this article for a large number of individual securities is therefore clearly different from the GRS test, both theoretically and statistically. Furthermore, as we shall see below, a test that exploits restrictions implied by the weak cross-sectional correlation of the errors is likely to have much better power properties than the GRS test that does not make use of such restrictions. Finally, being a multivariate F-test, the power of the GRS test is primarily driven by the time dimension, T, while for the analysis of a large number of assets or portfolios we need tests that have the correct size and are powerful for large N.

3 Large N Tests of Alpha in LFPMs

Under Gaussianity and $H0:α=0, SWv→dN(0,1)$ as $N→∞$⁠. To construct tests of large N panels, a suitable estimator of V is required. But as was noted in the introduction this is possible only if we are prepared to impose restrictions on the structure of V. In the case of LFPM regressions where the errors are at most weakly cross-sectionally correlated, this can be achieved by adaptive thresholding which sets to zero elements of V that are sufficiently small, or by use of shrinkage type estimators that put a substantial amount of weight on the diagonal elements of the shrinkage estimator of V. Fan, Liao, and Mincheva (2011, 2013) consider consistent estimation of V in the context of an approximate factor model. They assume V is sparse and propose an adaptive threshold estimator, denoted as $V^POET$⁠, which they show to be positive definite with satisfactory small sample properties. We refer to the feasible SW test statistic which replaces V with $V^POET$ as SW_POET test.⁷

3.1 A ^{^}J_α Test for Large N Securities

To make the $Jα$ test operational, we need to provide a large N consistent estimator of $θN2$⁠. Second, we need to show that, despite the fact that $Jα$ test is standardized assuming t_i has a standard t distribution, the test will continue to have satisfactory small sample performance even if such an assumption does not hold due to the non-Gaussianity of the underlying errors. More formally, in what follows we relax the Gaussianity assumption and assume that $ut=Qεt$⁠, where Q is an N × N invertible matrix$, εt=(ε1t,ε2t,…,εNt)′$⁠, and ${εit}$ is an IID process over i and t, with means zero and unit variances, and for some c > 0, $E(|εit|8+c)$ exists, for all i and t. Then, $E(utut′)=V=(σij)=QQ′$ and V is an N × N symmetric positive definite matrix, with $λmin(V)≥c>0$⁠. We allow for cross-sectional error heteroskedasticity, but assume that the errors are homoskedastic over time. This assumption can be relaxed by replacing the assumption of error independence by a suitable martingale difference assumption. This extension will not be attempted in this article.⁹

3.2 Sparsity Conditions on Error Correlation Matrix

3.3 Non-Gaussianity

3.4 Allowing for Error Cross-Sectional Dependence

3.5 Survivorship Bias

When applying the $J^α$ test, it is important to minimize the effect of survivorship bias. To this end, the GRS type tests of alpha consider a relatively small number of portfolios over a relatively large time period to achieve sufficient power. By making use of portfolios rather than individual securities, the GRS test is less likely to suffer from survivorship bias. By comparison, tests such as the $J^α$ test can suffer from the survivorship bias due to the fact that they are applied to individual securities directly and obtain power from increases in N as well as from T. To deal with the survivorship bias, we propose that the $J^α$ test is applied recursively to securities that have been trading for at least T time periods (days or months) at any given time t. The set of securities included in the $J^α$ test varies over time and dynamically takes account of exit and entry of securities in the market. The number of securities, $Nτ$⁠, used in the test at any point of time, $τ,$ depends on the choice of T, and declines as T is increased. It is clearly important that a balance is struck between T and $Nτ$⁠. Since the $J^α$ test is applicable even if N is much larger than T, and given that the power of the $J^α$ test rises both in N and T, then it is advisable to set T such that $minτ(Nτ)/T2$ is sufficiently small. This procedure is followed in the empirical application discussed in Section 6, where we set T = 60 and end up with $Nτ$ in the range $[464,487]$⁠, giving $minτ(Nτ)/T2=0.12$⁠.

3.6 Other Existing Tests

3.6.1 The GOS test

3.6.2 The GL F_max

They consider various versions of the test, and recommend the use of the maximum test which we will consider in our Monte Carlo exercise. The authors claim that their resampling test procedure is robust against non-normality and cross-sectional error dependence.¹⁹ Their test effectively makes use of wild bootstrap resampling aimed at preserving the sample residual cross-sectional correlations, and deals with nuisance parameters by the introduction of a bounds testing procedure.

3.6.3 The BS and SD tests in He et al. (2021)

4 Summary of the Main Theoretical Results

In this section, we provide the list of assumptions and a formal statement of the theorems for the size and power of the proposed $J^α$⁠. First, we state the assumptions required for establishing the results. 

 

• The unobserved factors $vt$ are serially independent and the $k$ elements are independent of each other, such that $vt∼IID(0,Ik), γ2,v=E(vst4)−3$⁠, and $sups,tE(vst8+c)<K$⁠, for some c > 0. The factor loadings, $γis$ for $s=1,2,…,k$⁠, are bounded, $supi,s|γis|<K$⁠, and the factors, $vt$⁠, are weak in the sense that

  $sups∑i=1N|γis|=O(Nδγ), with 0≤δγ<1/2.$

  (53)

• For any i and j, the T pairs of realizations, ${(ηi1,ηj1),(ηi2,ηj2),…,(ηiT,ηjT)},$ are independent draws from a common bivariate distribution with mean $E(ηit)=0, Var(ηit)=ση,ii, 0<c<ση,ii≤K$⁠, and the covariance $E(ηitηjt)=ση,ij$⁠.

Our main theoretical results are set out in the following theorems. The proofs of these theorems are provided in the Appendix, and necessary lemmas for the proofs are given in the Supplementary Material. 

        

5 Small Sample Evidence Based on Monte Carlo Experiments

We examine the finite sample properties of the $J^α$ test by Monte Carlo experiments, and compare its performance to the existing tests, which are discussed in Section 3.6. Specifically, we consider the GRS test, the GOS test, and a feasible version of the SW test, as well as the distribution-free $Fmax$ test and the BS and SD tests, which are defined by Equations (3), (47), (19), (48), (49), and (50), respectively. Computational details of these tests are given in Section M1.1 of the Supplementary Material.

5.1 Monte Carlo Designs and Experiments

To calibrate the empirical FF3 model, we estimated it using S&P500 security level monthly excess return for 120 months ending on April 2018. We chose the series with the full sample period, which left 457 securities. The results are summarized in Table 1.

We generate the factor loadings as $IIDU(0.3,1.8)$ for the market factor, $IIDU(−1.0,1.0)$ for the HML factor, and $IIDU(−0.6,0.9)$ for the SMB factor. In this way, we ensure that the means and standard deviations of the betas match their empirical counterparts and sufficient ranges of the estimates of $β′s$ reported in Table 1 for the FF3 model are covered in the experiments.

All the N return series are generated from $t=−49,−48,….0,1,2,….,T$⁠, with $fℓ,−50=0$ and $hℓ,−50=1$ for $ℓ=1,2,3$⁠. The first 50 observations are dropped to minimize the effects of the initial values and observations $rit, ft=(f1t,f2t,f3t)′$⁠, for $t=1,2,…,T$ are used in the MC experiments. Further details are provided in the Supplementary Material.

For the scenario called “Power 1,” we set $λ=μ$⁠, and generated α_i as $αi=ϖi∼IIDN(0,1)$ for $i=1,2,…,Nα$ with $Nα=⌊Nδα⌋$⁠; $αi=0$ for $i=Nα+1,Nα+2,…,N$⁠. We considered the values $δα=0.7$⁠. In another scenario called “Power 2,” we assume there are no pricing errors and set $ϖi=0$ for all i, but consider the case where $λ−μ=c(2.92,−0.63,−9.96)′$⁠, that match the estimates reported in Table 1 of GOS (p. 1011) for c = 1. To make the power of the tests for “Power 2” comparable for “Power 1,” we set c = 0.1. We do not consider the case both $λ≠μ$ and $ϖi≠0$⁠, as it is clear that in this case higher power will be achieved.

All combinations of T = 60, 120, 240 and N = 50, 100, 200 (and 500, 1000, 2000, 5000 for the $J^α$ test) are considered. All tests are conducted at the 5% significance level and all experiments are based on R = 2000 replications. To compute $ρ˜N,T2$ which enters the denominator of the $J^α$ statistic, given by Equation (46), we consider $p={0.05,0.1}$ and $δ={1,2}$⁠. The results are very insensitive to the choice of the values of $(p,δ)$ and the case for $(p,δ)=(0.05,1)$ is reported. It is worth noting that that the choice of p when computing $ρ˜N,T2$ is not governed or affected by the choice of the nominal size of the $J^α$ test.

5.2 Size and Power

Table 2 reports the size and power of the $J^α$⁠, GRS, GOS, SW, $Fmax$⁠, BS, and SD tests in the case of normal errors, under various degrees of cross-sectional error correlations, as measured by the exponent, $δγ$⁠.

First, consider Panel A of Table 2 which reports the size of the tests. The GRS test when applicable (namely when T > N) is an exact test and has the correct size. The empirical size of the $J^α$ test is also very close to the 5% nominal level for all combinations of N and T. Even when N = 200 and $δγ=0.5$⁠, the size of the $J^α$ test lies in the range 5.9–6.4% for different values of T. In contrast, both GOS and SW tests grossly over-reject the null hypothesis, and the degree of the over-rejection becomes more serious as N increases for a given T. In line with the discussion in Section 3.4, the size distortion of these tests is mitigated when T increases. The $Fmax$ test severely under-rejects the null hypothesis, with the size ranging between 0.0% and 0.4%. Although less pronounced than the $Fmax$ test, the BS test is very conservative and the size steadily drops as T (and N) rises. Again, although less pronounced than the GOS and SW tests, the SD test tends to over-reject the null hypothesis and the degree of the over-rejection becomes more serious as N increases for a given T.

The power of the tests based on the “Power 1” design is reported in Panel B of Table 2. The power of $J^α$ test is substantially higher than that of the GRS test. This is in line with our discussion at the end of Section 1, and reflects the fact that GRS assumes an arbitrary degree of cross-sectional error correlations and thus relies on a large time dimension to achieve a reasonably high power. In contrast, the power of the $J^α$ test is driven largely by the cross-sectional dimension. The power comparison of the GOS, SW, and SD tests with the $J^α$ test seems inappropriate, given their large size-distortions. Having said this, it is perhaps remarkable that the power of the $J^α$ test is comparable to the unadjusted power of the GOS, SW_POET, and SW_LW tests. The power of the $Fmax$ and BS tests is uniformly lower than the power of the $J^α$ test, likely due to the conservative nature of these tests. The power of the tests based on the “Power 2” design is reported in Panel C of Table 2. The properties of the tests with the “Power 2” design reported in Panel C of Table 2 are qualitatively very similar to those of the “Power 1” design. A detailed discussion of Table 2 is therefore omitted.

We now consider the case in which the errors are non-normal. The size results are summarized in Table 3. The results show that the size of the $J^α$ test and the GRS test, as well as the $Fmax$⁠, BS, and SD tests, is hardly affected by non-normality. The over-rejection of the GOS and SW tests tends to be somewhat magnified by non-normality.

Furthermore, the behavior of the test statistics is examined under the same DGP as that examined in Table 2, except that a spatial autoregressive component was incorporated into the error generation process. The results with such mixed factor-spatial errors are reported in Table 4. As can be seen, the size of the $J^α$ test and GRS test is well controlled, with a slight over-rejection for T = 60, which disappears when T is increased to 120. In contrast, the size distortion of GOS and SW seems to be amplified with this design. The size properties of the $Fmax$⁠, BS, and SD tests remain similar to those in Table 2.

Since the autoregressive conditional heteroskedasticity is commonly found in security returns, the effect of cross-sectionally correlated errors with GARCH(1,1) processes is also investigated. The size properties of the tests are summarized in Table 5. The results are almost identical to those using unconditionally time-series homoskedastic (but cross-sectionally heteroskedastic) errors reported in Table 2. This is to be expected as the LFPM is a static model and unconditional homoskedastic GARCH errors do not affect our theoretical results.

The experimental results so far confirm that the finite sample performance of the $J^α$ test is superior to the other tests we have considered. In the light of these promising results, we further investigate the properties of J-alpha tests, in particular the sensitivity of the choice of the values for {$δ,p$} and the effectiveness of the standardization employed by the $J^α$⁠.

First, we examine the sensitivity of the test to the choice of the value of {$δ,p$}. As mentioned, the $J^α$ we have considered employs δ = 1 and p = 0.1. To check whether this choice is appropriate, in the next experiment, we consider four combinations of {$δ,p$} using $δ=1,2, p=0.05,0.01$⁠. Table 6 summarizes the size and power results. As can be seen, the choice of p has little effect on the size and power characteristics. Meanwhile, the performance of the test is slightly sensitive to the choice of δ, but this effect quickly disappears as T increases. These experimental results support the use of the $J^α$ test with δ = 1 and p = 0.1.

Finally, an experiment was conducted to check the effectiveness of the standardization employed in the $J^α$⁠. In particular, we check the effectiveness of the centering $ti2−v/(v−2)$ employed by the $J^α$ test compared with $ti2−1$ employed by GOS, and the usefulness of estimating the cross-correlation of $ti2$ with the MT estimator $ρ˜N$⁠, respectively. For this purpose, two J-alpha test variants, $J˜α$ and $Jα(0)$⁠, are considered on top of the $J^α$ statistic. $J˜α$ is identical to $J^α$⁠, but replaces $ti2−v/(v−2)$ by $ti2−1$⁠. The second statistic, $Jα(0)$⁠, sets $ρ˜N$ equal to zero (i.e., does not control for cross-correlation). In the present experiment, to investigate the behavior of the $J^α$ test in more challenging environments, N is considered with larger values, that is, $N=500, 1000, 2000$ and 5000, while T is set to 60, 120, and 240 as before. The results are reported in Table 7, which reveal that the centering using $v/(v−2)$ as well as the control of error cross-correlations by the MT estimator play a very significant role in controlling the size of the test for large N (and large T as shown in Panel A of Table 2).

6 Empirical Application

6.1 Data Description

We consider the application of our proposed $J^α$ test to the securities in the S&P 500 index of large cap U.S. equities market. Since the index is primarily intended as a leading indicator of U.S. equities, the composition of the index is monitored by S&P to ensure the widest possible overall market representation while reducing the index turnover to a minimum. Changes to the composition of the index are governed by published guidelines. In particular, a security is included if its market capitalization currently exceeds US$5.3 billion, is financially viable, and at least 50% of their equity is publicly floated. Companies that substantially violate one or more of the criteria for index inclusion, or are involved in merger, acquisition, or significant restructuring are replaced by other companies.

In order to take account for the change to the composition of the index over time, we compiled returns on all the 500 securities that constitute the S&P 500 index each month over the period January 1984 to April 2018. The monthly return of security i for month t is computed as $rit=100(Pit−Pi,t−1)/Pi,t−1+DYit/12$⁠, where P_it is the end of the month price of the security and DY_it is the percent per annum dividend yield on the security. Note that index i depends on the month in which the security i is a constituent of S&P 500, τ, say, which is suppressed for notational simplicity.

The time-series data on the safe rate of return, and the market factors are obtained from Ken French’s data library web page. The one-month U.S. treasury bill rate is chosen as the risk-free rate (r_ft), the value-weighted return on all NYSE, AMEX, and NASDAQ stocks (from CRSP) is used as a proxy for the market return (r_mt), the average return on the three small portfolios minus the average return on the three big portfolios (SMB_t), the average return on two value portfolios minus the average return on two growth portfolios (HML_t), the difference between the returns on diversified portfolios of stocks with robust and weak profitability (RMW_t), and the difference between the returns on diversified portfolios of the stocks of low and high investment firms (CMA_t). SMB, HML, RMW, and CMA are based on the stocks listed on the NYSE, AMEX, and NASDAQ. All data are measured in percent per month. See Section M1.3 in the Supplementary Material for further details.

6.2 Month End Test Results (September 1989–April 2018)

Table 8 reports the rejection frequencies of the $J^α$ and GOS tests based on the CAPM, FF3, and FF5 models over the month ends, for the full sample periods, and three market disruption periods: (1) the Asian financial crisis (1997M07–1998M12), (2) the Dot-com bubble burst (2000M03–2002M10), and (3) the Great Recession (2007M12–2009M06) periods. Depending on the factor model (CAPM, FF3, or FF5) and nominal size (5% or 1%) considered, the $J^α$ test rejects the null hypothesis $H0:αi=0$⁠, from 24% to 30% of the total number of tests carried out, which is much smaller than the rejection rates of the GOS test that lie between 39% and 72%. The high rejection rates and their wide range in the GOS test may be due to the tendency of this test to over-reject when T is relatively small, as documented by Monte Carlo experiments in Section 5.

As to be expected, rejection rates in the top panel of Table 8 (based on 5% level) are larger than those in the bottom panel (based on 1% level), but the differences are of second-order importance, particularly when compared with the choice of the underlying asset pricing models. Focusing on the test results based on the 5% level, we note wide variations in the test outcomes across models (CAPM, FF3, and FF5) particularly in the case of sub-samples representing the Asian Financial Crisis and the Dot-com Bubble. The test outcomes for these two sub-samples critically depend on the choice of the asset pricing model, although as for the full sample results the GOS test gives much larger rejection rates. Given the sensitivity of the test outcomes to the choice of the asset pricing model, no firm conclusions can be made in relation to these financial crises. The results based on the $J^α$ only lead to substantial rejections only in the case of Dot-com Bubble period and when we base the test on the FF5 model.

The situation is very different when we consider the Great Recession period, where we find substantial rejection of the null of market efficiency irrespective of the model choice. Using the $J^α$ there is no pattern to the rejection rates across the models, and using CAPM given a rejection rate of 84% when compared with 95% for FF3 and 74% for FF5. The GOS rejection rates are much higher (100% for CAPM and FF3 and 95% for FF5). Due to its over-rejection tendency, the GOS test seems to be less discriminatory when we compare the GOS rejection rates across the different sample periods. This is particularly so in the case of the GOS tests based on the FF5 model. Overall, both tests provide strong evidence of pricing errors during the Great Recession, but $J^α$ test appears to provide more sensible results than the GOS test in this application.

7 Conclusion

In this article, we propose a simple test of LFPMs, the $J^α$ test, when the number of securities, N, is large relative to the time dimension, T, of the return series. It is shown that the $J^α$ test is more robust against error cross-sectional correlation than the SW tests based on an adaptive thresholding estimator of V, which is considered by Fan, Liao, and Yao (2015). It allows N to be much larger than T, when compared with alternative tests proposed in the literature. The proposed test also allows for a wide class of error dependencies including mixed weak-factor spatial autoregressive processes, and is shown to be robust to random time-variations in betas.

Using Monte Carlo experiments, designed specifically to match the distributional features of the residuals of Fama–French three factor regressions of individual securities in the S&P 500 index, we show that the proposed $J^α$ test performs well even when N is much larger than T, and outperforms other existing tests such as the tests of GOS et al. (2015) and GL. Also, in cases where N < T and the standard F-test due to GRS can be computed, we still find that the $J^α$ test has much higher power, especially when T is relatively small.

Application of the $J^α$ test to all securities in the S&P 500 index with 60 months of return data at the end of each month over the period September 1989–April 2018 clearly illustrates the utility of the proposed test. Statistically significant evidence against Sharpe–Lintner CAPM and Fama–French three and five factor models is found mainly during periods of financial crises and market disruptions.

Supplemental Data

Supplemental data are available at https://www.datahostingsite.com.

Appendix: Proofs of the Theorems

In this Appendix, we provide proofs of the theorems set out in Section 4 of the article. These proofs make use of lemmas which are provided, together with their proofs, in the Supplementary Material.

In sum, under Assumptions 1–3, $SNT→p0$⁠, so long as $0≤δγ<1/2, N/T2→0$ as N and $T→∞,$ jointly.▪ 

 

Hence, $supijφij≤1+|γ2,εη|$⁠, as required. ▪ 

 

Footnotes

References