Abstract

This article proposes a new approach to measuring trend output that exploits survey data on expectations to distinguish the effects of permanent and transitory shocks and to track the time-variation in the processes underlying the determination of output. The approach is illustrated using measures of output expectations and output uncertainties based on a business survey conducted for UK manufacturing. The measures are employed in a time-varying vector autoregression (VAR) to track trend output and to provide a compelling characterization of the output fluctuations in UK manufacturing over the last 20 years.

JEL codes: C32, E32

1 Introduction

The measurement of trend output suffers from two pervasive problems: separating permanent output movements from short-lived fluctuations in output, and accommodating the inherent instabilities in the drivers of the trend and changes in the nature of shocks to output over time. This article exploits the information available in surveys on output expectations to distinguish innovations with transitory effects on output from innovations with permanent effects, and it proposes the use of a ‘meta modelling’ time series technique, based on model averaging involving the survey data, to address the time-variation in the determination of the trend. We derive a real-time trend output series for UK manufacturing that exploits the survey expectations measures obtained from the Confederation of British Industry (CBI) and which illustrates clearly the importance of accommodating the instabilities in the drivers of the trend, the dramatic effects of the Global Financial Crisis (GFC) and the role played by uncertainty in explaining the trend.

There is, of course, a considerable time-series literature on trend-cycle decompositions, as reviewed in Hodrick (2020) for example. A popular measure of the permanent component is the Beveridge–Nelson (BN) trend, defined as the long-horizon expectation of the series (minus any deterministic drifts) given current information. Its popularity derives from its natural interpretation as the ‘steady-state’ outcome that will occur in the absence of any further shocks, and because every possible permanent measure must converge on the BN trend in expectation as the forecast horizon increases and the corresponding transitory element goes to zero.1 A frequently mentioned criticism of the BN trend is that, when based on the relatively simple univariate AR models typically supported by the data, the trend is excessively volatile (and sometimes more volatile than the series itself!). Kamber et al. (2018) (KMW) suggest a solution in the univariate context in which the AR model parameters are restricted to limit the size of the signal-to-noise ratio—i.e. the variance of the trend shocks relative to the variance of the overall forecast errors—to a ‘realistic’ level. When applied to US data, the approach delivers trends and cycles that broadly correspond to those of the NBER-published business cycle but, clearly, this solution is based on an arbitrarily chosen limit based on a prejudgment on the nature of the permanent and transitory components.

This article uses the BN trend as a measure of permanent output change but it resolves the problem raised above by using survey measures of expected future outputs alongside actual output data in vector autoregression (VAR) models (often termed ‘VAR-in-Expectations’ or VAR-E models). The expected future output series provides information on the evolution of output that complements the actual output data no matter how expectations are formed. The two series can be used together in a joint model which will deliver a more sophisticated time series representation of actual output than can be obtained in a univariate context. This eliminates the problem of the volatility in BN trends based on univariate AR models discussed above because the expected output growth series is relatively stable compared to the actual output growth series and together they provide an objective measure of the signal-to-noise ratio. This automatically delivers the smoothing of the BN trend suggested by KMW.2 Equally importantly, the expectations series also provides direct insight on what survey respondents consider to be the transitory and permanent components of a shock (since today’s output will be affected by permanent and short-lived shocks but only the former will be expected to influence output in the far future). The use of actual and expected outputs in a joint model allows for an explicit and separate characterization of the permanent and transitory shocks to output as perceived by the survey respondents and this directly informs the derived BN trend.3

An advantage of using direct measures of expectations in measuring trend output is that we can also investigate the role of uncertainty in business cycle dynamics and its contribution to the trend over time. Uncertainty refers to the extent to which something is not known and although many measures of uncertainty have been proposed in the literature, these are often based on an outside metric—stock price volatility or newspaper coverage,4 for example—which may be only tangentially related to the event of interest. Uncertainty measures based on individuals’ stated understanding of output growth, as reflected in surveys, unambiguously relate to the extent to which output growth is unknown and can be used alongside the direct measures of expectations to investigate the extent to which uncertainty influences trend output.

The second problem in measuring trend output arises if there is substantial time variation in the processes underlying the determination of output. The shocks to the macroeconomy associated with the GFC and the covid pandemic appear very different in nature to those experienced during the low levels of output volatility of the ‘Great Moderation’. Certainly, it seems unlikely that the relative importance of transitory and permanent shocks remained unchanged through these different episodes. Moreover, there is a broad consensus that there has been a slowdown in productivity growth across advanced economies. The timing of the slowdown is unclear: many associate the slowdown with GFC but others note that the data show a slowdown prior to GFC, and others see the recent decline as part of a much longer pattern, interrupted by a temporary improvement in mid1990’s–mid 2000’s due to digital technologies. And there are numerous candidate explanations for its cause, including: demographic changes (relating to migration, for example); changes in economic inequality and education provision; the ebb and flow of globalization and international trade; a reduction in investment opportunities because of increased credit constraints or the riskiness of ever-more-important intangibles; changes in the user cost of capital and cost of materials; and a simple deterioration in technological advance.5 Any or all of these explanations could be true, with some influences affecting trend growth slowly and incrementally over time and some more abruptly following a specific structural change. Of course, this introduces inherent instability in models of output growth and complicates the measurement of trend output derived from the models.

There have been a variety of approaches taken in the applied literature to deal with structural instabilities in models of output growth. Changes in the mean growth rate, in the dynamic response to shocks, and/or in the nature of the shocks can be accommodated through switching-regression or smooth-transition models, through time-varying parameter models, or through model averaging, where a number of alternative time series specifications are estimated and then combined with time-varying weights.6 We adopt a model averaging approach to deal with the inherent structural instability in output growth models, but we pay particular attention to finding the time frame for which the model is relevant. Specifically, we follow the suggestion of Pesaran and Timmermann (2007) to apply model averaging techniques to alternative models estimated over different estimation windows. The approach recognizes that, when it is uncertain whether or when a break has occurred in a relationship, there is a trade-off between using short samples or long samples of data in a rolling estimation exercise. This is because longer samples improve the precision of parameter estimates in the absence of breaks but are corrupted for longer in the presence of breaks. We consider the relationship between actual output growth and survey responses on growth and, at each point in the sample, we consider a range of models of the same form but estimated over different sample lengths and averaged using weights that change over time. This provides a very flexible form for capturing structural change, allowing for slowly evolving or abruptly changing regimes and we describe it as ‘meta modelling’ to highlight our emphasis on regime uncertainty compared to more standard model averaging exercises. Using the weights in the VAR models estimated using actual and expected future outputs delivers BN trends that can accommodate shifts due to any productivity slowdown or to changes in the relative importance of transitory and permanent shocks over time.

The article illustrates the use of these methods by describing a time series model of actual output, expected output and output uncertainties in the UK drawing on the data provided in the CBI’s survey of manufacturing businesses. In the next section, we motivate our modelling approach by reviewing the advantages of using direct measures in empirical work on the business cycle (and the potential dangers of omitting them) and explain how these advantages are exploited in the joint model of actual and expected outputs we use in deriving our output trend. The following section then introduces a novel method of deriving quantitative measures of expectations and uncertainty from the qualitative CBI survey responses which delivers the time-varying weights that capture the relevance of the past data over differing sample lengths. We then use these quantitative measures and the time-varying weights in a simple VAR-E model to capture UK business cycle dynamics between 2000q1 and 2019q4.7 The analysis provides a compelling characterization of the output fluctuations in UK manufacturing over the last 20 years showing, for example, that the time-variation in models is important, that a properly estimated model delivers reasonably clear business cycles typically lasting 2–3 years from peak to trough, and that output uncertainty provided a negative source of shocks to output outcomes resulting in output trend 1% lower than would have been the case following the GFC and still lower, by 1.5%–2%, due to the uncertainty shocks experienced in the second half of the 2010’s.

1.1 Related literature

There is a long tradition of using direct measures of expectations to study the expectation formation process—see Pesaran and Weale (2006) and Croushore (2010) for useful overviews—and there has been increasing recent interest in the role of agents’ use of information in generating macro dynamics that has focused attention on expectation formation processes. For example, the influential papers by Mankiw and Reis (2002), Sims (2003), and Woodford (2001) explore the consequences of various forms of information rigidity in rational expectations models while the empirical evidence on the nature and extent of information rigidities, based on the analysis of survey responses of professional forecasters, has been provided in Coibion and Gorodnichenko (2012, 2015), and Dovern et al. (2015, 2012), inter alia, to establish that these influences are important in practice. This literature is very crowded then, but modelling the joint determination of actual and expected outputs in a VAR will be a useful way of capturing the important feedbacks suggested by this literature.

Interest in the role of uncertainty in business cycle dynamics has also been studied extensively since Bloom’s (2009) seminal piece. Bloom (2014) provides an excellent overview. An enormous literature developed in response suggesting various alternative measures of uncertainty and exploring the extent to which this sort of macro dynamic is observed in practice, typically including the uncertainty measure in a VAR and identifying the effects of uncertainty shocks on output through an impulse response analysis. Prominent examples include Bloom (2009) itself, Jurado et al. (2015), Barrero et al. (2017), and Ludvigson et al. (2021). Papers that make reference to measures of uncertainty based on survey responses include Lahiri and Liu (2006), Mitchell et al. (2007), Lahiri and Sheng (2008, 2010), Boero et al. (2015), Bachmann et al. (2013), Clements (2017), Garratt et al. (2018), and Jo and Sekkel (2019), for example.

The literature discussing structural changes in the determinants of output growth is wide and varied. For example, the decline in output growth volatility during the Great Moderation generated many papers concerned with the changing influences of permanent and transitory shocks on the business cycle; see, for example, Romer (1999), Davis and Kahn (2008), and Keating and Valcarcel (2015) There is a complementary literature focused on the extent to which (usually monetary) policy changes, and hence the properties of the business cycle, change over time—exemplified by Cogley and Sargent (2005), Alcidi et al. (2011), and Lee et al. (2015)—and a related literature that emphasizes the role of learning in expectation formations and business cycle fluctuations; see Evans and Honkapohja (2001), Eusepi and Preston (2011), and Bordalo et al. (2020), for example. And, in addition to the debate regarding the productivity slowdown mentioned above, there is a more focused literature concerned with the ‘secular stagnation’ observed globally following the GFC; see, for example, the papers in the volume edited by Teulings and Baldwin (2014).

2 The use of survey expectations in measuring trend output

One reason why the direct measures of expectations obtained from surveys are useful in modelling output movements is because of the information they provide on the persistent effects of shocks: if, following a shock to observed output, reported expected future output returns to the pre-shock level, survey respondents are saying they believe the effects of the shock to be transitory; in contrast, if shocks continue to show in the reported expected future output level, the respondents are saying they are permanent. The ability to distinguish transitory from permanent shocks using survey data is important in measuring the output trend in two important respects: first, the effects of transitory shocks can be discounted to obtain a trend with sensible signal-to-noise ratio based on the data; and second, the extra information can be used to better uncover structural breaks in the growth process as changes in the relative importance of permanent and transitory shocks can be properly accommodated and changes in trend growth rates can be more easily discerned.

In this section, we consider these two advantages of using survey data in turn and describe an approach to modelling that can exploit the advantages to deliver an output trend measure. The basis of the modelling approach is a bivariate model of actual and expected growth where, for a vector process zt, the vector of BN trends, z¯t, is defined by the long-horizon expectations of the series (minus any deterministic drift) given current information

(2.1)

where g, the element of deterministic growth, is typically a vector of constants. If Δzt can be given a stationary moving average representation of the form

with C(L) a lag polynomial and εt a vector of iid shocks, then the BN trends can be expressed as

(2.2)

where C(1)εt represents the persistent or ‘infinite horizon’ effect of the shock experienced at t. In what follows, we illustrate the usefulness of surveys in this context with time-invariant parameters before then extending the approach to allow time-variation in the g and C(L).

2.1 Survey expectations and the persistence of shocks

Survey measures of expected future output growth tend to be relatively stable over time, compared to actual growth, and this comparison helps distinguish the permanent shocks that drive trend output from the transitory shocks that generate the cycles around trend. Measures of the size of shocks based on actual output data alone will overstate the true extent of the uncertainty surrounding output (being unable to take account of the known-to-be-transitory element) and univariate models estimated using only actual data will overstate the consequences of the shocks for future output movements. This translates into measures of trend output that are excessively volatile. The problem is readily solved using a bivariate VAR-E model of actual and expected output.

To illustrate these points, we can consider the most simple data generating process that will deliver a trend and a cycle and, in this context, explore the advantages of having direct measures of expectations and the difficulties in measuring the trend and cycle in the absence of direct measures. So, denote output growth ytyt1, where yt is (the logarithm of) output at time t, and consider a simple model in which growth is determined according to the following

(2.3)

depending on two types of shock, vt and ωt, the latter of which is known to have only short-lived effects and will be offset in the next period, with the effects of both propagated over time according to the size of the (assumed known) coefficient ρ. Assuming (for illustrative purposes) that expectations are formed with Full Information Rational Expectations (FIRE) and are captured precisely by the direct survey measures, then—given that ρ(ytyt1) is known—the expected value of (yt+1yt) formed in time t reveals the offset of the short-lived shock experienced in t since

(2.4)

where tyt+1e is the time t expectation of yt+1. The advantage of having direct measures of expectations from a survey is immediately apparent: the short-lived shocks are observed directly through the survey responses, and the permanent shocks are recoverable from the ‘adjusted’ actual series obtained by subtracting the effects of the short-lived shocks from the original actual series:

(2.5)

If the two types of shock are independent and have variance σv2 and σω2, respectively, then the ‘news’ arriving on (ytyt1) at time t is reflected in the expectational error ytt1yte=vt+ωt and the uncertainty surrounding growth - describing the extent to which output in t is not known in t1, i.e. σv2+σω2—can be readily calculated and split into its component parts.8

Problems with univariate specifications

In the absence of direct measures of expectations, modelling may be based on the actual output series alone which, in this illustration, can be captured by the autoregressive integrated moving average (ARIMA) process

(2.6)

where θ and the variance of ut,σu2, are obtained matching the variance and covariance terms of the two characterizations. With the details relegated to the Appendix, solving we find that θ[1,0], with the value depending on the relative size of the two types of shock (tending to 0 when the permanent shocks dominate and σv2σω2, and tending to 1 when the short-lived shocks are relatively important and σv2σω20), and that σu2>σv2+σω2 so that the uncertainty surrounding growth as obtained from the univariate MA specification overstates the ‘true’ uncertainty surrounding growth.

Applying the definitions of the BN trend in (2.1) to this simple illustration, we have

The overstatement of the uncertainty surrounding growth according to the univariate ARIMA specification translates into an overstatement of the persistent effect of shocks to output—and hence the BN trend. To see this, note that a 1% change in output growth, based on the composite shock vt+ωt, will typically involve a vt of size σv2σv2+σω2% and so the size of the increase in BN trend is 11ρ×σv2σv2+σω2 (often described as the ‘persistence measure’). The corresponding 1% shock in the univariate ARIMA representation results in output 1+θ1ρ% higher at the infinite horizon. As shown in the Appendix, (1+θ)σv2σv2+σω2=(θ2+θ3)(1+θ+θ2)>0, so the ARIMA version overstates the extent to which the 1% shock translates into a permanent increase in output. To gain a sense of the orders of magnitudes involved here, we note that, in the U.S. for example, the volatility of actual quarterly output growth is 0.60 while the expected one-quarter-ahead output growth provided by the Survey of Professional Forecasters take the values 0.23. If growth was characterized by the model of (2.3) with ρ=0.8, this conservatism in the expectations series—suggesting συσυ2+σω20.4—implies that the transitory shocks are five times more volatile than the permanent shocks and that we have θ=0.64 in the ARIMA specification of (2.6). Here, then a 1% increase in output on impact would raise output by 0.84% eventually and a gap measure based on the true BN trend (obtained when the two types of shock can be separately identified) are sensibly pro-cyclical. But the estimate of the long-run effect based on the univariate MA representation would be 1.80% and the corresponding BN trend would show more volatility than the actual output series itself.

As highlighted by KMW, these same problems in estimating the BN trend are arguably even more serious where the trend is based on a univariate AR representation of growth rather than the specification of (2.6). The above example again illustrates the problem: if the ARIMA(1,1) model of the illustration is approximated by an AR(1) or AR(2) representation of growth, the estimated value of the coefficient on the lagged dependent variable (ldv) is around 0.20 in the AR(1) case, or 0.28 for the sum of the ldv coefficients in the AR(2) case. This translates into persistence measures of 1.23 or 1.38, respectively, so again the estimated BN trends are more volatile than the actual series. KMW note that, in practice, the persistence value obtained from an AR specification is always greater than unity regardless of lag order if it is unrestricted, and they make the insightful suggestion, when calculating the BN trend, to impose restrictions on the AR parameters that constrain the implied signal-to-noise ratio to a reasonable value. This effectively imposes the ratio of the volatility of transitory shocks to that of permanent shocks to take a pre-specified value. In the absence of information on the signal:noise ratio, this constraint is rather arbitrary but, as shown above, the survey-based measures of expectations provide information on this which can be used to identify the effects of the transitory and permanent shocks directly. We turn now to the general approach to using the series therefore.

2.2 Accommodating permanent and transitory shocks in a VAR-E

The limitations of the univariate models of actual output arise because a single relationship is being used to capture the outcomes of two innovation processes. A bivariate VAR-E model of actual and expected growth—making use of direct measures of expectations alongside the actual series—is able to characterize the two processes and their interactions properly and avoids the econometric issues raised above. A VAR in actual and expected output growth is appropriate if output is stationary in differences and expectational errors are stationary (which will be true for any reasonable expectation formation process). For example, consider the first-order model9

(2.7)

This model can clearly accommodate the illustrative example of (2.3), with b12=1, b22=ρ, and b11=b21=0, and with ξ1t=vt+ωt and ξ2t=ρvt(1ρ)ωt. The FIRE assumption of the illustrative model is embedded within (2.7) by imposing the restriction that b12=1 and b11=0. But more sophisticated assumptions on the expectation formation process could be captured through an appropriate set of (less-rigid) restrictions in the first row of the VAR, while leaving the relationship unrestricted would give maximum flexibility in representing the expectation formation process.10 Equally, more complex models of output determination would be accommodated if no coefficient restrictions are imposed and/or higher order lags are included in the VAR-E model.

The VAR-E of (2.7) is readily written in levels form

where zt=(yt, tyt+1e) and εt=(ξ1t, ξ1t+ξ2t). As elaborated in the Appendix, the Φi are functions of the parameters in (2.7) which reflect the fact that, with expectational errors being stationary, the levels of actual and expected output series are cointegrated with cointegrating vector (1,1) and this can be made explicit writing the model in a cointegrating VAR form explaining Δzt. This also means that the associated MA representation

(2.8)

has the property that C(1) is reduced rank, so actual and expected outputs are both driven by the same single stochastic trend and, from (2.2), both share the same BN trend. The element of shocks that have a permanent effect on actual and expected output can be identified following the standard methods of Blanchard and Quah (1989) and, with the effects of the permanent and transitory shocks identified separately, measures of the persistence of shocks to the bivariate model can deliver values that are arbitrarily close to zero in contrast to the problems faced by univariate AR models.11

3 Survey expectations and structural change

The strength of the separate influences of permanent and transitory shocks could change over time. For example, it seems likely that the relative importance of permanent shocks increased during the period of the Global Financial Crisis when compared to more ‘normal’ times, and the way in which their effects were propagated over time—i.e. their dynamics—also probably changed. This means there is an inherent structural instability in the time series representations of output growth, and hence measures of the trend, even in the absence of a productivity slowdown, and a simple univariate time series representation of actual output considered alone could be seriously misleading in these circumstances. If, for example, occasional periods of crisis are associated with increased volatility in the permanent shocks (with the size of the short-lived shocks unchanged), the parameters of the univariate model—θ and σu2 in (2.6) or the corresponding parameters in the AR approximation—are actually time-varying and a simple time-invariant univariate model would be misspecified. Further, the misspecified errors of an estimated time-invariant specification will be correlated over time with the volatility in the errors so that the addition in the growth equation of an uncertainty variable that reflects this volatility would show as important even if uncertainty actually has no effect on output growth.

Surveys provide an excellent means of exposing the time variation in the time series representations of macro variables because they often include ‘backward-looking’ questions on the survey respondents’ recent experiences, as well as their expectations of the future. The relationship between the actual past outcome and the range of survey respondents’ past experiences can provide useful information on the size and nature of the shocks. For example, it could be argued that transitory shocks tend to be more firm- or sector-specific while permanent shocks have a more global impact on firms. In this case, equally sized negative transitory or permanent shocks would both result in an increase in the number of firms experiencing a fall in output but the increase would be larger in response to the permanent shock since the shift in mean is accompanied by a fall in the variance. This insight is pursued below where we discuss first how we might use a qualitative survey which includes backward- and forward-looking questions to derive quantitative measures of expectations and uncertainty and then how this quantification can be adapted to accommodate structural change.12

3.1 Deriving quantitative measures of expectations and uncertainty from a qualitative survey

The typical qualitative survey provides information on the question ‘Excluding seasonal variations, what has been the trend over the past three months and what are the expected trends for the next three months, with regard to volume of output (i.e. production)?’. Survey participants can respond that the trend has been one of ‘Up’, ‘Same’, ‘Down’, or ‘n/a’ over the two time frames. Then denote

while the corresponding variables referring to the expected trend over the next 3 months are denoted tRt+1e, tSt+1, and tFt+1e.

There have been a number of methods proposed for converting the information contained in these proportions into a time series for a quantitative measure of expectations.13 Many are based on the recognition that, if the average percentage increase in output for those individuals reporting a rise is α and the average decrease in output for those individuals reporting a fall is β, then the average increase in output across all individuals is

(3.1)

Further, if the average increases and decreases, α and β, remain relevant in expectation, then the expected future growth is given by

(3.2)

The ‘regression method’ for converting qualitative survey outcomes to a quantitative series is based on (3.1) and (3.2) where estimated values for α and β can be obtained by treating (3.1) as a relationship that holds over time with error and regressing ytyt1 on Rt and Ft. The estimated values can then be applied to (3.2) to obtain a time series for expected growth.14

Having derived the measures of expectations, a measure of uncertainty about output growth can be based on the size of the average expectational error, (yt+1tyt+1e)2. This provides a direct measure of the size of what was not known about yt+1 at time t which is precisely what we mean by ‘the uncertainty around yt+1’. The measure is often termed ‘ex post consensus uncertainty’. The disadvantage of using this measure is that it involves the actual outcome—hence the descriptor ‘ex post’—which is unknown at the time expectations are formed and so does not properly capture the uncertainty surrounding the reported expectation at the time it was reported. Lahiri and Sheng (2008) therefore suggest constructing a measure of ‘ex ante consensus uncertainty’ tyt+1u based on a GARCH model in which

(3.3)

and, for example,

so that the size of the expectational error in time t+1 is driven by the innovations observed at time t. The ex ante consensus uncertainty measure is tyt+1u=σt+1^, the estimated standard deviation of the innovations to the (de-meaned) expectational error at time t+1 conditioned on the information available at t.

3.2 Accommodating structural change in measuring expectations

The assumption in (3.1) that the average percentage increase of output among those experiencing a rise and the average percentage decrease in output among those experiencing a fall are both constant over time is extremely unlikely to hold true in practice. More realistically, α will be higher and β will be lower in good times, and vice versa in recessions. Similarly, changes in the range of outcomes across firms will cause α and β to change over time: with mean output growth positive but unchanged, a fall in the variance will increase α and decrease β. However, the formula at (3.1)–(3.2) remain relevant even if the average or variance changes over time so long as we replace α and β with time-varying αt and βt. Two possibilities for estimating values for αt and βt based on (3.1) are as follows:

  1. use a rolling sample window of, say, s periods and obtain time-varying values for α and β at each point in time T by estimating the models Ms,T defined by.
    (3.4)
    so that the coefficient attached to each period are based on the regression estimated over the most recent s periods; and
  2. use a ‘meta model’, again based on a set of rolling regressions, but allow the data to choose an appropriate sample window at each point to balance the advantages of longer samples (which provide more accuracy in estimated relationships) versus short samples (which are less vulnerable to the effects of structural breaks).

Both of these approaches will capture time variation in the α and β coefficients and improve the scaling in quantification. The simple rolling window approach is more straightforward but makes an arbitrary choice on the size of the sample window. As explained below, the ‘meta model’ is based on a more systematic choice of window size and is able to capture both smoothly evolving relationships and abrupt changes in the relationship between growth and the survey responses.

The ‘meta’ modelling approach is described in detail in Lee et al. (2015), where it is applied to the estimation of a Taylor rule, and Aristidou et al. (2019, 2022), where it applied in the context of forecasting output growth and explaining exchange rate movements, respectively. The approach deals with the uncertainty over the appropriate sample window through model averaging, assuming that there are S possible models that can be used to characterize output growth at time T. The models are all of the form in (3.1) but estimated over various sample windows Tsmax,,Tsmin with S=smaxsmin+1. Hence each model Ms,T links output growth to the backward-looking qualitative survey responses over the period Ts,,T, but we contemplate models that might be relevant only for the most recent smin periods or back to smax periods in the past.

The ‘meta model’ explaining ytyt1 over the full sample T_,,T¯ is then defined by

(3.5)

with weights ws,T capturing the relevance of the different candidate model at each point over the sample. A pragmatic approach to deriving model weights is to allow these to evolve over time, updating the weights in each period to reflect new evidence on whether the previously held model continues to be valid or whether an alternative new-born model is now appropriate. The approach can be formalized by writing, for any T and for s=smin,,smax1,

(3.6)

Here, the weight assigned at time T1 to the model based on data T1s to T1 is either transferred to the model with one additional observation—i.e. using data T1s to T—or to a new model based on the shorter sample of data Tr to T. If a model is rejected in favour of more than one shorter alternative, the weight can be split equally among the alternative models. In transferring the weights, the tests should be conducted comparing the null to successively shorter samples so the weights can be shifted down sequentially where the evidence is that a model based on a shorter sample outperforms a model based on a longer one. Given that the shorter models are all nested within the longer model, the validity of the null can be tested using standard F-tests of structural stability.

The estimated weights of the meta model show which of the individual models, distinguished by the sample length, provide the most likely characterization of the relationship between Δyt and the qualitative survey responses. The importance of the various models is reflected in the averaged coefficients

and changes in the size of the weights over time provide useful information on how the relationship has evolved. For example, the duration statistic

(3.7)

provides a time-T indication of the average duration of the relationship in place at that time. The use of model weights provides considerable flexibility in capturing the time-variation in the nature of the relationship between growth and the survey responses which can evolve smoothly over time, with weights shifted to progressively longer samples during periods of stability for example, or can change very abruptly if, for example, the weights all shift to a short sample following a significant structural break.

3.3 Accommodating structural change in measuring the BN trend

The weights derived above are driven by structural change either because the mean growth rate changes, through a productivity slowdown say, or because the cross-firm variation in experiences changes, through a shift in the relative importance of transitory and permanent shocks for example. At each point in time, the weights capture the relevance of the more distant and the more recent observations in the determination of growth. In what follows, we apply the weights in the estimation of the VAR-E models described in (2.7), extended to include a potential role for uncertainty. The extension to include uncertainty is natural here given the recent interest in the effects of uncertainty on growth and given the measure of uncertainty employed is based on the second moment of the expectational errors observed directly from the actual and expected output series.15

To be more specific, the ‘meta VAR-E’ model of order q that we consider has the form

(3.8)

where y~ts=ws,T×yts, ty~t+1e=ws,T×tytse, and ty~t+1u=ws,T×tyt+1u. Here each of the equations in the VAR is estimated using weighted least squares with weights defined by the survey-based exercise above. This estimator has the standard least squares motivation but observations are weighted so emphasis is given to the observations in short samples where there is evidence of structural breaks and to observations in longer samples where there was no evidence of changing means or changing cross-firm variation in the survey data. The model can be estimated working recursively through the data, delivering estimates of Bi,T, i=0,,q and for T=T_,,T¯. Then, through the same transformations as (2.8), we obtain the date-specific MA representation

with the CT(L) obtained from the parameters of the Bi,T16 The BN trend is then defined by

(3.9)

Here, the BN trend in T is equal to its value in the previous period plus the accumulated future effects of the timeT shock based on the parameters of the meta VAR-E model estimated over the previous smax observations (plus the deterministic term also obtained from that model). The implicit assumption is that, despite any structural breaks observed at that time and captured within the weights ws,T , the infinite horizon effect of past shocks—as embedded within last period’s BN trend—remains unchanged. Only the addition to the trend is influenced by the newly estimated model at T.

4 Actual output, expected output and trend output in UK manufacturing, 2000q1–2019q4

The empirical work of the article is conducted using actual quarterly output data provided by Office for National Statistics (ONS) and expectations data reported quarterly through the Industrial Trends Survey (ITS) conducted by the Confederation of British Industry (CBI). The actual output series are obtained from the ONS and represent the Gross Value Added series for the ‘Manufacturing sector’, covering the production industries excluding Energy and Water.17 The ITS is completed by businesses operating in UK manufacturing and traces its origins back to 1958, although our analysis focuses on the period 2000q1–2019q4, limiting attention to the pre-covid period on the grounds that the pandemic period requires special ‘extreme value’ treatment and that it is still too early to see the subsequent impact. The survey asks questions relating to recent and 3-month-ahead expectations on a range of economic magnitudes.18 Completion of each survey is voluntary, meaning that firms do not have to complete consecutive surveys, but some 2093 firms participated in the ITS over our sample period generating 29,692 survey responses. The sample frame is small relative to ONS’s own quarterly firm-level surveys and, in terms of the number of firms surveyed, the ITS disproportionately covers medium-sized and large firms. But the continuity among participating firms is reasonably good, as is the representation by sector (considering Primary vs. Secondary Manufacturing, say) and by location (considering the 11 standard UK regions, say) and, despite the lack of sophistication in the sampling design, the ITS provides the only serviceable source of direct measures of expectations over this period.19, 20

Figure 1 introduces the raw output data showing the evolution over time of the quarterly output growths ytyt1 where yt is the (logarithm of the) output level. The UK Manufacturing Sector was broadly the same size in 2020 as in 2000, with growth over the sample period averaging just 0.04% per annum. The slowdown associated with the Global Financial Crisis [GFC] of 2008/9 clearly played a considerable role in this, seeing output fall by 12% in the year from 2008q3. Figure 1 also plots the survey responses Rt and Ft published in the ITS survey in t. The broad co-movements in the data are very clear, with the proportions of firms reporting ‘Up’ and ‘Down’ rising and falling as output growth rises and falls, and this is reflected in the simple correlations between quarterly output growth and Rt and Ft which are, respectively, 0.43 and 0.54.

Quarterly output growth and survey responses.
Figure 1.

Quarterly output growth and survey responses.

4.1 Measures of expected output and output uncertainty in the presence of structural breaks

Figure 2 describes the quantitative expectations series obtained for Manufacturing output adopting the three alternative approaches for converting the qualitative survey expectations series into quantitative series discussed in the previous section. These are: the ‘balance statistic’ approach, assuming α and β in (3.1) and (3.2) are constant over time; the ‘rolling modelling’ approach in which α and β are allowed to vary over time with their estimates based on a rolling regression analysis of 25 quarters; and the ‘meta modelling’ approach in which α and β vary over time with their estimates based on estimated model averages. The derived quantitative expectations series for the methods, tyt+1eyt, are plotted against the actual outcome, yt+1yt.

Actual and expected output growth.
Figure 2.

Actual and expected output growth.

The figure shows the relative benefits of the more sophisticated quantification methods in each case. For example, the correlations between actual output growth and expected output growth obtained using the balance statistic approach take the statistically significant value of 0.51, so the quantification technique is certainly useful. But the expectations series based on the models incorporating time-variation in the parameters are better able to capture the more extreme periods of output growth and contraction than the other series, and the correlation with actual outcomes is 0.65 in the meta model for example.21 To illustrate the nature of the time-variation, Figure 3 plots the duration statistics defined in (3.7) that lie behind the meta model’s expectations series, showing four or five distinct periods during which the preferred sample period increased period-by-period—indicating model stability—to be replaced abruptly by shorter preferred sample lengths at specific break-points.22

Sample duration and differences in time-varying parameters.
Figure 3.

Sample duration and differences in time-varying parameters.

The uncertainty measure described at (3.3) can be obtained using the derived expectations series found using the parameters obtained through the meta modelling approach. The ex ante uncertainty measure is obtained from an estimated GARCH model applied to the expectation errors and, following a specification search, we found a fourth-order ARCH in the variance.23 The derived series is plotted in Figure 4, showing that there were periods of relatively high uncertainty through the early 2000’s—possibly related to uncertainty surrounding the historically high exchange rates experienced at that time—and then during the period covering the GFC and its aftermath and in then spiking in 2019, perhaps associated with the UK’s imminent departure from the EU in January 2020.

Uncertainty over output.
Figure 4.

Uncertainty over output.

4.2 VAR-E models of UK manufacturing output

The interplay between actual and expected output growth and the associated uncertainty is captured by the VAR-E model of (3.8). The model is again estimated in three ways to illustrate the importance of taking account of structural change: (i) MWHOL, estimated using the whole sample; (ii) MROLL, estimated recursively using a rolling window of 25 observations (with all observations given equal weight); and (iii) MMETA, estimated recursively using a rolling window of 25 observations with the three series adjusted according to the weights obtained in the meta modelling exercise above.24

To give an indication of the sort of VAR model underlying the analysis, Table 1 summarizes the results of estimating MWHOL, where a VAR of order 4 was found to be sufficient to capture the dynamics of the system. The results show strong interdependence between actual and expected outputs, with the relative volatility of the actual series and the relative stability of the expected series reflected in the large negative and the large positive coefficients found on their respective lags. Summing across the lags, the uncertainty variable has a relatively large, but statistically insignificant, downward effect in the actual output equation, although the overall, system-wide effect is obviously difficult to judge from the parameters alone. The VAR-E models underlying MROLL and MMETA are of a similar form but, as the meta-analysis of the survey data anticipated, there is considerable variability in these relationships over time and the MWHOL results provide only an imprecise indication of the relationships between the three series at any one time.25

Table 1.

MWHOL VAR-E model of output uncertainty and expected and actual output growth 2002q1–2019q4

 Dependent variable
 tyt+1utyt+1eytytyt1
Explanatory variables
kcoeffs ontkytk+1u0.3301.829***0.905
(0.48)(0.64)(1.44)
kcoeffs on(tkytk+1eytk)0.0501.009***0.275***
(0.04)(0.06)(0.12)
kcoeffs on(ytkytk1)0.0210.0400.725***
(0.07)(0.09)(0.21)
R20.3070.8880.285
Std. error of equation0.0030.0040.008
Std. deviation of dept. variable0.0030.0100.010
F-test (16, 172)4.75***85.51***4.28***
DW1.9891.8972.046
 Dependent variable
 tyt+1utyt+1eytytyt1
Explanatory variables
kcoeffs ontkytk+1u0.3301.829***0.905
(0.48)(0.64)(1.44)
kcoeffs on(tkytk+1eytk)0.0501.009***0.275***
(0.04)(0.06)(0.12)
kcoeffs on(ytkytk1)0.0210.0400.725***
(0.07)(0.09)(0.21)
R20.3070.8880.285
Std. error of equation0.0030.0040.008
Std. deviation of dept. variable0.0030.0100.010
F-test (16, 172)4.75***85.51***4.28***
DW1.9891.8972.046

Note. Results relate to the VAR-E model of order 4 at (3.16) in the text. Each column refers to one of the equations in the VAR-E. Each row refers to the sum of the coefficients on the lagged values of the explanatory variables. Standard error are in parentheses. Superscripts *, ** and *** highlight significance at the 10%, 5% and 1% levels of significance respectively.

Table 1.

MWHOL VAR-E model of output uncertainty and expected and actual output growth 2002q1–2019q4

 Dependent variable
 tyt+1utyt+1eytytyt1
Explanatory variables
kcoeffs ontkytk+1u0.3301.829***0.905
(0.48)(0.64)(1.44)
kcoeffs on(tkytk+1eytk)0.0501.009***0.275***
(0.04)(0.06)(0.12)
kcoeffs on(ytkytk1)0.0210.0400.725***
(0.07)(0.09)(0.21)
R20.3070.8880.285
Std. error of equation0.0030.0040.008
Std. deviation of dept. variable0.0030.0100.010
F-test (16, 172)4.75***85.51***4.28***
DW1.9891.8972.046
 Dependent variable
 tyt+1utyt+1eytytyt1
Explanatory variables
kcoeffs ontkytk+1u0.3301.829***0.905
(0.48)(0.64)(1.44)
kcoeffs on(tkytk+1eytk)0.0501.009***0.275***
(0.04)(0.06)(0.12)
kcoeffs on(ytkytk1)0.0210.0400.725***
(0.07)(0.09)(0.21)
R20.3070.8880.285
Std. error of equation0.0030.0040.008
Std. deviation of dept. variable0.0030.0100.010
F-test (16, 172)4.75***85.51***4.28***
DW1.9891.8972.046

Note. Results relate to the VAR-E model of order 4 at (3.16) in the text. Each column refers to one of the equations in the VAR-E. Each row refers to the sum of the coefficients on the lagged values of the explanatory variables. Standard error are in parentheses. Superscripts *, ** and *** highlight significance at the 10%, 5% and 1% levels of significance respectively.

The adequacy of the VAR-E models

Of course, the literature includes many papers suggesting various macro aggregates as potentially useful in explaining and forecasting output and, hence, in measuring the trend. And Evans and Reichlin’s (1994) (ER) paper on measuring business cycles specifically highlights the sensitivity of the BN’s cycle-trend variance to the inclusion of extra variables, noting that enlarging the information set used to forecast output growth typically increases the lower bound for the estimated ratio. It is important then to consider the statistical adequacy of the simple VAR-E model used here.

Table 2 reports the outcome of a series of variable exclusion tests relating to growth in consumption, Δct, in labour income, Δwt, in aggregate house prices, Δht, and in stock market prices, Δst. The list is suggested by Lettau and Ludvigson’s (2001) VAR analysis and, while obviously not comprehensive, includes some of the key macro variables involved in business cycle analysis. The table reports two sets of results. The first set considers regressions of the expectational errors, ytt1yt, on (four) lagged values of the variables in VAR-E and on (four) lagged values of the additional macro variables. In this first set of results, the expectations error are based on, alternatively, the balance statistic, the rolling regression method and the meta modelling method. The test statistics relate to the exclusion of all the variables (i.e. both those in the VAR-E and the macro variables), of just the VAR-E variables, and of just the macro variables (taken together or individually).

Table 2.

Test of exclusion restrictions

Dependent variableModelAll variablesVAR-E variablesMacro variablesIndividual macro variables (Δct,Δwt,Δhpt,Δftset, respectively)
Expectational errors
ytt1yteBalance0.001***0.009***0.2970.479, 0.089*, 0.284, 0.599
Rolling0.002***0.018**0.1940.255, 0.169, 0.136, 0.837
Meta0.001***0.002***0.2980.438, 0.108, 0.348, 0.592
Equation residuals
Actual Output residualsWHOL0.2980.438, 0.108, 0.348, 0.592
ROLL0.3300.743, 0.072, 0.362, 0.914
META0.3600.868, 0.089, 0.323, 0.472
Expected Output residualsWHOL0.039**0.041**, 0.631, 0.616, 0.061
ROLL0.055*0.226, 0.348, 0.458, 0.101
META0.086*0.275, 0.573, 0.422, 0.094*
Output Uncertainty residualsWHOL0.5290.771, 0.400, 0.201, 0.657
ROLL0.1300.603, 0.470, 0.888, 0.024
META0.0520.170, 0.460, 0.863, 0.007
Dependent variableModelAll variablesVAR-E variablesMacro variablesIndividual macro variables (Δct,Δwt,Δhpt,Δftset, respectively)
Expectational errors
ytt1yteBalance0.001***0.009***0.2970.479, 0.089*, 0.284, 0.599
Rolling0.002***0.018**0.1940.255, 0.169, 0.136, 0.837
Meta0.001***0.002***0.2980.438, 0.108, 0.348, 0.592
Equation residuals
Actual Output residualsWHOL0.2980.438, 0.108, 0.348, 0.592
ROLL0.3300.743, 0.072, 0.362, 0.914
META0.3600.868, 0.089, 0.323, 0.472
Expected Output residualsWHOL0.039**0.041**, 0.631, 0.616, 0.061
ROLL0.055*0.226, 0.348, 0.458, 0.101
META0.086*0.275, 0.573, 0.422, 0.094*
Output Uncertainty residualsWHOL0.5290.771, 0.400, 0.201, 0.657
ROLL0.1300.603, 0.470, 0.888, 0.024
META0.0520.170, 0.460, 0.863, 0.007

Notes. Statistics refer to the p-values of the F-tests of the various null hypotheses based on the various models described in the text. Superscripts *, **, and *** highlight significance at the 10%, 5%, and 1% levels of significance, respectively.

Table 2.

Test of exclusion restrictions

Dependent variableModelAll variablesVAR-E variablesMacro variablesIndividual macro variables (Δct,Δwt,Δhpt,Δftset, respectively)
Expectational errors
ytt1yteBalance0.001***0.009***0.2970.479, 0.089*, 0.284, 0.599
Rolling0.002***0.018**0.1940.255, 0.169, 0.136, 0.837
Meta0.001***0.002***0.2980.438, 0.108, 0.348, 0.592
Equation residuals
Actual Output residualsWHOL0.2980.438, 0.108, 0.348, 0.592
ROLL0.3300.743, 0.072, 0.362, 0.914
META0.3600.868, 0.089, 0.323, 0.472
Expected Output residualsWHOL0.039**0.041**, 0.631, 0.616, 0.061
ROLL0.055*0.226, 0.348, 0.458, 0.101
META0.086*0.275, 0.573, 0.422, 0.094*
Output Uncertainty residualsWHOL0.5290.771, 0.400, 0.201, 0.657
ROLL0.1300.603, 0.470, 0.888, 0.024
META0.0520.170, 0.460, 0.863, 0.007
Dependent variableModelAll variablesVAR-E variablesMacro variablesIndividual macro variables (Δct,Δwt,Δhpt,Δftset, respectively)
Expectational errors
ytt1yteBalance0.001***0.009***0.2970.479, 0.089*, 0.284, 0.599
Rolling0.002***0.018**0.1940.255, 0.169, 0.136, 0.837
Meta0.001***0.002***0.2980.438, 0.108, 0.348, 0.592
Equation residuals
Actual Output residualsWHOL0.2980.438, 0.108, 0.348, 0.592
ROLL0.3300.743, 0.072, 0.362, 0.914
META0.3600.868, 0.089, 0.323, 0.472
Expected Output residualsWHOL0.039**0.041**, 0.631, 0.616, 0.061
ROLL0.055*0.226, 0.348, 0.458, 0.101
META0.086*0.275, 0.573, 0.422, 0.094*
Output Uncertainty residualsWHOL0.5290.771, 0.400, 0.201, 0.657
ROLL0.1300.603, 0.470, 0.888, 0.024
META0.0520.170, 0.460, 0.863, 0.007

Notes. Statistics refer to the p-values of the F-tests of the various null hypotheses based on the various models described in the text. Superscripts *, **, and *** highlight significance at the 10%, 5%, and 1% levels of significance, respectively.

These tests are typical of those found in the literature investigating the rationality of expectations formation. If expectations are formed with FIRE, expectation errors should have no systematic content and none of the exclusion restrictions should be rejected. In the event, we see that the errors do show systematic content, but this is entirely down to the lagged values of the variables in VAR-E. So, while this rejects full information rationality, it does provide support for the rational expectations models incorporating information rigidities outlined in, for example, Coibion and Gorodnichenko (2012). Here, the survey measures of expectations reflect current and past values of rational expectations, weighted according to the nature of the information rigidities, so that lagged values of the variables in VAR-E would capture all the systematic patterns in the expectational error. The other macro variables will still fail to show significantly since they do not contribute to the information content beyond that of the current and past values of the rational expectations embedded within the surveys as appears to be the case here.

The second set of results in Table 2 considers the residuals from the VAR-E model explaining actual output growth, expected output growth and uncertainty, and regresses these residuals on the macro variables. The residuals are obtained alternatively from the MWHOL, MROLL, and MMETA models (where, for the latter two, the residuals relate to the end-of-sample observation in each rolling regression). The lagged variables in VAR-E are orthogonal to these residuals by construction—so no test results are presented—but the exclusion tests on the macro variables again show that these have little or no significant explanatory power (with only the stock market price showing, at the 10% level, as potentially useful in the expected growth equations of MROLL and MMETA). The lack of explanatory power of the macro variables in the actual output growth residuals mirrors the results of the first set of results. But the lack of explanatory power in the expected output growth (and uncertainty) residuals shows that the lagged VAR-E variables are also sufficient for capturing the evolution of the expected output (and uncertainty) series. As discussed earlier, the relative stability of the expected output series is likely due to its dependence on the fundamentals driving growth, abstracting from shorter-term cyclical influences. These results suggest the lagged variables in VAR-E are able to adequately isolate the effects of changes in fundamentals from transitory cyclical influences so that lagged macro variables provide no additional insights.26

The results of Table 2 provide reassurance on the statistical adequacy of the VAR-E model with the simple 3-variable VAR-E model providing a parsimonious characterization of the determination of actual and expected output and of output uncertainty which cannot be improved upon by the inclusion of (at least these specific) macro variables. Parsimony is very useful here as it minimizes parameter uncertainty, and hence the accuracy of the derived BN trends, and limits the potential ambiguity surrounding the cycle-trend variance ratio highlighted by ER.

The system-wide dynamic properties of the VAR-E models

The system-wide properties of the estimated models can be best characterized by looking at the dynamic responses of output, expected output and uncertainty to shocks. This can be done imposing a causal ordering of the shocks through a Choleski decomposition. For example, we might assume that output decisions are made against a backdrop of uncertainty, so that the uncertainty variables are determined prior to the output variables. And, since expected output (tyt+1yt) includes yt by definition, it is reasonable to assume that actual output is determined before expected output. This allows us to identify the separate effects of uncertainty shocks, shocks to current output and, finally, shocks to expected future output.

Figures 5, 6, and 7 provide an indication of the sort of system dynamics obtained from the VAR-E models, again focusing on the whole sample model MWHOL for illustrative purposes. These figures set out the response of actual output, expected output and uncertainty to shocks to, respectively, uncertainty, actual output and expected output under the Choleski ordering discussed above. The shocks in each case are taken to be of a ‘typical’ (one standard deviation) size so the effects of the shocks on output, say, are comparable across the figures and give a sense of the historical importance of the different types of shock. For example, Figure 5 shows that uncertainty shocks have a permanent negative effect on actual and expected output, with a typical uncertainty shock causing output to be 0.4% lower than it would have been in the absence of the shock and with adjustment to the new output level taking 2–3 years. The impact effect on actual output of a typical shock to actual output in Figure 6 is around 0.8% and also initiates adjustments over the subsequent 2 years, with the ultimate effect around twice as large at 1.4%. The shape of the actual output response to a typical shock to expected output in Figure 7 is similar. It is worth noting that, in each case, the actual and expected output series converge to the same levels—by construction—but this convergence is not monotonic and can take up to 18 months, and again provides little support for a FIRE interpretation of the model (in which the output response would mirror that of the expectations response after one quarter).

Response of actual output, expected output, and uncertainty over output to an uncertainty shock using MWHOL.
Figure 5.

Response of actual output, expected output, and uncertainty over output to an uncertainty shock using MWHOL.

Response of actual output, expected output, and uncertainty over output to an actual output shock using MWHOL.
Figure 6.

Response of actual output, expected output, and uncertainty over output to an actual output shock using MWHOL.

Response of actual output, expected output, and uncertainty over output to an expected output shock using MWHOL.
Figure 7.

Response of actual output, expected output, and uncertainty over output to an expected output shock using MWHOL.

While the figures for MWHOL provide an indication of the sort of dynamics captured by our VAR-E models, they do not accommodate the time-variation in the processes underlying the determination of output captured by MROLL or MMETA. To show the importance of this time-variation, Figures 8 and 9 illustrate the system dynamics obtained from the MMETA analysis, looking at the effects on actual output of the a one-standard-deviation shocks to actual output at different points in the sample. To highlight the issues, Figure 8 plots the impulse responses for four specific dates, two of which look similar to those from the MWHOL model (namely 2010q2 and 2013q3) but two of which have much more muted responses (namely 2007q1 and 2017q3). Figure 9 shows the impulse responses obtained at every point in the sample; while it is difficult to provide any economic interpretation of the differences in the shape of the responses across time, the figure shows the effects of shocks are very different at different points in the sample. This highlights the importance of accommodating structural change in the modelling and will clearly have a considerable effect on the associated BN trends.

Response of actual output to an actual output shock at different times using MMETA.
Figure 8.

Response of actual output to an actual output shock at different times using MMETA.

Response of actual output to an actual output shock across times using MrmMETA.
Figure 9.

Response of actual output to an actual output shock across times using MrmMETA.

4.3 UK manufacturing output trends

We are interested not only in the general dynamic properties of the system and the impact of typical shocks but also in the practical importance of the different types of shocks and their contribution to trend output. Figure 10 plots the three variants of the Beveridge–Nelson (BN) output trend obtained from MWHOL, MROLL, and MMETA alongside actual output. As discussed at (3.9), the BN trend describes, at each point, the output level that would be obtained assuming all current and past shocks have played out and that no further shocks occur.27 The trends obtained from the three models are related to each other but they are qualitatively and quantitatively distinct, establishing again the importance of accommodating structural changes in their estimation. For example, the trend based on MWHOL follows actual output relatively closely compared to those based on the time-varying models and output falls by just 2.4% relative to trend during 2009 according to the MWHOL model, but by 7.6% according to MROLL and by 9.7% according to MMETA. And while there are similarities between the trends based on the gaps based on MROLL and MMETA, with the correlation between their corresponding gaps equal to 0.36, only the gap based on MMETA shows distinct cycles, varying between 2% and 2% and typically lasting 2–3 years from peak to trough.

Actual and trend output using MWHOL, MROLL, and MMETA.
Figure 10.

Actual and trend output using MWHOL, MROLL, and MMETA.

The intuition here is that the simpler MWHOL and MROLL models fail to adequately accommodate structural changes, misinterpreting these as the effects of permanent shocks and overstating the contribution of signal to that of noise. The associated trends follow actual output too closely, highlighting the importance of tracking the changing output data generating process as in MMETA.

Figure 11 plots the trend obtained according to MMETA alongside the HP trend and one obtained using the KMW method. As noted earlier, KMW base their trend on a univariate AR specification for output growth estimated subject to restrictions that ensure a reasonable signal:noise ratio (denoted δ). The chosen value of the ratio can be fixed according to the investigator’s ‘dogmatic prior’ or selected by choosing the largest ratio obtained searching across the ratios obtained from the corresponding AR models. Interestingly, KMW find there is little difference in the gaps obtained with US data using a dogmatic prior of δ=0.05 or their selected measure δ=0.24, broadly corresponding to cases where the size of noise is between one and four times the size of the signal. This highlights the relative reliability of any gap estimated where the signal:noise ratio takes sensible values.

Output trends using MMETA, the HP filter and the KMW approach.
Figure 11.

Output trends using MMETA, the HP filter and the KMW approach.

As it turns out, this observation carries over to our data, with the correlation between the KMW gap measures obtained with δ=0.05 or δ=0.26 (as chosen following a grid search) equal to 0.99. More strikingly, the correlation between the KMW gap measure and the gap based on MMETA is also very high, at 0.95.28 Although the MMETA model allows the signal:noise ratio to be freely estimated and to vary over time, it delivers estimates of the ratios and associated gaps which are entirely in line with the ‘intuitive and reliable’ estimates obtained using KMW’s more restrictive method.

Finally here, we consider an exercise to highlight the role of uncertainty in the evolution of the output trend. Specifically, under the assumed Choleski ordering, the BN trend can be decomposed to show the contribution of the separate orthogonlized shocks to the overall trend. Figure 12 plots again the actual output level and the trend based on MMETA but now also plots the BN trend that would be obtained from that model assuming no uncertainty shocks occurred after 2008q2. This trend lies around 1% higher than the trend incorporating uncertainty shocks by 2011, showing that uncertainty played an important role in the downturn following GFC although—to put it into perspective—the larger part of the overall 5% reduction in the trend was due to the first moment shocks to actual and expected output. Having said that, the difference between the trends incorporating and excluding the effects of uncertainty shocks widens further to 1.5%–2% over the subsequent years, and especially from 2016 onwards, which might be attributable to the apprehension surrounding the outcome of the EU referendum.

Actual and trend output using MMETA with and without uncertainty shocks.
Figure 12.

Actual and trend output using MMETA with and without uncertainty shocks.

5 Concluding remarks

Measures of trend output, and the associated output gap, are key ingredients in policy making and in understanding macroeconomic dynamics. A popular and natural measure of trend output is the BN trend and the key to obtaining a reliable measure of the BN trend is basing it on a ‘reasonable’ signal:noise ratio that acknowledges the dominance of transitory shocks over permanent shocks in output movements. Survey measures of expected future outputs are relatively stable over time and we have argued that they can be used to identify transitory and permanent shocks to output, to judge the size of the signal:noise ratio and to derive reliable BN trends and associated gaps through a VAR-E.

Further, by analysing the relationship between output and backward-looking survey responses, the surveys also help to expose the time variation in the processes underlying output determination. The ‘meta’ modelling approach provides a very useful vehicle for characterizing this time variation, improves the measures of expected future outputs, and helps avoid misinterpreting structural changes as ‘signal’ in the estimation of the signal:noise ratio.

The empirical exercise of the article illustrates these features and the use of survey data in the context of UK Manufacturing, providing a compelling characterization of output fluctuations over the last 20 years, including insight on the role played by uncertainty.29 The reliable and intuitive trends and gaps presented in the article benefit from the dual ability of the VAR-E model presented here to capture the effects of transitory and permanent shocks and to track changes in the processes underlying output determination. It will be interesting in subsequent work to see how the derived gap measures perform in explaining inflation or interest rate behaviour or in other macroeconomic contexts.

Acknowledgments

We are grateful to Paul Mizen, James Morley, Kalvinder Shields, and Ben Wong for helpful discussions and to Nicola Grimwood, Ben Jones, and Anna Leach at the CBI for help and advice on accessing the CBI Suite of Business Surveys.

Funding

Financial support from the Office for National Statistics through the Economic Statistics Centre of Excellence is gratefully acknowledged.

Supplementary material

Supplementary material is available online at Journal of the Royal Statistical Society: Series A.

Appendix

A. 1 An illustrative model with transitory and permanent shocks

Assume the true model is

with two types of shock, vt and ωt being independent and with variance σv2 and σω2 respectively. The corresponding ARIMA process is

where θ and the variance of ut, σu2, are obtained matching the variance and covariance terms of the two characterizations; i.e. the solution to

( A1)

The expressions in (A1) provide

( A2)

from which it is readily shown that σω2+σv2<σu2 for any θ[1,0]. Eliminating σu2 from the expressions in (A1) delivers a quadratic in θ, the solution of which gives

establishing that θ[1,0] with the value depending on the relative size of the two types of shock. Further, from (A2), we can show that (1+θ)>σv2σv2+σω2 as

Finally, noting that the variance of tyt+1yt is ρ2σv2+(1ρ)2σω21ρ2, the ratio of the variance of actual output growth to that of expected output growth is

A.2 The relationship between VAR-E and MA representations of actual and expected growths

The vector containing actual and expected output growth can be written in levels noting that

where zt=(yt, tyt+1e). The VAR-E model in (2.7) can be readily written as

so that

where Φ1=M01(BM0M1), Φ2=M01BM1 and εt=M01(ξ1t, ξ2t)=(ξ1t, ξ1t+ξ2t). The model can the be readily written in the form of a cointegrating VAR in the differences Δzt=(Δyt, Δtyt+1e) as follows

where Π=(IΦ1Φ2)=[b121b12b22][1,1], reflecting the role of the cointegrating vector [1,1] which ensures the levels of actual and expected output are cointegrated and expectational errors are stationary.

The corresponding MA representation for Δzt is

where C(L)=C0+C1L+C2L2+ and C0=I, C1=Φ1I, C2=C1Φ1+Φ2, and Ci=Ci1Φ1+Ci2Φ2 for all i3. Summing the Ci’s we find

so

and C(1) takes the form

with the levels of actual and expected output driven by the single stochastic trend ξ1t+b121b12b22ξ2t.

In the illustrative example of Section 2.1, where B=[010ρ] and ξ1t=vt+ωt and ξ2t=ρvt(1+ρ)ωt, we have Φ1=[01ρ1+ρ], Φ2=0, Ci=ρi1[11ρρ] for i=1,2,, and εt=[vt+ωtρvt(1+ρ)ωt]. In this case,

and the long run trend is independent of the short-lived shocks ωt.

Assume now that the survey responses measure the FIRE with error so that

where tyt+1e remains the reported survey response and is distinguished from the FIRE tyt+1* by the measurement error ηt introduced in the time-t survey. Then, from (2.3),

so (2.7) holds with ξt defined by this error structure. Using the M01 transformation,

and

and the long-run properties of the system remain independent of short-lived shocks ωt and measurement errors ηt.

References

Alcidi
C.
,
Flamini
A.
, &
Fracasso
A.
(
2011
).
Policy regime changes, judgment and Taylor rules in the Greenspan era
.
Economica
,
78
(
309
),
89
107
.

Aristidou
,
C.
,
Lee
,
K.
, &
Shields
,
K.
(
2022
).
Fundamentals, regimes and exchange rate forecasts: Insights from a meta exchange rate model
.
Journal of International Money and Finance
,
123
(C),
102601
.

Aristidou
C.
,
Lee
K. C.
, &
Shields
K.
(
2019
).
Evaluating the use of real-time data in forecasting output levels and recessionary events in the US
.
Journal of the Royal Statistical Society A
,
182
(
1
),
131
163
.

Bachmann
R.
,
Elstner
S.
, &
Sims
E. R.
(
2013
).
Uncertainty and economic activity: Evidence from business survey data
.
American Economic Journal: Macroeconomics
,
5
(
2
),
217
249
.

Baker
S. R.
,
Bloom
N.
, &
Davis
S. J.
(
2016
).
Measuring economic policy uncertainty
.
The Quarterly Journal of Economics
,
131
(
4
),
1593
1636
.

Barrero
J. M.
,
Bloom
N.
, &
Wright
I
(
2017
).
Short and long run uncertainty, NBER Working Paper 23676
.

Blanchard
,
O
, &
Quah
,
D
. (
1989
).
The dynamic effects of aggregate supply and demand disturbances
.
American Economic Review
,
79
,
655
673
.

Bloom
N.
(
2009
).
The impact of uncertainty shocks
.
Econometrica
,
77
(
3
),
623
685
.

Bloom
N.
(
2014
).
Fluctuations in uncertainty
.
Journal of Economic Perspectives
,
28
(
2
),
153
176
.

Boero
G.
,
Smith
J.
, &
Wallis
K. F.
(
2015
).
The measurement and characteristics of professional forecasters’ uncertainty
.
Journal of Applied Econometrics
,
30
(
7
),
1029
1046
.

Bordalo
P.
,
Gennaioli
N.
,
Ma
Y.
, &
Shleifer
A.
(
2020
).
Overreaction in macroeconomic expectations
.
American Economic Review
,
110
(
9
),
2748
2782
.

Clements
M. P.
(
2017
).
Assessing macro uncertainty in real-time when data are subject to revision
.
Journal of Business and Economic Statistics
,
35
(
3
),
420
433
.

Cochrane
J. H.
(
1994
).
Permanent and transitory components of GNP and stock prices
.
The Quarterly Journal of Economics
,
109
(
1
),
241
265
.

Cogley
T.
, &
Sargent
T. J.
(
2005
).
Drifts and volatilities: Monetary policies and outcomes in the post WWII US
.
Review of Economic Dynamics
,
8
(
2
),
262
302
.

Coibion
O.
, &
Gorodnichenko
Y.
(
2012
).
What can survey forecasts tell us about information rigidities?
Journal of Political Economy
,
120
(
1
),
116
159
.

Coibion
O.
, &
Gorodnichenko
Y.
(
2015
).
Information rigidity and the expectations formation process: A simple framework and new facts
.
American Economic Review
,
105
(
8
),
2644
2678
.

Coyle
D.
, &
Mei
J.
(
2022
).
Productivity slowdown: Which sectors matter and why?
Productivity Institute WP
.

Croushore
D
(
2010
).
Philadelphia fed forecasting surveys: Their value for research
.
Business Review, Federal Reserve Bank of Philadelphia
,
3
,
1
11
.

Davis
S.
, &
Kahn
J.
(
2008
).
Interpreting the great moderation: Changes in the volatility of economic activity at the macro and micro levels
.
Journal of Economic Perspectives
,
22
(
4
),
155
180
.

Dovern
J.
,
Fritsche
U.
,
Loungani
P.
, &
Tamirisa
N.
(
2015
).
Information rigidities: Comparing average and individual forecast for a large international panel
.
International Journal of Forecasting
,
31
(
1
),
144
154
.

Dovern
J.
,
Fritsche
U.
, &
Slacalek
J.
(
2012
).
Disagreement among forecasters in G7 countries
.
Review of Economics and Statistics
,
94
(
4
),
1081
1096
.

Eusepi
S.
, &
Preston
B.
(
2011
).
Expectations, learning, and business cycle fluctuations
.
American Economic Review
,
101
(
6
),
2844
2872
.

Evans
G
, &
Honkapohja
S
(
2001
).
Learning and expectations in macroeconomics
.
Princeton University Press
.

Evans
G.
, &
Reichlin
L.
(
1994
).
Information, forecasts, and measurement of the business cycle
.
Journal of Monetary Economics
,
33
(
2
),
233
254
.

Garratt
,
A.
,
Lee
,
K.
, &
Shields
,
K.
(
2016a
).
Information rigidities and the news-adjusted output gap
.
Journal of Economic Dynamics and Control
,
70
(C),
1
17
.

Garratt
A.
,
Lee
K.
, &
Shields
K.
(
2016b
).
Forecasting global recessions in a GVAR model of actual and expected output
.
International Journal of Forecasting
,
32
(
2
),
374
390
.

Garratt
A.
,
Lee
K. C.
, &
Shields
K.
(
2018
).
The role of uncertainty, sentiment and cross-country interactions in G7 output dynamics
.
Canadian Journal of Economics
,
51
(
2
),
391
418
.

Garratt
A.
,
Robertson
D.
, &
Wright
S.
(
2006
).
Permanent vs transitory components and economic fundamentals
.
Journal of Applied Econometrics
,
21
(
4
),
521
542
.

Goldin
I.
,
Koutroumpis
P.
,
Lafond
F.
, &
Winkler
J.
(
2023
).
Why is productivity slowing down?
Journal of Economic Literature
62
(
1
),
196
.

Goodridge
P.
, &
Haskell
J.
(
2022
).
Accounting for the slowdown in UK innovation and productivity
.
Productivity Institute WP 022
.

Gordon
R
(
2016
).
The rise and fall of American growth: The U.S. standard of living since the civil war
.
Princeton University Press
.

Hodrick
R. J
(
2020
).
An exploration of trend-cycle decomposition methodologies in simulated data. NBER Working Paper 26750
.

Jo
S.
, &
Sekkel
R.
(
2019
).
Macroeconomic uncertainty through the lens of professional forecasters
.
Journal of Business and Economic Statistics
,
37
(
3
),
436
446
.

Jurado
K.
,
Ludvigson
S. C..
, &
Ng
S.
(
2015
).
Measuring uncertainty
.
American Economic Review
,
105
(
3
),
1177
1216
.

Kamber
G.
,
Morley
J.
, &
Wong
B.
(
2018
).
Intuitive and reliable estimates of the output gap from a univariate Beveridge–Nelson filter
.
The Review of Economics and Statistics
,
100
(
3
),
550
566
.

Keating
J.
, &
Valcarcel
V.
(
2015
).
The time-varying effects of permanent and transitory shocks to real output
.
Macroeconomic Dynamics
,
19
(
3
),
477
507
.

Koopman
S. J.
,
Lee
K. M.
, &
Wong
S. Y.
(
2006
).
Trend-cycle decomposition models with smooth-transition parameters: Evidence from U.S. economic time series, Chapter 8 in
.
Contributions to Economic Analysis
,
276
,
199
219
.

Lahiri
K.
, &
Liu
F.
(
2006
).
Modelling multi-period inflation uncertainty using a panel of density forecasts
.
Journal of Applied Econometrics
,
21
(
8
),
1199
1219
.

Lahiri
K.
, &
Sheng
F.
(
2010
).
Measuring forecast uncertainty by disagreement: The missing link
.
Journal of Applied Econometrics
,
25
(
4
),
514
538
.

Lahiri
K.
, &
Sheng
X.
(
2008
).
Evolution of forecast disagreement in a Bayesian learning model
.
Journal of Econometrics
,
144
(
2
),
325
340
.

Lee
K. C.
,
Mahony
M.
, &
Mizen
P.
(
2020
).
Nowcasting output movements and recessionary events using the CBI’s suite of business surveys; A technical note. Working Paper, University of Nottingham
.

Lee
K. C.
,
Morley
J.
, &
Shields
K.
(
2015
).
The meta Taylor rule
.
Journal of Money, Credit and Banking
,
47
(
1
),
73
98
.

Lettau
M.
, &
Ludvigson
S. C.
(
2001
).
Consumption, aggregate wealth and expected stock returns
.
The Journal of Finance
,
56
(
3
),
815
849
.

Ludvigson
S.
,
Ma
S.
, &
Ng
S.
(
2021
).
Uncertainty and business cycles: Exogenous impulse or endogenous response?
American Economic Journal: Macroeconomics
,
13
(
4
),
369
410
.

Mankiw
N. G.
, &
Reis
R.
(
2002
).
Sticky information versus sticky prices: A proposal to replace the new Keynesian Phillips curve
.
The Quarterly Journal of Economics
,
117
(
4
),
1295
1328
.

Mitchell
J.
,
Mouratidis
K.
, &
Weale
M.
(
2007
).
Uncertainty in UK manufacturing: Evidence from qualitative survey data
.
Economics Letters
,
94
(
2
),
245
252
.

Morley
J.
, &
Piger
J.
(
2008
).
Trend/cycle decomposition of regime-switching processes
.
Journal of Econometrics
,
146
(
2
),
220
226
.

Morley
J.
, &
Piger
J.
(
2012
).
The asymmetric business cycle
.
Review of Economics and Statistics
,
94
(
1
),
208
221
.

Oh
K. H.
,
Zivot
E.
, &
Creal
D.
(
2008
).
The relationship between the Beveridge–Nelson decomposition and other permanent–transitory decompositions that are popular in economics
.
Journal of Econometrics
,
146
(
2
),
207
219
.

Pesaran
M. H
(
1987
).
The limits to rational expectations
.
Blackwell
.

Pesaran
M. H.
, &
Timmermann
A.
(
2007
).
Selection of estimation window in the presence of breaks
.
Journal of Econometrics
,
137
(
1
),
134
161
.

Pesaran
,
M. H.
, &
Weale
,
M.
(
2006
). Survey expectations. In
G.
Elliott
,
C.
Granger
,
A.
Timmermann
(Eds.),
Handbook of Economic Forecasting
(Vol.
1
).
Elsevier
.

Romer
C. D.
(
1999
).
Changes in business cycles: Evidence and explanations
.
Journal of Economic Perspectives
,
13
(
2
),
23
44
.

Sims
C. A.
(
2003
).
Implications of rational inattention
.
Journal of Monetary Economics
,
50
(
3
),
665
690
.

Terasvirta
T.
, &
Anderson
H. M.
(
1992
).
Characterizing nonlinearities in business cycles using smooth transition autoregressive models
.
Journal of Applied Econometrics
,
7
(
S1
),
S119
S136
.

Teulings
C.
, &
Baldwin
R.
(
2014
).
Secular stagnation: Facts, causes and cures
.
CEPR Press
.

Woodford
M
(
2001
).
Imperfect common knowledge and the effects of monetary policy. In P. Aghion, R. Frydman, J. Stiglitz, & M. Woodford (Eds.), Knowledge, information, and expectations in modern macroeconomics: In honor of Edmund Phelps. Princeton University Press
.

Footnotes

1

Oh et al. (2008) show how the BN decomposition is closely related to decompositions based on unobserved components models of a variety of forms and can also be related to non-model based signal extraction filters such as the HP filter.

2

The general improvement in BN trend measures obtained through multivariate models is explored in Garratt et al. (2006).

3

The insights provided by the joint use of actual and expected series in modelling output are also exploited in Garratt et al. (2016a, 2016b).

4

Examples include Bloom (2009), Baker et al. (2016), and Barrero et al. (2017), inter alia.

7

We end the analysis before the effects of the coronavirus pandemic to focus attention on the modelling strategy and the usefulness of direct measures in this.

8

If the survey responses measure the FIRE with (unsystematic) error, there is an additional source of innovation to the actual and expected outputs. As shown in the Appendix, the effects of the short-lived shocks and measurement error are confounded in this case, but the effects of the permanent shocks can still be distinguished.

9

A general discussion of the relationship between the VAR-E and MA representations of actual and expected growths is given in the Appendix.

10

Different restrictions can be motivated by specific forms of information rigidity or learning processes for example. See Garratt et al. (2018) for further discussion.

11

If the measures were available, the bivariate model of (2.7) could be readily extended to include direct measures of expectations further into the future (for example tyt+2etyt+1e) or additional measures of expectations over the same forecast horizon from alternative surveys. Assuming all expectational errors are stationary, there would still be a single stochastic trend driving output and the BN trend.

12

The approach to accommodating structural change can be applied equally to quantitative survey data so long as there is information on the cross-firm distribution of outcomes/expectations as well as the average outcome/expectation.

13

See Pesaran (1987) for a more detailed discussion.

14

These expressions also provide the motivation for using the balance statisticBt=RtFt as an indicator of growth since, in the special case where α=β, (3.2) simplifies to ytyt1¯=α×Bt and tyt+1eyt¯=α×tBt+1 and actual and expected growth move proportionately with the backward-looking and forward-looking balance statistics, respectively.

15

An alternative modelling approach would be to capture the uncertainty based on the squared deviations of actual from expected output by embedding the VAR-E in a GARCH-in-Mean framework. The two-step approach we adopt, measuring uncertainty through the GARCH model at (3.3) and including this in the extended VAR-E, is computationally more convenient and easier to understand. Both approaches, when estimated using the meta modelling approach, allow for considerable time-variation in uncertainty and its effect on output.

16

The uncertainty variable ty~t+1u is assumed stationary so its inclusion in (3.8) simply adds a third source of shocks. There remains a single stochastic trend driving permanent movements in actual and expected output although some part of that might now be related to shocks to uncertainty.

17

Manufacturing’ covers Sections B, C, and F of the UK’s Standard Industrial Classification 2007.

18

A detailed description of the questions posed in the survey, the sample frame, the characteristics of the firm participants, etc. is provided in Lee et al. (2020).

19

The Bank of England’s Decision Maker Panel is an equivalent, but more carefully designed, survey of firms’ expectations but this was only established in 2016.

20

The data used in the analysis, and the code used to generate the results below, are available from online supplementary material.

21

The underlying regressions in these models were estimated subject to the restriction that α and β remain positive throughout. In practice, this meant restricting β=0.01 on those occasions when the unrestricted β became negative.

22

The shortest sample length considered reasonable for estimation was 2 years. This is also the minimum length of time assumed to be required for a break to be recognized (so that breaks occur 8 periods before the shift in weight). Break points were identified by rejecting the null of no break described in (3.6) based on F-tests operating at the 1% significance level.

23

The preferred specification took the form σt=0.62+0,35ϵt12+0.28ϵt220.13ϵt320.23ϵt42 . The inclusion of a lagged σt1 term was not helpful.

24

In practice, for observation T, Model MMETA weights were calculated as follows: Step 1—assign observations T8,,T a weight of 1; Step 2—assign observations i=T9,,T24 weights 1s=8iws,T based on the meta-modelling exercise; Step 3—to ensure the weighted least squares regressions can be estimated over 25 observations in all periods, add the weights of Step 2 to a set of declining linear weights running 1,2425,,125 from T to T25; and Step 4 scale the weights of Step 3 to sum to one.

25

There were occasions, in extreme periods such as the onset of GFC, where MROLL and MMETA delivered poorly determined parameter estimates implying dynamic instability in the system. In these cases, and with a focus on the estimation of BN trends, we imposed parameter restrictions to constrain the long-run effects of shocks to have the same long-run response as shocks in MWHOL.

26

If the survey asked for respondents’ expectations of output at the infinite horizon, it would provide a direct measure of the BN trend and no additional variable would help in its measurement. Cochrane (1994) makes a related point in a bivariate analysis of output and consumption in which the two are cointegrated (in the same way our actual and expected series are cointegrated) and consumption growth is close to unpredictable (as are our expectational errors). In Cochrane’s context, ‘consumption summarizes consumers’ information on long run GNP so other variables are superfluous’ and the direct measure of expected future output plays the same role here.

27

In practice, the estimated model provides measures of the shocks and, based on the estimated parameters, their accumulated effect which drives the change in the BN trend. The level of the BN trend is obtained assuming the average of the output gap is zero over the sample.

28

As a point of comparison, the correlation with the gap based on the HP trend is 0.76.

29

Clearly, the analysis does not cover the period of the pandemic. Firms’ survey responses showed an immediate and dramatic reaction to the outbreak of the coronavirus disease. But they did not fully reflect the unprecedented fall in UK manufacturing output—falling by 28% in the first quarter of the pandemic—or the subsequent interventions and dynamic responses. An ‘extreme value’ treatment of the pandemic period is left for future work.

Author notes

Conflicts of interest: None declared.

This is an Open Access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted reuse, distribution, and reproduction in any medium, provided the original work is properly cited.

Supplementary data