Bhatt, Ferguson, Flaxman, Gandy, Mishra, and Scott's reply to the Discussion of ‘The Second Discussion Meeting on Statistical aspects of the Covid-19 Pandemic’

We would like to thank the Royal Statistical Society, and all of the discussants and editors for facilitating an important discussion and for all of their important contributions during the pandemic.

Professor Diggle gives insightful comments on problems that arise through discretisation of space, and when different spatial scales are mixed in a model, as occurs commonly for pragmatic reasons. The effect of this in epidemiological models would be of interest to explore in more detail.

As to Prof. Diggle’s point of what reproduction numbers are exactly estimating; indeed the precise interpretation can differ amongst different models, they usually aim to give an average number of infections caused by an infected individual; the key difference is of course what group this is being averaged over, and if these groups become small then estimates can potentially become very noisy. To aid decision makers, reporting results at some aggregation level will probably always be necessary.

As Prof. Diggle correctly points out, whether case numbers (or other observations) are representative of the underlying epidemic is a key concern, particularly due to informed missingness. For example, limited availability of testing in the early parts of the pandemic led to unreliable case numbers. In our work, we often found that as long as the missingness is spatially and temporally stable, it is still possible to estimate reproduction numbers; however, when it comes to projecting population saturation due to acquired immunity, i.e. when the epidemic will turn, then it is indeed crucial to tie the observed counts to the underlying infections; population surveys, such as REACT seem to be a necessary (albeit imperfect) tool for this. We would also like to point out that our framework allows us to coherently integrate different observations, i.e. infections, case counts, hospitalisations, and deaths among others to provide a more reliable estimate (Pakkanen et al., 2023).

Finally, Prof. Diggle’s point about discretising space is an important area of future research. While the model-based geostatistics framework pioneered by Prof. Diggle is a sensible starting point, we caution that spatial correlation is likely to be highly complex and driven by an interaction of mobility, the economy, culture, behaviour, and other aspects. As a result, simple covariance functions are unlikely to fully resolve the spatial aspect.

Professor Richardson rightly points out that ascertainment bias is important and that randomised surveillance seems to be the main way of estimating the amount of bias. As mentioned above, assuming stability in the ascertainment bias, it is still possible to estimate reproduction numbers. Combining data from different sources is indeed possible in our model; conflicting evidence may point to deficiencies in the underlying model—thus it might sometimes be beneficial to present estimates from different types of data separately; differences in these estimates could lead to important insights about the state of the epidemic and possible model improvements, from including age structure (Monod et al., 2021) to modelling variants of concern (Faria et al., 2021).

A limitation of the two-stage approach is indeed the propagation of uncertainty. A joint approach as taken in Flaxman et al. (2020) is always preferable but at a considerably larger computational cost with the possibility of harder to sample posteriors. The two-stage approach therefore represents a compromise. One of the challenges in our model is that the key object of interest is $R(t)$⁠, for which no ground truth exists. Hence, it is not possible to directly assess residual confounding for $R(t)$⁠. As Prof. Richardson points out, causal interpretations almost always rely on strong assumptions, and thus should be used cautiously.

We agree with Prof. Richardson that caution should be taken with shrinkage priors; a sensible choice is the horseshoe prior by Piironen and Vehtari (2017) that balances variable selection and collinearity. As to the source of the mobility data, we used the Community Mobility Report provided by Google (https://www.google.com/covid19/mobility/).

Dr Chind points out the value of research’s reproducibility, which is an important issue. Thankfully, it is becoming standard that code is being provided with papers, and we always strive to make code available in our own research. The policies of the journals of the RSS do indeed strongly encourage this. As pointed out, successfully executing the code provided with articles can sometimes be difficult; one key problem is that the programming languages and packages used in the code evolve. One promising approach for long-term reproducibility is to provide code in containers that fix the environment in which the code runs (see e.g. Clyburne-Sherin et al., 2019).

Professor Lawson raises several points. Indeed, models can have varying degree of fit, particularly around peaks and early on in the pandemic. This can be problematic, as estimates can be heavily influenced by factors not accounted for in models. This is why models need to have sufficient flexibility to adapt to the observed data to provide fits throughout an epidemic. One example of this is that observed counts are generally overdispersed compared to Poisson counts and this is why we generally work with negative binomial observation models. As pointed out, a similar effect could be achieved in a Bayesian setting by including a latent variable for each observation; due to computational efficiencies we did not choose to do this.

Professor Mateu and Dr Briz-Red raise a series of very important points. We again stress a critical area of research is into the spatial aspect of spread, both in terms of fine grained data (such as mobility) but also new analytical models with a mechanism for the spatial spread of contagion.

The contribution by Dr Bong, Prof. Ventura, and Prof. Wassermann raises very interesting and valid points. The issue of identifiability is very important; it might indeed be easier to discuss it in a frequentist setting but it would be present in both frequentist and Bayesian models. Even very simple models, with few parameters, lead to identifiability issues, e.g. a model based on cases cannot distinguish between fast initial growth of an epidemic due to high $R0$ vs. many imported infections. In our experience, the inclusion of epidemiologically informed priors in a Bayesian framework is the most effective way to ensure identifiability.

We agree that causal questions could be answered using different tools, all of which have their own assumptions, advantages, and drawbacks. We look forward to reading their derivation of the g-formula and Marginal Structural Models for our model. A formal consideration of mediation in our model is important, and could go beyond mobility to consider, for example, social factors such as attitudes and risk perceptions. We hope their causal approaches will generate future avenues of research, and be put to the test in real settings.

The complexities of mechanistic models of infectious disease spread, implemented within a statistical framework, requires a careful assessment of the mechanism, robust inference, attention to causality and confounding, and an accounting of identifiability. Pandemic response leaves little time to fully explore the interplay among these factors and their impact on estimates. We hope that statistical research undertaken while we await the next pandemic can be used to refine analytic methods and decisively map the data sources, analytical approaches, diagnostics, and questions so as to begin resolving the issues discussed here.

References

Author notes