Abstract
Ecosystem models, specifically multispecies dynamic models, have been increasingly used to project impacts of fishing activity on the trophodynamics of ecosystems to support ecosystem-based fisheries management. Uncertainty is unavoidable in modelling processes and needs to be recognized and properly quantified before models are utilized. Uncertainty was assessed in this study for a multispecies size-spectrum model that quantifies community structure and ecological characteristics. The uncertainty was assumed to result from errors in fish life-history and metabolic scale parameters, environmental variability, fishing variability, and sampling errors. Given the same level of imprecision, metabolic scale parameters had the dominant influence on the uncertainty of the size spectrum modelling results, followed by life-history parameters. Both types of errors led to “scenario uncertainty”, suggesting the possible existence of alternative states of community structure. Environmental variability, fishing variability, and observation errors resulted in “statistical uncertainty”, implying that such uncertainty can be described adequately in statistical terms. The results derived from such a simulation study can provide guidance for identifying research priorities to help narrow the gap in scientific knowledge and reduce the uncertainty in fisheries management.
Introduction
Fishing activity is believed to have a great influence on species compositions and ecosystem trophodynamics (Jennings and Polunin, 1996; Jennings and Kaiser, 1998). With the growing realization of the need to consider broader ecosystem and socio-economic effects of fisheries, ecosystem-based fisheries management has long been proposed to address crises in fisheries worldwide (Garcia et al., 2003; Hall and Mainprize, 2004; Garcia and Cochrane, 2005; Scandol et al., 2005). The tenet of such an approach is to recognize the complex dynamic interactions among ecological, economical, and social components of ecosystem and to promote sustainability and biodiversity in an effort to optimize ecosystem services (Garcia et al., 2003; Pikitch et al., 2004).
The inherent complexity and dynamics of ecosystems are often poorly understood (Link et al., 2009; Sainsbury et al., 2000), because the complexity involves different levels of interactions among components (Levin, 1999). To support fisheries management “towards broader ecosystem objectives”, ecosystem models have been increasingly used to project marine ecosystem dynamics while explicitly considering interactions among species and other ecological components (Plagányi, 2007; Greenstreet et al., 2010; Houle et al., 2012).
Fisheries management has traditionally been based on models projecting the consequences of regulations (Beverton and Holt, 1957). Scientific research is often expected to provide accurate predictions; however, “uncertainty” is an unavoidable part of the modelling process, highlighted by an increasing attention to the “precautionary principle” in management processes (Foster et al., 2000). Information with high uncertainty is less valuable and even risky for management purposes as it may lead to detrimental decisions. A better understanding of uncertainty and its potential impacts is thus essential for informing fisheries management (Walker et al., 2003; Stelzenmüller et al., 2015).
Uncertainty exists in every step of modelling due to a poor understanding of ecosystem structure and function. Parameters need to be estimated by a model fitting process in quantitative models, which requires sufficient and representative data that are not always available. The majority of global fisheries are reported to lack enough catch or survey data needed for adequate stock assessment and many stocks are not assessed at all. The proportion of assessed fish stocks ranges between 10 and 50% in developed countries and is much lower in developing countries (Costello et al., 2012).
Although uncertainty is influential and ubiquitous in both science and management, the definition and classification is not commonly agreed upon in different research fields (Walker et al., 2003; Refsgaard et al., 2007). Walker et al. (2003) generally defined uncertainty as “any deviation from the unachievable ideal of completely deterministic knowledge of the relevant system” and characterized uncertainty with three dimensions: “location” (i.e. the context where uncertainty takes place), “level” (i.e. the continuum between deterministic knowledge and total ignorance), and “nature” (i.e. inherent variability or imperfection of knowledge). This framework is helpful to recognize and control uncertainty (Planque et al., 2011). However, evaluating uncertainty for an ecosystem model is often difficult due to complex model structure and the large number of parameters with multiple error sources. The structure and magnitude of uncertainty are poorly understood and rarely dealt with systematically for most ecological modelling studies (Refsgaard et al., 2007; Stelzenmüller et al., 2015). This has led to a tendency of “overconfidence in model projections” (Brander et al., 2013).
Sources of uncertainty can be characterized in different ways, such as by context and framing, input, model structure, parameter estimation, and model technique (Walker et al., 2003; Refsgaard et al., 2007). Planque et al. (2011) summarized the major sources of uncertainty in predictive models as “observation process, conceptual and numerical model formulations, parameter estimation, model evaluation, inappropriate spatial and temporal scales, and potential adaptation of living systems”.
In this study, we distinguished “process error” and “observation error” concerning the “location” of uncertainty (Walker et al., 2003) and referred to process errors as the errors involved in model building and observation errors as sampling inaccuracy (Planque et al., 2011). Errors were also classified into two main categories based on the nature of uncertainty in this study: (i) epistemic uncertainty resulting from imperfect knowledge, which may be reduced by more research efforts, e.g. limited data, measurement error, imperfect models, etc.; and (ii) stochastic uncertainty, or ontological uncertainty, generated by inherent variability, which is non-reducible, e.g. climate variability, social, economic, and cultural dynamics (Walker et al., 2003; Refsgaard et al., 2007). The purpose of this classification is to distinguish the approaches in dealing with stochastic and epistemic uncertainty.
We developed a Monte Carlo simulation approach and used a multispecies size-spectrum model as an example to illustrate the application of this approach to evaluate and quantify uncertainty. The size-spectrum model is a biological process model with relatively simple but robust assumptions. It formulates biological processes at an individual level and explicitly considers the trophic interactions among marine species (Andersen and Beyer, 2006, 2015; Pope et al., 2006). There is a growing literature describing such ecosystem models that aim to explore foodweb and ecosystem properties (Silvert and Platt, 1978; Benoît and Rochet, 2004; Law et al., 2009; Hartvig et al., 2011), elucidate processes determining fish population and ecosystem dynamics (Rossberg, 2012), simulate impacts of fishing on trophic cascades (Andersen and Pedersen, 2010), evaluate fisheries reference points (Andersen and Beyer, 2015), and predict the dynamics of fish communities (Blanchard et al., 2014; Thorpe et al., 2015).
The use of multispecies models is limited in tactical management because uncertainty undermines the confidence of stakeholders. More transparent evaluation of model uncertainty can be helpful to build the trust and to facilitate the usages. We assessed uncertainty of the size-spectrum model resulting from observation and process errors and evaluated the consequent uncertainty associated with community structure and ecological characteristics. The objectives of this study are to evaluate the impacts of input imprecision on model performance and to identify the structure, characteristics, and magnitude of resultant uncertainty in model prediction. We (i) quantified the contribution of various error sources to the total uncertainty and identified the dominant sources and (ii) examined how uncertainty increased with the error in model parameters and assessed the efficiency of reducing input error in improving modelling precision. This study provides guidance for identifying research priorities to reduce uncertainty in fisheries management (Stelzenmüller et al., 2015) and help narrow the gap in scientific knowledge for developing the management policy (Bradshaw and Borchers, 2000; Bijlsma et al., 2011).
Materials and methods
Size-spectrum model
This model describes the main biological processes of food intake, metabolism, growth, reproduction, and mortality at the individual level with submodels (Supplementary Table S1). The main assumptions of this model include: (i) food preference follows a lognormal selection curve in terms of the weight ratio between predators and prey (Ursin, 1973); (ii) predator–prey encounter is described by the “Andersen–Ursin” encounter model, in which the available food is determined by integrating over all species and the background resources (Andersen and Ursin, 1977); (iii) the consumed food is assimilated with an efficiency α and used for standard metabolism, and the remaining energy is divided between growth and reproduction (Hartvig et al., 2011); (iv) the stock–recruitment relationship is described by the Beverton–Holt recruitment model (Beverton and Holt, 1957); and (v) there are four sources of mortality, predation, starvation, fishing, and a constant background mortality (Peters, 1983; Hartvig et al., 2011). The multispecies size spectrum model is implemented by the R package “mizer” (Scott et al. 2014).
Data sources and model parameterization
We developed the multispecies size-spectrum model using data collected from fish community surveys in Haizhou Bay. Haizhou Bay is located in a temperate area of China (Figure 1) and historically served as a productive fishing ground, providing nursery and spawning habitat for many commercially important fish species. However, fisheries production in Haizhou Bay has dropped dramatically since the end of last century causing most trawling fisheries to relocate. Large changes have occurred in species compositions and biomass in the bay (Tang et al., 2011).
Location and sampling design of survey in Haizhou Bay, China. Stratified random surveys were conducted in March, May, July, September, and December, 2011.
A stratified random bottom trawl survey was conducted to evaluate the fish community in Haizhou Bay in March, May, July, September, and December of 2011, with 24 stations randomly assigned in each month. A stow-net fisheries survey was conducted simultaneously to assess fishing effort, as stow-net fisheries currently yield the majority of catch in Haizhou Bay (Qi, 2013). The 23 most common species were included in the model, accounting for over 90% of the total biomass of all species caught on the trawl survey. Each species was characterized by a set of life-history trait parameters describing growth, predation, mortality, and reproduction. These parameters were obtained from FishBase (www.fishbase.org), previous studies, and reports, as well as being derived from the bottom trawl survey results (see Supplementary material for details). Spatial interactions among species were estimated with the “overlap index” from the survey data (Blanchard et al., 2014). The unknown parameters, resource carrying capacity and maximum recruitment in the “Beverton–Holt” stock–recruitment relationship, were estimated by minimizing the sum of squared errors between log-transformed trawl survey data and model predictions (Blanchard et al., 2014).
Fishing mortality was determined by three factors: catchability, selectivity, and fishing effort. Selectivity of the stow-net was assumed to be a “knife-edge” shape (Scott et al., 2014), in which the “knife-edge size” was estimated from the survey data. Constant fishing effort for all species and species-specific catchability were estimated by comparing catch from the stow-net and bottom trawl surveys. It was assumed that stock abundances were well represented by abundance indices derived from the bottom trawl survey and that fishing mortality results only from the stow-net fishery.
Simulation design
This study was designed to evaluate uncertainty in modelling of fish community dynamics. We focused on uncertainty resulting from the utilization of imprecise parameters rather than model structure, which can only be validated within multiple model inference. However, the importance of structural uncertainty is emphasized for its basic influence on model prediction (Farcas and Rossberg, submitted study available on http://arxiv.org/abs/1412.0199). The uncertainty from model parameters was assessed using a Monte Carlo simulation approach (Refsgaard et al., 2007), which simulated error by randomly drawing parameters from defined statistical distributions (Andersen and Pedersen, 2010; Houle et al., 2012).
Regarding the nature of uncertainty (Walker et al., 2003), epistemic uncertainty was addressed as imprecise parameters of fish life-history and metabolic scale, whereas stochastic uncertainty was addressed as environmental and fishing variability as well as observation errors. Specifically, the parameters of the size-spectrum model were grouped into four categories: (i) fish life history, including size at maturity (Wmat), predation preference (β, σ), von Bertalanffy growth parameter K (kvb), maximum recruitment (Rmax), and background mortality rate (Z0); (ii) metabolic scale, including the allometric exponents for maximum food intake (n), standard metabolism (p), and search volume (q); (iii) environmental variability, represented by background carrying capacity (kappa), reproductive efficiency, and spatial overlap coefficients; and (iv) fishing variability, including fishing efforts, catchability, and size-selectivity (“knife-edge”). Error terms randomly drawn from normal distributions with the mean of zero and the assigned standard deviations were added to these parameters to mimic input errors. The observation error was assumed to follow a lognormal distribution (Gavaris, 1980; Stedinger, 1980) and was simulated by multiplying the model predictions with lognormally distributed errors.
Six scenarios corresponding to the above-mentioned parameter categories were considered in this study (Table 1). The simulations were conducted for three levels of fishing effort (0.5, 1, and 2 times current F, corresponding to low, medium, and high fishing effort, respectively), as previous studies suggested that the response of fish communities to fishing depends on scenario settings (Travers et al., 2006). As a first step to exhibit the pattern of uncertainty in model prediction, parameters were assigned standard deviation equal to 5% of their values, and the model was run 1000 times to evaluate uncertainty (Houle et al., 2012). Thereafter, the level of standard deviation was simulated from 0 to 10% with an interval of 1% (referred to as “sigma” below). These levels of variability were decided rather arbitrarily because variability is often hard to quantify for ecosystems (Chen and Paloheimo, 1998; Walters, 1998); however, the settings were sufficient to exploring uncertainty patterns. The model was simulated for 100 years, and the last 10 years of each simulation run were used to assess the uncertainty of model prediction (Houle et al., 2012; Blanchard et al., 2014).
Simulation scenarios considered in this study.
| Scenarios | Category | Location | Nature | Parameters |
|---|---|---|---|---|
| 1 | Metabolic scale | Process | Epistemic | Exponent for maximum food intake, standard metabolism, and search volume |
| 2 | Life history | Process | Epistemic | Maturity size, predation preference, growth rate, maximum recruitment, and background mortality rate |
| 3 | Environmental variability | Process | Stochastic | Background carrying capacity, efficiency of reproduction, and spatial interaction coefficients |
| 4 | Fishing variability | Process | Stochastic | Fishing efforts, catchability, and selective size |
| 5 | Sampling error | Observation | Stochastic | Predicted abundance |
| 6 | Compound sources | – | – | All above |
| Scenarios | Category | Location | Nature | Parameters |
|---|---|---|---|---|
| 1 | Metabolic scale | Process | Epistemic | Exponent for maximum food intake, standard metabolism, and search volume |
| 2 | Life history | Process | Epistemic | Maturity size, predation preference, growth rate, maximum recruitment, and background mortality rate |
| 3 | Environmental variability | Process | Stochastic | Background carrying capacity, efficiency of reproduction, and spatial interaction coefficients |
| 4 | Fishing variability | Process | Stochastic | Fishing efforts, catchability, and selective size |
| 5 | Sampling error | Observation | Stochastic | Predicted abundance |
| 6 | Compound sources | – | – | All above |
Each scenario was simulated with parameter errors (sigma) ranging from 0.01 to 0.1 under low, medium, and high levels of fishing efforts (i.e. 0.5, 1, and 2 times of estimated current fishing effort).
Simulation scenarios considered in this study.
| Scenarios | Category | Location | Nature | Parameters |
|---|---|---|---|---|
| 1 | Metabolic scale | Process | Epistemic | Exponent for maximum food intake, standard metabolism, and search volume |
| 2 | Life history | Process | Epistemic | Maturity size, predation preference, growth rate, maximum recruitment, and background mortality rate |
| 3 | Environmental variability | Process | Stochastic | Background carrying capacity, efficiency of reproduction, and spatial interaction coefficients |
| 4 | Fishing variability | Process | Stochastic | Fishing efforts, catchability, and selective size |
| 5 | Sampling error | Observation | Stochastic | Predicted abundance |
| 6 | Compound sources | – | – | All above |
| Scenarios | Category | Location | Nature | Parameters |
|---|---|---|---|---|
| 1 | Metabolic scale | Process | Epistemic | Exponent for maximum food intake, standard metabolism, and search volume |
| 2 | Life history | Process | Epistemic | Maturity size, predation preference, growth rate, maximum recruitment, and background mortality rate |
| 3 | Environmental variability | Process | Stochastic | Background carrying capacity, efficiency of reproduction, and spatial interaction coefficients |
| 4 | Fishing variability | Process | Stochastic | Fishing efforts, catchability, and selective size |
| 5 | Sampling error | Observation | Stochastic | Predicted abundance |
| 6 | Compound sources | – | – | All above |
Each scenario was simulated with parameter errors (sigma) ranging from 0.01 to 0.1 under low, medium, and high levels of fishing efforts (i.e. 0.5, 1, and 2 times of estimated current fishing effort).
Quantification of uncertainty
The modelling uncertainty was quantified using two approaches: (i) multivariate analysis on community data and (ii) variance analysis on ecological indicators (White et al., 2014). Multidimensional scaling (MDS) was used to explore the pattern of uncertainty in the simulation based on projected species biomass (Hout et al., 2013). MDS is an ordination technique commonly used to visualize the level of similarity between individual observations within an ecological dataset (Kenkel and Orlóci, 1986; James, 1990; Clarke, 1993). This method is built on a similarity or distance matrix (Bray–Cutis similarity index used in this study, Bray and Curtis, 1957) and uses an iterative process and ordinal distance to increase robustness for irregular data distribution and sampling variance (Rice, 2000). MDS can be used to reduce the dimension of a dataset for easy visualization with two- or three-dimensional plots and while preserving the distances between observations as well as possible (James, 1990). A “stress” value indicates how well information is preserved in dimensional reduction with higher stress values (>0.2) suggesting poor representation (Clarke and Warwick, 1994).
For this study, the projections of fish community composition (species biomass) from different scenarios were treated as n-dimensional data, and the first two dimensions of MDS were used for displaying similarity among simulations. The distributional patterns of individual simulation runs from various scenarios were used for identifying the relative vulnerability of the model to different sources of uncertainty. The MDS method was implemented with “metaMDS” function in R package “vegan”.
Fish community characteristics were described by a set of ecological indicators, which have been widely used in environmental conservation and fisheries management (e.g. Christensen and Publishing, 2000; Rochet and Trenkel, 2003; Jennings, 2005; Powers and Monk, 2010; Shin et al., 2010). These indicators are related to various management objectives, including conserving biodiversity and maintaining ecosystem function (Shin et al., 2010). We used the following seven ecological indicators to characterize abundance, body size, and trophodynamics of fish communities: total biomass (Biomass), Shannon–Wiener diversity index (H′), W-statistic, Large Fish Indicator (LFI), mean body weight (meanW), slope of community size spectrum (Beta), and mean trophic level (MTL) (Table 2). These indicators were calculated within the “mizer” package in Rand derived from the model outputs.
Definitions of ecological indicators that characterize fish community structure.
| Indicators | Symbol | Definition | Sources |
|---|---|---|---|
| Total biomass | Biomass | The sum of biomass of all individuals | Murawski (2000), Fulton et al. (2005) |
| Shannon–Wiener diversity index | H′ | H′ = −∑pi ln(pi), pi indicates the biomass proportion of species i in fish community | Shannon (1948) |
| W statistic | W-statistic | The area between biomass and abundance cumulative curves across all species, W = ∑ Bcum−Bcum−Ncum/50(S−1) Bcum and Ncum are the cumulative biomass and abundance, respectively, and S the total number of species | Clarke and Warwick (1994), Yemane et al. (2005) |
| Large fish indicator | LFI | The total biomass of all individuals with the body weight >100 g divided by total biomass of the community | Greenstreet et al. (2010) |
| Mean body weight | meanW | Mean body weight of all individuals in the community | Rochet and Trenkel (2003) |
| Slope of community size spectrum | Beta | log(Nw) = α + βlog(W). The slope of a linear regression of log (abundance in each size class, Nw) vs. log (body weight) | Rice (1996) |
| Mean trophic level | MTL | The average trophic levels of the community weighted by the biomass of each species MTL = ∑ TLi(Bi/B), TLi is the trophic level of species i obtained from FishBase, Bi the biomass of species i, and B the community biomass | Pauly et al. (1998) |
| Indicators | Symbol | Definition | Sources |
|---|---|---|---|
| Total biomass | Biomass | The sum of biomass of all individuals | Murawski (2000), Fulton et al. (2005) |
| Shannon–Wiener diversity index | H′ | H′ = −∑pi ln(pi), pi indicates the biomass proportion of species i in fish community | Shannon (1948) |
| W statistic | W-statistic | The area between biomass and abundance cumulative curves across all species, W = ∑ Bcum−Bcum−Ncum/50(S−1) Bcum and Ncum are the cumulative biomass and abundance, respectively, and S the total number of species | Clarke and Warwick (1994), Yemane et al. (2005) |
| Large fish indicator | LFI | The total biomass of all individuals with the body weight >100 g divided by total biomass of the community | Greenstreet et al. (2010) |
| Mean body weight | meanW | Mean body weight of all individuals in the community | Rochet and Trenkel (2003) |
| Slope of community size spectrum | Beta | log(Nw) = α + βlog(W). The slope of a linear regression of log (abundance in each size class, Nw) vs. log (body weight) | Rice (1996) |
| Mean trophic level | MTL | The average trophic levels of the community weighted by the biomass of each species MTL = ∑ TLi(Bi/B), TLi is the trophic level of species i obtained from FishBase, Bi the biomass of species i, and B the community biomass | Pauly et al. (1998) |
The seven ecological indicators quantify abundance, body size, and trophodynamic of fish community. Biomass estimates used to calculate the ecological indicators are derived from the prediction of the size-spectrum model, which accounted for individuals beyond a threshold of body size defined in Supplementary Table S2.
Definitions of ecological indicators that characterize fish community structure.
| Indicators | Symbol | Definition | Sources |
|---|---|---|---|
| Total biomass | Biomass | The sum of biomass of all individuals | Murawski (2000), Fulton et al. (2005) |
| Shannon–Wiener diversity index | H′ | H′ = −∑pi ln(pi), pi indicates the biomass proportion of species i in fish community | Shannon (1948) |
| W statistic | W-statistic | The area between biomass and abundance cumulative curves across all species, W = ∑ Bcum−Bcum−Ncum/50(S−1) Bcum and Ncum are the cumulative biomass and abundance, respectively, and S the total number of species | Clarke and Warwick (1994), Yemane et al. (2005) |
| Large fish indicator | LFI | The total biomass of all individuals with the body weight >100 g divided by total biomass of the community | Greenstreet et al. (2010) |
| Mean body weight | meanW | Mean body weight of all individuals in the community | Rochet and Trenkel (2003) |
| Slope of community size spectrum | Beta | log(Nw) = α + βlog(W). The slope of a linear regression of log (abundance in each size class, Nw) vs. log (body weight) | Rice (1996) |
| Mean trophic level | MTL | The average trophic levels of the community weighted by the biomass of each species MTL = ∑ TLi(Bi/B), TLi is the trophic level of species i obtained from FishBase, Bi the biomass of species i, and B the community biomass | Pauly et al. (1998) |
| Indicators | Symbol | Definition | Sources |
|---|---|---|---|
| Total biomass | Biomass | The sum of biomass of all individuals | Murawski (2000), Fulton et al. (2005) |
| Shannon–Wiener diversity index | H′ | H′ = −∑pi ln(pi), pi indicates the biomass proportion of species i in fish community | Shannon (1948) |
| W statistic | W-statistic | The area between biomass and abundance cumulative curves across all species, W = ∑ Bcum−Bcum−Ncum/50(S−1) Bcum and Ncum are the cumulative biomass and abundance, respectively, and S the total number of species | Clarke and Warwick (1994), Yemane et al. (2005) |
| Large fish indicator | LFI | The total biomass of all individuals with the body weight >100 g divided by total biomass of the community | Greenstreet et al. (2010) |
| Mean body weight | meanW | Mean body weight of all individuals in the community | Rochet and Trenkel (2003) |
| Slope of community size spectrum | Beta | log(Nw) = α + βlog(W). The slope of a linear regression of log (abundance in each size class, Nw) vs. log (body weight) | Rice (1996) |
| Mean trophic level | MTL | The average trophic levels of the community weighted by the biomass of each species MTL = ∑ TLi(Bi/B), TLi is the trophic level of species i obtained from FishBase, Bi the biomass of species i, and B the community biomass | Pauly et al. (1998) |
The seven ecological indicators quantify abundance, body size, and trophodynamic of fish community. Biomass estimates used to calculate the ecological indicators are derived from the prediction of the size-spectrum model, which accounted for individuals beyond a threshold of body size defined in Supplementary Table S2.
The ecological indicators were calculated for each simulation and their probability distributions derived from 1000 runs were compared with the “true” values of community indicators derived from model projections without error. The uncertainty resulting from different levels of input error was examined by plotting coefficients of variation (CV, standard deviation divided by the mean) of each ecological indicator against sigma. Linear regression models were fitted between CV and sigma to evaluate the average influence of parameter variation on the estimated ecological indicators. The analyses were conducted for all three levels of fishing effort (0.5F, F, and 2F) and the six uncertainty scenarios defined in Table 1.
Results
Uncertainty of community structure
The model with a medium level of fishing effort (i.e. the estimation of current effort) projected that the community reached an equilibrium status after 60 years of simulation, which closely matched the observed trawl survey data (Supplementary Figure S2). The first two dimensions of MDS were displayed to visually compare the similarity of the projected fish community with uncertainty (Figure 2). Each point represented the projection of an individual simulation run, and their distances indicated the level of dissimilarity. The background colour denoted the density of the scattered points. The results showed that variations in the estimated community structure resulting from Scenario 3 (i.e. environmental variability), Scenario 4 (i.e. fishing variability) and Scenario 5 (i.e. sampling error) were symmetrically distributed around the true state. The uncertainty in Scenario 1 (i.e. imprecision of metabolic scale parameters) and Scenario 2 (i.e. imprecision of life-history parameters) formed two clusters with the true state centring at the larger cluster. Scenario 6 (considering all sources of uncertainty) showed a similar pattern to Scenario 1 results with a cluster on one side of the MDS plot and spreading out on the other side. The stress value associated with the MDS plot suggested that the bicluster patterns in Scenarios 1, 2, and 6 were good representations of original data and that the symmetrical distributions in Scenarios 3, 4, and 5 were less representative.
MDS analysis of community structure in simulations under the medium level of fishing efforts (i.e. estimated current fishing effort). The sources of uncertainty included variation in metabolic scale, life history, environment, fishing activity, and observation error. The crosses indicate the “true” community without errors in model parameters. This figure is available in black and white in print and in colour at ICES Journal of Marine Science online.
The variation arising from metabolic scale and life-history parameters dominated the shape of the compound MDS plots given the same level of parameter variations (Figure 3). Variation caused by other sources of uncertainty was relatively limited, except environmental variability (i.e. Scenario 3) under high fishing efforts. The uncertainty caused by metabolic scale and life-history variability also increased with the increasing fishing effort.
MDS analysis of fish community with uncertainty from metabolic scale, life history, environmental variability, fishing activity, and observation. Simulations were conducted under low, medium, and high levels of fishing efforts (i.e. 0.5, 1, and 2 times of estimated current fishing effort). This figure is available in black and white in print and in colour at ICES Journal of Marine Science online.
Uncertainty of ecological indicators
The simulations under medium fishing effort in Scenarios 3, 4, and 5 (i.e. environment uncertainty, fishing variability, and observation errors, respectively) yielded indicators with tight symmetric distributions around the true values, whereas indicators from Scenarios 1 and 2 (i.e. errors in metabolic scale and life-history parameters) showed a wider distribution (Figure 4). Bimodal shapes were detected in Scenarios 1 and 2 for all indicators except for mean body weight, in which one peak overwhelmed the other. The result was consistent with the pattern exhibited in the MDS analysis. Scenario 6, which included all sources of uncertainty (Table 1), showed a similar distribution as Scenario 1 (i.e. errors in metabolic scale), suggesting that errors in the metabolic scale was the dominant source of uncertainty.
Frequency distribution of seven ecological indicators of fish community under six scenarios of uncertainty sources (metabolic scale, life history, environmental variability, fishing variability, observation error, and compound errors). The vertical lines indicate “true” values without errors in simulations. This figure is available in black and white in print and in colour at ICES Journal of Marine Science online.
The relative impacts of various sources of uncertainty on ecological indicators were evaluated with CVs. An increase in parameter variation (sigma) led to an increased CV for all sources of uncertainty, but their relative impacts differed consistently (Figure 5). Given the same level of variation, process errors from metabolic scale parameters had the most influence compared with other sources, followed by life-history parameters, whereas observation and fishing variability had the least impact. However, it should be noted that the pattern could be reversed when the input variation was small for metabolic scale parameters whereas large for other parameters.
CVs of seven ecological indicators with respect to different levels of errors in parameters for the six scenarios defined in Table 1. Simulations were conducted in low, medium, and high levels of fishing efforts.
The seven ecological indicators used in this study showed various levels of sensitivity to uncertainty of different resources. W-statistic and LFI were the most sensitive to input errors, followed by total biomass and biodiversity indices, whereas the mean body weight and the mean trophic level showed the least variation. Uncertainty levels were generally consistent under the three fishing efforts for each indicator, except the W-statistic that varied widely under high fishing effort (Figure 5).
Linear regressions between CVs of indicators and sigma (standard deviation in model parameters) showed a large slope ranging from 0.43 to 6.28 in Scenarios 1 and 2, indicating that the uncertainty resulting from imprecise metabolic scale and life-history parameters increased faster than others (Table 3). The slope values were less than 1 in Scenarios 3 and 4 and less than 0.1 in Scenario 5, indicating a limited influence of environmental variability, fishing variability, and observation errors. The regression models for Scenario 1 had intercepts that were significantly different from zero (Table 3), probably resulting from the non-linearity between the input error and output variation.
Linear regression of CVs of ecological indicators and the levels of impression in model parameters (sigma).
| Indicators | Metabolic scale | Life history | Environment | Fishing | Observation |
|---|---|---|---|---|---|
| Biomass | 3.72 (0.14) | 2.97 | 1.26 | 0.36 | 0.04 |
| H‘ | 0.74 (0.23) | 3.19 | 1.55 | 0.84 | 0.02 |
| W-statistic | 2.41 (0.50) | 4.78 | 5.02 | 3.21 | 0.07 |
| LFI | 6.28 (0.18) | 5.01 | 1.36 | 0.56 | 0.04 |
| meanW | 1.66 | 1.33 | 0.46 | 0.29 | 0.06 |
| Beta | 1.92 (0.14) | 1.33 | 0.47 | 0.19 | 0.01 |
| MTL | 0.43 (0.08) | 1.33 | 0.51 | 0.24 | 0.01 |
| Indicators | Metabolic scale | Life history | Environment | Fishing | Observation |
|---|---|---|---|---|---|
| Biomass | 3.72 (0.14) | 2.97 | 1.26 | 0.36 | 0.04 |
| H‘ | 0.74 (0.23) | 3.19 | 1.55 | 0.84 | 0.02 |
| W-statistic | 2.41 (0.50) | 4.78 | 5.02 | 3.21 | 0.07 |
| LFI | 6.28 (0.18) | 5.01 | 1.36 | 0.56 | 0.04 |
| meanW | 1.66 | 1.33 | 0.46 | 0.29 | 0.06 |
| Beta | 1.92 (0.14) | 1.33 | 0.47 | 0.19 | 0.01 |
| MTL | 0.43 (0.08) | 1.33 | 0.51 | 0.24 | 0.01 |
The increasing rate of the impacts of each category of uncertainty sources (Scenarios 1–5 defined in Table 1) on the indicators was represented by the slopes of linear regressions. The intercepts significantly differing from zero were listed in brackets (α = 0.05).
Linear regression of CVs of ecological indicators and the levels of impression in model parameters (sigma).
| Indicators | Metabolic scale | Life history | Environment | Fishing | Observation |
|---|---|---|---|---|---|
| Biomass | 3.72 (0.14) | 2.97 | 1.26 | 0.36 | 0.04 |
| H‘ | 0.74 (0.23) | 3.19 | 1.55 | 0.84 | 0.02 |
| W-statistic | 2.41 (0.50) | 4.78 | 5.02 | 3.21 | 0.07 |
| LFI | 6.28 (0.18) | 5.01 | 1.36 | 0.56 | 0.04 |
| meanW | 1.66 | 1.33 | 0.46 | 0.29 | 0.06 |
| Beta | 1.92 (0.14) | 1.33 | 0.47 | 0.19 | 0.01 |
| MTL | 0.43 (0.08) | 1.33 | 0.51 | 0.24 | 0.01 |
| Indicators | Metabolic scale | Life history | Environment | Fishing | Observation |
|---|---|---|---|---|---|
| Biomass | 3.72 (0.14) | 2.97 | 1.26 | 0.36 | 0.04 |
| H‘ | 0.74 (0.23) | 3.19 | 1.55 | 0.84 | 0.02 |
| W-statistic | 2.41 (0.50) | 4.78 | 5.02 | 3.21 | 0.07 |
| LFI | 6.28 (0.18) | 5.01 | 1.36 | 0.56 | 0.04 |
| meanW | 1.66 | 1.33 | 0.46 | 0.29 | 0.06 |
| Beta | 1.92 (0.14) | 1.33 | 0.47 | 0.19 | 0.01 |
| MTL | 0.43 (0.08) | 1.33 | 0.51 | 0.24 | 0.01 |
The increasing rate of the impacts of each category of uncertainty sources (Scenarios 1–5 defined in Table 1) on the indicators was represented by the slopes of linear regressions. The intercepts significantly differing from zero were listed in brackets (α = 0.05).
Discussion
The complex dynamics of marine ecosystems create imposing challenges for fisheries management and the success of management depends heavily on whether we can recognize the limitation and precision of our understanding of marine ecosystems (Fogarty and Rose, 2014). In the process of building models, limited knowledge of model structure is explicitly expressed as assumptions in model formulations, although assumption validation is often insufficient and difficult. However, the uncertainty associated with model parameters is less clearly defined, especially for multispecies models that require many parameters (Planque et al., 2011; Brander et al., 2013). The size-spectrum model used in this study is a typical multispecies model in that it has simple but robust assumptions and captures the main processes and interactions of fish communities (Blanchard et al., 2005; Daan et al., 2005; Andersen and Beyer, 2006; Hartvig et al., 2011). Growth and recruitment dynamics are unconstrained and the model allows the emergence of non-linear effects (Houle et al., 2012, 2013), which creates complex and non-linear uncertainty in model prediction. This model is also built on many parameters and has a calibration process (i.e. estimation of resource carrying capacity and maximum recruitment) that requires sufficient data, thus exposing it to potential overfitting issues that are pervasive for most complex models (Silver, 2012; Fogarty and Rose, 2014). All these characteristics make it a good candidate for studying the uncertainty of multispecies models. The approach developed and conclusion derived in this study can be applicable to many ecological models of similar nature and complexity.
This study evaluated uncertainty of the multispecies size-spectrum model resulting from various sources. The results of the MDS analysis and ecological indicators illustrated that, given the same level of imprecision, the parameters of metabolic scale, followed by life-history parameters, have the most influence on model prediction uncertainty. Environmental and fishing variation have limited impacts under medium and low fishing effort, and observation errors lead to a low level of uncertainty. The pattern is generally consistent under three levels of fishing effort. The symmetrical distributions for Scenarios 3, 4, and 5 (i.e. environmental variability, fishing variability, and observation errors, respectively, defined in Table 1) in Figure 2 imply that parameter uncertainty for these three scenarios led to “statistical uncertainty”, which means the uncertainty can be described adequately in statistical terms (Walker et al., 2003). The bimodal clustering distributions for Scenarios 1 and 2 suggest “scenario uncertainty”, implying a range of possible outcomes (i.e. alternative states of community structure in this study; Walker et al., 2003). It should be noted that the pattern is a primary result and that extensive studies on relatively smaller (considering linear sensitivity) and larger (considering multiple stability) variations of the model are desired to address emerging dynamics of the size-spectrum model (Rossberg et al., 2013). We compared this study with Thorpe et al. (2015) to evaluate possible differences between the two size-based models. Their operational model was developed on a length-class structure with the assumption that growth is unconstrained by food resources, i.e. a deterministic von Bertalanffy growth rate. The study tested the contribution of seven parameters (including diet matrix, predation size selection, natural mortality, spawner–recruit parameters, growth efficiency, and asymptotic length) to the uncertainty of the model output. Their results suggested a pattern of statistical rather than scenario uncertainty, which was not consistent with our results as these parameters were mostly classified as life-history traits in our study. The difference may be attributed to the fixed growth rate in their model, since there is evidence that natural foodwebs are self-organized to structural instability (Rossberg, 2013), which is implicitly assumed in our size-spectrum model. Although this comparison suggests that our conclusion might not hold for other models with different assumptions, the divergence between these two studies can be inferred as the effect of model structure, which should be highlighted in future studies.
Many methods and tools have been developed for assessing uncertainty. Refsgaard et al. (2007) reviewed 14 commonly used methods from a decision-making perspective: data uncertainty engine, error propagation equations, expert elicitation, extended peer review (review by stakeholders), inverse modelling (parameter estimation), inverse modelling (predictive uncertainty), Monte Carlo analysis, multiple model simulation, NUSAP, quality assurance, scenario analysis, sensitivity analysis, stakeholder involvement, and uncertainty matrix. These methods can address a wide range of uncertainty for different purposes; however, none of them can address all sources or aspects of uncertainty in modelling (Refsgaard et al., 2007). In fact, the precision of fish stock projections is hard to assess, because such projections generally cover a broad temporal scale (years or decades) and the dynamic may be history-dependent (Levin, 1999). In addition, the majority of global fisheries can only be assessed with limited precision and frequency (Costello et al., 2012; Carruthers et al., 2014), making it difficult to assess the quality of projections (Brander et al., 2013). The Monte Carlo simulation approach developed in this study was intended to provide an example for assessing uncertainty; however, it is worth noting that we addressed part of the greater picture of modelling uncertainty.
The parameters of our model were treated in groups rather than individually so as to gain general sense of how uncertainty results from different sources. This approach may introduce certain problems for interpretation. The number of parameters considered in each category is not equal, i.e. Scenario 2 (life history) involves six parameters and the others categories involve three (Table 1), which makes the comparison among scenarios less statistically meaningful. Furthermore, autocorrelations in parameter estimation were not considered in our simulations, and the pattern of uncertainty can be hidden if an individual parameter interacts, synergistically or antagonistically, with other parameters in the same category. However, we used this approach for a practical consideration. Multiple parameters in the same category can be estimated in one study or several similar studies and improved simultaneously, which make our approach meaningful for model improvements. For example, the estimation of most life-history parameters can be improved with biological studies; fishing parameters can be better estimated with studies on fishing gear and stock assessment and metabolic parameters can be improved with physiological research. The results of this study can thus provide guidance for identifying research priorities to reduce key uncertainty in fisheries management (Stelzenmüller et al., 2015).
Our approach categorized one source of uncertainty as environmental variability, but the size-spectrum model does not explicitly include environmental variables such as temperature, salinity, and dissolved oxygen into modelling (Andersen and Beyer, 2006; Scott et al., 2014). It is worth noting that incorporating multiple environmental variables into mechanistic modelling is exceedingly difficult as environmental effects on ecosystems are often non-linear and mixed, and it is impossible to include all environmental variables (Pauly, 1980; Pepin, 1991; Charnov and Gillooly, 2004; Pörtner and Knust, 2007). In our simulations, the joint effects of environmental variables are simplified to the dynamics of resource carrying capacity, reproductive efficiency and spatial overlaps of species distribution (Scenario 3; Table 1), which are most directly influenced by environmental variables (Huntley and Lopez, 1992; Humston et al., 2000; Pörtner et al., 2001; Sponaugle and Grorud-Colvert, 2006; Farrell, 2009). Although environmental conditions influence every biological process and associated parameters, this categorization intends to distinguish the nature of uncertainty, i.e. describing the inherent variability of environment (“stochastic” uncertainty) in contrast to the more consistent life-history and metabolic scale parameters (“epistemic” uncertainty). However, the results should be interpreted with cautions as environmental variability was only partially addressed in our simulation.
This study demonstrates how parameters can result in uncertainty at different levels of imprecision. The sigma (error term ranging from 1 to 10%) in the simulation was used to represent multilevel of imprecision. Combined with different fishing efforts, this setting is intended to cover a wide range of potential outcomes of the model and to explore the pattern and structure of uncertainty in a general perspective. It should be emphasized that uncertainty patterns depend on both the intrinsic sensitivity as we addressed in our simulation and the extent to which the parameters vary in reality. In this sense, the range of sigma from 0.01 to 0.1 is arbitrary, as the level of imprecision is supposed to be ecosystem-specific and have inherent differences across various sources. For example, the level of observation error tends to depend heavily on the quantity of survey data, which are commonly insufficient due to limited financial resources especially for developing countries (Stockwell and Peterson, 2002). Fish life-history parameters such as those in the von Bertalanffy growth equation are supposed to be properly estimated with limited data, whereas natural or anthropogenic environmental variability is more difficult to assess or reduce. The level of uncertainty reflects the limitation of our knowledge of ecosystem dynamics (Walker et al., 2003). Interpretation of the influences of various uncertainty sources refers to the “baseline” variation, i.e. the uncertainty of model output needs to be estimated according to the specific ecosystem variability.
Effective fisheries management calls for reduced uncertainty in ecosystem dynamic models, which is often achieved by conducting additional studies and collecting more data (Refsgaard et al., 2007). Knowing the relative influences of different sources of uncertainty may elucidate the most cost-effective strategy to carry out studies (Walker et al., 2003). In this study, the intercepts and slopes of the linear regressions between CVs of indicators and sigma of parameters may provide a clue (Table 3). For example, the large slopes such as LFI in Scenarios 1 and 2 and W-statistic in Scenario 2 suggest that improving the estimation of metabolic scale and life-history parameters would be the top priority for reducing uncertainty of these ecological characteristics. On the other hand, the gradual slope in the observation error scenario (Scenario 5; Table 1) suggests that reducing the sampling error should have lower priority. In addition, the obvious intercepts for Scenario 6 (compound sources of uncertainty) suggest that a considerable uncertainty exists even when the parameters are properly estimated, implying that such uncertainty is relatively “unreducible” with the current model structure. In future studies, a broader comparison of multiple models with different concepts and structures could greatly improve the validation of models.
Acknowledgements
We thank Katherine Thompson of University of Maine and three anonymous referees for comments and suggestions that improved this paper. This study was funded by the Special Fund for Agroscientific Research in the Public Interest (Grant No.201303050) and the Fundamental Research Funds for the Central Universities (Grant No. 201022001 and 201262004). The senior author's study at the University of Maine is supported by the China Scholarship Council, Ocean University of China, and University of Maine.
References
Author notes
Handling editor: Ken Andersen
