When to conduct, and when not to conduct, management strategy evaluations Open Access

Abstract

The basis of natural resource management is decision making under uncertainty while balancing competing objectives. Within fisheries management, a process described as management strategy evaluation (MSE) is becoming increasingly requested globally to develop and test management procedures. In a fisheries or other natural resource context, a management procedure is a rule that predetermines the management response given feedback from the resource and is simulation tested to be robust to multiple uncertainties. MSEs are distinguished from other risk or simulation analyses by the explicit testing of the feedback mechanism that applies decision rule-based management advice back to the simulated population or ecosystem. Stakeholder input is frequently cited as a best practice in the MSE process, since it fosters communication and facilitates buy-in to the process. Nevertheless, due to the substantial additional cost, time requirement, and necessary scientific personnel, full stakeholder MSEs remain relatively uncommon. With this communication, we provide guidance on what constitutes an MSE, when MSEs should be undertaken or where simpler approaches may suffice, and how to prioritize the degree of stakeholder participation.

Introduction

At the basis of natural resource management is decision making under uncertainty while balancing competing objectives (Kochenderfer, 2015; Hemming et al., 2022). In these situations, the key to informed decision making is to define the objectives, elucidate tradeoffs, and test the performance of management actions prior to implementation, to the extent possible. Globally, management strategy evaluation (MSE) is increasingly applied as a formal process for fisheries decision making. MSE is not the only tool for structured decision making in natural resources, as numerous others exist (Hemming et al., 2022), nor is it the best tool for all applications. Since it directly fits into the fisheries advice framework of providing management advice, MSE has seen more applications to fisheries. Some countries, such as South Africa or Australia, have a long history of using MSE to select management procedures (Punt et al., 2016; De Moor et al., 2022), and ∼30 MSEs have been conducted in Europe by ICES from 2013 to 2018 (ICES, 2019). The number of MSEs is also growing for transboundary stocks such as Atlantic bluefin tuna (Carruthers and Butterworth, 2018), North Pacific albacore (ISC, 2021), Pacific halibut (IPHC), and Pacific hake (Jacobsen et al., 2021). These MSEs also range considerably in the question they seek to answer, the complexity of the process, the extent to which stakeholders are engaged, and in who qualifies for inclusion as such.

While the United States has not employed MSE as extensively to implement management decisions, its use recently expanded (DeVore and Gilden, 2019). US federal fisheries management is mandated by the Magnuson-Stevens Fishery Conservation and Management Act (MSA), codified in National Standard 1 guidelines, and is implemented by the regional Fishery Management Councils according to defined rules for setting Allowable Biological Catch (ABC) and Annual Catch Limits (ACLs). Most US regional management bodies already have catch (né harvest; Bohnsack et al., 2020) control rules (CCRs; Table 1) and conceptual management objectives (Table 1) in place as defined in Fishery Management Plans (e.g. PFMC, 2019). Some of these CCRs have been informed by MSEs. For instance, in the United States, MSE informed the selection of CCRs for the northern subpopulation of Pacific sardine (Punt et al., 2016), Sacramento River winter-run chinook salmon (SRWCW, 2017), and Atlantic herring (Deroba et al., 2019), among others.

Globally, when management possesses effective controls, it has been successful in ending overfishing, rebuilding overfished stocks, and achieving optimum yields (Hilborn et al., 2020). So, a natural question that fishery managers may ask is: “Why do we need MSE when our current management works?” While standard tenets of fisheries management (reduce fishing mortality and stocks build) works in many situations, society will increasingly encounter environmental non-stationarity, growing demand for resources, and more complicated fishery management decisions balancing multispecies and ecosystemic interactions. These situations will challenge status quo advice and management frameworks. In this essay, we, a group of MSE practitioners, provide guidance to prioritize MSE applications, focusing on defining situations where the payoff is likely to be worth the investment.

First, we define MSE as: “A simulation-based, analytical framework used to evaluate the performance of multiple candidate management procedures (Table 1) relative to the prespecified management objectives” (Miller et al., 2019). Structurally, MSEs consist of a closed-feedback loop between operating models (Table 1) and a management procedure, which may be derived from an estimation model or can be entirely empirical, such as based on a survey. The goal is to screen the performance of multiple management procedures against the desired objectives under a range of uncertainties, often revealing tradeoffs among alternative actions. Performance metrics are measurable quantities from the operating model that indicate whether the management procedure met the management objectives, and may include proxies for biological or socioeconomic realities (e.g. socio-economic submodels are typically not modelled explicitly). More so, the intent of an MSE is eventual adoption of a form of action, usually management related, though desk MSEs may support actions, such as data collection, that inform but are not direct management actions. Essential components of the process are that the management problem be well defined, and that there be clear conceptual and quantifiable operational management objectives. In situations where these do not exist, it is critical to have consistent, engaged stakeholder input to define these as part of the MSE process (Miller et al., 2019). Such input is also often critical to obtaining buy-in for proposed management actions (Goethel et al., 2019).

MSEs fall along a continuum based upon the degree of stakeholder and analyst participation required:  

• Desk MSEs can be conducted by a single or several analysts as a computer exercise and require little or no stakeholder input as both conceptual and operational management objectives are predefined. The product may qualitatively inform management or resource allocation but may or may not result in adoption of a new management procedure. The time commitment is as little as 2–12 months for one to two full-time employees, though this estimate may vary widely with analyst experience and the complexity of the research question. Desk MSEs may focus on general research questions, such as testing the performance of CCRs or reference points (Table 1; e.g., Wiedenmann et al., 2013; Punt and Ralston, 2007), or specific applications for management where the management objectives and alternative management procedures are clearly articulated from the outset (e.g. Wetzel and Punt, 2017).

• Full Stakeholder MSEs require iterative stakeholder-analyst interaction. Conceptual and operational management objectives may need to be defined as part of the process, and the process is intended to produce management action (e.g. Feeney et al., 2019). This is often highly resource intensive, requiring at least a 6–36 month time commitment (Butterworth, 2007, Sampedro et al., 2017), possibly taking several years (e.g. Kolody et al., 2008; ICCAT, 2022), and requiring multiple full-time dedicated staff. However, some regions with long experience with MSE, such as South Africa, have been able to adopt management procedures with quite limited resources (Butterworth, 2007). The stakeholder-engagement component of an MSE may be accomplished through existing groups and processes, such as regional Fishery Management Councils in the United States or Regional Fishery Management Organizations (RFMOs) internationally [e.g. North Atlantic Fisheries Organization (NAFO); and other regional management bodies], or may require establishing new communication pathways and advisory groups (e.g. the Management Strategy Advisory Board of the International Pacific Halibut Commission).

• Intermediate MSE approaches. A range of intermediate options exist between a desk MSE and full stakeholder MSE that may better balance the need for stakeholder input and cost of dedicated meetings for this purpose. Teleconference engagements with stakeholders may be an inferior substitute for in-person meetings, but are likely also a low-cost alternative in situations where stakeholder engagement is needed. Costs of in-person meetings might potentially be reduced if they are scheduled in conjunction with other meeting processes involving stakeholder travel to a central location, such as fishery management body meetings or other stakeholder workshops. For example, stakeholder-driven conceptual mapping exercises informed MSEs for red grouper in the southeast United States (Grüss et al., 2016; Harford et al., 2018). Another consideration is whether MSEs should be integrated into existing management processes, which, as for the Pacific Fishery Management Council examples highlighted above, can facilitate stakeholder engagement, eventual implementation, and reduce costs.

• Not MSE. Additionally, it is necessary to define a class of explorations that are not MSEs, which include various simulations of how different data, modelling assumptions, or model structures affect the bias and precision of assessments. We explicitly delineate these here, as often a large number of research questions regarding whether models, model outcomes, or assessment results are sensitive to certain factors can be evaluated without propagating the uncertainty through a management model, and therefore do not require the full closed-loop simulations that are the primary engine of MSE (e.g. Crone et al., 2019; Lee et al., 2019).

We bring up the concept of “Not MSE” not for the purposes of creating unnecessary distinctions, but rather to focus prior discussion calling for an MSE on the critical question: What is the research question, and can it be answered faster and cheaper with alternative means of decision making (Hemming et al., 2022)? Analysts should consider:

• whether alternate approaches (e.g. power analysis or risk analysis) could sufficiently answer the research question at a lower cost;

• whether an MSE would duplicate existing or ongoing research efforts;

• whether existing research can be leveraged for the development of the operating model; and

• whether the system has appropriate data to conduct the type of MSE requested and develop management procedures of the desired complexity, or whether the MSE and resulting management procedures could be developed through a data-limited approach (Carruthers et al., 2014; Carruthers and Hordyk, 2018) more aligned with the data availability of the system.

The complexity, time, and resources required to conduct an MSE must be balanced with the priorities of the organization. Once a decision is made to move forward with implementing an MSE, a decision on the level of stakeholder involvement must be made. Here, we highlight (I) the types of questions and situations in which full stakeholder MSEs should be prioritized, and (II) questions and situations where simpler approaches may answer the question faster and cheaper.

(I) High-priority situations for MSEs

As decision makers must carefully consider the allocation of resources to address management decisions, we outline a series of six non-mutually exclusive situations that are likely to require investment in MSE and varying degrees of stakeholder engagement. As scientists should not and cannot define societal objectives for a fishery, stakeholder input should be prioritized where management objectives are not clearly defined or ranked or where several objectives are in opposition to each other. In other words, this includes situations outlined in Hare (2020), where the problematicity of the issue (Turnbull and Hoppe, 2019) or the political distance between stakeholder positions is great. In such scenarios, stakeholder and manager input should help to identify where within the MSE-identified management tradeoff space the management procedure should target. Any given real-world scenario may fall into multiple categories.

Adoption of binding management advice versus exploring management options

MSE can be undertaken with or without adoption of management advice. We prioritize full stakeholder inclusive MSE for situations that will set binding management advice. Stakeholder time and, critically, that of the decision makers who will ultimately decide upon and implement such action, is too precious of a commodity to invest in MSE for the ultimate decisions not to matter. Further, a key theory of the nature of human decision making (e.g. the urgency-gating hypothesis, Cisek et al., 2009) posits that when the decision is difficult, complicated, or the path unclear, humans will often seek more information and delay, unless the urgency of decision requires action. Absent the urgency of a decision, such as pending quota advice or allocations, the tradeoffs elucidated in the MSE remain hypothetical, and the difficult, and necessary, decision of adopting a management action likely will remain unmade, effectively dissipating much of the power of MSE. Hence, we recommend that the full measure of stakeholder engagement be used primarily for situations of pending decisions and the less intensive versions of MSE outlined above be prioritized in other situations.

When there is a really difficult policy decision

When a policy decision is painful, difficult, complex (e.g. direct consideration of ecosystem or multisector interactions), or fraught with uncertainty, decision makers need a structured framework for stakeholder engagement to quantify objectives and to develop potential management strategies or alternative actions, while accounting for the risk associated with different actions (Francis and Shotton, 1997). Risk is defined as the probability of an unfavourable occurrence happening times the consequence if it does happen (e.g. exposure × consequence, Hobday et al., 2011). Such policy decisions are exacerbated for controversial or highly valuable species, and may have major societal implications such as the billion-dollar conflicts between water, hydropower provision, and Pacific salmon, as well as many of the examples that follow. Full stakeholder MSE is powerful, as it offers a structured process for identifying operational management objectives of diverse participants and decision making by identifying management procedures that provide acceptable performance across a set of often-difficult tradeoffs.

When there are heretofore intractable stakeholder conflicts

One major aspect of MSE is that it involves stakeholder participation focused on quantifying operational management objectives, several of which may be in direct conflict. This definition and explicit quantification puts these conflicts “on the table” and lays bare the societal tradeoffs inherent in them. While MSE itself cannot always reconcile these conflicts, it provides a framework for transparent and equitable decision making under conflict (Holland, 2010; Punt et al., 2016), even in situations where “some problems have no solutions, only better and worse ways of dealing with imperfections” (Samuelson, 2018).

Stakeholder conflicts may emerge from many situations, including (1) accounting for technical interactions in multispecies fisheries where one species is captured as bycatch in another (Punt et al., 2005; Dichmont et al., 2008; Ono et al., 2018); (2) opposing management objectives (e.g. recreational opportunity versus commercial yield; Mapstone et al., 2008; Van Beveren et al., 2020); and (3) where there are competing interests for the species, such as between the ecosystem and industrial utilization of forage species (Deroba et al., 2019; Feeney et al., 2019), between the human utilization of forage species and utilization by species that depend on them as prey (Hannesson et al., 2009), between habitat and a fishery (oysters; e.g. Wilberg et al., 2013), between different user groups, transboundary resource conflicts (Song et al., 2017), and including intrinsic, non-extractive values (Lee, 2010; Feeney et al., 2019).

When there are disenfranchised stakeholders

This situation occurs when certain stakeholders are disenfranchised either by not having been part of the original group for whom management was developed or by having no representation in the process. Here, we explicitly classify the ecosystem as an obvious competing and disenfranchised stakeholder for our single-species management paradigm (e.g. Dichmont et al., 2008; Masi et al., 2018). The New England Fishery Management Council herring MSE (Deroba et al., 2019; Feeney et al., 2019) provides a clear example where competing stakeholders, who originally would not have been enfranchised in single-species herring management, were identified, including bluefin tuna fishers (concerned about herring as forage), lobstermen (who need herring as bait), and NGOs. Quantification of fisher behaviour in response to management procedures may also elucidate unforeseen consequences that should be considered within a full MSE (Yletyinen et al., 2018).

When scientific uncertainty threatens the integrity of the current management approach or when status quo management is clearly failing (“known unknowns”)

Every fisheries management decision is made under uncertainty. When these uncertainties have the potential to impact the ability of management measures to achieve the stated objectives, their influence should be explicitly conveyed. We classify these uncertainties, which we are aware of but cannot neatly parameterize, as the “known unknowns”. Atlantic bluefin tuna represents a situation where scientific uncertainty regarding the mixing of two populations, one approximately ten times the size of the other, may threaten the integrity of single-area-based management (Fromentin et al., 2014). In many cases, non-stationary environmental dynamics are expected to change the basis of fisheries productivity (Punt et al., 2016; Haltuch et al., 2019; Karp et al., 2019), but we do not fully understand how these changes will manifest in the future. For example, ocean acidification may affect calcareous-shell-forming organisms, or increasing temperatures may induce distributional shifts of species across management boundaries (Pinsky et al., 2018; Karp et al., 2019; Cisneros-Montemayor et al., 2020). Management of transboundary species, which may be subjected to international fishery agreements, may fall within this category. Further “known unknown” scenarios include those in which the magnitude of total removals is unknown due to international fishing pressure (Van Beveren et al., 2020), high proportion of recreational effort (Shertzer et al., 2019), or illegal, unregulated, or unreported fishing. MSE propagates scientific uncertainties through to the decision-making process, which allows fisheries managers to more transparently manage risks. Desk MSEs may be sufficient to identify potential risks; however, a full stakeholder MSE will be needed when scientific uncertainties result in a high likelihood of failure of existing management, such that objectives need to be reconsidered.

When there are conditions that make future projections unclear (“unknown unknowns”)

Under scenarios in which there are so many unknowns that it becomes challenging to proceed with conventional management, a full stakeholder MSE can serve to identify a management path forward. We define these scenarios afflicted by “unknown unknowns” as those that fall on the extreme end of “wicked problems” with high path dependence (Jentoft and Chuenpagdee, 2009; Hare, 2020). Such scenarios include management related to future climate scenarios, where uncertainty within the system is so high that all dimensions of the problem are yet undefinable. Fisheries management actions relative to a changing climate are plagued by many “unknown unknowns”, from both natural resources and human responses. However, it is precisely in this realm where stakeholder input and ecological knowledge (Berkström et al., 2019; Stori et al., 2019), or the “words of the lagoon” (sensu Johannes, 1980), may be as relevant as existing scientific data in defining future states of nature (e.g. multiple operating models that may bracket true uncertainty) and prioritizing management objectives and actions (Hare, 2020). Operating models in MSEs can project potential, but uncertain, effects of climate change on the biological system (recruitment, growth, movement, or natural mortality; Punt et al., 2014; Haltuch et al., 2019; Jacobsen et al., 2021), but a greater challenge remains in how the human system will respond. Fundamentally, fisheries management is human behavioural modification to achieve human objectives; hence, the solutions must consider this dimension explicitly. Consequently, understanding how humans have responded to environmental changes in the past may provide essential context for defining management procedures for an uncharted future.

(II) Other situations where a full stakeholder MSE may be requested but simpler approaches may suffice

When management objectives and their relative rankings are clearly articulated (such as within a US Fishery Management Plan) and when uncertainties are less extreme, the expense of full stakeholder involvement in an MSE likely exceeds the value added. Here, the problematicity or political distance (Turnbull and Hoppe, 2019) between management objectives is low. In such cases, a desk MSE may be an appropriate approach, or other quantitative or qualitative risk management tools may suffice, such as scenario planning or risk analysis (Sethi, 2010). Below, we articulate five situations where a full stakeholder MSE is probably not necessary.

When an empirical management procedure approach might improve upon status quo management

Many short-lived environmentally driven or recruitment-based (Rice and Browman, 2014) stocks do not lend themselves to the usual 2–5 year time delay between the assessment and the provision of management advice (Shertzer and Prager, 2007). In such fisheries where the conventional best-assessment paradigm is conceptually inappropriate or failing, empirical management procedures (Table 1) may offer a more suitable management approach (e.g. Rademeyer et al., 2007). Empirical management procedures typically rely on a stock indicator (e.g. index of abundance or average length) to adjust total allowable catch advice. Even standard assessment catch advice could be routinely updated with an interim assessment approach (Huynh et al., 2020; Kuriyama et al., 2020), providing more timely and responsive advice. Because of the straightforward link between an abundance indicator and resulting management advice, empirical management procedures are easily understandable and, consequently, are generally favoured by stakeholders. Empirical management procedures have a natural appeal for data-limited scenarios (De Oliveira et al., 2008; Carruthers et al., 2014; Harford et al., 2016; Sagarese et al., 2019). Moreover, even data-rich integrated modelling approaches may not provide the optimal tool for managing a fishery, given the demands for increased throughput and the increasing complexity of assessment models. Here, empirical management procedures and interim approaches, simulation tested to be robust to uncertainties through the equivalent of a desk MSE (Punt, 2010), may provide more timely, cost-effective advice. In these cases, the operational management objectives have already been agreed upon, they are clear, the data used are relatively straightforward compared to conventional assessments, and the empirical management procedure is expected to improve performance relative to all management objectives over the current management approach, such that no new trade-offs are anticipated. The difference is in the timing of how the primary data are used to adjust recommended catches in a more dynamic and responsive manner.

To adopt or modify a CCR when time and resources are limited

Most best management practices recommend adoption of a CCR to adjust catch as a function of stock status (ISSF, 2018), and, ideally, any CCR would be developed through an MSE process, incorporating iterative stakeholder feedback and management procedure tuning. However, this process is costly, time-consuming, and will not justify the expenditure for a large number of stocks for which generic, simple, or interim CCRs are likely to be “pretty-good” approximations (e.g. Thorpe and De Oliveira, 2019; and see Deroba and Bence, 2008 and Punt, 2010 for details and performance of commonly applied CCRs). While the optimal shape of a given CCR might vary and benefit from explicit stock-specific determinations (Zhang et al., 2011), until such time as this can be done, “perfect is the enemy of the good” for a large number of stocks. In these cases, generic CCRs (Table 1; e.g. the 40:10 rule used for many stocks managed by the US Pacific Fishery Management Council) could be adopted on an interim basis until such time as full, stock-specific MSE-vetted CCRs are obtained. Indeed, some Fishery Management Plans have default generic control rules in place for those stocks that are not actively managed (PFMC, 2019). Furthermore, if the operational management objectives are already clearly specified (e.g. maintain the stock at the biomass that produces maximum sustainable yield, avoid overfishing, etc.), then testing of alternate CCRs may only require a desk MSE. However, some stakeholder engagement would likely occur before the adoption of any new CCR in practice.

Mainly tactical decisions regarding allocation of survey and scientific resources

These include a range of evaluations related to maximizing the value of information needed to support management while minimizing resource expenditure, including identifying necessary sample sizes, allocation of survey resources, and frequency of surveys and assessments (Carruthers et al., 2016; Harford and Babcock, 2016; Li et al., 2016; Ono et al., 2018). In these cases, a desk MSE could be used to inform decisions about these observation and assessment aspects of a management procedure and how they affect the management procedures’ ability to achieve management objectives. If the resulting management implications are not of interest, the investment in developing an MSE framework just to address these types of questions may not be warranted. Many of these questions may be better addressed more simply with a simulation approach, such as the observing system simulation experiment (OSSE) framework developed in atmospheric sciences (Atlas et al., 1985) or others (Siegfried et al., 2016; Zimmerman and Enberg, 2017; Nesslage et al., 2020). The R program ss3sim (Anderson et al., 2014) for simulation testing Stock Synthesis models is an excellent example of simulation testing without the feedback loop.

When stakeholders desire information for an external purpose

Many international regional fisheries management organizations (RFMOs) are incentivized to conduct MSEs, since MSE-derived CCRs are one consideration for many sustainability certifications. Situations such as these, where MSEs are requested to serve the interests of particular stakeholder groups rather than management needs, should be of low priority. In these situations, stakeholders pay consulting companies to evaluate the status of a fishery and to apply for certain sustainability certifications. When such certifications are driving motivations for conducting MSE, then the lead beneficiaries of such certification should surely pay to support or co-fund an MSE, and a desk MSE would suffice if management objectives are already clearly defined.

Research/scientific questions not intended to directly support management advice

A substantial number of MSE studies have been conducted in the absence of stakeholder input and without clear requests from managers. These include a large number of published research papers, many of which focus on improving the assessment process (e.g. Irwin et al., 2008; Carruthers et al., 2016), evaluating the inclusion of environmental factors in assessments (Harford et al., 2018; Haltuch et al., 2019), and developing modelling capacity and tools for MSEs with a scope beyond single-species management (Kaplan et al., 2021). In almost every case, these are desk MSEs and can be considered part of the necessary research and development process for improving stock assessment practice and the provision of management advice.

As the interdisciplinary nature of fisheries management continues to broaden (Phillipson and Symes, 2013), the scope of MSEs may need to expand. Indeed, for some applications, this expansion has already begun, for example, taking the shape of ecosystem-level MSEs (e.g. Fulton et al., 2014). In the face of a changing climate, projecting the efficacy of fisheries management approaches may require tighter links to ocean models and observation systems, where the oceanographic simulation capacity of OSSEs (Atlas et al., 1985) might be valuable for creating more realistic operating models. Ecosystem-based fisheries management (Pikitch et al., 2004) and dynamic ocean management strategies (Maxwell et al., 2015) can also be explored within the desk MSE framework to develop and validate the simulation models before attempting a full stakeholder MSE. However, many management problems cannot be simulation tested, and/or development of the required operating models and simulations would take too long or be too resource intensive to inform management  actions. Consider, for example, a situation where a multispecies management procedure is desired but managers or stakeholders require detailed, fleet-specific information on interactions with unmanaged, data-poor, or protected stocks, stocks migrating through the area, interactions with stocks managed by other entities, and climate interactions, and where there exist governance challenges to implementing the management procedure. Here, alternative methods, such as conceptual modelling, scenario planning, or Bayesian belief networks (e.g. Hoshino et al., 2016), may lend promise for finding solutions that allow a more tractable MSE to proceed. Ultimately, whether or not these simulation experiments require stakeholder involvement depends on whether the management objectives are clearly articulated and whether the relative ranking of objectives will influence the resulting management decisions.

Accordingly, attention should be paid to the necessity of defining exceptional circumstances and timelines to revisit implemented management procedures (e.g. every 4 years in South Africa; de Moor et al., 2022). Where MSE results are used to alter management actions, exceptional circumstance provisions and management procedure reviews will be necessary, particularly as we increasingly expect future non-stationary dynamics to prevail. However, these supplementary reviews may not be required where simpler desk MSE or other approaches are used to answer more general research questions.

Conclusion

RFMOs, national fisheries agencies, and other scientific and management entities worldwide are in the process of expending considerable capital on MSE. In many cases, these efforts are going towards the most valuable and most heavily resourced fisheries that usually already have solid conventional management and previously defined management objectives. Given the limited resources to conduct MSE, we offer that analysts and decision makers should carefully consider whether the management or scientific question can be answered more efficiently with a tool that is less intensive than an MSE, and prioritize the highest profile, highest conflict, and paradigm-changing management decisions for an MSE with extended stakeholder engagement.

For example, many MSEs worldwide are conducted to develop a catch (né harvest) control rule for fisheries with existing management objectives. Involving stakeholders when management goals are already identified induces redundancy within the management process and effectively delays management procedure implementation while running the risk of inducing stakeholder fatigue. The MSE process becomes substantially more costly and time-consuming without adding compensatory value. Furthermore, it may even be counter-productive by providing a forum for inherently management-based decisions, such as allocations, to then be litigated through the scientific process. Moreover, spreading limited technical resources thinly across too many simultaneous projects risks compromising the integrity of all of them. Participatory processes are intended to increase buy-in and legitimacy of the technical work of an MSE, but if progress is slowed such that involved stakeholders become frustrated, then the goals of stakeholder engagement may not be achieved.

Full stakeholder MSEs are required when management objectives are not clearly stated or are incomplete, when conflicting objectives are going to result in clear winners and losers in the adoption or modification of a management procedure, or when certain parties may be underrepresented in existing management. Stakeholder participation should also be prioritized if their knowledge can contribute to revealing uncertainties to which management should be robust. Full stakeholder participation should not be prioritized when management objectives are implicit or known, or when MSE results are not intended to be used to directly guide management advice. These applications can often be conducted more efficiently as desk MSEs (e.g. Carruthers et al., 2016; Li et al., 2016; Zimmerman and Enberg, 2017; Carruthers and Hordyk, 2019), as MSEs with intermediate stakeholder input, or may not even require the full MSE feedback loop (e.g. Siegfried et al.,2016).

We recognize the controversial nature of a statement that proposes to limit stakeholder involvement in MSEs despite numerous papers that propose this as a key element of the process (Goethel et al., 2019; Miller et al., 2019). Both Miller et al. (2019) and Goethel et al. (2019) outline processes and examples of stakeholder engagement, and we build upon both to prioritize when and to what degree this engagement should occur. Almost invariably, stakeholders will have an opportunity for input at some point in the process before a management procedure is implemented, but the degree to which stakeholders should be involved in the technical aspects of conducting an MSE needs careful consideration. We propose this not to disenfranchise the stakeholder process, but rather because stakeholder attention is such a precious resource that we want it focused upon the highest-profile, paradigm-changing, and decision-based applications. In conclusion, we hope that this essay will assist our community to be prudent in the allocation of what is the scarcest of all human commodities, our time.

Acknowledgement

The authors thank Jason Link for initiating the concept for this essay, Mandy Karnauskas for reviewing an earlier version, and the journal reviewers for their valuable reviews.

Conflict of Interest

The views and opinions expressed or implied in this article are those of the authors and do not necessarily reflect the position of the National Oceanic and Atmospheric Administration. The authors declare no conflicts of interest.

Data availability

No data were generated or used in this study.

Funding

The authors are funded by NOAA Fisheries, and have no additional funding to report.

References