Management strategy evaluation: best practices

Management strategy evaluation (MSE) involves using simulation to compare the relative effectiveness for achieving management objectives of different combinations of data collection schemes, methods of analysis and subsequent processes leading to management actions. MSE can be used to identify a ‘best’ management strategy among a set of candidate strategies, or to determine how well an existing strategy performs. The ability of MSE to facilitate fisheries management achieving its aims depends on how well uncertainty is represented, and how effectively the results of simulations are summarized and presented to the decision-makers. Key challenges for effective use of MSE therefore include characterizing objectives and uncertainty, assigning plausibility ranks to the trials considered, and working with decision-makers to interpret and implement the results of the MSE. This paper explores how MSEs are conducted and characterizes current ‘best practice’ guidelines, while also indicating whether and how these best practices were applied to two case-studies: the Bering–Chukchi–Beaufort Seas bowhead whales (Balaena mysticetus; Balaenidae) and the northern subpopulation of Pacific sardine (Sardinops sagax caerulea; Clupeidae).

[1]  N. Lo,et al.  Assessment of the Pacific sardine resource in 2009 for U. S. management in 2010: Executive Summary , 2009 .

[2]  Marie-Joëlle Rochet,et al.  Simulation-based management strategy evaluation: ignorance disguised as mathematics? , 2009 .

[3]  David C. Smith,et al.  Lessons in modelling and management of marine ecosystems: the Atlantis experience , 2011 .

[4]  S. E. Wayte Management implications of including a climate-induced recruitment shift in the stock assessment for jackass morwong (Nemadactylus macropterus) in south-eastern Australia , 2013 .

[5]  J. Isaacs,et al.  ABUNDANCE OF PELAGIC FISH DURING THE 19TH AND 20TH CENTURIES AS RECORDED IN ANAEROBIC SEDIMENT OFF THE CALIFORNIAS , 1974 .

[6]  André E. Punt,et al.  Developing management procedures that are robust to uncertainty: lessons from the International Whaling Commission , 2007 .

[7]  Nokome Bentley,et al.  Contrasting Paradigms for Fisheries Management Decision Making: How Well Do They Serve Data-Poor Fisheries? , 2009 .

[8]  Doug S Butterworth,et al.  Evaluation of a class of possible simple interim management procedures for the Namibian hake fishery , 2001 .

[9]  Jean-Marc Fromentin,et al.  A MANAGEMENT STRATEGY EVALUATION FRAMEWORK FOR MEDITERRANEAN ATLANTIC BLUEFIN TUNA , 2009 .

[10]  André E. Punt,et al.  Management strategy evaluation for rock lobster, Jasus edwardsii, off Victoria, Australia: Accounting for uncertainty in stock structure , 2009 .

[11]  Han'guk Susan Kwahakhoe Fisheries and aquatic sciences , 2009 .

[12]  Doug S Butterworth,et al.  Technical details underlying the Management Strategy Evaluation process leading to selection of a Management Procedure for Western Component (4Xopqrs5) pollock , 2011 .

[13]  André E. Punt,et al.  Development and evaluation of a cpue-based harvest control rule for the southern and eastern scalefish and shark fishery of Australia , 2011 .

[14]  Natalie Dowling,et al.  Risk management tools for sustainable fisheries management under changing climate: a sea cucumber example , 2013, Climatic Change.

[15]  Doug S Butterworth,et al.  Towards rational harvesting of the South African anchovy considering survey imprecision and recruitment variability , 1987 .

[16]  James N. Ianelli,et al.  Evaluating management strategies for eastern Bering Sea walleye pollock (Theragra chalcogramma) in a changing environment , 2011 .

[17]  José A. A. De Oliveira,et al.  Lessons for fisheries management from the EU cod recovery plan , 2013 .

[18]  N. Bunnefeld,et al.  Management strategy evaluation: a powerful tool for conservation? , 2011, Trends in ecology & evolution.

[19]  Edward A. Codling,et al.  On scientists’ discomfort in fisheries advisory science: the example of simulation-based fisheries management-strategy evaluations , 2010 .

[20]  C. Davies,et al.  Salvaged pearls: lessons learned from a floundering attempt to develop a management procedure for Southern Bluefin Tuna , 2008 .

[21]  Doug S Butterworth,et al.  South coast rock lobster OMP 2014: initial specifications , 2014 .

[22]  Steen Christensen,et al.  Evaluation of management strategies - a bioeconomic approach applied to the Greenland Shrimp Fishery , 1997 .

[23]  T. Quinn,et al.  Using perceptions of data accuracy and empirical weighting of information: assessment of a recreational fish population , 2000 .

[24]  Éva E Plagányi,et al.  Models for an ecosystem approach to fisheries , 2007 .

[25]  M. Wilberg,et al.  An Evaluation of Harvest Control Rules for Data-Poor Fisheries , 2013 .

[26]  André E. Punt,et al.  The performance of a management procedure for rock lobsters, Jasus edwardsii, off western Victoria, Australia in the face of non-stationary dynamics , 2013 .

[27]  V Dayal,et al.  Report of the scientific committee. , 2000, Journal of the Indian Medical Association.

[28]  Doug S Butterworth,et al.  Tips and tricks in designing management procedures , 2007 .

[29]  S. Cox,et al.  Practical stakeholder-driven harvest policies for groundfish fisheries in British Columbia, Canada , 2008 .

[30]  André E. Punt,et al.  Experiences in the evaluation and implementation of management procedures , 1999 .

[31]  André E. Punt,et al.  The impact of regime shifts on the performance of management strategies for the Gulf of Alaska walleye pollock (Theragra chalcogramma) fishery , 2009 .

[33]  Susan J Johnston,et al.  Development and application of management procedures for fisheries in southern Africa , 1999 .

[34]  Doug S Butterworth,et al.  Pretesting the Likely Efficacy of Suggested Management Approaches to Data-Poor Fisheries , 2010 .

[35]  André E. Punt,et al.  Design of operational management strategies for achieving fishery ecosystem objectives , 2000 .

[36]  André E. Punt,et al.  Incorporating ecosystem forcing through predation into a management strategy evaluation for the Gulf of Alaska walleye pollock (Theragra chalcogramma) fishery , 2010 .

[37]  André E. Punt,et al.  Harvest Strategy Evaluation for School and Gummy Shark , 2005 .

[38]  William N. Venables,et al.  Beyond biological performance measures in management strategy evaluation: Bringing in economics and the effects of trawling on the benthos , 2008 .

[39]  Doug S Butterworth,et al.  Generic management procedures for data-poor fisheries: forecasting with few data , 2015 .

[40]  Rebecca L. Lewison,et al.  Evaluating sustainability of fisheries bycatch mortality for marine megafauna: a review of conservation reference points for data-limited populations , 2013, Environmental Conservation.

[41]  Laurence T. Kell,et al.  Evaluation of the robustness of maximum sustainable yield based management strategies to variations in carrying capacity or migration pattern of Atlantic bluefin tuna (Thunnus thynnus) , 2007 .

[42]  José A. A. De Oliveira,et al.  Fisheries management under climate and environmental uncertainty: control rules and performance simulation , 2014 .

[43]  David B. Sampson,et al.  A Monte Carlo Evaluation of the Stock Synthesis Assessment Program , 1998 .

[44]  D. Howell,et al.  Unquantifiable uncertainty in projecting stock response to climate change: Example from North East Arctic cod , 2013 .

[45]  José A. A. De Oliveira,et al.  Is the management procedure approach equipped to handle short-lived pelagic species with their boom and bust dynamics? The case of the South African fishery for sardine and anchovy , 2011 .

[46]  Natalie Dowling,et al.  Developing harvest strategies for low-value and data-poor fisheries: Case studies from three Australian fisheries , 2008 .

[47]  André E. Punt,et al.  Impacts of spatial uncertainty on performance of age structure-based harvest strategies for blue eye trevalla (Hyperoglyphe antarctica) , 2011 .

[48]  De Oliveira,et al.  The development and implementation of a joint management procedure for the South African pilchard and anchovy resources , 2002 .

[49]  On an approach for comparing the implications of alternative fish stock assessments, with application to the stock of Cape hake Merluccius spp. off northern Namibia , 1991 .

[50]  Richard D. Methot,et al.  Stock synthesis: A biological and statistical framework for fish stock assessment and fishery management , 2013 .

[51]  D. Butterworth A commentary on: salvaged pearls: lessons learned from a floundering attempt to develop a management procedure for Southern Bluefin Tuna , 2008 .

[52]  Doug S Butterworth,et al.  Greenland Halibut MSE Results for Updated SCAA Reference Case and Robustness Test Operating Models , 2010 .

[53]  R. D. Scott,et al.  Evaluation of multiple management objectives for Northeast Atlantic flatfish stocks: sustainability vs. stability of yield , 2005 .

[54]  Ralf Goericke,et al.  Re-assessment of the stock-recruit and temperature-recruit relationships for Pacific sardine (Sardinops sagax) , 2010 .

[55]  Elizabeth A. Fulton,et al.  An Integrated Approach Is Needed for Ecosystem Based Fisheries Management: Insights from Ecosystem-Level Management Strategy Evaluation , 2014, PloS one.

[56]  A. Maccall,et al.  How useful is the ratio of fish density outside versus inside no-take marine reserves as a metric for fishery management control rules? , 2011 .

[57]  James R. Bence,et al.  Influence of Age-Selective Surveys on the Reliability of Stock Synthesis Assessments , 1993 .

[58]  J. George,et al.  Age and growth estimates of bowhead whales (Balaena mysticetus) via aspartic acid racemization , 1999 .

[59]  R. Hilborn,et al.  Can information from marine protected areas be used to inform control-rule-based management of small-scale, data-poor stocks? , 2011 .

[60]  Other Report of the ICES Advisory Committee, 2010 , 2010 .

[61]  William N. Venables,et al.  Management strategies for short lived species: The case of Australia's Northern Prawn Fishery 1. Accounting for multiple species, spatial structure and implementation uncertainty when evaluating risk , 2006 .

[62]  Doug S Butterworth,et al.  Why a management procedure approach? Some positives and negatives , 2007 .

[63]  Richard Hillary An introduction to FLR fisheries simulation tools , 2009 .

[64]  André E. Punt,et al.  The impact of climate change on the performance of rebuilding strategies for overfished groundfish species of the U.S. west coast , 2011 .

[65]  Anthony D. M. Smith,et al.  Implementing effective fisheries-management systems – management strategy evaluation and the Australian partnership approach , 1999 .

[66]  D. S. Butterworth,et al.  Evolution of operational management procedures for the South African West Coast rock lobster (Jasus lalandii) fishery , 2005 .

[67]  Elizabeth A. Fulton,et al.  Multispecies fisheries management and conservation: tactical applications using models of intermediate complexity , 2014 .

[68]  A. Raftery,et al.  Assessment of the Bering-Chukchi-Beaufort Seas stock of bowhead whales using the BALEEN II model in a Bayesian synthesis framework , 1995 .

[69]  Assessment of the Pacific Sardine Resource in 2011 for U.S. Management in 2012 , 2013 .

[70]  André E. Punt,et al.  The Management Strategy Evaluation Approach and the Fishery for Walleye Pollock in the Gulf of Alaska , 2008 .

[71]  André E. Punt,et al.  Evaluating empirical decision rules for southern rock lobster fisheries: A South Australian example , 2012 .

[72]  Bo Sølgaard Andersen,et al.  TEMAS: fleet-based bio-economic simulation software to evaluate management strategies accounting for fleet behaviour , 2007 .

[73]  Catherine M. Dichmont,et al.  EDITOR'S CHOICE: Evaluating marine spatial closures with conflicting fisheries and conservation objectives , 2013 .

[74]  Jeremy D. Prince,et al.  A Management Strategy for Sedentary Nearshore Species that Uses Marine Protected Areas as a Reference , 2010 .

[75]  K. Shirakihara,et al.  A feedback management procedure based on controlling the size of marine protected areas , 2005, Fisheries Science.

[76]  A. Punt,et al.  Evaluating the impact of buffers to account for scientific uncertainty when setting TACs: application to red king crab in Bristol Bay, Alaska , 2012 .

[77]  Nokome Bentley,et al.  Moving Fisheries from Data-Poor to Data-Sufficient: Evaluating the Costs of Management versus the Benefits of Management , 2009 .

[78]  Martin A. Pastoors,et al.  Validating management simulation models and implications for communicating results to stakeholders , 2007 .

[79]  Randall M. Peterman,et al.  Possible solutions to some challenges facing fisheries scientists and managers , 2004 .

[80]  Carl J. Walters,et al.  Adaptive Management of Renewable Resources , 1986 .

[81]  Benoit Mesnil,et al.  An evaluation of the implicit management procedure used for some ICES roundfish stocks , 2005 .

[82]  Legal Minimum Lengths and the Management of Abalone Fisheries , 2013 .

[83]  Martin A. Pastoors,et al.  The effect of management choices on the sustainability and economic performance of a mixed fishery: a simulation study , 2008 .

[85]  W. Venables,et al.  Management strategies for short lived species: The case of Australia's Northern Prawn Fishery 3. Factors affecting management and estimation performance , 2006 .

[86]  R. I. C. C. Francis,et al.  "Risk" in fisheries management: a review , 1997 .

[87]  A. E. Punt,et al.  The effects of future consumption by the Cape fur seal on catches and catch rates of the Cape hakes. 4. Modelling the biological interaction between Cape fur seals Arctocephalus pusillus pusillus and the Cape hakes Merluccius capensis and M. paradoxus , 1995 .

[88]  Darren Dennis,et al.  Integrating indigenous livelihood and lifestyle objectives in managing a natural resource , 2013, Proceedings of the National Academy of Sciences.

[89]  Doug S Butterworth,et al.  Developing and refining a joint management procedure for the multispecies South African pelagic fishery , 2004 .

[90]  Benoit Mesnil,et al.  An evaluation of multi-annual management strategies for ICES roundfish stocks , 2006 .

[91]  David C. M. Miller,et al.  "Satisficing" and trade-offs: evaluating rebuilding strategies for Greenland halibut off the east coast of Canada , 2010 .

[92]  Nils Bunnefeld,et al.  New directions in management strategy evaluation through cross-fertilization between fisheries science and terrestrial conservation , 2010, Biology Letters.

[93]  Philippe Grosjean,et al.  FLR: an open-source framework for the evaluation and development of management strategies , 2007 .

[94]  T. Essington,et al.  Assembly rules for aggregate-species production models: simulations in support of management strategy evaluation , 2012 .

[95]  K. Hiramatsu,et al.  Developing a management procedure robust to uncertainty for southern bluefin tuna: a somewhat frustrating struggle to bridge the gap between ideals and reality , 2010, Population Ecology.

[96]  André E. Punt,et al.  Use of multiple selectivity patterns as a proxy for spatial structure , 2014 .

[97]  Management of an invasive marine species: defining and testing the effectiveness of ballast-water management options using management strategy evaluation , 2008 .

[98]  Andrew A. Rosenberg,et al.  Uncertainty and Risk Evaluation in Stock Assessment Advice for U.S. Marine Fisheries , 1994 .

[99]  Beatriz A. Roel,et al.  Management procedures in a fishery based on highly variable stocks and with conflicting objectives: experiences in the South African pelagic fishery , 1998, Reviews in Fish Biology and Fisheries.

[100]  André E. Punt,et al.  Evaluation of management tools for Australia's South East Fishery.3. Towards selecting appropriate harvest strategies , 2002 .

[101]  André E. Punt,et al.  Which ecological indicators can robustly detect effects of fishing , 2005 .

[102]  Doug S Butterworth,et al.  Complex assessments or simple management procedures for efficient fisheries management: a comparative study , 2015 .

[103]  André E. Punt,et al.  Management strategy evaluation for line fishing in the Great Barrier Reef: balancing conservation and multi-sector fishery objectives , 2008 .

[104]  J. Cooke,et al.  Improvement of fishery-management advice through simulation testing of harvest algorithms , 1999 .

[105]  Cody S. Szuwalski,et al.  Fisheries management for regime-based ecosystems: a management strategy evaluation for the snow crab fishery in the eastern Bering Sea , 2013 .

[106]  Richard D. Methot,et al.  Does MPA mean 'Major Problem for Assessments'? Considering the consequences of place-based management systems , 2006 .

[107]  C. Dichmont,et al.  A Case Study in Successful Management of a Data-Poor Fishery Using Simple Decision Rules: the Queensland Spanner Crab Fishery , 2010 .

[108]  Panayiota Apostolaki,et al.  Evaluating the performance of survey-based operational management procedures , 2010 .

[109]  Tore Schweder,et al.  On the effect on cod and herring fisheries of retuning the Revised Management Procedure for minke whaling in the greater Barents Sea , 1998 .

[110]  Thomas R Carruthers,et al.  Spatial surplus production modeling of Atlantic tunas and billfish. , 2011, Ecological applications : a publication of the Ecological Society of America.

[111]  Carl J. Walters,et al.  Use of Gaming Procedures in Evaluation of Management Experiments , 1994 .

[112]  André E. Punt,et al.  Purported flaws in management strategy evaluation: basic problems or misinterpretations? , 2010 .

[113]  C. Ulrich,et al.  Challenges in integrating short-term behaviour in a mixed-fishery Management Strategies Evaluation frame: A case study of the North Sea flatfish fishery , 2010 .

[114]  G. Tuck Are bycatch rates sufficient as the principal fishery performance measure and method of assessment for seabirds , 2011 .

[115]  Sean C. Anderson,et al.  ss3sim: An R Package for Fisheries Stock Assessment Simulation with Stock Synthesis , 2014, PloS one.

[116]  R. Hilborn,et al.  Exploitation rate reference points for West Coast rockfish: Are they robust and are there better alternatives? , 2002 .

[117]  Daniel Goodman,et al.  Ecosystem-based fisheries management: some practical suggestions , 2007 .

[118]  André E. Punt,et al.  The evaluation of two management strategies for the Gulf of Alaska walleye pollock fishery under climate change , 2009 .

[119]  G. Dinardo,et al.  Accounting for uncertainty in the development of harvest strategies for the Northwestern Hawaiian Islands lobster trap fishery , 1999 .

[120]  Paul R. Wade,et al.  CALCULATING LIMITS TO THE ALLOWABLE HUMAN‐CAUSED MORTALITY OF CETACEANS AND PINNIPEDS , 1998 .

[121]  André E. Punt,et al.  Evaluating methods for setting catch limits in data-limited fisheries , 2014 .

[122]  J. Bence,et al.  Comparison of virtual population analysis and statistical kill-at-age analysis for a recreational, kill-dominated fishery , 2005 .

[123]  Mitsuyo Mori,et al.  Modelling the krill-predator dynamics of the Antarctic ecosystem , 2005 .

[124]  David C. Smith,et al.  Human behaviour: the key source of uncertainty in fisheries management , 2011 .

[125]  Doug S Butterworth,et al.  A history of recent bases for management and the development of a species-combined Operational Management Procedure for the South African hake resource , 2008 .

[126]  David C. Smith,et al.  Scientific tools to support the practical implementation of ecosystem-based fisheries management , 2007 .