Validation metrics for turbulent plasma transport

Developing accurate models of plasma dynamics is essential for confident predictive modeling of current and future fusion devices. In modern computer science and engineering, formal verification and validation processes are used to assess model accuracy and establish confidence in the predictive capabilities of a given model. This paper provides an overview of the key guiding principles and best practices for the development of validation metrics, illustrated using examples from investigations of turbulent transport in magnetically confined plasmas. Particular emphasis is given to the importance of uncertainty quantification and its inclusion within the metrics, and the need for utilizing synthetic diagnostics to enable quantitatively meaningful comparisons between simulation and experiment. As a starting point, the structure of commonly used global transport model metrics and their limitations is reviewed. An alternate approach is then presented, which focuses upon comparisons of predicted local fluxes, ...

[1]  Hugh W. Coleman,et al.  Comprehensive Approach to Verification and Validation of CFD Simulations—Part 1: Methodology and Procedures , 2001 .

[2]  Habib N. Najm,et al.  Uncertainty Quantification and Polynomial Chaos Techniques in Computational Fluid Dynamics , 2009 .

[3]  R. Nazikian,et al.  Fast ion induced shearing of 2D Alfvén eigenmodes measured by electron cyclotron emission imaging. , 2011, Physical review letters.

[4]  R. Budny,et al.  A STANDARD DT SUPERSHOT SIMULATION , 1994 .

[5]  M. Barnes,et al.  Direct multiscale coupling of a transport code to gyrokinetic turbulence codes , 2009, 0912.1974.

[6]  N Hawkes,et al.  A key to improved ion core confinement in the JET tokamak: ion stiffness mitigation due to combined plasma rotation and low magnetic shear. , 2011, Physical review letters.

[7]  F. Halpern,et al.  Approaching the investigation of plasma turbulence through a rigorous verification and validation procedure: A practical examplea) , 2015 .

[8]  Gregory W. Hammett,et al.  Toroidal gyrofluid equations for simulations of tokamak turbulence , 1996 .

[9]  T. S. Hahm,et al.  Zonal flows in plasma—a review , 2005 .

[10]  Paolo Ricci,et al.  Methodology for turbulence code validation: Quantification of simulation-experiment agreement and application to the TORPEX experiment , 2011 .

[11]  M. Greenwald,et al.  Multi-scale gyrokinetic simulation of Alcator C-Mod tokamak discharges , 2014 .

[12]  W. Horton.  Drift waves and transport , 1999 .

[13]  Henrik Bindslev,et al.  Chapter 7: Diagnostics , 2007 .

[14]  David Borland,et al.  Rainbow Color Map (Still) Considered Harmful , 2007 .

[15]  Laurent Villard,et al.  A global collisionless PIC code in magnetic coordinates , 2007, Comput. Phys. Commun..

[16]  G. Hammett,et al.  A Landau fluid model for electromagnetic plasma microturbulence , 2001 .

[17]  H. Sugama,et al.  Nonlinear electromagnetic gyrokinetic equation for plasmas with large mean flows , 1998 .

[18]  A. Werner,et al.  Modelling of JET Diagnostics Using Bayesian Graphical Models , 2011 .

[19]  D. P. Stotler,et al.  Validation in fusion research: Towards guidelines and best practices , 2008, 0801.2787.

[20]  M. Nakata,et al.  Cross-Scale Interactions between Electron and Ion Scale Turbulence in a Tokamak Plasma. , 2015, Physical review letters.

[21]  J. Rice,et al.  Impurity transport, turbulence transitions and intrinsic rotation in Alcator C-Mod plasmas , 2014 .

[22]  W. Nevins,et al.  System for simulating fluctuation diagnostics for application to turbulence computations , 2006 .

[23]  Mark Nornberg,et al.  Determination of Zeff by integrating measurements from x-ray tomography and charge exchange recombination spectroscopy , 2015 .

[24]  R. Waltz,et al.  VALIDATING SIMULATIONS OF CORE TOKAMAK TURBULENCE: CURRENT STATUS AND FUTURE DIRECTIONS , 2008 .

[25]  R. W. Harvey,et al.  Benchmarking of codes for electron cyclotron heating and electron cyclotron current drive under ITER conditions , 2008 .

[26]  Mike Kotschenreuther,et al.  Comparison of initial value and eigenvalue codes for kinetic toroidal plasma instabilities , 1995 .

[27]  C. Bourdelle,et al.  Electromagnetic stabilization of tokamak microturbulence in a high-β regime , 2014, 1409.1963.

[28]  J. Madsen.  Full-F gyrofluid model , 2013 .

[29]  P. Roache.  QUANTIFICATION OF UNCERTAINTY IN COMPUTATIONAL FLUID DYNAMICS , 1997 .

[30]  Paul T. Bonoli,et al.  Review of recent experimental and modeling progress in the lower hybrid range of frequencies at ITER relevant parameters , 2014 .

[31]  C. Fuchs,et al.  Integrated Data Analysis of Profile Diagnostics at ASDEX Upgrade , 2010 .

[32]  C. Bourdelle,et al.  Ion temperature profile stiffness: non-linear gyrokinetic simulations and comparison with experiment , 2013, 1303.2217.

[33]  Gregory W. Hammett,et al.  Gyro‐Landau fluid models for toroidal geometry , 1992 .

[34]  F Jenko,et al.  Nonlinear stabilization of tokamak microturbulence by fast ions. , 2013, Physical review letters.

[35]  D. J. Campbell,et al.  Chapter 1: Overview and summary , 1999 .

[36]  D. Russell,et al.  Comparison of scrape-off layer turbulence simulations with experiments using a synthetic gas puff imaging diagnostic , 2011 .

[37]  Atsushi Fukuyama,et al.  Transport simulation on L-mode and improved confinement associated with current profile modification , 1995 .

[38]  Charlson C. Kim,et al.  Comparisons and physics basis of tokamak transport models and turbulence simulations , 2000 .

[39]  L. Lao,et al.  Edge radial electric field structure in quiescent H-mode plasmas in the DIII-D tokamak , 2003 .

[40]  T. Evans.  Resonant magnetic perturbations of edge-plasmas in toroidal confinement devices , 2015 .

[41]  Choong-Seock Chang,et al.  Full-f gyrokinetic particle simulation of centrally heated global ITG turbulence from magnetic axis to edge pedestal top in a realistic tokamak geometry , 2009 .

[42]  B. Scott.  Derivation via free energy conservation constraints of gyrofluid equations with finite-gyroradius electromagnetic nonlinearities , 2007, 0710.4899.

[43]  Plasma.  Chapter 3: MHD stability, operational limits and disruptions , 1999 .

[44]  M. Greenwald,et al.  Fidelity of reduced and realistic electron mass ratio multi-scale gyrokinetic simulations of tokamak discharges , 2015 .

[45]  Timothy G. Trucano,et al.  Verification and Validation in Computational Fluid Dynamics , 2002 .

[46]  Jeff M. Candy,et al.  The effect of ion-scale dynamics on electron-temperature-gradient turbulence , 2007 .

[47]  R. Waltz,et al.  A gyro-Landau-fluid transport model , 1997 .

[48]  Frank Jenko,et al.  Electron temperature gradient driven turbulence , 1999 .

[49]  S. J. Wukitch,et al.  Lower hybrid current drive experiments on Alcator C-Mod : Comparison with theory and simulation , 2007 .

[50]  R. Nazikian,et al.  Radial structure of Alfvén eigenmodes in the DIII-D tokamak through electron-cyclotron-emission measurements. , 2006, Physical review letters.

[51]  H. Doerk,et al.  Gyrokinetic studies of core turbulence features in ASDEX Upgrade H-mode plasmas , 2015 .

[52]  W. A. Peebles,et al.  L-mode validation studies of gyrokinetic turbulence simulations via multiscale and multifield turbulence measurements on the DIII-D tokamak , 2011 .

[53]  Yasuhiro Idomura,et al.  Computation-Communication Overlap Techniques for Parallel Spectral Calculations in Gyrokinetic Vlasov Simulations , 2013 .

[54]  P. Bonoli,et al.  Validation of full-wave simulations for mode conversion of waves in the ion cyclotron range of frequencies with phase contrast imaging in Alcator C-Mod , 2015 .

[55]  L. Lao,et al.  PREDICTIVE CAPABILITY OF MHD STABILITY LIMITS IN HIGH PERFORMANCE DIII-D DISCHARGES , 2002 .

[56]  L. Lao,et al.  Theory and Simulation Basis for Magnetohydrodynamic Stability in DIII-D , 2005 .

[57]  Jeff M. Candy,et al.  Tokamak profile prediction using direct gyrokinetic and neoclassical simulation , 2009 .

[58]  Yang Chen,et al.  Electromagnetic gyrokinetic δf particle-in-cell turbulence simulation with realistic equilibrium profiles and geometry , 2007, J. Comput. Phys..

[59]  K. H. Burrell,et al.  Effects of E×B velocity shear and magnetic shear on turbulence and transport in magnetic confinement devices , 1997 .

[60]  P. T. Bonoli,et al.  Ion cyclotron range of frequencies mode conversion electron heating in deuterium–hydrogen plasmas in the Alcator C-Mod tokamak , 2003 .

[61]  Jeff M. Candy,et al.  A unified method for operator evaluation in local Grad–Shafranov plasma equilibria , 2009 .

[62]  David J. Schlossberg,et al.  2D properties of core turbulence on DIII-D and comparison to gyrokinetic simulations , 2012 .

[63]  T. L. Rhodes,et al.  Advances in validating gyrokinetic turbulence models against L- and H-mode plasmas a) , 2011 .

[64]  Arnold H. Kritz,et al.  Physics basis of Multi-Mode anomalous transport module , 2013 .

[65]  William Dorland,et al.  Quantitative predictions of tokamak energy confinement from first‐principles simulations with kinetic effects , 1995 .

[66]  P. T. Bonoli,et al.  Role of trapped electron mode turbulence in internal transport barrier control in the Alcator C-Mod Tokamak , 2004 .

[67]  P. Bonoli,et al.  Observation and modelling of ion cyclotron range of frequencies waves in the mode conversion region of Alcator C-Mod , 2005 .

[68]  W. Dorland,et al.  Comparing simulation of plasma turbulence with experiment. II. Gyrokinetic simulations , 2002 .

[69]  T. C. Luce,et al.  Application of dimensionless parameter scaling techniques to the design and interpretation of magnetic fusion experiments , 2008 .

[70]  M. Greenwald.  Verification and validation for magnetic fusiona) , 2010 .

[71]  Tom Fawcett,et al.  An introduction to ROC analysis , 2006, Pattern Recognit. Lett..

[72]  T. L. Rhodes,et al.  Measurements of the cross-phase angle between density and electron temperature fluctuations and comparison with gyrokinetic simulations , 2010 .

[73]  W. Heidbrink,et al.  Fast-ion Dα measurements of the fast-ion distribution (invited). , 2010, The Review of scientific instruments.

[74]  J. Candy,et al.  Linear and nonlinear verification of gyrokinetic microstability codes , 2011 .

[75]  T. L. Rhodes,et al.  Reduced electron thermal transport in low collisionality H-mode plasmas in DIII-D and the importance of TEM/ETG-scale turbulence , 2012 .

[76]  K. Matsuda,et al.  Ray tracing study of the electron cyclotron current drive in DIII-D using 60 GHz , 1989 .

[77]  X. Garbet,et al.  Global full-f gyrokinetic simulations of plasma turbulence , 2007 .

[78]  W. Heidbrink,et al.  CORRIGENDUM: The behaviour of fast ions in tokamak experiments , 1994 .

[79]  E. Joffrin,et al.  Chapter 6: Steady state operation , 2007 .

[80]  A. Peeters,et al.  Trapped electron mode driven electron heat transport in JET: experimental investigation and gyro-kinetic theory validation , 2015 .

[81]  Wei Chen,et al.  Toward a Better Understanding of Model Validation Metrics , 2011 .

[82]  W. Houlberg,et al.  Bootstrap current and neoclassical transport in tokamaks of arbitrary collisionality and aspect ratio , 1997 .

[83]  F. Jenko,et al.  Scale separation between electron and ion thermal transport. , 2008, Physical review letters.

[84]  G. Staebler,et al.  Comparing simulation of plasma turbulence with experiment , 2002 .

[85]  A. Diallo,et al.  Electrostatic instabilities, turbulence and fast ion interactions in the TORPEX device , 2010 .

[86]  William Dorland,et al.  Gyrofluid turbulence models with kinetic effects , 1993 .

[87]  G. Staebler,et al.  Observation of a critical gradient threshold for electron temperature fluctuations in the DIII-D Tokamak. , 2012, Physical review letters.

[88]  J. Decker,et al.  Fast electron bremsstrahlung in axisymmetric magnetic configuration , 2008 .

[89]  O. Meneghini,et al.  Fullwave Simulations of Lower Hybrid Waves Coupled to 3D Fokker-Planck Solver: Comparison with Alcator C-Mod Experiment , 2011 .

[90]  K. Taylor.  Summarizing multiple aspects of model performance in a single diagram , 2001 .

[91]  Harold P. Furth,et al.  Finite‐Resistivity Instabilities of a Sheet Pinch , 1963 .

[92]  B Ph van Milligen,et al.  Integrated data analysis at TJ-II: the density profile. , 2011, The Review of scientific instruments.

[93]  J. L. Luxon,et al.  A design retrospective of the DIII-D tokamak , 2002 .

[94]  A. Manini,et al.  Profile stiffness and global confinement , 2004 .

[95]  Perkins,et al.  Fluid moment models for Landau damping with application to the ion-temperature-gradient instability. , 1990, Physical review letters.

[96]  D. Hatch,et al.  Damped eigenmode saturation in plasma fluid turbulence , 2011 .

[97]  E. Frieman,et al.  Nonlinear gyrokinetic equations for low-frequency electromagnetic waves in general plasma equilibria , 1981 .

[98]  T. Hahm,et al.  Turbulent transport reduction by zonal flows: massively parallel simulations , 1998, Science.

[99]  R. Bell,et al.  Simulation of microtearing turbulence in national spherical torus experimenta) , 2012 .

[100]  M. A. Van Zeeland,et al.  Fast imaging of transients and coherent magnetohydrodynamic modes in DIII-D , 2009 .

[101]  F. Jenko,et al.  Multiscale features of density and frequency spectra from nonlinear gyrokinetics , 2008 .

[102]  R. Fonck,et al.  Spatial transfer function for the beam emission spectroscopy diagnostic on DIII-D , 2006 .

[103]  Jet Efda Contributors,et al.  First scenario development with the JET new ITER-like wall , 2013 .

[104]  Dennis G. Whyte,et al.  Nonlinear gyrokinetic simulations of the I-mode high confinement regime and comparisons with experimenta) , 2015 .

[105]  J. Manickam,et al.  Chapter 3: MHD stability, operational limits and disruptions , 2007 .

[106]  A. Leonard.  Edge-localized-modes in tokamaksa) , 2014 .

[107]  J. Kinsey,et al.  A theory-based transport model with comprehensive physicsa) , 2006 .

[108]  R. Groebner,et al.  Study of the L-mode tokamak plasma "shortfall" with local and global nonlinear gyrokinetic δf particle-in-cell simulation , 2014 .

[109]  R. E. Waltz,et al.  Gyro-Landau fluid equations for trapped and passing particles , 2005 .

[110]  R. J. Hastie,et al.  Sawtooth Instability in Tokamak Plasmas , 1997 .

[111]  D. Spong.  3D toroidal physics: Testing the boundaries of symmetry breakinga) , 2014 .

[112]  T. Fujita,et al.  Chapter 2: Plasma confinement and transport , 2007 .

[113]  F. Hinton,et al.  Theory of plasma transport in toroidal confinement systems , 1976 .

[114]  W. Dorland,et al.  Fluid models of phase mixing, Landau damping, and nonlinear gyrokinetic dynamics , 1992 .

[115]  F. Imbeaux,et al.  A new gyrokinetic quasilinear transport model applied to particle transport in tokamak plasmas , 2007 .

[116]  Youssef M. Marzouk,et al.  Improved profile fitting and quantification of uncertainty in experimental measurements of impurity transport coefficients using Gaussian process regression , 2015 .

[117]  Martin Greenwald,et al.  Synergistic cross-scale coupling of turbulence in a tokamak plasma , 2014 .

[118]  R. Fitzpatrick,et al.  Interaction of tearing modes with external structures in cylindrical geometry (plasma) , 1993 .

[119]  Martin Greenwald,et al.  Multi-scale gyrokinetic simulation of tokamak plasmas: enhanced heat loss due to cross-scale coupling of plasma turbulence , 2015 .

[120]  Zhihong Lin,et al.  Verification and validation of linear gyrokinetic simulation of Alfvén eigenmodes in the DIII-D tokamak , 2012 .

[121]  J. P. Christiansen,et al.  A review of the dimensionless parameter scaling studies , 1996 .

[122]  C. Domier,et al.  Investigation of fast particle driven instabilities by 2D electron cyclotron emission imaging on ASDEX Upgrade , 2011 .

[123]  Jeff M. Candy,et al.  Implementation and application of two synthetic diagnostics for validating simulations of core tokamak turbulence , 2009 .

[124]  R. Budny,et al.  Local Transport in Joint European Tokamak Edge-Localized, High-Confinement Mode Plasmas with H, D, DT, and T Isotopes , 2000 .

[125]  T. L. Rhodes,et al.  Electron profile stiffness and critical gradient studies , 2012 .

[126]  Frank Jenko,et al.  The European turbulence code benchmarking effort: turbulence driven by thermal gradients in magnetically confined plasmas , 2008 .

[127]  N. Wiener.  The Homogeneous Chaos , 1938 .

[128]  T. Fujita,et al.  The 2008 Public Release of the International Multi-tokamak Confinement Profile Database , 2008 .

[129]  O. Meneghini,et al.  Electron temperature critical gradient and transport stiffness in DIII-D , 2013 .

[130]  H. Bindslev,et al.  A Technique for Improving the Relative Accuracy of JET ECE Temperature Profiles , 1988 .

[131]  F Jenko,et al.  Saturation of gyrokinetic turbulence through damped eigenmodes. , 2011, Physical review letters.

[132]  P. Terry,et al.  Role of stable eigenmodes in saturated local plasma turbulence , 2006 .

[133]  Jeff M. Candy,et al.  Full linearized Fokker–Planck collisions in neoclassical transport simulations , 2011 .

[134]  F. J. Casson,et al.  The nonlinear gyro-kinetic flux tube code GKW , 2009, Comput. Phys. Commun..

[135]  Participant Teams,et al.  ITER: burning plasma physics experiment , 2003 .

[136]  E. D'Azevedo,et al.  Time dependent evolution of RF-generated non-thermal particle distributions in fusion plasmas , 2014 .

[137]  C. C. Petty,et al.  Sizing up plasmas using dimensionless parametersa) , 2006 .

[138]  T. L. Rhodes,et al.  Validation studies of gyrofluid and gyrokinetic predictions of transport and turbulence stiffness using the DIII-D tokamak , 2013 .

[139]  T. Tala,et al.  Experimental study of the ion critical-gradient length and stiffness level and the impact of rotation in the JET tokamak. , 2009, Physical review letters.

[140]  W. W. Heidbrink,et al.  Basic physics of Alfvén instabilities driven by energetic particles in toroidally confined plasmas , 2008 .

[141]  Federico David Halpern,et al.  Simulation of plasma turbulence in scrape-off layer conditions: the GBS code, simulation results and code validation , 2012 .

[142]  Jeff M. Candy,et al.  Coupled ion temperature gradient and trapped electron mode to electron temperature gradient mode gyrokinetic simulations , 2007 .

[143]  Paolo Ricci,et al.  Langmuir probe-based observables for plasma-turbulence code validation and application to the TORPEX basic plasma physics experiment , 2009 .

[144]  T. Görler,et al.  A flux-matched gyrokinetic analysis of DIII-D L-mode turbulence , 2014 .

[145]  R. Prater,et al.  Heating and current drive by electron cyclotron waves , 2003 .

[146]  T. Trucano,et al.  Verification, Validation, and Predictive Capability in Computational Engineering and Physics , 2004 .

[147]  T. L. Rhodes,et al.  Measurements of core electron temperature and density fluctuations in DIII-D and comparison to nonlinear gyrokinetic simulations , 2008 .

[148]  Jet Efda Contributors,et al.  Electron heat transport studies , 2006 .

[149]  P. Terry,et al.  Suppression of turbulence and transport by sheared flow , 2000 .

[150]  F. Jenko,et al.  On the Validation of Gyrokinetic L-Mode Simulations , 2016 .

[151]  Matthew F. Barone,et al.  Measures of agreement between computation and experiment: Validation metrics , 2004, J. Comput. Phys..

[152]  J. Manickam,et al.  Nonlocal neoclassical transport in tokamak and spherical torus experiments , 2006 .

[153]  R. Fonck,et al.  High sensitivity beam emission spectroscopy for core plasma turbulence imaging (invited) , 2006 .

[154]  E. Doyle,et al.  Comparison of turbulence measurements from DIII-D low-mode and high-performance plasmas to turbulence simulations and models , 2001 .

[155]  F. Hinton,et al.  Effect of finite aspect ratio on the neoclassical ion thermal conductivity in the banana regime , 1982 .