Verification and validation technologies for gas dynamic simulations
This paper presents a review of the literature on terminology, methodology and technology of verification and validation processes related to the field of computational gas dynamics. The interdisciplinarity of the used approaches that are developed by the efforts of specialists in various branches of knowledge such as operations research, statistical analysis, and computational physics is highlighted. Such fundamental problems as comprehension of the verification procedure features for computational code and solution; comprehension of the validation process specificity for model and computational solution, identification of sources of errors and uncertainty, definition of the role of validation in predictive modeling are discussed. The basic principles of the validation based on identification and quantification of errors in the computer simulation model and numerical solutions obtained using this model are formulated. Particular attention is paid to methods for estimating the accuracy of calculations. The importance of computational software testing in the framework of verification is highlighted. The validation strategy that involves a hierarchical representation of the complex interrelated physical processes occurring in the simulated engineering system is described. Approaches to planning and conducting validation experiments are presented. Methods for estimating the experimental error, random and systematic errors are described. The necessity of using nondeterministic calculations in the validation activity to take into account the factor of experimental uncertainties in the numerical simulation process is substantiated.
verification, validation, computational gas dynamics, computational aerodynamics, numerical simulation, software systems
В работе представлен обзор литературы, посвященный вопросам терминологии, методологии и технологии процессов верификации и валидации применительно к области вычислительной газовой динамики. Подчеркивается междисциплинарность используемых подходов, которые развиваются усилиями специалистов различных отраслей знания – исследования операций, статистического анализа, вычислительной физики. Обсуждаются такие фундаментальные проблемы, как раскрытие особенностей процедур верификации программного кода и численного решения, специфики процессов валидации модели и численного решения; выявление источников неопределенностей и ошибок; определение роли валидации в предсказательном моделировании. Формулируются основные принципы верификации, которые заключаются в выявлении и количественной оценке погрешности компьютерной модели и получаемых с ее помощью численных решений. Особое внимание уделяется методам оценки точности вычислений, подчеркивается важность тестирования программного обеспечения в рамках верификации. Излагается стратегия валидации, предполагающая иерархическое представление сложных взаимосвязанных физических процессов, протекающих в моделируемой инженерной системе. Приводятся подходы к планированию и проведению валидационных экспериментов. Описываются методы оценки экспериментальной погрешности, случайных и систематических ошибок. Обосновывается необходимость использования недетерминированных расчетов в процессе валидации для учета фактора экспериментальных неопределенностей в численном моделировании.
1. Харитонов А.М. О верификации и валидации моделей и методов численного моделирования пространственных течений // Материалы Международной конференция “Современные проблемы прикладной математики и механики: теория, эксперимент и практика”, Новосибирск, 30 мая – 4 июня, 2011. 2. Железнякова А.Л., Суржиков С.Т. На пути к созданию модели виртуального ГЛА. I. – М.: ИПМех РАН, 2013. – 160 c. 3. Wilcox D.C. Formulation of the k-omega Turbulence Model Revisited // AIAA Journal, 2008, V. 46, №11, pp. 2823–2838. 4. Wilcox D.C. Turbulence Modeling for CFD. 3rd edition, DCW Industries, Inc., La Canada CA, 2006. 5. Launder B.E., Spalding D.B. The numerical computation of turbulent flows // Computer Methods in Applied Mechanics and Engineering, V.3, №2, 1974, pp. 269–289. 6. Menter F.R. Two-Equation Eddy-Viscosity Turbulence Models for Engineering Applications // AIAA Journal, V.32, №8, August 1994, pp. 1598–1605 7. Железнякова А.Л., Суржиков С.Т. Расчет гиперзвукового обтекания тел сложной формы на неструктурированных тетраэдральных сетках c использованием схемы AUSM // ТВТ, 2014, т. 52, № 2, с. 283–293. 8. Железнякова А.Л., Суржиков С.Т. Применение метода расщепления по физическим процессам для расчета гиперзвукового обтекания пространственной модели летательного аппарата сложной формы // ТВТ, 2013, том 51, № 6, с. 897–911. 9. Иванов И.Э., Крюков И.А. Метод расчета вязких пространственных течений на нерегулярных сетках // Материалы IX Международной конференции по неравновесным процессам в соплах и струях (NPNJ-2012), Алушта, 25-31 мая, 2012. Изд-во “МАИ-ПРИНТ”, Москва, 2012, с. 240,241. 10. Ермаков М.К. Многопроцессорное моделирование аэродинамики гиперзвукового летательного аппарата на трехмерных неструктурированных сетках // Материалы XXIII Научно-технической конференции по аэродинамике, п. Володарского Московской обл., 1-2 марта 2012, с. 104,105. 11. Ермаков М.К. Моделирование гиперзвукового обтекания летательного аппарата на суперкомпьютере “Ломоносов” // 6-я Всероссийская школа-семинар “Аэрофизика и физическая механика классических и квантовых систем”: Сборник научных трудов. - М: ИПМех РАН, 2012, с. 18–23. 12. Железнякова А.Л., Кузенов В.В., Петрусев А.С., Суржиков С.Т. Расчет аэротермодинамики двух типов моделей спускаемых космических аппаратов. // Физико-химическая кинетика в газовой динамике, 2010, Т.9. http://chemphys.edu.ru/media/published/025.pdf 13. Железнякова А.Л., Суржиков С.Т. Численное моделирование поля течения при входе в атмосферу земли спускаемого аппарата с аэродинамическим качеством // Вестник МГТУ им. Н.Э. Баумана. - Сер. “Машиностроение”, 2009, №2, с. 3–25. 14. Железнякова А.Л., Суржиков С.Т. Численное моделирование гиперзвукового обтекания модели летательного аппарата Х-43 // Препр. ИПМех им. А.Ю. Ишлинского РАН, 2010, № 950, 82 с. 15. Железнякова А.Л., Кузенов В.В., Петрусев А.С., Суржиков С.Т. Численный анализ конвективного нагрева двух моделей спускаемых космических аппаратов // Вестник МГТУ им. Н.Э. Баумана. - Сер. “Машиностроение”, 2009, №3, с. 3–15. 16. Железнякова А.Л., Суржиков С.Т. Численное моделирование гиперзвукового обтекания модели летательного аппарата Х-43 // Вестник МГТУ им. Н.Э. Баумана. - Сер. “Машиностроение”, 2010, №1, с. 3–19. 17. Суржиков С.Т. Аналитические методы построения конечно-разностных сеток для расчета аэротермодинамики спускаемых космических аппаратов // Вестник МГТУ им. Н.Э. Баумана. - Сер. “Машиностроение”, 2004, № 2, c. 24–50. 18. Суржиков С.Т. Радиационно-конвективный теплообмен космического аппарата сферической формы в углекислом газе // ТВТ, 2011, Т.49, № 1, c. 92–107. 19. Котов Д.В., Суржиков С.Т. Расчет течений вязкого и невязкого газа на неструктурированных сетках с использованием схемы AUSM // Вычислительная механика сплошных сред, 2011, Т.4, №1, с. 36–54. 20. Popper K.R. The Logic of Scientific Discovery, Basic Books, New York, 1959. 21. Popper K.R. Conjectures and Refutations: The Growth of Scientific Knowledge, Routledge and Kegan, London, 1969. 22. Carnap R. Testability and Meaning // Philosophy of Science, 1963, V.3, pp. 420–468. 23. Kleindorfer G.B., O’Neill L., Ganeshan R. Validation in Simulation: Various Positions in the Philosophy of Science // Management Science, 1998, V.44, №8, pp. 1087–1099. 24. Balci O., Sargent R.G. A Bibliography on the Credibility Assessment and Validation of Simulation and Mathematical Models // Simuletter, 1984, V.15, №3, pp. 15–27. 25. Hamilton M.A. Model Validation: An Annotated Bibliography // Communications in Statisitcs-Theory and Methods, 1991, V.20, №7, pp. 2207–2266. 26. Oberkampf W.L. Bibliography for Verification and Validation in Computational Simulation, Sandia National Laboratories, SAND98-2041, Albuquerque, NM, 1998. 27. Вентцель Е.С. Исследование операций: задачи, принципы, методология: учебное пособие. – 5-е изд. – М.: КНОРУС, 2013. 28. Barkley D., Henderson R.D. Three-Dimensional Floquet Stability Analysis of the Wake of a Circular Cylinder // Journal of Fluid Mechanics, 1996, V.322, pp. 215–241. 29. Karniadakis G.E., Triantafyllou G.S. Three-Dimensional Dynamics and Transition to Turbulence in the Wake of Bluff Objects // Journal of Fluid Mechanics, 1992, V.238, pp. 1 –30. 30. Verhoff A. Far-Field Computational Boundary Conditions for Three-Dimensional External Flow Problems, AIAA-96-0892 / AIAA 34th Aerospace Sciences Meeting, Reno, NV, 1996. 31. Verhoff A., Cary A. Analytical Euler Solutions for 2D Flows with Corners using Asymptotic Methods, AIAA-98-2687 / 2nd AIAA Theoretical Fluid Mechanics Meeting, Albuquerque, NM, 1998. 32. Roache P.J. Scaling of High-Reynolds-Number Weakly Separated Channel Flows / in Numerical Aspects of Physical Aspects of Aerodynamic Flows, T. Cebecci Ed. Springer-Verlag, New York, 1982, pp. 87 –98. 33. Roache P.J. Need for Control of Numerical Accuracy // Journal of Spacecraft and Rockets, 1990, V.27, №2, pp. 98–102. 34. Roache P.J. Quantification of Uncertainty in Computational Fluid Dynamics / in Annual Review of Fluid Mechanics, J.L. Lumley and M. Van Dyke Eds. Annual Reviews, Inc., Palo Alto, CA, 1997, pp. 126–160. 35. Roache P.J. Verification and Validation in Computational Science and Engineering, Hermosa Publishers, Albuquerque, NM, 1998. 36. Roache P.J., Knupp P.M. Completed Richardson Extrapolation // Communications in Numerical Methods in Engineering, 1993, V.9, pp. 365–374. 37. Celik I., Hassan Y., Hughes D., Johnson R., Sommerfeld M. Experimental and Computational Aspects of Validation of Multiphase Flow CFD Codes / FED-Vol. 180, The American Society of Mechanical Engineers, United Engineering Center, New York, 1994. 38. Ferziger J.H., Peric M. Further Discussion of Numerical Errors in CFD // International Journal for Numerical Methods in Fluids, 1996, V.23, pp. 1263–1274. 39. Blackwell B.F., Armaly B.F. Computational Aspects of Heat Transfer: Benchmark Problems / ASME HTD-V.258, American Society of Mechanical Engineers, New York, 1993. 40. Blackwell B.F., Pepper D.W. Benchmark Problems for Heat Transfer Codes / ASME HTD-V.222, American Society of Mechanical Engineers, New York, 1992. 41. Mittal R., Balachandar S. Effect of Three-Dimensionality on the Lift and Drag of Nominally Two-Dimensional Cylinders // Physics of Fluids, 1995, V.7, №8, pp. 1841–1865. 42. Mittal R., Balachandar S. Vortical Structures in Bluff Body Wakes, AIAA-95-0867 // AIAA 33rd Aerospace Sciences Meeting, Reno, NV, 1995. 43. Blottner F.G. Variable Grid Scheme Applied to Turbulent Boundary Layers // Computer Methods in Applied Mechanics and Engineering, 1974, V.4, pp. 179–194. 44. Blottner F.G. Investigation of Some Finite-Difference Techniques for Solving the Boundary Layer Equations // Computer Methods in Applied Mechanics and Engineering, 1975, V.6, pp. 1–30. 45. Blottner F.G. Accurate Navier-Stokes Results for the Hypersonic Flow Over a Spherical Nosetip // Journal of Spacecraft and Rockets, 1990, V.27, №2, pp. 113–122. 46. Shih T.M., Tan C.H., Hwang B.C. Effects of Grid Staggering on Numerical Schemes, International Journal for Numerical Methods in Fluids, 1989, V.9, pp. 193 –212. 47. Zhang X.D., Pelletier D., Trepanier J.Y., Camarero R. Verification of Error Estimators for the Euler Equations, AIAA-2000-1001 / 38th AIAA Aerospace Sciences Meeting, Reno, NV, 2000. 48. Zhang X.D., Trepanier J.Y., Camarero R. An A Posteriori Error Estimation Method Based on Error Equations, AIAA-97-1889 / 13th Computational Fluid Dynamics Conference, Snowmass Village, CO, 1997, pp. 383–397. 49. Ethier C.R., Steinman D.A. Exact Fully 3D Navier-Stokes Solutions for Benchmarking // International Journal for Numerical Methods in Fluids, 1994, V.19, pp. 369 –375. 50. Terrill R.M., Colgan T. Some Simple Analytic Solutions of the Navier-Stokes Equations // International Journal of Engineering Science, 1991, V.29, №1, pp. 55–68. 51. Grinstein F.F. Open Boundary Conditions in the Simulation of Subsonic Turbulent Shear Flows // Journal of Computational Physics, 1994, V.115, pp. 43 –55. 52. Nordstrom J. Accurate Solutions of the Navier-Stokes Equations Despite Unknown Outflow Boundary Data // Journal of Computational Physics, 1994, V.120, pp. 184 –205. 53. Boerstoel J.W. Numerical Accuracy Assessment // AGARD-CP-437, Fluid Dynamics Panel Symposium: Validation of Computational Fluid Dynamics, Lisbon, Portugal, 1988. 54. Yoshizawa A. Laminar Viscous Flow Past a Semi-Infinite Flat Plate // Journal of the Physical Society of Japan, 1970, V.28, №3, pp. 776–779. 55. DiMascio A., Paciorri R., Favini B. Convergence of Two Numerical Schemes for Turbulent Boundary Layer Computations, AIAA 98-3009 // 29th AIAA Fluid Dynamics Conference, Albuquerque, NM, 1998. 56. Venkateswaran S., Merkle C.L. Evaluation of Artificial Dissipation Models and Their Relationship to the Accuracy of Euler and Navier-Stokes Computations / Sixteenth International Conference on Numerical Methods in Fluid Dynamics, Arcachon, France, 1998, pp. 427–432. 57. Shih T.M. A Procedure to Debug Computer-Programs // International Journal for Numerical Methods in Engineering, 1985, V.21, №6, pp. 1027–1037. 58. Davis R.T. Laminar Incompressible Flow Past a Semi-Infinite Flat Plate // Journal of Fluid Mechanics, 1967, V.27, №4, pp. 691 –704. 59. Jameson A., Martinelli L. Mesh Refinement and Modeling Errors in Flow Simulation // AIAA Journal, 1998, V.36, №5, pp. 676 –686. 60. Botta E.F.F., Dijkstra D., Veldman A.E.P. The Numerical Solution of the Navier-Stokes Equations for Laminar, Incompressible Flow Past a Parabolic Cylinder // Journal of Engineering Mathematics, 1972, V.6, №1, pp. 63–81. 61. Shimazaki K., Himeno Y., Baba N. Quantification of Uncertainty in Computational Fluid Dynamics / FED-Vol.-158, Fluids Engineering Conference, Washington, DC, 1993, pp. 19–28. 62. Van De Vooren A.I., Dijkstra D. The Navier-Stokes Solution for Laminar Flow Past a Semi-Infinite Flat Plate // Journal of Engineering Mathematics, 1970, V.4, №1, pp. 9–27. 63. Barragy E., Carey G.F. Stream Function-Vorticity Driven Cavity Solution Using p Finite Elements // Computers and Fluids, 1997, V.26, №5, pp. 453–468. 64. Wang C.Y. Exact Solutions of the Steady-State Navier-Stokes Equations / in Annual Review of Fluid Mechanics, J.L. Lumley and M. Van Dyke Eds. Annual Reviews, Inc., Palo Alto, CA, 1991, pp. 159 –177. 65. Zingg D.W. Grid Studies for Thin-Layer Navier-Stokes Computations of Airfoil Flowfields // AIAA Journal, 1992, V.30, №10, pp. 2561–2564. 66. AGARD. Quality Assessment for Wind Tunnel Testing, NATO Advisory Group for Aerospace Research & Development (AGARD), AGARD-AR-304, 1994. 67. AGARD. A Selection of Experimental Test Cases for the Validation of CFD Codes, NATO Advisory Group for Aerospace Research & Development, AGARD-AR-303-Vol. II, 1994. 68. AGARD. A Selection of Experimental Test Cases for the Validation of CFD Codes, NATO Advisory Group for Aerospace Research & Development, AGARD-AR-303-Vol. I, 1994. 69. Marvin J.G. Accuracy Requirements and Benchmark Experiments for CFD Validation // AGARD-CP-437, Fluid Dynamics Panel Symposium: Validation of Computational Fluid Dynamics, Lisbon, Portugal, 1988. 70. Marvin J.G. CFD Validation Experiments for Hypersonic Flows, AIAA-92-4024 // AIAA 17th Aerospace Ground Testing Conference, Nashville, TN, 1992. 71. Marvin J.G. Perspective on Computational Fluid Dynamics Validation // AIAA Journal, Vol. 33, No. 10, 1995; 1778-1787. 72. Barber T.J. Role of Code Validation and Certification in the Design Environment // AIAA Journal, Vol. 36, No. 5, 1998; 752-758. 73. Oberkampf W.L., Aeschliman D.P. Joint Computational/Experimental Aerodynamics Research on a Hypersonic Vehicle: Part 1, Experimental Results // AIAA Journal, 1992, V.30, №8, pp. 2000 –2009. 74. Oberkampf W.L., Aeschliman D.P., Henfling J.F., Larson D.E. Surface Pressure Measurements for CFD Code Validation in Hypersonic Flow, AIAA Paper 95-2273 // 26th AIAA Fluid Dynamics Conf., San Diego, CA, 1995. 75. Oberkampf W.L., Aeschliman D.P., Henfling J.F., Larson D.E., Payne J.L. Surface Pressure Measurements on a Hypersonic Vehicle, AIAA Paper 96-0669 // 34th Aerospace Sciences Meeting, Reno, NV, 1996. 76. Oberkampf W.L., Blottner F.G. Issues in Computational Fluid Dynamics Code Verification and Validation // AIAA Journal, 1998, V.36, №5, pp. 687–695. 77. Oberkampf W.L., Blottner F.G., Aeschliman D.P. Methodology for Computational Fluid Dynamics Code Verification/Validation, AIAA Paper 95-2226 // 26th AIAA Fluid Dynamics Conf., San Diego, CA, 1995. 78. Bradley R.G. CFD Validation Philosophy // AGARD-CP-437, Fluid Dynamics Panel Symposium: Validation of Computational Fluid Dynamics, Lisbon, Portugal, 1988. 79. Kammeyer M.E. Wind Tunnel Facility Calibrations and Experimental Uncertainty, AIAA 98-2715 // 20th AIAA Advanced Measurement and Ground Testing Technology Conference, Albuquerque, NM, 1998. 80. Mehta U.B. Computational Requirements for Hypersonic Flight Performance Estimates // Journal of Spacecraft and Rockets, 1990, V.27, №2, pp. 103–112. 81. Mehta U.B. Guide to Credible Computational Fluid Dynamics Simulations, AIAA Paper 95-2225 // 26th AIAA Fluid Dynamics Conference, San Diego, CA, 1995. 82. Mehta U.B. Guide to Credible Computer Simulations of Fluid Flows // Journal of Propulsion and Power, 1996, V.12, № 5, pp. 940 – 948. 83. Deiwert G.S. Issues and Approach to Develop Validated Analysis Tools for Hypersonic Flows: One Perspective / NASA Ames Research Center, NASA-TM-103937, 1992. 84. Van Wie D.M., Rice T. Quantification of Data Uncertainties and Validation of CFD Results in the Development of Hypersonic Airbreathing Engines, AIAA 96-2028 // 27th AIAA Fluid Dynamics Conference, New Orleans, LA, 1996. 85. Cosner R.R. The Role of Validation in the CFD Process at McDonnell Douglas/St. Louis, AIAA-96-2273 // 19th AIAA Advanced Measurement and Ground Testing Technology Conference, New Orleans, LA, 1996. 86. Cosner R.R. Experimental Data Needs for Risk Management in CFD Applications, AIAA Paper 98-2781 // 20th AIAA Advanced Measurement and Ground Testing Technology Conference, Albuquerque, NM, 1998. 87. Martellucci A. The Challenging Process of Validating CFD Codes, AIAA-90-1402 // AIAA 16th Aerodynamic Ground Testing Conference, Seattle, WA, 1990. 88. Paciorri R., Dieudonne W., Degrez G., Charbonnier J.-M., Deconinck H. Exploring the Validity of the Spalart-Allmaras Turbulence Model for Hypersonic Flows // Journal of Spacecraft and Rockets, 1998, V.35, №2, pp. 121–126. 89. Holden M.S., Moselle J.R., Sweet S.J., Martin S.C. A Database of Aerothermal Measurements in Hypersonic Flow for CFD Validation, AIAA Paper 96-4597 // AIAA 7th International Space Planes and Hypersonic Systems and Technologies Conf., Norfolk, VA, 1996. 90. Gosman A.D. Quality Assurance for Industrial CFD Codes, AIAA 98-2637 // 29th AIAA Fluid Dynamics Conference, Albuquerque, NM, 1998. 91. Springer A.M. Comparison of the Aerodynamic Characteristics of Similar Models in two Different Size Wind Tunnels at Transonic Speeds, AIAA-98-2875 // 20th AIAA Advanced Measurement and Ground Testing Technology Conference, Albuquerque, NM, 1998. 92. Benek J.A., Kraft E.M., Lauer R.F. Validation Issues for Engine – Airframe Integration // AIAA Journal, 1998, V.36, №5, pp. 759–764. 93. Reed H.L., Haynes T.S., Saric W.S. Computational Fluid Dynamics Validation Issues in Transition Modeling // AIAA Journal, 1998, V.36, №5, pp. 742–751. 94. Walker M.A., Oberkampf W.L. Joint Computational/Experimental Aerodynamics Research on a Hypersonic Vehicle: Part 2, Computational Results, AIAA Journal, 1992, V.30, №8, pp. 2010–2016. 95. Aeschliman D.P., Oberkampf W.L. Experimental Methodology for Computational Fluid Dynamics Code Validation / Sandia National Laboratories, SAND95-1189, Albuquerque, NM, 1997. 96. Aeschliman D.P., Oberkampf W.L. Experimental Methodology for Computational Fluid Dynamics Code Validation // AIAA Journal, 1998, V.36, №5, pp. 733–741. 97. Aeschliman D.P., Oberkampf W.L., Blottner F.G. A Proposed Methodology for CFD Code Verification, Calibration, and Validation / Paper 95-CH3482-7, 16th International Congress on Instrumentation for Aerospace Simulation Facilities, Dayton, OH, 1995. 98. Dolling D.S. Problems in the Validation of CFD Codes Through Comparison with Experiment // AGARD-CP-514, Fluid Dynamics Panel Symposium: Theoretical and Experimental Methods in Hypersonic Flows, Torino, Italy, 1992, pp. 1–19. 99. Dolling D.S. High-Speed Turbulent Separated Flows: Consistency of Mathematical Models and Flow Physics // AIAA Journal, 1998, V.36, №5, pp. 725–732. 100. Sindir M.M., Lynch E.D. Overview of the State-of-Practice of Computational Fluid Dynamics in Advanced Propulsion System Design, AIAA Paper 97-2124 // 28th AIAA Fluid Dynamics Conference, Snowmass, CO, 1997. 101. Veazey D.T., Hopf J.C. Comparison of Aerodynamic Data Obtained in the Arnold Engineering Development Center Wind Tunnels 4T and 16T, AIAA 98-2874 // 20th AIAA Advanced Measurement and Ground Testing Technology, Albuquerque, NM, 1998. 102. Coleman H.W., Stern F. Uncertainties and CFD Code Validation // Journal of Fluids Engineering, 1997, V.119, pp. 795–803. 103. Bertin J.J., Martellucci A., Neumann R.D., Stetson K.F. Developing a Data Base for the Calibration and Validation of Hypersonic CFD Codes - Sharp Cones, AIAA Paper 93-3044 // 24th AIAA Fluid Dynamics Conf., Orlando, FL, 1993. 104. Settles G.S., Dodson L.J. Supersonic and Hypersonic Shock/Boundary-Layer Interaction Database // AIAA Journal, 1994, V.32, №7, pp. 1377–1383. 105. ГОСТ Р ИСО 9000-2015. Системы менеджмента качества. Основные положения и словарь. Национальный стандарт Российской Федерации. 106. ГОСТ Р ИСО 9001-2015. Системы менеджмента качества. Требования. Национальный стандарт Российской Федерации. 107. ISO 9000:2015. Quality management systems. Fundamentals and vocabulary. 108. ISO 9001:2015. Quality management systems. Requirements. 109. Schlesinger S. Terminology for Model Credibility // Simulation, Vol. 32, No. 3, 1979; 103-104. 110. Guide for the Verification and Validation of Computational Fluid Dynamics Simulations, American Institute of Aeronautics and Astronautics, AIAA-G-077-1998, Reston, VA, 1998. 111. Годунов С.К., Рябенький В.С. Разностные схемы. Введение в теорию. 2-е изд. – М.: Наука, 1977. – 440 с. 112. Рябенький В.С., Филиппов А.Ф. Об устойчивости разностных уравнений. – М.: Гостехиздат, 1956. – 172 c. 113. Lax P.D., Richtmyer R.D. Survey of the stability of linear finite difference equations // Comm. Pure Appl. Math, 1956. V.9, pp. 267–293. 114. Кулямин В.В. Методы верификации программного обеспечения. – М.: Институт Системного Программирования РАН, 2008. – 111 с. 115. Г.Дж. Майерс. Искусство тестирования программ. – М.: Финансы и статистика, 1982. – 176 c. 116. Г.Дж. Майерс. Надежность программного обеспечения. – М: Мир, 1980. – 359 c. 117. Laney C.B. Computational Gasdynamics, Cambridge University Press, 2007. – 613 p. 118. Yee H.C., Sweby P.K. Aspects of Numerical Uncertainties in Time Marching to Steady-State Numerical Solutions // AIAA Journal, 1998, V.36, №5, pp. 712–724. 119. Ferziger J. H., Peric M. Computational Methods for Fluid Dynamics, 3rd edition, Berlin: Springer, 2002. – 431 p. 120. Richtmeyer R.D., Morton K.W. Difference Methods for Initial-Value Problems, Interscience, New York, 1967. – 420 с. 121. von Neumann J. Proposal and Analysis of a Numerical Method for the Treatment of Hydrodynamical Shock Problems / Nat. Res. Com. Report AM-551, 1944. 122. Hirsch C. Numerical Computation of Internal and External Flows: Fundamentals of Numerical Discretization, John Wiley, New York, NY, 1988. 123. Hirsch C. Numerical Computation of Internal and External Flows: Computational Methods for Inviscid and Viscous Flows, John Wiley, New York, 1990. – 714 p. 124. Alvin K.F., Oberkampf W.L., Rutherford B.M., Diegert K.V. Methodology for Characterizing Modeling and Discretization Uncertainties in Computational Simulation, Sandia National Laboratories, SAND2000-0515, Albuquerque, NM, 2000. 125. Carpenter M.H., Casper J.H. Accuracy of Shock Capturing in Two Spatial Dimensions // AIAA Journal, 1999, V.37, №9, pp. 1072–1079. 126. Roy C.J., McWherter-Payne M.A., Oberkampf W.L. Verification and Validation for Laminar Hypersonic Flowfields, AIAA 2000-2550, Fluids 2000 Conference, Denver, CO, 2000. 127. de Vahl Davis G. Natural Convection of Air in a Square Cavity: A Benchmark Numerical Solution // International Journal for Numerical Methods in Fluids, 1983, V.3, pp. 249–264. 128. Hatton L. The T Experiments: Errors in Scientific Software // IEEE Computational Science and Engineering, 1997, V.4, №2, pp. 27–38. 129. ГОСТ Р ИСО/МЭК 25001-2017. Информационные технологии (ИТ). Системная и программная инженерия. Требования и оценка качества систем и программного обеспечения (SQuaRE). Планирование и управление. Национальный стандарт Российской Федерации. 130. ГОСТ Р ИСО/МЭК 25010-2015. Информационные технологии (ИТ). Системная и программная инженерия. Требования и оценка качества систем и программного обеспечения (SQuaRE). Модели качества систем и программных продуктов. Национальный стандарт Российской Федерации. 131. ISO/IEC 25000:2014. Systems and software engineering. Systems and software Quality Requirements and Evaluation (SQuaRE). Guide to SQuaRE. 132. ISO/IEC 25010:2011. Systems and software engineering. Systems and software Quality Requirements and Evaluation (SQuaRE). System and software quality models. 133. Wallace D.R., Ippolito L.M., Cuthill B.B. Reference Information for the Software Verification and Validation Process / Rept. 500-234, 1996. 134. Стандартный глоссарий терминов, используемых в тестировании программного обеспечения. Версия 2.3 (от 9 июля 2014 года) / ред. Erik van Veenendaal, А. Александров. Russian Software Testing Qualifications Board. 135. IEEE 610.12:1990. Standard Glossary of Software Engineering Terminology. 136. IEEE 1028:1997. Standard for Software Reviews and Audits. 137. Роуч П. Вычислительная гидродинамика, 1977. – 606 c. 138. Nishida H., Satofuka N. Higher-Order Solutions of Square Driven Cavity Flow Using a Variable-Order Multi-Grid Method // International Journal for Numerical Methods in Engineering, 1992, V.34, №2, pp. 637–653. 139. Gervais J.J., Lemelin D., Pierre R. Some Experiments with Stability Analysis of Discrete Incompressible Flow in the Lid-Driven Cavity // International Journal for Numerical Methods in Fluids, 1997, V.24, №5, pp. 477–492. 140. Schreiber F., Keller H.B. Driven Cavity Flows by Efficient Numerical Techniques // Journal of Computational Physics, 1983, V.49, №2, pp. 310–333. 141. Guerrero J.S.P., Cotta R.M. Integral Transform Solution for the Lid-Driven Cavity Flow Problem in Streamfunction-Only Formulation // International Journal for Numerical Methods in Fluids, 1992, V.15, №4, pp. 399–409. 142. Hansen E.B., Kelmanson M.A. An Integral Equation Justification of the Boundary Conditions of the Driven-Cavity Problem // Computers and Fluids, 1994, V.23, №1, pp. 225–240. 143. Balachandar S., Mittal R., Najjar F.M. Properties of the Mean Recirculation Region in the Wakes of Two-Dimensional Bluff Bodies // Journal of Fluid Mechanics, 1997, V.351, pp. 167–199. 144. Baber R. The Spine of Software. Designing Provably Correct Software: Theory and Practice, Wiley, New York, 1987. – 316 p. 145. Encyclopedia of Software Engineering / Marciniak J.J. ed. Wiley, New York, 1994. – 1929 p. 146. DeMillo R.A., McCracken W.M., Martin R.J., Passafiume J.F. Software Testing and Evaluation, Benjamin/Cummings, Menlo Park, CA, 1987. 147. Rook P. Software Reliability Handbook, Elsevier Science Publishers, New York, 1990. – 564 p. 148. Dahl O. Verifiable Programming. Prentice-Hall, Englewood Cliff, NJ, 1992. 149. Oberkampf W.L., Aeschliman D.P., Tate R.E., Henfling J.F. Experimental Aerodynamics Research on a Hypersonic Vehicle / Sandia National Laboratories, SAND92-1411, Albuquerque, NM, 1993. 150. Adrian R.J. Scattering particle characteristics and their effect on pulsed laser measurements of fluid flow: speckle velocimetry vs. particle image velocimetry // Applied Optics, 1984, V.23, pp. 1690–1691. 151. Алексеенко С.В., Бильский А.В., Маркович Д.М. Применение метода цифровой трассерной визуализации для анализа турбулентных потоков с периодической составляющей // Приборы и техника эксперимента, 2004, №. 5, c. 145–153. 152. Seitzman J.M., Hanson R.K. Planar Fluorescence Imaging in Gases, Chapter 6 in Instrumentation for Flows with Combustion, ed. A.M.K.P. Taylor, Academic Press, London, 1993. 153. AIAA. Assessment of Experimental Uncertainty With Application to Wind Tunnel Testing, American Institute of Aeronautics and Astronautics, S-071A-1999, Reston, VA, 1999. 154. Coleman H.W., Steele W.G. Experimentation and Uncertainty Analysis for Engineers / 2nd ed., John Wiley & Sons, New York, 1999. – 205 p. 155. Box G.E.P., Hunter W.G., Hunter J.S. Statistics for Experimenters: An Introduction to Design, Data Analysis, and Model Building, Wiley, New York, 1978. 156. Cullen A.C., Frey H.C. Probabilistic Techniques in Exposure Assessment: A Handbook for Dealing with Variability and Uncertainty in Models and Inputs, Plenum Press, New York, 1999. – 336 p. 157. Gamerman D. Markov Chain Monte Carlo, Chapman & Hall, London, 1997. – 342 p. 158. Kleijnen J.P.C. Statistical Tools for Simulation Practitioners, Marcel Dekker, Inc., New York, 1987. – 430 p. 159. Mehta U.B. Guide to Credible Computer Simulations of Fluid Flows // Journal of Propulsion and Power, 1996, V.12, №5, pp. 940–948. 160. Cukier R.I., Levine H.B., Shuler K.E. Nonlinear Sensitivity Analysis of Multiparameter Model Systems // Journal of Computational Physics, 1978, V.26, №1, pp. 1–42. 161. Iman R.L., Helton J.C. An Investigation of Uncertainty and Sensitivity Analysis Techniques for Computer Models // Risk Analysis, 1988, V.8, №1, pp. 71–90. 162. Saltelli A., Scott M. The Role of Sensitivity Analysis in the Corroboration of Models and Its Link to Model Structural and Parametric Uncertainty // Reliability Engineering and System Safety, 1997, V.57, pp. 1–4. 163. Beck M.B. Water Quality Modeling: A Review of the Analysis of Uncertainty // Water Resources Research, 1987, V.23, №8, pp. 1393–1442. 164. Chlond A., Wolkau A. Large-Eddy Simulation of a Nocturnal Stratocumulus-Topped Marine Atmospheric Boundary Layer: An Uncertainty Analysis // Boundary-Layer Meteorology, 2000, V.95, №1, pp. 31–55. 165. Du J., Mullen S.L., Sanders F. Short-Range Ensemble Forecasting of Quantitative Precipitation // Monthly Weather Review, 1997, V.125, №10, pp. 2427–2459. 166. Palmer T.N. Predicting Uncertainty in Forecasts of Weather and Climate // Reports on Progress in Physics, 2000, V.63, pp.71–116. 167. Lehmann E.L., Romano J.P. Testing Statistical Hypotheses, 3rd edition, Springer, New York, 2005. – 786 p. 168. Thompson J.F., Soni B. K., Weatherill N. P. Handbook of Grid Generation. CRC Press, 1998.
