Working with Data Repeatability
“A measurement process is in a state of statistical control if the resulting observations from the process, when collected under any fixed experimental conditions within the scope of the a priori well-defined conditions of the measurement process, behave like random drawings from some fixed distribution with fixed location [mean] and fixed scale [standard deviation] parameters.”
Belanger, Brian, Measurement Assurance Programs: Part I – Introduction, NBS SP 676-I, May 1984
Uncertainty Quantification for Simulation Databases
The Ares Uncertainty Quantification team at NASA Langley Research Center (LaRC) is working to identify and quantify the uncertainties associated with wind tunnel measurements, CFD computations and other data products. The team’s primary focus has been to robustly identify and quantify the errors associated with each component of the aerodynamic simulation database and combine the errors appropriately. 1-3 Properly combining uncertainties from numerous data sources results in more precise error bounds on the overall database thus assisting guidance and control engineers in the design of vehicle control laws.
ViGYAN Experience
ViGYAN’s experience in applying statistical quality control principles extends to real-world, complex, and often sparse data sets. For example, wind tunnel data (force/moment or pressure data) has many potential sources of uncertainty between the numerous measurement instruments and complex data collection and reduction software. 4 Additionally, resource (tunnel time, personnel, funding) limitations also can make it difficult to obtain as many replicate data points as one would like.
In the ViGYAN wind tunnel, we develop individual plans for customers that allows for data fidelity while taking into account costs and time constraints. ViGYAN strives to acquire sufficient repeat data sets throughout the duration of each wind tunnel test entry in order to provide each customer with an assessment of the repeatability/reproducibility of their data. ViGYAN engineers use Statistical Quality Control principles to analyze the stability of our wind tunnel measurement process.
ViGYAN personnel support the Ares Uncertainty Quantification team and the wind tunnel Check Standard data analysis at NASA LaRC.
Wind Tunnel Data Repeatability/Reproducibility
For at least the last six years, ViGYAN has provided wind tunnel data repeatability estimates to every customer who has requested it. The level and detail of the estimate depends on the amount of repeat data obtained for each test.
Here are a few examples of tests for which ViGYAN has provided uncertainty estimates:
- Drag measurements of architectural screens/meshes (numerous tests)
- All the Mars Lander tests
- 1/25th-scale LAV test for Orbital Sciences Corporation
ViGYAN also supports the Check Standard test process NASA LaRC has implemented in several wind tunnel facilities. Currently ViGYAN’s main efforts in uncertainty quantification center around the Ares Uncertainty Quantification team, helping to analyze Check Standard LaRC wind tunnel data.
For additional information, please consult the following publications:
Hemsch, M.J., Hanke, J.L., Walker, E.L., Houlden, H.P., “Detailed Uncertainty Analysis for Ares I Ascent Aerodynamics Wind Tunnel Database”, AIAA-2008-4259, June 2008. download full .pdf
Hemsch, M.J., Hanke, J.L., Houlden, H.P., “Quality Assurance and Uncertainty Quantification for Ares I A101 and A103 Ascent Aero Databases”, Ares-AD-TA-0011, April 18, 2008.
Hemsch, M.J., Houlden, H.P., Hanke, J.L., Walker, E., “Detailed Uncertainty Analysis for Ares I Ascent Aerodynamics Wind Tunnel Databases”, AIAA-2008-4259. download full .pdf
Hemsch, M.J., Tuttle, D., Houlden, H.P., Graham, A.B., “Measurement of Force Balance Repeatability and Reproducibility in the NTF”, AIAA-2004-771. download full .pdf
Glossary
Repeatability (short term variation) – refers to variation in repeated data measurements obtained under the same measurement conditions (i.e., same measurement procedure, instrumentation, conditions, or over a short period of time).
Reproducibility (long-term variation) – refers to variation in repeated data measurements obtained under changed conditions of measurement (i.e., different measurement process, instrumentation, test conditions, observer, or over a longer period of time).