The Theory and Practice of Precision Measurement

Abstract: Astrophysics projects are producing bigger and more complicated data sets with bleeding edge instrumentation that is not fully understood. At the same time, the relevant astrophysics communities expect these projects to deliver more and more precise (and accurate!) measurements. I discuss a range of considerations and techniques—all both theoretically rigorous and also useful in practice—for making good measurements in the presence of noise and unknown foregrounds and backgrounds, with instruments that must be empirically characterized. Information theory, causal inference, survey design, hierarchical models, machine learning, and old-school frequentism will all be harmed in different ways. Examples will be taken from the domains of cosmology, stellar dynamics, and exoplanets.