Information Field Theory
(IFT)

Information field theory (IFT) is information theory, logic under uncertainty, applied to fields. A field can be any quantity defined over some space, such as the air temperature over Europe, the magnetic field strength in the Milky Way, or the matter density in the Universe. IFT describes how data and knowledge can be used to infer field properties. Mathematically it is a statistical field theory and exploits many of the tools developed for such. Practically, it is a framework for signal processing and image reconstruction. IFT is fully Bayesian. How else can infinitely many field degrees of freedom be constrained by finite data? It can be used without the knowledge of Feynman diagrams. There is a full toolbox of methods. It reproduces many known well working algorithms. This should be reassuring. And, there were certainly previous works in a similar spirit, like Bayesian Field Theory (BFT). See below for IFT & BFT publications and previous works. Anyhow, in many cases IFT provides novel rigorous ways to extract information from data.

Highlights

Algorithmic improvements for radio interferometry

Radio telescopes observe the sky in a very indirect fashion. Sky images in the radio frequency range therefore have to be computed using sophisticated algorithms. Scientists at the MPI for Astrophysics have developed a series of improvements for these algorithms, which help to improve the telescopes' resolution considerably.

  • Learn more
  • Artificial intelligence combined

    Artificial intelligence expands into all areas of the daily life, including research. Neural networks learn to solve complex tasks by training them on the basis of enormous amounts of examples. Researchers at the Max Planck Institute for Astrophysics in Garching have now succeeded in combining several networks, each one specializing in a different task, to jointly solve tasks using Bayesian logic in areas none was originally trained on. This enables the recycling of expensively trained networks and is an important step towards universally deductive artificial intelligence.

    • Learn more
    • Next generation imaging

      The Information Field Theory Group at the Max Planck Institute for Astrophysics has released a new version of the NIFTy software for scientific imaging. NIFTy5 generates an optimal imaging algorithm from the complex probability model of a measured signal. Such algorithms have already proven themselves in a number of astronomical applications and can now be used in other areas as well.

      The embarrassment of false predictions -
      How to best communicate probabilities?

      Complex predictions such as election forecasts or the weather reports often have to be simplified before communication. But how should one best simplify these predictions without facing embarrassment? In astronomical data analysis, researchers are also confronted with the problem of simplifying probabilities. Two researchers at the Max Planck Institute for Astrophysics now show that there is only one mathematically correct way to measure how embarrassing a simplified prediction can be. According to this, the recipient of a prediction should be deprived of the smallest possible amount of information.

      Galactic anatomy with gamma rays

      The anatomy of the Milky Way as seen in gamma light is full of mysteries. For example, there are gigantic bubbles of unknown origin above and below the center of the Milky Way that emit a lot of this high-energy radiation. A new method for imaging, developed at the Max Planck Institute for Astrophysics, now divided the Galactic gamma-radiation into three fundamental components: radiation from point sources, radiation from reactions of energetic protons with dense cold gas clouds, and radiation from electrons scattering light in the thin, hot, Galactic gas. The anatomic insights gained unravel some Galactic mysteries. Thus, it appears that the gamma-ray bubbles are simply outflows of ordinary, hot gas from the central region of the Milky Way.

      New all-sky map shows the magnetic fields of the Milky Way with the highest precision

      With a unique new all-sky map, scientists at MPA have made significant progress toward measuring the magnetic field structure of the Milky Way in unprecedented detail. Specifically, the map is of a quantity known as Faraday depth, which among other things, depends strongly on the magnetic fields along a particular line of sight. To produce the map, data were combined from more than 41,000 individual measurements using a novel image reconstruction technique. The work was a collaboration between scientists at the Max Planck Institute for Astrophysics (MPA), who are specialists in the new discipline of information field theory, and a large international team of radio astronomers. The new map not only reveals the structure of the galactic magnetic field on large scales, but also small-scale features that provide information about urbulence in the galactic gas.

      D3PO: Denoising, Deconvolving, and Decomposing Photon Observations

      A common problem for scientists analysing astronomical images is the separation of diffuse and point-like components. This analysis has now become easier: scientists at the Max Planck Institute for Astrophysics have recently published the D3PO algorithm, which removes noise effects and instrumental artefacts from the observed images, while simultaneously separating diffuse and point-like contributions.

      Resolving the radio sky

      Radio astronomers obtain extremely high resolution sky images by using interferometers, instruments where several single radio telescopes are linked together. However, optimal data analysis procedures for such an instrument are significantly more involved than for a single telescope. Scientists from the Max Planck Institute for Astrophysics have now developed the algorithm RESOLVE which solves a number of outstanding problems in radio imaging.

      Data analysis and steam engines

      As astronomical telescopes become more and more sensitive, the analysis techniques become ever more sophisticated. But do we need a new theoretical approach for a modern image reconstruction method? Not necessarily, a well-known theory, originally developed for a better understanding of steam engines, does the trick: thermodynamics. Two researchers at the Max Planck Institute for Astrophysics have now shown that the so called Gibbs energy in thermodynamics, known for more than a century, can be applied to the development of new, optimal imaging techniques.

      Mathematics of digital senses: Information Field Theory for signal recognition

      The correct interpretation of signals through our senses is not only an essential problem of living creatures, but also of fundamental scientific relevance. Scientists at the Max-Planck-Institute for Astrophysics have shown that mathematical methods from particle physics can be used for developing image reconstruction techniques. These yield optimal results even for incomplete, defective, and distorted data. Information Field Theory, which is used to develop such image reconstruction techniques, provides us with algorithms, i.e. mathematical instructions, for computing complicated perception processes in engineering and science, such as in cosmology.

IFT Introduction

Theory

IFT Applications

  • Studying Bioluminescence Flashes with the ANTARES Deep Sea Neutrino Telescope
    Nico Reeb et al. (2021), arXiv:2107.08063
    • Non-parametric Bayesian Causal Modeling of the SARS-CoV-2 Viral Load Distribution vs. Patient's Age
      Matteo Gurdiani et al. (2021), arXiv:2105.13483
    • The Galactic Faraday rotation sky 2020
      Sebastian Hutschenreuter et al. (2021), arXiv:2102.01709
    • Reconstructing non-repeating radio pulses with Information Field Theory
      Christoph Welling, Philipp Frank, Torsten A. Enßlin, Anna Nelles (2021), arXiv:2102.00258
    • Comparison of classical and Bayesian imaging in radio interferometry
      Philipp Arras, Richard A. Perley, Hertzog L. Bester, Reimar Leike, Oleg Smirnov, Rüdiger Westermann, Torsten A. Enßlin (2020), arXiv:2008.11435
    • Resolving nearby dust clouds
      Reimar H. Leike, Martin Glatze, Torsten A. Enßlin (2020), Astronomy & Astrophysics, Volume 639, A138, 10.1051/0004-6361/202038169, arXiv:2004.06732
    • The variable shadow of M87*
      Philipp Arras, Philipp Frank, Philipp Haim, Jakob Knollmüller, Reimar Leike, Martin Reinecke, Torsten A. Enßlin (2020), arXiv:2002.05218
    • Unified Radio Interferometric Calibration and Imaging with Joint Uncertainty Quantification
      Philipp Arras, Philipp Frank, Reimar Leike, Rüdiger Westermann, Torsten A. Enßlin (2019), Astronomy & Astrophysics, Volume 627, id.A134, 12 pp. 10.1051/0004-6361/201935555; arXiv:1903.11169
    • The Galactic Faraday depth sky revisited
      Sebastian Hutschenreuter, Torsten A. Enßlin, Astronomy & Astrophysics, Volume 633, id.A150, 16 pp.d eJournal, arXiv:1903.06735
    • Charting nearby dust clouds using Gaia data only
      Reimar Leike, Torsten A. Enßlin, A&A 631, A32 (2019); doi:10.1051/0004-6361/201935093; arXiv:1901.05971
    • The primordial magnetic field in our cosmic backyard
      Sebastian Hutschenreuter, Sebastian Dorn, Jens Jasche, Franco Vazza, Daniela Paoletti, Guilhem Lavaux, Torsten A. Enßlin, Classical and Quantum Gravity, Volume 35, Issue 15, article id. 154001 (2018) DOI: 10.1088/1361-6382/aacde0 arXiv:1803.02629
    • Radio Imaging With Information Field Theory
      Philipp Arras, Jakob Knollmüller, Henrik Junklewitz, Torsten A. Enßlin, submitted arXiv:1803.02174
    • Denoising, Deconvolving and Decomposing multi-Dimensional Photon Observations- The D4PO Algorithm
      Daniel Pumpe, Martin Reinecke, Torsten A. Enßlin, Astronomy & Astrophysics 619 , 119 (2018) arXiv:1802.02013
    • Inferring Galactic magnetic field model parameters using IMAGINE - An Interstellar MAGnetic field INference Engine
      Theo Steininger, Torsten A. Enßlin, Maksim Greiner, Tess Jaffe, Ellert van der Velden, Jiaxin Wang, Marijke Haverkorn, Jörg R. Hörandel, Jens Jasche, Jörg P. Rachen, submitted arXiv:1801.04341
    • Bayesian weak lensing tomography: Reconstructing the 3D large-scale distribution of matter with a lognormal prior
      Vanessa Böhm, Stefan Hilbert, Maksim Greiner, Torsten A. Enßlin, Physical Review D, Volume 96, Issue 12, id.123510 arXiv:1701.01886 DOI
    • Search for quasi-periodic signals in magnetar giant flares
      Daniel Pumpe, Michael Gabler, Theo Steininger, Torsten A. Enßlin, Astronomy & Astrophysics, Volume 610, id.A61, 12 pp. eJournal, arXiv:1708.05702
    • Information Field Theory with INTEGRAL/SPI data
      Mahsa Ghaempanah, Dissertation, LMU München: Faculty of Physics (2017) thesis
    • Cosmic expansion history from SN Ia data via information field theory
      Natalia Porqueres, Torsten A. Enßlin, Maksim Greiner, Vanessa Boehm, Sebastian Dorn, Pilar Ruiz-Lapuente, Alberto Manrique, submitted arXiv:1608.04007
    • Tomography of the Galactic free electron density with the Square Kilometer Array
      Maksim Greiner, Dominic Schnitzeler, Torsten A. Enßlin, A&A 590, A59 (2016) arXiv:1512.03480
    • Stochastic determination of matrix determinants
      Sebastian Dorn, Torsten A. Enßlin, Phys. Rev. E 92, 013302 (2015) arXiv:1504.02661
    • Using rotation measure grids to detect cosmological magnetic fields - a Bayesian approach
      V. Vacca, N. Oppermann, T. Enßlin, J. Jasche, M. Selig, M. Greiner, H. Junklewitz, M. Reinecke, M. Brueggen, E. Carretti, L. Feretti, C. Ferrari, C. A. Hales, C. Horellou, S. Ideguchi, M. Johnston-Hollitt, R. F. Pizzo, H. Roettgering, T. W. Shimwell, K. Takahashi, A&A 591, A13 (2016) arXiv:1509.00747
    • Estimating extragalactic Faraday rotation
      Niels Oppermann, Henrik Junklewitz, Maksim Greiner, Torsten A. Enßlin, Takuya Akahori, Ettore Carretti, Bryan M. Gaensler, Ariel Goobar, Lisa Harvey-Smith, Melanie Johnston-Hollitt, Luke Pratley, Dominic H. F. M. Schnitzeler, Jeroen M. Stil, Valentina Vacca, (2015) A&A 575, id.A118, 25 arXiv:1404.3701
    • A Bayesian method for pulsar template generation
      M. Imgrund, D.J. Champion, M. Kramer, H. Lesch, MNRAS (June 01, 2015) 449 (4): 4162 arXiv:1501.03497
    • All-sky reconstruction of the primordial scalar potential from WMAP temperature data
      Sebastian Dorn, Maksim Greiner, Torsten A. Enßlin, JCAP02 (2015) 041 arXiv:1412.8315 data
    • The Denoised, Deconvolved, and Decomposed Fermi gamma-ray Sky - An Application of the D3PO Algorithm
      Marco Selig, Valentina Vacca, Niels Oppermann, Torsten A. Enßlin, A&A 581, A126 (2015) arXiv:1410.4562 data
    • Log-transforming the matter power spectrum
      Maksim Greiner, Torsten A. Enßlin, A&A 574, A86 (2015) arXiv:1312.1354
    • Estimating extragalactic Faraday rotation
      Niels Oppermann, Henrik Junklewitz, Maksim Greiner, Torsten A. Enßlin, Takuya Akahori, Ettore Carretti, Bryan M. Gaensler, Ariel Goobar, Lisa Harvey-Smith, Melanie Johnston-Hollitt, Luke Pratley, Dominic H. F. M. Schnitzeler, Jeroen M. Stil, Valentina Vacca, accepted ba A&A, arXiv:1404.3701 arXiv:1404.3701 data
    • A new approach to multi-frequency synthesis in radio interferometry
      Henrik Junklewitz, Michael Bell, Marco Selig, Torsten A. Enßlin, Astronomy & Astrophysics, Volume 581, id.A5 (2015) arXiv:1401.4711
    • RESOLVE: A new algorithm for aperture synthesis imaging of extended emission in radio astronomy
      Henrik Junklewitz, Michael Bell, Marco Selig, Torsten A. Enßlin, submitted, arXiv:1311.5282 arXiv:1311.5282
    • D3PO - Denoising, Deconvolving, and Decomposing Photon Observations
      Marco Selig, Torsten A. Enßlin, accepted by Physical Review E, arXiv:1311.1888 arXiv:1311.1888
    • A fast and precise way to calculate the posterior for the local non-Gaussianity parameter f_nl from Cosmic Microwave Background observations
      Sebastian Dorn, Niels Oppermann, Rishi Khatri, Marco Selig, Torsten A. Enßlin, Phys. Rev. D 88, 103516 (2013) arXiv:1307.3884
    • The XENON100 exclusion limit without considering Leff as a nuisance parameter
      Jonathan H. Davis, Celine Boehm, Niels Oppermann, Torsten A. Enßlin, Thomas Lacroix, Physical Review D, vol. 86, Issue 1, id. 015027, 2012 arXiv:1203.6823
    • An improved map of the Galactic Faraday sky
      Niels Oppermann, et al., 2012, Astronomy & Astrophysics, Volume 542, id.A93 (2012) arXiv:1111.6186
    • Improving stochastic estimates with inference methods: calculating matrix diagonals
      Marco Selig, Niels Oppermann, Torsten A. Enßlin, Phys. Rev. E 85, 021134 (2012) arXiv:1108.0600
    • Probing Magnetic Helicity with Synchrotron Radiation and Faraday Rotation
      Niels Oppermann, Henrik Junklewitz, Georg Robbers, Torsten A. Enßlin 2011, Astronomy and Astrophysics, 530, id.A89 arXiv:1008.1246
    • Bayesian analysis of spatially distorted cosmic signals from Poissonian data
      Cornelius Weig, Torsten A. Enßlin 2010, MNRAS 409, 1393 arXiv:1003.1311
    • Bayesian non-linear large scale structure inference of the Sloan Digital Sky Survey data release 7
      Jens Jasche, Francisco S. Kitaura, Cheng Li, Torsten A. Enßlin 2010, MNRAS 409, 355 arXiv:0911.2498
    • Fast Hamiltonian sampling for large-scale structure inference
      Jens Jasche, Francisco S. Kitaura 2010, MNRAS 407, 29 arXiv:0911.2496
    • Bayesian power-spectrum inference for Large Scale Structure data
      Jens Jasche, Francisco S. Kitaura, Benjamin D. Wandelt, Torsten A. Enßlin 2010, MNRAS 406, 60 arXiv:0911.2493
    • Cosmic Cartography of the Large-Scale Structure with Sloan Digital Sky Survey Data Release 6
      Francisco S. Kitaura, Jens Jasche, Cheng Li, Torsten A. Enßlin, R.Benton Metcalf, Benjamin D. Wandelt, Gerard Lemson, Simon D.M. White 2009, MNRAS 400, 183 arXiv:0906.3978

Dynamics

  • Dynamical field inference and supersymmetry
    Margret Westerkamp, Igor Ovchinnikov, Torsten A. Enßlin, 2020, submitted, arXiv:2010.15414
  • Probabilistic simulation of partial differential equations
    Philipp Frank, Torsten A. Enßlin, 2020, submitted, arXiv:2010.06583
  • Field dynamics inference for local and causal interactions
    Philipp Frank, Reimar Leike, Torsten A. Enßlin, 2021, Annalen der Physik, 2000486, arXiv:1902.02624
  • Towards information-optimal simulation of partial differential equations
    Reimar Leike, Torsten A. Enßlin, 2018, Physical Review E, 97, 033314; journal article, arXiv:1709.02859
  • Consistency and convergence of simulation schemes in Information field dynamics
    Martin Dupont, Torsten A. Enßlin, 2018, Physical Review E, Vol. 98, No. 4, DOI: 10.1103/PhysRevE.98.043307, arXiv:1802.00971
  • Field dynamics inference via spectral density estimation
    Philipp Frank, Theo Steininger, Torsten A. Enßlin, Phys. Rev. E 96, 052104 (2017) arXiv:1708.05250
  • Supersymmetric theory of stochastic ABC model
    Igor Ovchinikov, Yuquan Sun, Torsten A. Enßlin and Kang L. Wang, J. Phys. Commun. (2018) doi.org/10.1088/2399-6528/aac94a
  • Dynamic system classifier
    Daniel Pumpe, Maksim Greiner, Ewald Mueller, Torsten A. Enßlin, Physical Review E (Vol.94, No.1) DOI: 10.1103/PhysRevE.94.012132 arXiv:1601.07901
  • Kinematic dynamo, supersymmetry breaking, and chaos
    Igor Ovchinikov, Torsten A. Enßlin, Physical Review D, Volume 93, Issue 8, id.085023 (2016) arXiv:1512.01651
  • Mathematical foundation of Information Field Dynamics
    Christian Muench, master thesis, Technical University Munich arXiv:1412.1226
  • Simulation of stochastic network dynamics via entropic matching
    Tiago Ramalho, Marco Selig, Ulrich Gerland, Torsten A. Enßlin, Phys. Rev. E 87, 022719 (2013) arXiv:1209.3700
  • Information field dynamics for simulation scheme construction
    Torsten A. Enßlin, 2013, Phys. Rev. E 87, 013308 arXiv:1206.4229

IFT Tools

Bayesian Density Estimation for Poisson Data

(open source code) .

NIFTy5: Numerical Information Field Theory v5

NIFTy (Numerical Information Field Theory) facilitates the construction of Bayesian field reconstruction algorithms for fields being defined over multidimensional domains. A NIFTy algorithm can be developed for 1D field inference and then be used in 2D or 3D, on the sphere, or on product spaces thereof. NIFTy5 is a complete redesign of the previous framework (ascl:1302.013), and requires only the specification of a probabilistic generative model for all involved fields and the data in order to be able to recover the former from the latter. This is achieved via Metric Gaussian Variational Inference, which also provides posterior samples for all unknown quantities jointly. (open source code) .

HMCF - Hamiltonian Monte Carlo Sampling for Fields - A Python framework for HMC sampling with NIFTy

HMCF "Hamiltonian Monte Carlo for Fields", is a software add-on for the NIFTy "Numerical Information Field Theory" framework implementing Hamilton Monte Carlo (HMC) sampling in Python. HMCF as well as NIFTy are designed to address field in- ference problems especially in - but not limited to - astrophysics. They are optimized to deal with the typically high number of degrees of freedom as well as their correlation structure. HMCF adds an HMC sampler to NIFTy that automatically adjusts the many free pa- rameters steering the HMC sampling machinery such as integration step size and the mass matrix according to the requirements of field inference. Furthermore, different convergence measures are available to check whether the burn-in phase has finished. Multiprocessing in the sense of running individual Markov chains (MC) on several cores is possible as well. A primary application of HMCF is to provide samples from the full field posterior and to verify conveniently approximate algorithms implemented in NIFTy. (open source code) .

NIFTy 3 - Numerical Information Field Theory - A Python framework for multicomponent signal inference on HPC clusters

NIFTy, "Numerical Information Field Theory", is a software framework designed to ease the development and implementation of field inference algorithms. Field equations are formulated independently of the underlying spatial geometry allowing the user to focus on the algorithmic design. Under the hood, NIFTy ensures that the discretization of the implemented equations is consistent. This enables the user to prototype an algorithm rapidly in 1D and then apply it to high-dimensional real-world problems. This paper introduces NIFTy 3, a major upgrade to the original NIFTy framework. NIFTy 3 allows the user to run inference algorithms on massively parallel high performance computing clusters without changing the implementation of the field equations. It supports n-dimensional Cartesian spaces, spherical spaces, power spaces, and product spaces as well as transforms to their harmonic counterparts. Furthermore, NIFTy 3 is able to treat non-scalar fields. The functionality and performance of the software package is demonstrated with example code, which implements a real inference algorithm from the realm of information field theory. NIFTy 3 is open-source software available under the GNU General Public License v3 (GPL-3) at https://gitlab.mpcdf.mpg.de/ift/NIFTy/ .

NIFTY, Numerical Information Field Theory,

is a versatile library designed to enable the development of signal inference algorithms that operate regardless of the underlying spatial grid and its resolution. Its object-oriented framework is written in Python, although it accesses libraries written in Cython, C++, and C for efficiency.
NIFTY offers a toolkit that abstracts discretized representations of continuous spaces, fields in these spaces, and operators acting on fields into classes. Thereby, the correct normalization of operations on fields is taken care of automatically without concerning the user. This allows for an abstract formulation and programming of inference algorithms, including those derived within information field theory. Thus, NIFTY permits its user to rapidly prototype algorithms in 1D and then apply the developed code in higher-dimensional settings of real world problems. The set of spaces on which NIFTY operates comprises point sets, n-dimensional regular grids, spherical spaces, their harmonic counterparts, and product spaces constructed as combinations of those.

Further literature

  • DIP -- Diagnostics for Insufficiencies of Posterior calculations in Bayesian signal inference
    Sebastian Dorn, Niels Oppermann, Torsten A. Enßlin, Phy. Rev. E 88 arXiv:1307.3889
  • Lectures on Probability, Entropy, and Statistical Physics
    A. Caticha arXiv:0808.0012
  • MAGIC: Exact Bayesian Covariance Estimation and Signal Reconstruction for Gaussian Random Fields
    B. Wandelt arXiv:ph/0401623