## Information Field Theory

(IFT)

Information field theory (IFT) is information theory, logic under uncertainty, applied to fields. A field can be any quantity defined over some space, such as the air temperature over Europe, the magnetic field strength in the Milky Way, or the matter density in the Universe. IFT describes how data and knowledge can be used to infer field properties. Mathematically it is a statistical field theory and exploits many of the tools developed for such. Practically, it is a framework for signal processing and image reconstruction. IFT is fully Bayesian. How else can infinitely many field degrees of freedom be constrained by finite data? It can be used without the knowledge of Feynman diagrams. There is a full toolbox of methods. It reproduces many known well working algorithms. This should be reassuring. And, there were certainly previous works in a similar spirit, like Bayesian Field Theory (BFT). See below for IFT & BFT publications and previous works. Anyhow, in many cases IFT provides novel rigorous ways to extract information from data.

## Highlights

**The embarrassment of false predictions - **

How to best communicate probabilities?

How to best communicate probabilities?

Complex predictions such as election forecasts or the weather reports often have to be simplified before communication. But how should one best simplify these predictions without facing embarrassment? In astronomical data analysis, researchers are also confronted with the problem of simplifying probabilities. Two researchers at the Max Planck Institute for Astrophysics now show that there is only one mathematically correct way to measure how embarrassing a simplified prediction can be. According to this, the recipient of a prediction should be deprived of the smallest possible amount of information.

**Galactic anatomy with gamma rays
**

The anatomy of the Milky Way as seen in gamma light is full of mysteries. For example, there are gigantic bubbles of unknown origin above and below the center of the Milky Way that emit a lot of this high-energy radiation. A new method for imaging, developed at the Max Planck Institute for Astrophysics, now divided the Galactic gamma-radiation into three fundamental components: radiation from point sources, radiation from reactions of energetic protons with dense cold gas clouds, and radiation from electrons scattering light in the thin, hot, Galactic gas. The anatomic insights gained unravel some Galactic mysteries. Thus, it appears that the gamma-ray bubbles are simply outflows of ordinary, hot gas from the central region of the Milky Way.

**New all-sky map shows the magnetic fields of the Milky Way with the highest precision**

With a unique new all-sky map, scientists at MPA have made significant progress toward measuring the magnetic field structure of the Milky Way in unprecedented detail. Specifically, the map is of a quantity known as Faraday depth, which among other things, depends strongly on the magnetic fields along a particular line of sight. To produce the map, data were combined from more than 41,000 individual measurements using a novel image reconstruction technique. The work was a collaboration between scientists at the Max Planck Institute for Astrophysics (MPA), who are specialists in the new discipline of information field theory, and a large international team of radio astronomers. The new map not only reveals the structure of the galactic magnetic field on large scales, but also small-scale features that provide information about urbulence in the galactic gas.

**D³PO: Denoising, Deconvolving, and Decomposing Photon Observations**

A common problem for scientists analysing astronomical images is the separation of diffuse and point-like components. This analysis has now become easier: scientists at the Max Planck Institute for Astrophysics have recently published the D³PO algorithm, which removes noise effects and instrumental artefacts from the observed images, while simultaneously separating diffuse and point-like contributions.

**Resolving the radio sky**

Radio astronomers obtain extremely high resolution sky images by using interferometers, instruments where several single radio telescopes are linked together. However, optimal data analysis procedures for such an instrument are significantly more involved than for a single telescope. Scientists from the Max Planck Institute for Astrophysics have now developed the algorithm RESOLVE which solves a number of outstanding problems in radio imaging.

**Data analysis and steam engines **

As astronomical telescopes become more and more sensitive, the analysis techniques become ever more sophisticated. But do we need a new theoretical approach for a modern image reconstruction method? Not necessarily, a well-known theory, originally developed for a better understanding of steam engines, does the trick: thermodynamics. Two researchers at the Max Planck Institute for Astrophysics have now shown that the so called Gibbs energy in thermodynamics, known for more than a century, can be applied to the development of new, optimal imaging techniques.

**Mathematics of digital senses: Information Field Theory for signal recognition **

The correct interpretation of signals through our senses is not only an essential problem of living creatures, but also of fundamental scientific relevance. Scientists at the Max-Planck-Institute for Astrophysics have shown that mathematical methods from particle physics can be used for developing image reconstruction techniques. These yield optimal results even for incomplete, defective, and distorted data. Information Field Theory, which is used to develop such image reconstruction techniques, provides us with algorithms, i.e. mathematical instructions, for computing complicated perception processes in engineering and science, such as in cosmology.

## IFT Introduction

**Why information field theory *is* a field theory**

Torsten A. Enßlin, May 2015. link**Information field theory, an introduction in a nutshell**

Torsten A. Enßlin, in MaxEnt 2012, the 32nd Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering, arxiv: 1301.2556**Astrophysical data analysis with information field theory, advanced recipes, also in a nutshell**

Torsten A. Enßlin, accepted chapter to the conference proceedings for MaxEnt 2013, to be published by AIP, arxiv: 1405.7701**Bayesian Field Theory: Nonparametric Approaches to Density Estimation, Regression, Classification, and Inverse Quantum Problems**

J. C. Lemm, arxiv:physics/9912005 and more of his works**Bayesian Field Theory applied to scattered data interpolation and inverse problems.**

C. Farmer. Algorithms for Approximation, pages 147-166, 2007. eBook**Information field theory for cosmological perturbation reconstruction and non-linear signal analysis**

Torsten A. Enßlin, Mona Frommert, Francisco S. Kitaura 2009, Phys. Rev. D 80, 105005 arxiv: 0806.3474**A path-integral approach to Bayesian inference for inverse problems using the semiclassical approximation**

Joshua C Chang, Van Savage, Tom Chou, J Stat Phys (2014) 157: 582 arxiv: 1312.2974- IFT lecture notes and crash course notes

## IFT Publications

**Inference of signals with unknown correlation structure from non-linear measurements**

Jakob Knollmüller, Theo Steininger, Torsten A. Enßlin, submitted arxiv: 1711.02955**Towards information optimal simulation of partial differential equations**

Reimar H. Leike, Torsten A. Enßlin, submitted arxiv: 1709.02859**Field dynamics inference via spectral density estimation**

Philipp Frank, Theo Steininger, Torsten A. Enßlin, Phys. Rev. E 96, 052104 (2017) arxiv: 1708.05250**Noisy independent component analysis of auto-correlated components**

Jakob Knollmüller, Torsten A. Enßlin, Phys. Rev. E 96, 042114 (2017) arxiv: 1705.02344**Correlated signal inference by free energy exploration**

Torsten A. Enßlin, Jakob Knollmüller, submitted arxiv: 1612.08406**Optimal Belief Approximation**

Reimar Leike, Torsten A. Enßlin, Entropy 2017, 19, 402; doi:10.3390/e19080402 arxiv:1610.09018**Operator Calculus for Information Field Theory**

Reimar Leike, Torsten A. Enßlin, Physical Review E, Volume 94, Issue 5, id.053306 (2016) arxiv:1605.00660**Dynamic system classifier**

Daniel Pumpe, Maksim Greiner, Ewald Mueller, Torsten A. Enßlin, Physical Review E (Vol.94, No.1) DOI: 10.1103/PhysRevE.94.012132 arxiv: 1601.07901**Mathematical foundation of Information Field Dynamics**

Christian Muench, master thesis, Technical University Munich arxiv: 1412.1226**Signal inference with unknown response: calibration uncertainty renormalized estimator**

Sebastian Dorn, Torsten A. Enßlin, Maksim Greiner, Marco Selig, Vanessa Boehm, PRE 91, 013311 (2015) arxiv: 1410.6289**Improving self-calibration**

Torsten A. Enßlin, Henrik Junklewitz, Lars Winderling, Maksim Greiner, Marco Selig, 2014, PRE 90, id.043301 arxiv: 1312.1349**Reconstruction of Gaussian and log-normal fields with spectral smoothness**

Niels Oppermann, Marco Selig, Michael R. Bell, Torsten A. Enßlin, 2013, Phys. Rev. E 87, 032136 arxiv: 1210.6866**Information field dynamics for simulation scheme construction**

Torsten A. Enßlin, 2013, Phys. Rev. E 87, 013308 arxiv: 1206.4229**Reconstructing signals from noisy data with unknown signal and noise covariances**

Niels Oppermann, Georg Robbers, Torsten A. Enßlin, 2011, Physical Review E 84, 041118 arxiv: 1107.2384**Reconstruction of signals with unknown spectra in information field theory with parameter uncertainty**

Torsten A. Enßlin, Mona Frommert 2011, Physical Review D 83, 105014 arxiv: 1002.2928**Inference with minimal Gibbs free energy in information field theory**

Torsten A. Enßlin, Cornelius Weig 2010, Physical Review E 82, 051112 arxiv: 1004.2868 and Comment on Paper and Reply to Comment- and more related papers

## IFT Applications

**Cosmic expansion history from SN Ia data via information field theory**

Natalia Porqueres, Torsten A. Enßlin, Maksim Greiner, Vanessa Boehm, Sebastian Dorn, Pilar Ruiz-Lapuente, Alberto Manrique, submitted arxiv: 1608.04007**Tomography of the Galactic free electron density with the Square Kilometer Array**

Maksim Greiner, Dominic Schnitzeler, Torsten A. Enßlin, A&A 590, A59 (2016) arxiv: 1512.03480**Stochastic determination of matrix determinants**

Sebastian Dorn, Torsten A. Enßlin, Phys. Rev. E 92, 013302 (2015) arxiv: 1504.02661**Using rotation measure grids to detect cosmological magnetic fields - a Bayesian approach**

V. Vacca, N. Oppermann, T. Enßlin, J. Jasche, M. Selig, M. Greiner, H. Junklewitz, M. Reinecke, M. Brueggen, E. Carretti, L. Feretti, C. Ferrari, C. A. Hales, C. Horellou, S. Ideguchi, M. Johnston-Hollitt, R. F. Pizzo, H. Roettgering, T. W. Shimwell, K. Takahashi, A&A 591, A13 (2016) arxiv: 1509.00747**Estimating extragalactic Faraday rotation**

Niels Oppermann, Henrik Junklewitz, Maksim Greiner, Torsten A. Enßlin, Takuya Akahori, Ettore Carretti, Bryan M. Gaensler, Ariel Goobar, Lisa Harvey-Smith, Melanie Johnston-Hollitt, Luke Pratley, Dominic H. F. M. Schnitzeler, Jeroen M. Stil, Valentina Vacca, (2015) A&A 575, id.A118, 25 arxiv: 1404.3701**A Bayesian method for pulsar template generation**

M. Imgrund, D.J. Champion, M. Kramer, H. Lesch, MNRAS (June 01, 2015) 449 (4): 4162 arxiv: 1501.03497**All-sky reconstruction of the primordial scalar potential from WMAP temperature data**

Sebastian Dorn, Maksim Greiner, Torsten A. Enßlin, JCAP02 (2015) 041 arxiv: 1412.8315 data**The Denoised, Deconvolved, and Decomposed Fermi gamma-ray Sky - An Application of the D3PO Algorithm**

Marco Selig, Valentina Vacca, Niels Oppermann, Torsten A. Enßlin, A&A 581, A126 (2015) arxiv: 1410.4562 data**Log-transforming the matter power spectrum**

Maksim Greiner, Torsten A. Enßlin, A&A 574, A86 (2015) arxiv: 1312.1354**Estimating extragalactic Faraday rotation**

Niels Oppermann, Henrik Junklewitz, Maksim Greiner, Torsten A. Enßlin, Takuya Akahori, Ettore Carretti, Bryan M. Gaensler, Ariel Goobar, Lisa Harvey-Smith, Melanie Johnston-Hollitt, Luke Pratley, Dominic H. F. M. Schnitzeler, Jeroen M. Stil, Valentina Vacca, accepted ba A&A, arXiv:1404.3701 arxiv: 1404.3701 data**A new approach to multi-frequency synthesis in radio interferometry**

Henrik Junklewitz, Michael Bell, Marco Selig, Torsten A. Enßlin, Astronomy & Astrophysics, Volume 581, id.A5 (2015) arxiv: 1401.4711**RESOLVE: A new algorithm for aperture synthesis imaging of extended emission in radio astronomy**

Henrik Junklewitz, Michael Bell, Marco Selig, Torsten A. Enßlin, submitted, arXiv:1311.5282 arxiv: 1311.5282**D3PO - Denoising, Deconvolving, and Decomposing Photon Observations**

Marco Selig, Torsten A. Enßlin, accepted by Physical Review E, arXiv:1311.1888 arxiv: 1311.1888**A fast and precise way to calculate the posterior for the local non-Gaussianity parameter f_nl from Cosmic Microwave Background observations**

Sebastian Dorn, Niels Oppermann, Rishi Khatri, Marco Selig, Torsten A. Enßlin, Phys. Rev. D 88, 103516 (2013) arxiv: 1307.3884**Simulation of stochastic network dynamics via entropic matching**

Tiago Ramalho, Marco Selig, Ulrich Gerland, Torsten A. Enßlin, Phys. Rev. E 87, 022719 (2013) arxiv: 1209.3700**The XENON100 exclusion limit without considering Leff as a nuisance parameter**

Jonathan H. Davis, Celine Boehm, Niels Oppermann, Torsten A. Enßlin, Thomas Lacroix, Physical Review D, vol. 86, Issue 1, id. 015027, 2012 arxiv: 1203.6823**An improved map of the Galactic Faraday sky**

Niels Oppermann, et al., 2012, Astronomy & Astrophysics, Volume 542, id.A93 (2012) arxiv: 1111.6186**Improving stochastic estimates with inference methods: calculating matrix diagonals**

Marco Selig, Niels Oppermann, Torsten A. Enßlin, Phys. Rev. E 85, 021134 (2012) arxiv: 1108.0600**Probing Magnetic Helicity with Synchrotron Radiation and Faraday Rotation**

Niels Oppermann, Henrik Junklewitz, Georg Robbers, Torsten A. Enßlin 2011, Astronomy and Astrophysics, 530, id.A89 arxiv: 1008.1246**Bayesian analysis of spatially distorted cosmic signals from Poissonian data**

Cornelius Weig, Torsten A. Enßlin 2010, MNRAS 409, 1393 arxiv: 1003.1311**Bayesian non-linear large scale structure inference of the Sloan Digital Sky Survey data release 7**

Jens Jasche, Francisco S. Kitaura, Cheng Li, Torsten A. Enßlin 2010, MNRAS 409, 355 arxiv: 0911.2498**Fast Hamiltonian sampling for large-scale structure inference**

Jens Jasche, Francisco S. Kitaura 2010, MNRAS 407, 29 arxiv: 0911.2496**Bayesian power-spectrum inference for Large Scale Structure data**

Jens Jasche, Francisco S. Kitaura, Benjamin D. Wandelt, Torsten A. Enßlin 2010, MNRAS 406, 60 arxiv: 0911.2493**Cosmic Cartography of the Large-Scale Structure with Sloan Digital Sky Survey Data Release 6**

Francisco S. Kitaura, Jens Jasche, Cheng Li, Torsten A. Enßlin, R.Benton Metcalf, Benjamin D. Wandelt, Gerard Lemson, Simon D.M. White 2009, MNRAS 400, 183 arxiv: 0906.3978

## IFT Tools

### NIFTy 3 - Numerical Information Field Theory - A Python framework for multicomponent signal inference on HPC clusters

NIFTy, "Numerical Information Field Theory", is a software framework designed to ease the development and implementation of field inference algorithms. Field equations are formulated independently of the underlying spatial geometry allowing the user to focus on the algorithmic design. Under the hood, NIFTy ensures that the discretization of the implemented equations is consistent. This enables the user to prototype an algorithm rapidly in 1D and then apply it to high-dimensional real-world problems. This paper introduces NIFTy 3, a major upgrade to the original NIFTy framework. NIFTy 3 allows the user to run inference algorithms on massively parallel high performance computing clusters without changing the implementation of the field equations. It supports n-dimensional Cartesian spaces, spherical spaces, power spaces, and product spaces as well as transforms to their harmonic counterparts. Furthermore, NIFTy 3 is able to treat non-scalar fields. The functionality and performance of the software package is demonstrated with example code, which implements a real inference algorithm from the realm of information field theory. NIFTy 3 is open-source software available under the GNU General Public License v3 (GPL-3) at https://gitlab.mpcdf.mpg.de/ift/NIFTy/ .

### NIFTY, “Numerical Information Field Theory”,

is a versatile library designed to enable the development of signal inference algorithms that operate regardless of the underlying spatial grid and its resolution. Its object-oriented framework is written in Python, although it accesses libraries written in Cython, C++, and C for efficiency.NIFTY offers a toolkit that abstracts discretized representations of continuous spaces, fields in these spaces, and operators acting on fields into classes. Thereby, the correct normalization of operations on fields is taken care of automatically without concerning the user. This allows for an abstract formulation and programming of inference algorithms, including those derived within information field theory. Thus, NIFTY permits its user to rapidly prototype algorithms in 1D and then apply the developed code in higher-dimensional settings of real world problems. The set of spaces on which NIFTY operates comprises point sets, n-dimensional regular grids, spherical spaces, their harmonic counterparts, and product spaces constructed as combinations of those.

- NIFTy 1- A versatile Python library for signal inference
- NIFTy 3 (supporting D20)
- D3PO- Denoising, Deconvolving, and Decomposing Photon Observations
- Resolve- a new approach to multifrequency synthesis in radio interferomerty
- Charm- cosmic history agnostic reconstruction method
- D20- a distributed data object for parallel high-performance computing in Python PAPER
- Keepers

## Secondary literature

**DIP -- Diagnostics for Insufficiencies of Posterior calculations in Bayesian signal inference**

Sebastian Dorn, Niels Oppermann, Torsten A. Enßlin, Phy. Rev. E 88 arxiv: 1307.3889**Lectures on Probability, Entropy, and Statistical Physics**

A. Caticha arxiv: 0808.0012**MAGIC: Exact Bayesian Covariance Estimation and Signal Reconstruction for Gaussian Random Fields**

B. Wandelt arxiv: ph/0401623