Information, physics, and computation by Marc Mezard and Andrea Montanari in 2009
computer science, physics, algorithms, information geometry
“At the core of the background necessary to understand the free energy principle is the relationship between free energy as it is used in statistical physics versus its use in probabilistic inference which is encapsulated in the notion of variational inference. The best explanation I know of this relationship is in the formulation of Gibbs free energy via a variational principle in Information, physics, and computation by Marc Mezard and Andrea Montanari in 2009.”
I. Background
1. Introduction to information theory
2. Statistical physics and probability theory
3. Introduction to combinatorial optimization
4. A probabilistic toolbox
- 4.1 Many random variables: a qualitative preview
- 4.2 Large deviations for independent variables
- 4.3 Correlated variables
- 4.4 The Gibbs free energy
- 4.5 The Monte Carlo method
- 4.6 Simulated annealing
- 4.7 Appendix: A physicists’s approach to Sanov’s theorem