Resources and References
This page lists various resources that may be of use in understanding the underlying concepts behind Infer.NET, both at the introductory and more technical level. Two excellent self-contained starting points are:
- Chris Bishop’s book Pattern Recognition and Machine Learning: http://research.microsoft.com/en-us/um/people/cmbishop/prml/. Chapters 8 (Graphical Models) and 10 (Approximate Inference) are particularly relevant.
- Zoubin Ghahramani’s tutorial paper: http://learning.eng.cam.ac.uk/zoubin/papers/ul.pdf
Both describe graphical models in general, specific models which we provide as Infer.NET examples, and the approximate inference algorithms available in Infer.NET.
Probabilistic models
- Kevin Murphy’s book Machine Learning: a Probabilistic Perspective gives excellent coverage of probabilistic Machine Learning with plenty of examples - it is targeted at undergraduates as well as graduates.
- Kevin Murphy also has a good tutorial paper that covers probabilistic models and Bayesian networks: http://www.cs.ubc.ca/~murphyk/Bayes/bnintro.html
- The chapters on Uncertainty and Probablistic Modelling (13 to 15) in Russel and Norvig’s standard AI text (3rd edition) have been recommended on the forum.
- Daphne Koller teaches an online course on Probabilistic Graphical Models: https://www.coursera.org/specializations/probabilistic-graphical-models
- Sebastian Thrun and Peter Norvig gave an online course course “Introduction to Artificial Intelligence” which covers many of the Infer.NET concepts at: https://www.ai-class.com/
- Radford Neal has a tutorial that explains how the Bayesian approach to machine learning (used by Infer.NET) differs from other approaches.
Approximate Inference
- Chapters 16 (Message Passing) and 33 (Variational Methods) of David Mackay’s textbook (Information Theory, Inference, and Learning Algorithms: http://www.inference.phy.cam.ac.uk/mackay/itila/book.html) are relevant.
- The key reference for Expectation Propagation is Tom Minka’s paper http://research.microsoft.com/en-us/um/people/minka/papers/ep/minka-ep-uai.pdf and for Variational Message Passing is John Winn’s paper https://www.microsoft.com/en-us/research/wp-content/uploads/2016/02/bishop-vibes-jmlr-04.pdf
- Tom also maintains a website with links to papers on EP: http://research.microsoft.com/en-us/um/people/minka/papers/ep/roadmap.html