Will I love Information Theory, Inference and Learning Algorithms?

The best parts of this book really represent a gold standard in pedagogical clarity.

Although it’s now twenty years old, there is still much to learn from this rather unconventional book that covers the boundary between machine learning, information theory, and Bayesian methods. There are also odd tangents and curiosities, some of which work better than others but are never dull.

Just writing this review makes me want to go back to it and squeeze more out of it.

This is considered to be a modern classic on information theory. It is a fairly readable text that roams far and wide over many topics. MacKay was extraordinarily clever, curious, and generous (which is why he made the book freely available). His wide range of interests included coding theory and artificial neural networks (machine learning), which occupy a few chapters of the book. In describing the difference between Bayesian and frequentist statistical methods, MacKay pulls no punches, and he can be heard laughing through gritted teeth as he tries to make sense of the nonsensical. This is one of those… show more.