Bayesian Networks and Decision Graphs (Information Science and Statistics)

Finn V. Jensen

1 A very good introduction to Bayesian networks
I am very pleased to have found a book that gives a modern, sound, and self-contained introduction to Bayesian networks. The only prerequisite is basic knowledge of probability. This makes sense because a Bayesian network is essentially a directed graph whose vertex set is a collection of random variables, while an edge from one variable X to another variable Y represents a belief that X has a causative effect on Y. For example, X could be the pregnancy status of a cow, while Y could be a blood test administered to the cow. Vertex Y would contain a contingency table that reflects the conditional probability of Y in terms of X. The author does well in explaining this, as well as adequately treating many of the practical issues surrounding Bayesian networks, such as design issues, network learing and tuning, and some basic algorithms (e.g. bucket elimination and junction trees) that aid in the efficient updating of variable probabilities due to new evidence that may instantiate or change the distribution of one or more variables.
The author also provides a good introduction to decision graphs, a close relative of Bayesian networks.

The aspect of Bayesian networks that I find most attractive is the fact that there is a "rational" way of designing a network, based on hypothesis, informational, and mediating variables, and their "causal" relationships. Unlike neural networks in which one is almost forced to guess the appropriate structure of the network, every node in a Bayesian network correpsonds with a state or quantity that can be measured either directly or indirectly through other variables. Thus, changes in a system model should only induce local changes in a Bayesian network, where as system changes might require the design and training of an entirely new neural network.

Another aspect of Bayesian networks that I find very compelling is the way in which they seem quite amendable to learning and the presentation of new evidence. This is true since knowledge updating is done locally (through variables), while the effects of those changes are witnessed globally through appropriate belief-updating algorithms.

On the downside, it should be noted that the operation of belief-updating is in general NP-hard, thus there exists a valid concern about the computational efficiency of Bayesian networks. Contrast this with the fact that once a nueral network has been trained, it is quite easy to compute. One would hope that these concerns will subside with more research, for the above mentioned benefits of Bayesian networks leads me to believe that these networks will have quite an influence on the future directions of machine learning.

Although this book will not go down in history as the definitive reference for Bayesian networks, it serves as a good conduit for explaining this quite interesting area of learning at a time when such few complete and modern references exist.

2 A lot about very little
The book covers many topics, but doesn't really cover them well. I would not recommend this book. I have learned litte from it.

3 Accessible introduction to Bayesian Networks
Among currently available introduction to Bayesian networks (also known as Bayes Net, Bayesian Belief Nets), this book is probably one of the most accessible. The book is divided into part I and II. Part I is intended for BN users (practitioners) and Part II more towards BN developers and researchers, as it contains algorithmic introduction of BN.

Prerequisites of the book as stated in the preface include Graph Theory and Calculus, both at introductory level. I personally did not have exposure to Graph theory, but I was able to understand most of the material without any help. Necessary probability theory is developed, but basic probability knowledge is also a prerequisite to digest the material to a reader without prior exposure of Probability as it shapes the core of the material in the book.

The strength of this text is in Part I where the author provides several examples to illustrate use of Bayesian Networks, Influence Diagrams and other models. I find it useful Influence Diagram as an extension of Bayesian Networks.

Most answers to Exercises at the end of each chapter are provided at the author's homepage, except answers of the last chapter. Answers that require graphical modeling software are also provided in Hugin format. (Hugin Lite can be downloaded from Hugin site.)

The downsides are that writing of the text is somewhat awkward, obscuring readers from understanding, that model building chapter could have been discussed more thoroughly, that material in Learning is barely present, and that definitions are sometimes not introduced upon the first encounter but they appear later in chapters. More different and complex examples could have been discussed to illustrate the material. Note: the author provides a page for Learning at his homepage.

Although this is an introduction to Bayesian Networks and Influence Diagrams, a reader should be equipped with some level of abstract thinking in order to digest the material.

This book is suitable for self-study. It has motivations for the uninitiated. References are provided at the end of the book and I was able to find some of them online. A notable is "A tutorial on Learning with Bayesian Networks" by Heckerman, to fill in the part of Learning in this book.

Other books at this level from users' perspective are:
Edwards, Introduction to Graphical Modeling (Utilizes software MIM.)
Clemen, et al., Making Hard Decisions (Uses Palisade Decision Tools suite. The book discusses Influence Diagrams but not Bayesian Networks.)

Further studies after completion of this book include:
Cowell, et al., Probabilistic Networks and Expert Systems
Lauritzen, Graphical Models
Pearl, Probabilistic Reasoning in Intelligent Systems
Pearl, Causality

4 Not worth the money
Chapter 1 is a nice introduction to probability. Chapter 2 is readable. Chapter 3 is poorly presented, and you feel sad for having wasted so much money on a book with only one intelligible chapter.

5 Hard to read, lacks clarity
Boy, this book is HARD to read for somebody without a PhD in Statistics. The language oftentimes is far from clear. Examples and explanations are not obvious at all. The author makes a lot of implicit assumptions. I had to get a few other books on Bayesian networks in order to understand author's explanations.

6 Just what I needed
I found this book to be an excellent introduction to the material, at precisely the right level for me. I don't have much of a background in either Bayesian networks or probability theory, and I've been looking for book with more depth than the typical introductory AI textbook. Jensen's book provided the practical and thorough introduction I needed.

It's certainly not an easy book. While the text is mostly clear and concise, there are occasional lapses and jumps that I attribute to language difficulties. But with concentration and focus, the information is all there.

Definitely a book in which it's best to work through the exercises at the end of each chapter.

7 You need know about the subject to understand the book
You need to understand the subject to understand the book. But if you already understand the subject, why do you need to read the book?

Poorly organized and written. Theorems based on terms are given before the definitions are supplied. The introductory example doesn't explain what is going on.

2 stars: read if you need the subject and this is the only book on the subject; but otherwise read something rated at least 3.