Parallel and Distributed Programming Using C++

Cameron Hughes | Tracey Hughes | Tracy Hughes

1 Don't worth the money
This book is a collection of many topics about MPI, PVM, Pthread, UML, CORBA, etc. None of them explains thoroughly. All the parts don't have coherence and don't have a complete example about parallel and distributed application. Readers would be better off just Google these topics and will find free documents which are far better than this book. It wastes your money to buy this book.

2 Too shallow and wide-focused
I gave it 3 stars only because there are few books on the subject; it deserves only 2.

The main problem I see with this book is that it tries to cover too much ground, even with subjects that it should not touch upon: there are whole chapters (7 and 9, for example) that are more about C++ than parallel or distributed programming, and one chapter (10) is about UML. Although it is interesting to see C++ and UML techniques applied to parallel and distributed programming, this leaves little space left for the coverage of MPI, for example. PVM and pthreads are given somewhat more attention, but still not nearly enough.

Another problem is that the examples are mostly incomplete code fragments. There are few complete, running examples. I hoped to see bigger examples in the last chapters (they seem like case studies), but again they are made of too much "talk" and no code.

The chapters are mostly independent of one another; it can be good if all you need is a quick first read on one of the subjects, but it gets annoying if you try to read it sequentially. Exercise for the reader: count how many times the PRAM model is cited AND explained.

To say something good about it, the book has some nice general musings about the nature and fundamental problems of concurrent systems. But it is not a coherent whole and is not particularly good at explaining any of the myriad subjects it touches upon.

Bottomline: It's neither an applied book about specific technologies, nor a general treatment. I would not buy this book unless it were very cheap. Take a look at it at the bookstore or borrow from a library, but don't spend your money.

To people wanting a general treatment about concurrency, it's principles, problems and solution space, I recommend "Concepts, Techniques and Models of Computer Programming" by Peter van Roy and Seif Haridi. It's thoroughly educational, and half of it is about concurrency (but no specific treatment of pthreads, PVM or MPI).

3 Excellent introduction to the topics. Fills a real need.
C++ has no keyword support for parallelism. One must rely on libraries for parallel and distributed applications in C++. This turns out to be a good thing because the C++ programmer isn't stuck with only one concurrency model supported by the language. Instead there are several standard options: POSIX (pthreads), MPI, PVM and CORBA. This book relies on the new Single UNIX Specification Standard which encompasses the POSIX standards for UNIX/Linux. You won't find anything in here that is specific to Microsoft Windows programming or the WIN32 API.

This book is a thorough and readable introduction to its topics. Its main strength its emphasis on design. Parallel and distributed applications are becoming the norm and doing them right impacts every phase of the software development process. Good, up-front design is necessary to avoid the serious problems one can encounter in this environment. The authors do a very effective job in helping the reader to first think about the problem they are trying to solve in terms of the inherent complexities and problems that parallel programing has over the sequential programming model that many programmers are used to.

Coverage of each topic in the table of contents is very well done. It offers just the right amount of background information, code examples and well designed diagrams and tables to give the reader a good understanding of concepts and good design, not just a "how to" explanation. UML diagrams are put to good use. There's a small appendix covering the UML notation used in the book and a larger one that contains manual pages for the POSIX API for process and thread creation and control.

Overall an excellent book. It would make a good textbook for students and a good tutorial and reference for experienced programmers.

4 Excellent introduction to parallel programming
I'm currently an undergraduate computer science student here at Youngstown State University, and in the course of research work for a Physics professor here, I have spent the last year struggling with learning what were very new concepts to me. I actually know Cameron in "real life," and on the occaisons I would run into him and try to get some advice on how to get my MPI scripts running correctly, etc., his reponse would invariably be "wait for the book, it will all be in there, I promise." Muddling through several poorly-written primers on programming for our cluster, and material on programming scattered on various part of the web, I can emphatically state my excitement that the book is finally done.

I consider myself to be very much an amateur programmer, which is important. With the increasingly widespread use of inexpensive computer clusters, many amateurs like myself find themselves in the position of having to learn something completely new. Moreover, many researchers don't particularly want to invest a great deal of time in something that is little more than a tool for them to get their "real" work done. For those of us who fall into that category, I think Camereon's book is a godsend. It covers all of the major areas of interest in a technically thorough way, without being too challenging or offputting for a beginner. The book starts with some of the basics of how a UNIX(Linux) system works, and how parallelism and multiproccessing fit in. By the time you start coding, you have a firm grasp of just what is going on. If you're familiar with C++, but have never done any parallel programming, this book is for you.

My personal opinion is that anyone becoming a new cluster user(They've been regularly doling clusters out here in Ohio) should recieve a copy of this book. A wonderful introduction. My only real complaint has been that Cameron didn't write it a year ago :-D

5 Covers the topics VERY well
This book covers the MPI (Message Passing Interface) library, the PVM (Parallel Virtual Machine) library, and the MICO (CORBA) library. If any these libraries were new to you, then this book is would be very beneficial to you.

So many books have been written on PTHREADS, threading, multiprocessing, etc, and yet, this book seems to cover more grounds, teach more concepts, and show you more simple but useful examples that any other one's that I have seen. The books on PTHREADS cover threading, there are a number of books on multiprocessing, networking (Stevens book comes to mind), but as far as generic multiprocessing, and parallel processing is concerned, this book is the key. Hughes and Hughes have written another similar book, but that book mainly covered C++ and threading (hence the title of the book). This book covers other great C++ tools such as the PVM (parallel virtual machine), which to my limited knowledge has not been covered greatly in any other books.

The first couple of chapters cover basic Operating System stuff. I don't know if it's because I have working for a number of years, or not, but the it actually made sense. We all have read the ever popular Modern Operating Systems book, but that book is somewhat dry. Maybe I think that say since I HAD to read the book for my class, but that feeling hasn't changed. This book covered the topics that you NEED to know, if you want to be able to write a program that takes advantage of some of the libraries mentioned above (MPI, PVM, MICO, and PTHREADS). It talks about the benefits, differences, and ideas behind parallel and distributed programming.

The next two chapters are probably the best chapters I have read on the topics of Multiprocessing and Multithreading respectively. The chapters are titled "Dividing C++ Programs into multiple tasks," and, "Dividing C++ Programs into multiple Threads."
I cannot say enough about how good these two chapters cover the topics. They start by the definition of both models, and they peel the onion as each chapter moves along. By the end of the chapter, you have seen examples, different design models for each method, and the pros and cons of each design model. Following these two chapters, a chapter is dedicated to synchronization and issues that go along with that in multithreaded/multiprocessing applications.

PVM, the Parallel Virtual Machine, is covered in chapter 6. This chapter must be my favorite chapter in the entire book. The authors give the following description for PVM:
"The PVM represents a collection of networked computers to the developer as a single logical machine with parallel capabilities."

The authors goes further and says:
"The PVM library is perhaps the easiest way to add parallel programming capabilities to the C++ language."

In general, the PVM is broken down into seven categories:
i) Process Management and Control: Routines used to manage and control PVM process
ii) Message Packing and Sending: Routines used to pack messages into a send buffer and send messages from one PVM process to another
iii) Message Unpacking and Receiving: Routines used to receive messages and unpack the data from the active buffer
iv) Task Signaling: Routines used to signal and notify PVM processes about the occurrence of an event
v) Message Buffer Management: Routines used to initialize, empty, dispose, and otherwise manage buffers between PVM processes
vi) Information and Utility Functions: Routines used to return information about a PVM process and perform other important tasks
vii) Group Operation: Routines used [in] joining, leaving, and otherwise managing processes in a group.

The authors then go in to detail covering the two programming models used commonly with the PVM: SPMD (Single Program Multiple Data), and MPMD (Multiple Program Multiple Data):

"In the SPMD model, the tasks will execute the same set of instructions but on different pieces of data. In the MPMD model, each task executes different instructions on different data"

The topics covered in PVM also apply to the chapter that covered MPI, the Message Passing Interface covered in Chapter 10. The authors go a step further in this chapter and talk about how some of C++ features such as polymorphism, generic programming and templates, and operator overload can be used in conjunction with the MPI to make programming simpler and more efficient.

The authors go into detail in the next few chapters about the Software Engineering aspects of the topics covered. Modeling aspects are covered first, and then applications and design. UML is used to show the readers how to represent concurrent and parallel designs. Flow charts, activity diagrams, class diagrams and state diagrams are used to represent the examples that were covered throughout the book.

A closer look at C++ and Object Orientation covers some advanced topics in the book. Various C++ class types are used in examples to show the modeling aspect and then the designing aspect of concurrent and parallel programming.

The last chapter of the book uses all the concepts covered in the book to implement a full-blown blackboard application. The BB is an approach to collaborative problem solving. The BB is used to record, coordinate, and communicate the efforts of two or more software-based problem solvers. The authors design and implement the BB using various architectural methodologies covered in the book: using the PVM, Threads, and CORBA. The examples certainly put everything that were covered in the book in perspective and enable the user to understand and grasp the topic at hand very simply.

C. Hughes and T Hughes picked a very difficult topic to cover and they did one fantastic job at doing so. The examples are great. The topics are covered very thoroughly, and yet are very understandable. Any series C++ programmer, developer, or architect would benefit lots from this book.

6 Great Book!!
Neat approach when discussing parallel and distributed programming, very good book. Keep up the good work.

Raise the right hand.

7 Concentrate on top-down design
If you have ever battled solving a deadlock in a parallel or distributed environment, you may have found that such problems can be qualitatively different and sometimes much harder than fixing a "simple" sequential program. In the latter, you can follow through in a stepwise fashion. And there are now powerful Integrated Development Environments to aid you in this. Most of us have little experience with truly distributed computing. I exclude much client-server or web access code. While a browser or client is indeed physically separated from the server that it queries, in practice, most of the logic on either end of the network connection is conventional sequential logic. Far from being a drawback, this is in fact a crucial simplifying point that lets many easily program.

But there comes a time when you have a problem that needs a truly distributed or parallel approach. A danger is that if you are not fluent in such an outlook, then debugging is HARD. Even worse, if you are designing a system and bungle it, this just spawns hard bugs down the road. (Keeps somebody employed, perhaps!) What the authors offer here is a comprehensive top-down methodology that starts by attacking the latter. They stress that picking the right software architecture to implement is vital. They show how this can be researched, to find the best fit to your problem.

Most of the book necessarily deals with the particulars of network coding. Semaphores, multithreading, different address spaces... Voluminous and understandable. But these are tactics. For the greatest leverage of your time, the introductory chapters on top level issues may be the most useful. Strategic.