Fast heterogeneous binary data interchange

As distributed applications have become more widely used, they more often need to leverage the computing power of a heterogeneous network of computer architectures. Modern communications libraries provide mechanisms that hide at least some of the complexities of binary data interchange among heterogeneous machines. However, these mechanisms may be cumbersome, requiring that communicating applications agree a-priori on precise message contents, or they may be inefficient, using both "up" and "down" translations for binary data. Finally, the semantics of many packages, particularly those which require applications to manually "pack" and "unpack" messages, result in multiple copies of message data, thereby reducing communication performance. This paper describes PBIO (Portable Binary Input/Output), a novel messaging middleware which offers applications significantly more flexibility in message exchange while providing an efficient implementation that offers high performance.

[1]  Karsten Schwan,et al.  DataExchange: High Performance Communications in Distributed Laboratories , 1998, Parallel Comput..

[2]  Message Passing Interface Forum MPI: A message - passing interface standard , 1994 .

[3]  Jack Dongarra,et al.  Pvm 3 user's guide and reference manual , 1993 .

[4]  Guy T. Almes,et al.  The Impact of Language and System on Remote Procedure Call Design , 1986, ICDCS.

[5]  Michael Burrows,et al.  Performance of Firefly RPC , 1989, SOSP '89.

[6]  Ian T. Foster,et al.  The Nexus Approach to Integrating Multithreading and Communication , 1996, J. Parallel Distributed Comput..

[7]  Jeffrey C. Mogul,et al.  The packer filter: an efficient mechanism for user-level network code , 1987, SOSP '87.

[8]  David D. Clark,et al.  Architectural considerations for a new generation of protocols , 1990, SIGCOMM '90.

[9]  Scott Pakin,et al.  Efficient layering for high speed communication: Fast Messages 2.x , 1998, Proceedings. The Seventh International Symposium on High Performance Distributed Computing (Cat. No.98TB100244).

[10]  Greg Eisenhauer Portable Self-Describing Binary Data Streams , 1994 .

[11]  Dawson R. Engler,et al.  VCODE: a retargetable, extensible, very fast dynamic code generation system , 1996, PLDI '96.

[12]  David Clark,et al.  Architectural considerations for a new generation of protocols , 1990, SIGCOMM 1990.

[13]  Beth Schroeder,et al.  Dataexchange: High Performance Communications in Distributed Laboratories 1 Communication Substrates for High Performance Applications , 1997 .

[14]  Todd A. Proebsting,et al.  USC: a universal stub compiler , 1994, SIGCOMM 1994.