1
External Data
Representation
(XDR)
2
Introduction
Examines a de facto standard for external
data representation and presentation as
well as a set of library procedures used to
perform data conversion
Describes the general motivations for
using an external data representation and
the details of one particular
implementation
3
Introduction (cont.)
The next chapter shows how an external
data representation standard helps
simplify client and server communication
It illustrates how a standard makes it
possible to use a single,uniform remote
access mechanism for client-server
communication
4
Representation of Data
Fig,20.1 shows little endian (if it stores
the LSB of an integer at the lowest
memory address) and big endian
representation for 32-bit integers
Programmers who create client and server
software must contend with data
representation because both endpoints
must agree on the exact representation for
all data sent across the communication
channel between them; otherwise,
conversion is required
6
Asynchronous Conversion
and the N-Squared Problem
Designs that follow the paradigm of
converting directly from one representation
to the server’s representation are said to use
asymmetric data conversion because only
one side needs to perform conversion
7
Asynchronous Conversion and
the N-Squared Problem (cont.)
If client-server software is designed to
convert between the client’s native data
representation and the server’s native
data representation asymmetrically,the
number of versions of the software grows
as the square of the number of
architectures (N-Squared Conversion
Problem)
8
Network Standard Byte
Order
The TCP/IP protocols use the same
representation for all the data sent across
a network in a protocol header (headers
represent integer data in a standard,
machine-independent form)
Such designs employ symmetric data
conversion
9
Network Standard Byte
Order (cont.)
Symmetric data representation has an
important consequence,only one version
of protocol software is needed
The standard representation used for data
traversing the network is known as the
external data representation
10
Network Standard Byte
Order (cont.)
The chief advantage lies in flexibility,
neither the client and the server needs to
understand the architecture of the other
The total programming effort required
will be proportional to the number of
machine architectures
The chief disadvantage of symmetric
conversion is computation overhead even
if the client and the server both operate at
the same architecture
11
Network Standard Byte
Order (cont.)
Because the external representation may
add information to the data or align it
with word boundaries,the conversion
may result in a larger stream of bytes
than necessary
Most programmers agree that using
symmetric conversion is worthwhile
12
Network Standard Byte
Order (cont.)
It simplifies programming,reduces errors,
and increases interoperability among
programs
It also makes network management and
debugging easier because the network
manager can interpret the contents of
packets without knowing the
architectures of the sending and receiving
machines
13
A De Facto Standard External
Data Representation
XDR,Sun’s eXternal Data Representation
has become a de facto standard for most
client-server applications
XDR specifies data formats for most of the
data types that clients and servers exchange.
For example,XDR specifies that 32-bit
binary integers should be represented in,big
endian” order
14
XDR Data Types
Fig,20.2 lists the data types for which XDR
defines a standard representation
XDR allows the programmer to compose
aggregate types from other types
For example,XDR allows an array of
structure,each of which can have multiple
fields that can be an array,structure,or union
Thus,XDR provides representation for most
of the structure that a C programmer can
specify
16
Implicit Types
The XDR standard specifies how a data
object should be encoded for each of the data
types listed in Fig,20.2
However,the encoding contains only the data
items and do not contain additional bits to
identify their types
Thus,clients and servers using XDR must
agree on the exact format of messages they
will exchange
17
Software Support for Using
XDR
As studied before,one method programmers
can use to perform the conversion,insert a
function call in the code to convert each data
item in a message to external form (e.g.,
XDR) before sending the message,and insert
a function call to convert each data item to
internal form when a message arrives
To eliminate potential conversions,an
implementation of XDR includes library
routines that perform the necessary
conversions
18
XDR Library Routines
XDR library routines for a given
machine can convert data items from the
computer’s native representation to the
XDR standard representation and vice
versa
Most implementations of XDR use a
buffer paradigm that allows a
programmer to create a complete
message in XDR form
19
Building a Message One
Piece at a Time
The buffer paradigm XDR uses requires a
program to allocate a buffer large enough to
hold the external representation of a message
and to add item (i.e.,fields) one at a time
In Linux,a program first invokes procedure
xdrmem_create to allocate a buffer in a
memory and inform XDR that it intends to
compose an external representation it
Xdrmem_create initializes the memory so it
represents an XDR stream that can be used to
encode or decode
20
Building a Message One
Piece at a Time (cont.)
The declarations and calls needed to create an
XDR stream using C are shown in Fig,20.2a
Once a program has created an XDR stream,it
can call individual XDR conversion routines to
convert native data objects into external form
For example,a program invokes xdr_int by
passing it a pointer to an XDR stream and a
pointer to an integer,as shown in Fig,20.2b
Fig,20.3 illustrates how the call to xdr_int
shown in the sample code adds 4 bytes of data
to the XDR stream
24
Conversion Routines in the
XDR Library
The table in Fig,20.4 lists the XDR conversion
routines
The receiving application must calls
xdrmem_create to create a memory buffer that
will hold an XDR stream,and places the
incoming message in a buffer area
For example,if the receiver has established an
XDR stream used for input (i.e.,the call specified
XDR_DECODE),it can extract a 32-bit integer
and convert it to the native representation by
calling xdr_int,as shown in Fig,20.4
26
Conversion Routines in the
XDR Library (cont.)
Unlike the conversion,htons and ntohs
that Linux provides,individual XDR
conversion routines do not specify the
direction of conversion
Instead,they require the program to
specify the direction when creating the
XDR stream
27
XDR Stream,I/O,and TCP
The code fragments above create an XDR
stream associated with a buffer in memory
After items have been converted to external
form and placed in the buffer,the
application must call an I/O function like
write to send it across a TCP connection
It is possible to arrange to have XDR
conversion routines send data across a TCP
connection automatically each time they
convert a data item to external form
28
XDR Stream,I/O,and TCP
(cont.)
An application program first creates a TCP
socket,and then calls fdopen to attach a
standard I/O stream to the socket
The application calls xdrstdio_create to create
an XDR stream and connect it to the existing
I/O descriptor
Each time the application calls an XDR
conversion routine,the conversion
automatically performs a buffered read or
write operation using the underlying
descriptor
29
Records,Record Boundaries,
and Datagram I/O
The XDR mechanism works well when
connected to a TCP socket because both
XDR and TCP implementations use the
stream abstraction
To make XDR work with UDP,the
alterative design provides an application
with a record-oriented interface
To use the record-oriented interface,a
program calls function xdrrec_create when
creating a record-oriented XDR stream
30
Records,Record Boundaries,
and Datagram I/O (cont.)
The call includes two arguments,inproc
and outproc,that specify an input
procedure and an output procedure
When converting to external form,each
conversion routine check the buffer
If the buffer becomes full,the conversion
routine calls outproc to send the existing
buffer contents and make space for the
new data
31
Records,Record Boundaries,
and Datagram I/O (cont.)
Each time the application calls a
conversion routine to convert from
external form to the native representation,
the routine checks the buffer to see if it
contains data
If the buffer is empty,the conversion
routine calls inproc to obtain more data
32
Records,Record Boundaries,
and Datagram I/O (cont.)
To use XDR with UDP,an application
creates a record-oriented XDR stream
It arranges for the input and output
procedures associated with the stream
to call read and write (or recv and
send)
Record-oriented streams allow an
application to mark record boundaries
33
Summary
Client-server interaction can be
asymmetric or symmetric
Asymmetric conversion requires either the
client or the server to convert between its
own representation and the other
machine’s native representation
Symmetric conversions uses a standard
network representation and requires both
the client and server to convert between
the network standard and the local
representation
34
Summary (cont.)
Client and server programs can use XDR
routines to convert data to external form
before sending it and to internal form after
receiving it
The conversion routine can be associated
with input and output using TCP or UDP
Although some protocols in the TCP/IP
suite use ASN.1 (Abstract Syntax Notation
One,by ISO) representation,most
application programmers use XDR
External Data
Representation
(XDR)
2
Introduction
Examines a de facto standard for external
data representation and presentation as
well as a set of library procedures used to
perform data conversion
Describes the general motivations for
using an external data representation and
the details of one particular
implementation
3
Introduction (cont.)
The next chapter shows how an external
data representation standard helps
simplify client and server communication
It illustrates how a standard makes it
possible to use a single,uniform remote
access mechanism for client-server
communication
4
Representation of Data
Fig,20.1 shows little endian (if it stores
the LSB of an integer at the lowest
memory address) and big endian
representation for 32-bit integers
Programmers who create client and server
software must contend with data
representation because both endpoints
must agree on the exact representation for
all data sent across the communication
channel between them; otherwise,
conversion is required
6
Asynchronous Conversion
and the N-Squared Problem
Designs that follow the paradigm of
converting directly from one representation
to the server’s representation are said to use
asymmetric data conversion because only
one side needs to perform conversion
7
Asynchronous Conversion and
the N-Squared Problem (cont.)
If client-server software is designed to
convert between the client’s native data
representation and the server’s native
data representation asymmetrically,the
number of versions of the software grows
as the square of the number of
architectures (N-Squared Conversion
Problem)
8
Network Standard Byte
Order
The TCP/IP protocols use the same
representation for all the data sent across
a network in a protocol header (headers
represent integer data in a standard,
machine-independent form)
Such designs employ symmetric data
conversion
9
Network Standard Byte
Order (cont.)
Symmetric data representation has an
important consequence,only one version
of protocol software is needed
The standard representation used for data
traversing the network is known as the
external data representation
10
Network Standard Byte
Order (cont.)
The chief advantage lies in flexibility,
neither the client and the server needs to
understand the architecture of the other
The total programming effort required
will be proportional to the number of
machine architectures
The chief disadvantage of symmetric
conversion is computation overhead even
if the client and the server both operate at
the same architecture
11
Network Standard Byte
Order (cont.)
Because the external representation may
add information to the data or align it
with word boundaries,the conversion
may result in a larger stream of bytes
than necessary
Most programmers agree that using
symmetric conversion is worthwhile
12
Network Standard Byte
Order (cont.)
It simplifies programming,reduces errors,
and increases interoperability among
programs
It also makes network management and
debugging easier because the network
manager can interpret the contents of
packets without knowing the
architectures of the sending and receiving
machines
13
A De Facto Standard External
Data Representation
XDR,Sun’s eXternal Data Representation
has become a de facto standard for most
client-server applications
XDR specifies data formats for most of the
data types that clients and servers exchange.
For example,XDR specifies that 32-bit
binary integers should be represented in,big
endian” order
14
XDR Data Types
Fig,20.2 lists the data types for which XDR
defines a standard representation
XDR allows the programmer to compose
aggregate types from other types
For example,XDR allows an array of
structure,each of which can have multiple
fields that can be an array,structure,or union
Thus,XDR provides representation for most
of the structure that a C programmer can
specify
16
Implicit Types
The XDR standard specifies how a data
object should be encoded for each of the data
types listed in Fig,20.2
However,the encoding contains only the data
items and do not contain additional bits to
identify their types
Thus,clients and servers using XDR must
agree on the exact format of messages they
will exchange
17
Software Support for Using
XDR
As studied before,one method programmers
can use to perform the conversion,insert a
function call in the code to convert each data
item in a message to external form (e.g.,
XDR) before sending the message,and insert
a function call to convert each data item to
internal form when a message arrives
To eliminate potential conversions,an
implementation of XDR includes library
routines that perform the necessary
conversions
18
XDR Library Routines
XDR library routines for a given
machine can convert data items from the
computer’s native representation to the
XDR standard representation and vice
versa
Most implementations of XDR use a
buffer paradigm that allows a
programmer to create a complete
message in XDR form
19
Building a Message One
Piece at a Time
The buffer paradigm XDR uses requires a
program to allocate a buffer large enough to
hold the external representation of a message
and to add item (i.e.,fields) one at a time
In Linux,a program first invokes procedure
xdrmem_create to allocate a buffer in a
memory and inform XDR that it intends to
compose an external representation it
Xdrmem_create initializes the memory so it
represents an XDR stream that can be used to
encode or decode
20
Building a Message One
Piece at a Time (cont.)
The declarations and calls needed to create an
XDR stream using C are shown in Fig,20.2a
Once a program has created an XDR stream,it
can call individual XDR conversion routines to
convert native data objects into external form
For example,a program invokes xdr_int by
passing it a pointer to an XDR stream and a
pointer to an integer,as shown in Fig,20.2b
Fig,20.3 illustrates how the call to xdr_int
shown in the sample code adds 4 bytes of data
to the XDR stream
24
Conversion Routines in the
XDR Library
The table in Fig,20.4 lists the XDR conversion
routines
The receiving application must calls
xdrmem_create to create a memory buffer that
will hold an XDR stream,and places the
incoming message in a buffer area
For example,if the receiver has established an
XDR stream used for input (i.e.,the call specified
XDR_DECODE),it can extract a 32-bit integer
and convert it to the native representation by
calling xdr_int,as shown in Fig,20.4
26
Conversion Routines in the
XDR Library (cont.)
Unlike the conversion,htons and ntohs
that Linux provides,individual XDR
conversion routines do not specify the
direction of conversion
Instead,they require the program to
specify the direction when creating the
XDR stream
27
XDR Stream,I/O,and TCP
The code fragments above create an XDR
stream associated with a buffer in memory
After items have been converted to external
form and placed in the buffer,the
application must call an I/O function like
write to send it across a TCP connection
It is possible to arrange to have XDR
conversion routines send data across a TCP
connection automatically each time they
convert a data item to external form
28
XDR Stream,I/O,and TCP
(cont.)
An application program first creates a TCP
socket,and then calls fdopen to attach a
standard I/O stream to the socket
The application calls xdrstdio_create to create
an XDR stream and connect it to the existing
I/O descriptor
Each time the application calls an XDR
conversion routine,the conversion
automatically performs a buffered read or
write operation using the underlying
descriptor
29
Records,Record Boundaries,
and Datagram I/O
The XDR mechanism works well when
connected to a TCP socket because both
XDR and TCP implementations use the
stream abstraction
To make XDR work with UDP,the
alterative design provides an application
with a record-oriented interface
To use the record-oriented interface,a
program calls function xdrrec_create when
creating a record-oriented XDR stream
30
Records,Record Boundaries,
and Datagram I/O (cont.)
The call includes two arguments,inproc
and outproc,that specify an input
procedure and an output procedure
When converting to external form,each
conversion routine check the buffer
If the buffer becomes full,the conversion
routine calls outproc to send the existing
buffer contents and make space for the
new data
31
Records,Record Boundaries,
and Datagram I/O (cont.)
Each time the application calls a
conversion routine to convert from
external form to the native representation,
the routine checks the buffer to see if it
contains data
If the buffer is empty,the conversion
routine calls inproc to obtain more data
32
Records,Record Boundaries,
and Datagram I/O (cont.)
To use XDR with UDP,an application
creates a record-oriented XDR stream
It arranges for the input and output
procedures associated with the stream
to call read and write (or recv and
send)
Record-oriented streams allow an
application to mark record boundaries
33
Summary
Client-server interaction can be
asymmetric or symmetric
Asymmetric conversion requires either the
client or the server to convert between its
own representation and the other
machine’s native representation
Symmetric conversions uses a standard
network representation and requires both
the client and server to convert between
the network standard and the local
representation
34
Summary (cont.)
Client and server programs can use XDR
routines to convert data to external form
before sending it and to internal form after
receiving it
The conversion routine can be associated
with input and output using TCP or UDP
Although some protocols in the TCP/IP
suite use ASN.1 (Abstract Syntax Notation
One,by ISO) representation,most
application programmers use XDR
