CSIE,NTUT,Taiwan1
Concurrent Server Design
Multiple-Threads v.s,
Single-thread
Chuan-Ming Liu
Computer Science and Information Engineering
Spring 2004,NTUT
TAIWAN
CSIE,NTUT,Taiwan2
Examples
Using Threads for Concurrency (TCP)
Single-Thread,Currency Servers (TCP)
Using Threads for
Concurrency (TCP)
CSIE,NTUT,Taiwan4
Multiple Threads
Master thread for connection requests
Slave threads for handling communication
with clients
Thread of execution:
an abstraction of independent computation
Single process can contain one or more
threads
CSIE,NTUT,Taiwan5
Threads vs,Processes
Creation of a new process using fork( )
is expensive (time & memory).
A thread (sometimes called a
lightweight process) does not require
lots of memory or startup time.
CSIE,NTUT,Taiwan6
Multiple Threads
Each process can include many threads.
All threads of a process share,
memory (program code and global data)
open file/socket descriptors
signal handlers and signal dispositions
working environment (current directory,
user ID,etc.)
CSIE,NTUT,Taiwan7
Thread-Specific Resources
Each thread has it’s own:
Thread ID (integer)
Stack
Registers
Program
Counter
Threads within the same process can
communicate using shared memory.
Must be done carefully!
CSIE,NTUT,Taiwan8
Posix Threads
We will focus on Posix Threads - most
widely supported threads programming API.
CSIE,NTUT,Taiwan9
fork()
Process A
Global
Variables
Code
Stack
Process B
Global
Variables
Code
Stack
fork()
CSIE,NTUT,Taiwan10
Thread Creation
pthread_create(
pthread_t *tid,
const pthread_attr_t *attr,
void *(*func)(void *),
void *arg);
Func() is the function to be called.
When func() returns the thread is terminated.
CSIE,NTUT,Taiwan11
Process A
Thread 1
Global
Variables
Code
Stack
Process A
Thread 2
Stack
pthread_create()
pthread_create()
CSIE,NTUT,Taiwan12
pthread_create()
The return value is 0 for OK.
nonzero error number on error.
Thread ID is returned in tid
CSIE,NTUT,Taiwan13
Thread IDs
Each thread has a unique ID,a thread can
find out it's ID by calling
pthread_self().
Thread IDs are of type pthread_t
which is usually an unsigned int,When
debugging it's often useful to do
something like this,
printf("Thread%u:\n",pthread_self());
CSIE,NTUT,Taiwan14
Thread Arguments
When func()is called,the value arg
specified in the call to pthread_create()
is passed as a parameter.
Func()can have only 1 parameter,and it
can't be larger than the size of a void *.
CSIE,NTUT,Taiwan15
Thread Arguments (cont.)
Complex parameters can be passed by
creating a structure and passing the
address of the structure,
The structure can't be a local variable (of
the function calling pthread_create)!!
- threads have different stacks!
CSIE,NTUT,Taiwan16
Thread args example
struct { int x,y } 2ints;
void *blah( void *arg) {
struct 2ints *foo = (struct 2ints *) arg;
printf("%u sum of %d and %d is %d\n",
pthread_self(),foo->x,foo->y,
foo->x+foo->y);
return(NULL);
}
CSIE,NTUT,Taiwan17
Thread Lifespan
Once a thread is created,it starts executing
the function func()specified in the call
to pthread_create().
If func() returns,the thread is terminated.
A thread can also be terminated by calling
pthread_exit().
If main()returns or any thread calls
exit()all threads are terminated.
CSIE,NTUT,Taiwan18
Linux Threads
Dynamic Creation,pthread_create( )
Concurrent execution,all threads appears to
execute at the same time
Preemption,voluntarily giving up the resource
Private local variables
Shared globals
Shared file descriptors
Coordinate and Synchronization functions
CSIE,NTUT,Taiwan19
Advantages of Threads
Increased efficiency
Lower context switching overhead
Shared memory
All threads can share a global memory
Easy to communicate and access data
between threads
CSIE,NTUT,Taiwan20
Disadvantages of Threads
More efforts on the synchronization
between threads due to shared globals
Thread safety maintenance
Thread safety,two or more threads call the
library function concurrently,the results
should be ok
Lack of robustness due to the
termination issued by any thread which
causes the whole process terminates
CSIE,NTUT,Taiwan21
int counter=0;
void *pancake(void *arg) {
counter++;
printf("Thread %u is number %d\n",
pthread_self(),counter);
}
main() {
int i; pthread_t tid;
for (i=0;i<10;i++)
pthread_create(&tid,NULL,pancake,NULL);
}
Shared Global Variables
CSIE,NTUT,Taiwan22
DANGER! DANGER! DANGER!
Sharing global variables is dangerous -
two threads may attempt to modify the
same variable at the same time.
Just because you don't see a problem
when running your code doesn't mean
it can't and won't happen!!!!
CSIE,NTUT,Taiwan23
pthreads API includes support for mutual
exclusion primitives that can be used to
protect against this problem.
The general idea is to lock something
before accessing global variables and to
unlock as soon as you are done.
Shared socket descriptors should be
treated as global variables!!!
Avoiding Problems
CSIE,NTUT,Taiwan24
pthread_mutex
A global variable of type
pthread_mutex_tis required:
pthread_mutex_t counter_mtx=
PTHREAD_MUTEX_INITIALIZER;
Initialization to
PTHREAD_MUTEX_INITIALIZER
is required for a static variable!
A mutex is dynamically initialized by
calling pthread_mutex_init
CSIE,NTUT,Taiwan25
Locking and Unlocking
To lock use:
pthread_mutex_lock(pthread_mutex_t &);
To unlock use:
pthread_mutex_unlock(pthread_mutex_t &);
Both functions are blocking!
CSIE,NTUT,Taiwan26
Semaphore
Semaphore (counting semaphore)
A synchronization mechanism to
generalize mutual exclusive to cases
where N copies of a resource are available
Initialized dynamically with function
sem_inithaving an argument which
specifies an initial count N
CSIE,NTUT,Taiwan27
Sem_wait and Sem_post
Thread calls sem_wait before
beginning to use one copy of the
resource
Thread calls sem_post when
returning its copy for other threads to
use
CSIE,NTUT,Taiwan28
Condition Variables
For a situation where
Set of threads is using a mutex for
accessing some resource
Once a thread acquires the resource,it
needs to wait for a particular condition to
occur
CSIE,NTUT,Taiwan29
Condition Variables
Pthreads API supports condition
variables,which allow one thread to
wait (sleep) for an event generated by
any other thread,
This allows us to avoid the busy
waiting problem.
CSIE,NTUT,Taiwan30
Condition Variables (cont.)
A condition variable is always used
with mutex.
pthread_cond_wait(pthread_cond_t *cptr,
pthread_mutex_t *mptr);
pthread_cond_signal(pthread_cond_t *cptr);
don’t let the word signal confuse you -
this has nothing to do with Unix signals
CSIE,NTUT,Taiwan31
Condition Variables (cont.)
A condition variable is a thread
synchronization mechanism used in
conjunction with a mutex,While it
waits on a condition,a thread
temporarily gives up its hold on the
mutex; the thread reacquires the mutex
after the condition is signaled
CSIE,NTUT,Taiwan32
Thread Summary
Threads are awesome,but dangerous,You
have to pay attention to details or it's easy to
end up with code that is incorrect (doesn't
always work,or hangs in deadlock).
Posix threads on LINUX provides support
for mutual exclusion,semaphore,condition
variables and thread-specific data.
CSIE,NTUT,Taiwan33
Example
The TCPmtechod.cexample
The call to accept() blocks the
server thread until a connection arrives
When a connection request arrives,a
slave thread is generated to handle the
connection
The newly slave thread executes the
procedure TCPechod() which is
passed to it when it is created
CSIE,NTUT,Taiwan34
Monitor and Control
The monitor in this example is
implemented as a separate thread that
executes procedure prstats()
Monitor and slave threads use the global
data structure,stats,to communicate
A monitor provides information on
demand and can allow the manager to
control the server
Single-Thread,
Concurrent Servers
(TCP)
CSIE,NTUT,Taiwan36
Data-Driven Processing
When I/O dominates the cost of
processing a request,synchronous I/O
provides apparent concurrency among
clients
TCP connection can be a form of I/O
The server can use the arrival of data to
trigger processing a request
Not timeslicing – unless the load is high
CSIE,NTUT,Taiwan37
Data-Driven with a Single
Thread
Single thread implementation requires less
switching between thread and process
context
Using asynchronous I/O with select
system call
A server creates a socket for each connection
and calls select to wait for the data arrival
Since select is for all kinds of I/O,it can
wait for new connections at the same time
CSIE,NTUT,Taiwan38
Process Structure
Server
OS
Application
Processes
(or threads)
Socket for
connection
requests
Sockets for
individual
connections
CSIE,NTUT,Taiwan39
Process Structure (cont.)
Select uses a bit mask to specify
which descriptor in the set is ready
Server uses the order in which
descriptors become ready to decide
how to proceed
Server uses the descriptors to tell the
master and slaves
CSIE,NTUT,Taiwan40
Example
The TCPmechod.c example
Using FD_ZERO,FD_SET,FD_ISSET,
and FD_CLR to manage the slaves and
master