Skip to content

Commit

Permalink
update tcp/server prose
Browse files Browse the repository at this point in the history
  • Loading branch information
Malini Das committed Dec 23, 2014
1 parent bd4b3d0 commit 0bad99c
Showing 1 changed file with 4 additions and 2 deletions.
6 changes: 4 additions & 2 deletions ci/chapter.rst
Original file line number Diff line number Diff line change
Expand Up @@ -114,9 +114,11 @@ The dispatcher is a separate service used to delegate testing tasks. It listens

When dispatch.py is executed, the 'serve' function is called. This starts the dispatcher server, and two other threads. One thread runs the 'runner_checker' function, and other thread runs the 'redistribute' function. The 'runner_checker' function is used to periodically ping each registered test runner to make sure they are still responsive. If they become unresponsive, then that runner will be removed from the pool, and its commit hash will be dispatched to the next available runner. It will log the commit hash in the 'pending_commits' variable. The 'redistribute' function is used to dispatch any of those commit hashes logged in 'pending commits'. When 'redistribute' runs, it checks if there are any commit hashes in 'pending_commits'. If so, it calls the 'dispatch_tests' function with the commit hash. The 'dispatch_tests' function is used to find an available test runner from the pool of registered runners. If one is available, it will send a 'runtest' message to it with the commit hash. If none are currently available, it will wait 2 seconds and repeat this process. Once dispatched, it logs which commit hash is being tested by which test runner in the 'dispatched_commits' variable. If this commit hash is in the 'pending_commits' variable, then it will remove it from this list, since it was successfully re-dispatched.

The dispatcher server uses the SocketServer module. The default TCPServer provided by SocketServer cannot handle the case where the dispatcher is talking to one connection, say from a test runner, and then a new connection comes in, say from the repository observer. If this happens, the repository observer will have to wait for the first connection to complete before it will be serviced. This is not ideal for our case, since the dispatcher server must be able to directly and swiftly communicate with all test runners and the repository observer.
The dispatcher server uses the SocketServer module, which is a very simple server that is part of the standard library. There are four basic server types in the SocketServer module, TCP, UDP, UnixStreamServer and UnixDatagramServer. We will be using a TCP based socket server so we can ensure continuous , ordered streams of data to between servers, as UDP does not ensure this.

In order for the dispatcher server to handle simultaneous connections, it uses the ThreadingTCPServer custom class, which adds threading ability to the default SocketServer. This means that anytime the dispatcher receives a connection request, it spins off a new thread just for that connection. This allows the dispatcher to handle multiple requests at the same time.
The default TCPServer provided by SocketServer can only handle one request at a time, and therefore cannot handle the case where the dispatcher is talking to one connection, say from a test runner, and then a new connection comes in, say from the repository observer. If this happens, the repository observer will have to wait for the first connection to complete and disconnect before it will be serviced. This is not ideal for our case, since the dispatcher server must be able to directly and swiftly communicate with all test runners and the repository observer.

In order for the dispatcher server to handle simultaneous connections, it uses the ThreadingTCPServer custom class, which adds threading ability to the default SocketServer. This means that anytime the dispatcher receives a connection request, it spins off a new process just for that connection. This allows the dispatcher to handle multiple requests at the same time.

The dispatcher server works by defining handlers for each request. This is defined by the DispatcherHandler class, which inherits from SocketServer's BaseRequestHandler. This base class just needs us to define the 'handle' function, which will be invoked whenever a connection is requested. The 'handle' function defined in DispatcherHandler is our custom handler, and it will be called on each connection. It looks at the incoming connection request (self.request holds the request information), and parses out what command is being requested of it. It handles four commands: 'status', 'register', 'dispatch', and 'results':

Expand Down

0 comments on commit 0bad99c

Please sign in to comment.