Thursday, September 21, 2006

Efficient serving using flow control

Generally, different parts of Kamaelia systems run at different throughputs. You may be able to read file faster than you can serve it to your clients (because they or you have slow network connections). They may not be able to decompress images at the rate they can read their data into memory. By slowing down data sources to reflect the rate at which later components can process/transmit that data, we can save a lot of memory.
Unpausing of components upon recipient.recv is due to be added to Axon in the next few weeks. This is very important for HTTPServer as it allows it to handle bottlenecks efficiently.
Three ways a file could be sent over a network:
  • Example A - without considering bottlenecks
    FileReader ---> TCPClient -> -> (slow network) -> -> TCPServer
    If the rate at which one can read a file from disk is faster than the rate at which the network can send (as it generally is), then the file ends up being stored in memory in TCPClient's inbox queue.

  • Example B - considering bottlenecks but without unpausing upon recv
    RateAdaptiveFileReader ---> TCPClient -> -> (slow network) -> -> TCPServer

    Now let us assume that we have a file reader that won't read any more from disk until the number of chunks of data in its outbox goes below a certain value. If the file reader pauses when its outbox length reaches this value, having nothing to do, it won't be woken when messages are removed from its outbox, so most of the file will never be read. Thus the file reader cannot pause, so it has to poll its outbox repeatedly. Polling is a bad thing - it uses large amounts of CPU unnecessarily. This is how HTTPServer currently works.

  • Example C - considering bottlenecks with unpausing upon recv
    RateAdaptiveFileReader ---> TCPClient -> -> (slow network) -> -> TCPServer
    As with Example B, except that the file reader can pause while it's waiting for the TCPClient to send stuff on. This is how HTTPServer is intended to be (and can be made to be with a few small changes).

Labels: ,

2 Comments:

Blogger michael said...

Agreed that polling is bad. You wouldn't have to do it if unpausing on data being taken out an outbox hadn't been left out when the optimisations happened. That's a bug, it needs fixing. End of story really, well, aside from writing, patching, testing, using. :-)

7:02 PM  
Anonymous Anonymous said...

I also agree polling is bad

8:52 PM  

Post a Comment

<< Home