P2P Streaming Success

The P2P streaming example I've been working on for Kamaelia is now in a working state. It allows any octet stream to be distributed automatically using the BitTorrent protocol. I have tested it over a LAN and it correctly shared and reconstructed my source stream in real-time.

Octet Stream -> Seed -> Swarm
Swarm <--> Peer

Though further testing is required, this approach could theoretically scale to thousands of peers per seed/tracker.

Proposed system for subcomponent flexibility

Many Kamaelia components need to create their own subcomponents at run-time, e.g. my HTTP client spawns a TCPClient to connect to the server. However, this can cancel out much of the flexibility that a component-based architecture provides. As an alternative to hard-coding the class of each subcomponent, I propose that every subcomponent should be created from by a function provided as an argument at that components creation.

For example, in SingleShotHTTPClient:

def __init__(self, starturl, postbody = "", connectionclass = TCPClient):
...
self.connectionclass = connectionclass

And whenever a TCPClient was created in the old code, connectionclass is now called with the same parameters. The possibilities added by this are great:

• for debugging, TCPClient could be replaced with a local console masquerading as a TCP connection.
• TCPClient could also be replaced by a SOCKSTCPClient allowing it to go via a SOCKS proxy.
• for logging purposes, connectionclass could be a wrapper function (rather than a class) that wrote the host addresses accessed to a file before spawning and returning a TCPClient
• etc.
  

Most importantly, all these things could be done without having to change the code of HTTPClient.py. I propose that this practice should become standard.

Packaging and finishing touches

I finally received the initial GSoC payment last week - thanks to Google for getting it sorted.
I'm working on packaging up my Kamaelia components - including the Torrent, HTTP, Util and IRC directories of /Sketches/RJL. Documentation is almost complete and I'm making the final touches to my components with a view to getting it all done by the first week of August.

Then, with some Python magic you'll be able to do the following to use my HTTPClient, as if it was part of the main-tree:

import Kamaelia.Community.RJL.Namespace
from Kamaelia.Protocol.HTTP.HTTPClient import SimpleHTTPClient

HTTP and streaming

I've made some big changes to my HTTP code. HTTPParser no longer waits for the end of a response/request before sending information about it on - it now follows the format:

• HEADER
• BODYCHUNK
• BODYCHUNK
• ...
• BODYCHUNK
• END

This allows memory to be saved and lower latency processing. As a result of this modification, my HTTP client now allows HTTP streaming.

My HTTP client has also been refactored to support this change - it is now split into a SingleShotHTTPClient which handles a single URL before terminating and a SimpleHTTPClient which uses the SSHC with a Carousel component to handle several requests one after another. I've also added a Kamaelia component called IcecastClient which can stream MP3 audio from a SHOUTcast/Icecast server. As the SHOUTcast protocol is almost identical to HTTP, this component subclasses SSHC.

Documentation will be my next focus.