Overlay Weaver

Overlay Weaver is an overlay construction toolkit, which supports overlay algorithm designers in addition to application developers.

For application developers, the toolkit provides a common API for higher-level services such as distributed hash table (DHT) and multicast. Applications relying on the common API depend no specific transport protocol, database implementation and routing algorithm.

The toolkit provides multiple routing algorithms, Chord, Kademlia, Koorde, Pastry and Tapestry. These algorithms could be implemented only in hundreds lines of code because of routing layer decomposition. Routing layer under the higher-level services has been decomposed into multiple components, routing driver, routing algorithm and messaging service. The decomposition also facilitates implementation of a new algorithm. A newly implemented algorithm can be tested, evaluated and compared on emulator, which can host hundreds of thousands of virtual nodes. It enables large-scale emulation and fair comparison between algorithms.

Keywords: Overlay Network, Peer-to-Peer (P2P), Distributed Hash Table (DHT), Large-scale Emulation

News

May 17, 2009

Version 0.9.4 released.

• Web interface of DHT shell can show a routing path on a geographical map (Google Maps). Screenshots and demo page has screenshots. Nodes on PlanetLab shows such a map: http://oweaver.nyuld.net:3998/
• Fixed a bug in Pastry implementation. It occasionally prevents a node from working on a real network.
• Berkeley DB Java Edition was updated from 3.3.75 to 3.3.82.

May 1, 2009

Version 0.9.2 released.

• Emulator uses thread pools to exploit parallelism between "control" commands.
• Kinds of useful thread pool implementations (ow.util.concurrent.*) were provided.
• Overload / clock-jump detection and adaptation mechanisms were improved.
• Chord chooses a better next hop in a finger table. It is along Chord papers.
• How background processes take intervals was changed in Pastry and Koorde.
• Apache XML-RPC (library) was updated from 3.1.1 to 3.1.2.
• Jetty was updated from 6.1.16 to 6.1.17.

March 31, 2009

Demonstration videos provided on the "Screenshots, Video and Demo" page.

• "Overlay Visualizer"
• "DNS Emulation with over 500 nodes on PlanetLab"

December 7, 2008

Version 0.9 released.

• Provides an Overlay-Weaver-based memcached implementation (bin/owmemcached). It is rather a technology demonstration than practical one today.
• A DHT shell can run as an OASIS-enabled service (web site) with a --oasis option. We can reach a node on PlanetLab via http://oweaver.nyuld.net:3998/.
• Jetty was updated from 6.1.11 to 6.1.14.
• Berkeley DB Java Edition was updated from 3.3.69 to 3.3.75.

January 6, 2008

DHT Access Tools 0.6 released.

• Put command accepts multiple key-value pairs.

[More ...]