Progress report
It's been a while since last update, so I wanted to show a little bit of what's happening in the repositories.
Read moreProgress report: Audio services, congestion control
network report structured-secure-streams uvvy
It's been a while since last update, so I wanted to show a little bit of what's happening in the repositories.
Read moreOur lives are increasingly mediated by, filtered through, and experienced via technology. A plethora of mechanisms capture, store, examine, manipulate, derive intelligence from, and share our information. This happens either with our knowledge, consent, and intent or without.
Aral Balkan site
Structured Secure Streams provide secure encrypted and authenticated data connection between endpoints. It's a simple userspace library written in c++14 and boost. It uses standard UDP to provide reliable delivery, multiple streams, quick connection setup, end-to-end connection encryption and authentication.
SSS is based on experimental, unfinished project under UIA - SST.
SSS is an experimental transport protocol designed to address the needs of modern applications that need to juggle many asynchronous communication activities in parallel, such as downloading different parts of a web page simultaneously and playing multiple audio and video streams at once.
Read moreAbsolutely every network needs a way to find a node's peers, for establishing connections, propagating updates and maintaining network integrity this is essential.
Overlay networks based on DHT or similar mechanisms (Kademlia, Chord) use peer nodes to figure this information out. Usually in such network you have a node ID, 128 or 160 bits in length, which uniquely identifies the node and node's position in routing tables. Based on this ID you simply look up the IP address. But wait, look it up from where?
Read moreStructured Secure Streams library is getting in shape. With C++11 and Boost it's relatively simple to write even without the tremendous help of Qt. Although I had to write some helpers which are factored into a separate library - grab arsenal if you want - some nice things there are byte_array, settings_provider, binary literals, hash_combine, make_unique, contains() predicate, base32 encoder-decoder and opaque big/little endian integer types.
To continue work on the structured streams transport I decided to get rid of the XDR data representation, as well as slightly awkward boost.serialization library.
Just a very simple thing to try and build Clang with polly, LLVM, libcxx, lldb and lld from trunk. Isn't it?
Using git, because cloning a git repo with full history is still faster than checking out svn repo with serf. Yay!
These instructions are not for copy-paste, they show general idea and should work with minor changes.
Read moreHere's a new technology from BitTorrent that might be bad news because I'm working on about the same features in MettaNode and at the same time might be good news because my implementation is going to be open source from the start, and I still plan to use less of so-called servers in the infrastructure. But for the much loved p2p world it's absolutely awesome, gone to install and play with it.
PS. OSX client is pretty unstable, crashes, loses added folders, but that's apparently a beta.
PPS. It quickly went into proprietary no-sources-available mode, not relevant for me anymore.
In the mean time, I'm slowly rewriting Bryan Ford's SST (Structured Streams Transport) library, using modern C++ and boost.asio, in the hopes that it will be easier to port to Metta. I called it libsss (Structured Secure Streams).
As this work progresses I also plan to enter the description of this protocol into an RFC document, so there will be some reference point for alternative implementations. Current specification progress is available in libsss repo on github.
I'd like to take the chance to thank Aldrin D'Souza for his excellent C++ wrapper around openssl crypto functions. He kindly licensed it for free use under BSD License.
update: Oct 2014 repository moved.
Some issues that need tackling in design of file sharing (see Brendan's post here):
The issue of trust: right now, the file is only distributed across a range of devices you manually allow to access your data. This doesn't solve the problem per se, but just makes it easier to tackle for the initial implementation. The data and metadata could be encrypted with asymmetric schemes (private keys), but that doesn't give full security.
Read moreWhile I'm still dabbling with fixing some SSS issues here and there I thought I'd post an old excerpt from assocfs design document.
It's a non-hierarchical filesystem - in other words, associative filesystem. It's basically a huge graph database. Every object is addressed by its hash (content addressable, like git), knowing the hash you can find it on disk. For more conventional searches (for those who does not know or does not care about the hash) there is metadata - attributes, drawn from an ontology and associated with a particular hashed blob.
Read moreTurns out the problem was on the server side setup. After moving the server to Amazon EC2 cloud and setting up UDP firewall rules punching started working. At least that takes some burden off my shoulders. The regserver connection is not very robust, that should probably be modified to force-reconnect the session once you open the search window again.
Read moreThere's a slight fault with uvvy not quite punching through home routers' NAT. While the UDP punching technique described by Bryan Ford should generally work, it doesn't account for the port change hence the announced endpoint addresses as seen by the regserver are invalid. Responses don't go back because the reply port number is different from what router's NAT assigns.
Read moreOver time I’ve come to believe that software should not be patentable; most any idea in software is something that a moderately experienced programmer could invent, and the definition of patents requires that patents be non-obvious. Nuff said.
Patrick Wyatt of Blizzard
I've ported events, sequencers and event-based communication primitives from Nemesis. It's a little bit messy at the moment (mostly because of mixing C and C++ concepts in one place), but I'm going to spend the autumn time on cleaning it up and finishing the dreaded needs_boot.dot dependencies to finally bootstrap some domains and perform communication between them. Obviously, the shortest term plan is timer interrupt, primitive kernel scheduler which activates domains and events to move domains between blocked and runnable queues.
There's some interesting theory behind using events as the main synchronization mechanism, described here in more detail.
For the vacation time I've printed some ANSA documents, which define architectural specifications for distributed computation systems and is very invaluable source of information for designing such systems. The full list of available ANSA documents can be found here. A System Designer's Introduction to ANSA is available here. Good reading.