Skip to main content

Creating Resilient Protocols with Data-Oriented Transport

Researcher: Dave Andersen


Creating Resilient Protocols with Data-oriented Transport

Traditional Internet protocols for transferring data perform poorly or not at all in adversarial or challenging environments—those with maliciously induced failures, excessive packet loss, or frequent link failures that interrupt connectivity. This proposal describes Data-Oriented Transport (DOT), a novel, data-centric approach to data transfer that can greatly improve the availability and performance of these protocols under harsh network conditions.

DOT performs data transfers on behalf of protocols such as SMTP and HTTP, providing two benefits:

  1. Data retrieval via alternate means.
  2. Forward progress despite interruptions.

DOT treats a data transfer as a set of uniquely self-identifying chunks of content, identifying them by the hash of their contents and first transmitting this list of hashes to the recipient. Each chunk can be independently received and verified from the source, from another entity that has the same content, or from the recipient’s own cache.

By providing a common data transfer method that is centered around the identity of the data being transferred,

DOT unchains data from a particular source: receivers can obtain the data from any available data source in the network. In doing so, DOT enables data transfers to succeed despite the failure of the original sender, or the network link between sender and receiver. Because the recipient can receive data chunks from its or its neighbors caches,

DOT can help reduce the amount of data copied across links burdened by low bandwidth (e.g., satellite links or cellular links) or floods of traffic.

This ability to retrieve data is not limited to just alternate paths, but can also extend to alternate media. As Andy

Tanenbaum famously noted, “Never underestimate the bandwidth of a station wagon full of [storage] hurtling down the highway.” This adage is truer today than it was in 1988: The capacity of storage devices has been increasing more rapidly than has the speed of the average user’s network connection, particularly in challenged environments. A major goal of DOT is to allow a user to transparently assist a file transfer by physically transporting storage from one location to another. When the storage is “docked” at a location closer to the recipient, the transfer can immediately take advantage of this new, better data source. Physical media transfer is a valuable back-up mechanism in failure or jamming-prone environments; DOT aims to make this mechanism transparent.

When used underneath a protocol such as SMTP which re-transmits failed email messages in the whole, DOT provides a way for partial transfers to still make forward progress. The ability to use a series of partial successes can be critical to making useful progress in an environment in which connections fail regularly. In an adversarial environment, an attacker can easily stop forward progress by interrupting transfers just before they complete (similar timing-based attacks have been demonstrated, for example, against TCP [2]).