Tiny Packet Programs

Today, there are several network `functions' needed to make any network work. These network functions can be broadly divided into control and data plane functions on the basis of the timescales at which they operate. Control plane functions typically operate at seconds to minutes (e.g., routing, traffic engineering, etc.), whereas data plane functions operate at packet and round-trip timescales (e.g., congestion control).

With SDN, the path to introducing new control plane functions is conceptually clear: Write new software running on commodity servers, controlling commodity switches through an open interface such as OpenFlow.

For new data plane functions that typically need hardware support for performance reasons, the path is less clear. You are stuck with whatever your vendor gives you. Moreover, for tasks such as congestion control for which there is no one right answer, what do you do even if you own an ASIC team?

We think TPPs strike a delicate balance between what is possible in hardware, across vendors, across device characteristics, while being sufficiently powerful to break the impasse in introducing new data plane tasks.

In the paper, we show how you can implement the following tasks simply by having new software at the end-hosts sending specially crafted TPPs:

• Congestion control: Rate Control Protocol (max-min fair version), Rate Control Protocol (proportional fair version).
• CONGA: A distributed network traffic load balancing mechanism available in Cisco's next-generation ASICs. If you have a TPP-enabled network, you can get benefits similar to CONGA just by using TPPs.
• NetSight: A platform for detailed diagnosis of packet provenance in a large network. You can learn more about NetSight in our NSDI 2014 paper (pdf).
• Performance diagnosis: Identifying links that have high queueing occupancy for a short period of time (also known `micro-bursts').
• High speed, low-overhead monitoring: A detailed breakdown of per-link latency for any specific packet.

We think TPPs are a good fit for a number of other data plane functions below, but we haven't had a chance to implement them yet:

• Other congestion control schemes: XCP, FCP, QCP, MCP, EyeQ, or perhaps feeding richer signals into Remy.
• Fast network updates: If TPPs take half a round-trip time to update network state (once the new state is computed, of course), can we mitigate the effect of transient network state?
• DeTail: Reducing tail latency for flows by quickly switching to alternate least-congested paths (pdf).
• Packet drop location: Figuring out exactly where a packet was dropped, either due to congestion, or due to corrupted bits.
• Dynamic flow prioritisation: Today, flows are prioritised into bins statically based on headers. With TPPs, one can collect finer statistics such as link utilisation, flow size distribution (on a per-link basis) to dynamically set flow prioritisation thresholds on a per-link basis. With a low-latency interface, these configurations could potentially be updated quickly in response to traffic changes.

If you have new ideas, please let me know!

Despite the benefits, the TPP is not a panacea. You cannot do the following tasks using TPPs, although there might be ways you can get reasonably close to achieving similar functionality.

• Per-packet scheduling: TPP does not affect packet scheduling at a per-packet level, so you cannot implement say deficit round robin.
• Event-driven notifications: The TPP specification doesn't enable notifications whenever the network changes its state. For instance, you can't get notifications and take action when a switch updates its forwarding entry in hardware. Alternatively, if your goal is to take real-time actions on changes to network state, you can simply poll the switch state.
• Define new header schemes: You can't use TPPs to tell the switch how to interpret new header formats (say IPv7). This paper might be a better fit for that task: forwarding metamorphosis.