From Oscilloscope to Wireshark: A UDP Story (2022)

From Oscilloscope to Wireshark: A UDP Story

UDP is a transport-level protocol for sending messages through an IP network.

It sits at level 4 in the OSI model:

| 7 | Application | |||||| | 6 | Presentation | |||||| | 5 | Session | |||||| 4 | Transport 3 | Network | 2 | Data link | 1 | Physical | |

Like many of you, I've got hardware on my desk that's sending UDP packets, and the time has come to take a closer look at them.

Most "low-level" networking tutorials will bottom out somewhere at "use tcpdump

to see raw packets". We'll be starting a bit lower in the stack; specifically, here:

This is a high-speed active differential probe soldered to an Oxide Computer Company rack switch. We're going all the way down to the metal.

(Huge thanks to Eric for the careful soldering that made this possible!)

Looking at the signals on an oscilloscope, we see data zooming down the wires:

The rest of this post will take us from these raw voltage waveforms all the way to decoded UDP packets. Hold on tight, we're going from L1 all the way to L4.

First, a bit of context

I work at Oxide Computer Company, writing embedded software for a rack-scale computer.

Over the past few months, I've been focused on the management network, which is a low-speed network between each server's Service Processor. The service processor is roughly equivalent to a baseboard management controller; it allows for lights-out management of the rack.

The heart of the management network is the VSC7448, a 52-port, 80G ethernet switch chip. This is the slower switch, and it's still a beast; here's the dev kit:

This decoding work was part of tracking down a nasty bug which caused a subset of links to only work some of the time. The root cause turned out to be a misconfiguration of the switch IC, but the hunt was an interesting dive into the physical layer of modern networking.

Loading the waveforms

That's enough context, back to work!

The oscilloscope doesn't have a built-in QSGMII analyzer (and we'll want to do fairly sophisticated processing of the data), so I wanted to export waveform data to my computer.

How much data should I capture? Analog waveforms can easily add up to multiple gigabytes, so I'd like to capture a small amount while still catching a packet or two.

I knew that a device on the network was emitting about 30K UDP packets per second, or one packet every 33 µs. I configured the oscilloscope to collect 100M samples at 1 TSPS (tera-sample per sec"