Skip to content- Sources: primary
- Summary: Every Fly host used to hold a BPF map of all migrated machines, hundreds of thousands of entries fed from Corrosion, and hosts held very different counts, showing that sync was failing. Now a VXLAN unknown-address event reaches s6pnforwarderd, which queries local Corrosion state and writes one kernel route, leaving 100 to 200 routes per host, with udp-xdp only checking network membership and stamping an fwmark so the path fails closed. The cost is a first connection that can take about a second because of the TCP retransmit timeout, which Fly plans to fix by holding the first packets during lookup.
- Why it matters: It shows a concrete way to replace eventually consistent state replicated into the kernel with lazy lookups that fail closed and can be observed.
- Background: VXLAN carries Ethernet frames inside UDP so a layer 2 segment can span an IP network, and it needs a mapping from addresses to remote tunnel endpoints. eBPF programs cannot call into userspace, so any lookup table they use must be loaded ahead of time. Corrosion is Fly's gossip-replicated SQLite store for fleet state.