In order to implement router style semantics, Vyatta allows setting many different configuration variables and then applying them all at once with a commit command. Currently, this is implemented by a combination of shell magic and unionfs. The problem is that keeping unionfs up to date and fixing the resulting crashes is major pain.

There must be better alternatives, current options include:

• Replace unionfs with aufs which has less users yelling at it and more developers.
• Use a filesystem like btrfs which has snapshots. This changes the model and makes api's like "what changed?" hard to implement.
• Move to a pure userspace model using git. The problem here is that git as currently written is meant for users not transactions.
  
• Use combination of copy, bind mount, and rsync.
• Use a database for configuration. This is easier for general queries but is the most work. Conversion from existing format would be a pain.
  

Looks like a fun/hard problem. Don't expect any resolution soon.

Added CLI support for TCP MD5 (via Quagga) to the upcoming Vyatta release. It worked fine under testing (VM) but wouldn't operate with IOS. Reduced the problem down by making some useful utilities:

• Patch for Netcat to support MD5
• Standalone using libpcap to validate MD5 option in capture file

It turned out that the sender was generating wrong MD5 option after the initial SYN handshake. When data is finally sent, the problem is that the data in the kernel is fragmented because the underlying device supports scatter/gather but the md5_calc doesn't do scatter gather.

I have high hopes for the first Linux Plumbers Conference. Unlike an academic conference with papers, or an un-conference with no agenda; the plumbers conference is using a mini-conference format to break down by topic. There is even a Call For Speakers to get speakers in topic areas.

First time conferences have a different feel, more rough edges, but more passion and fun. So I hope it works out. There is no particular networking track, mostly because the other areas seemed to need more work.

For the next release of Vyatta, I wanted to enable the Trie algorithm for routing in our kernel. Since FIB Trie is compatible with the previous hash, I expected no change. Well the day after enabling it caused an immediate failure in the regression test. The regression test plays back a full BGP input stream into the router and polls for the result. The problem was that the Trie to a long time, ... a really long time, to dump the routes. For the full 163395 routes, the dump was taking 20 seconds vs 1/2 sec for FIB_HASH.

As expected the problem turned out to be an N^2 lookup. The code was basically:
walk tree to find a route, and put it in buffer; if buffer is full, then give up, then go
back and walk to the last location. Since the trie has nice fast lookup function the change to just record the last route dumped (rather than offset), then use the lookup to find the location. This dropped the dump down to under 3/10 sec.

Finding this took several days, mostly because of looking at the profile; see "nextleaf" is the hotspot, so let's look at that. The real breakthrough came when I realized there were other operations that were walking the tree, like collecting stats, but they were fast. The next diversion was figuring out all the other suboptimal behaviour (ip route flush calls fflush for each route), which although slow weren't the real issue.

Started out for first day at Vyatta. There is a lot of overlap between what I know from doing iproute2 utilities so hopefully the learning curve won't be too steep.