Serial Experimentation


TL;DR: Using two Serial-to-USB adaptors to get a ~1Mbit point to point TCP/IP connection.


So, you know when you get an idea stuck in your head and you just need to follow it through to its conclusion to be able to move on?

I've watched some retro-Internet and telecom-history related Youtube recently and I was thinking about how Ethernet came along and ate like ... everything. Routers and telecom equipment used to deal with all sorts of framing and physical connectors and formats. X.25 (over X.21), ISDN delivered via Basic Rate or Primary Rate Interfaces (the latter being delivered over E1, at least in Europe), ATM (over SONET, which was itself used for transporting DS1 etc.).

The chances are very strong that your Internet access is delivered via PPPoE — meaning it's fundamentally the same protocol we used on dial-up modems, but stuffed into Ethernet frames and then (in my case) delivered via fibre-optic SFP to my switch.

We, the user, do not really benefit from this even being delivered over Ethernet. I presume it helps the ISPs, but for my purposes this connection is point-to-point.

You know what else is point-to-point? Serial ports.

So then I was thinking how fast could a serial port actually go? What if in an alternative history we just got faster and faster serial to deliver our Internet, instead of terminating DOCSIS or ADSL or Ethernet into our homes? (I know that the bandwidth of voice telephone networks wouldn't have kept pace — but I had ISDN during a brief window between dial-up and while ADSL was struggling to be born — and it was also delivered over "really-quite-fast-RS-232-with-PPP").

I know that RS-422 could go faster than RS-232, and it's specified as two differential pairs. This makes is significantly more immune to stray electro-magnetic noise. (The serial port of classic Apple Macintosh computers was RS-422). Wikipedia says 10 megabaud is possible, and even in the 90's Macs ran at 230Kbps for LocalTalk.

I started browsing AliExpress, deciding how much money I would be willing to spend on the totally pointless folly of "having an insanely fast serial port" and saw that many of the adaptors use the CHG340 chipset, which I'm familiar with from RS-232. Then I looked up the datasheet and realised these are exactly the same chipset as the RS-232 adaptors. A little more searching showed me that RS-422 shows up exactly the same way as an RS-232 port on Linux (and presumably, on Windows, Solaris, etc.) — could I just set a really high baud rate on some USB serial adaptors that I already have?

Modern Linux has a non-POSIX termios2 interface which lets one specify arbitrary baud rates! I know from working on microcontrollers that not all baud rates are possible on all hardware: it's going to depend on the speed of the clock on the device — baud rate is usually determined by taking the basic clock rate and dividing it down to arrive at the bit rate that you need. This means choosing rates which are multiples of the common (9600, 57600, etc.) rates or which are easily divided from common clocks (like 16MHz) would work best.

I have both FTDI FT232R and Silicon Labs CP210x USB to serial adaptors on hand. Looking at the Linux kernel sources the FT232R it seems it will try and support arbitrary baud rates up to 3 megabaud. Neat; but my version of these boards has mini-USB ports and only one of mine isn't already connected up to a project. The CP210x can do up to 2 megabaud depending on the variant. These have a full-size USB-A connector; much more friendly.

Okay! Let's wire them up crossed (i.e. TX and RX swapped on one side). Over these very short distances I'm going to assume EMI isn't an issue and because these are TTL serial adaptors they don't need to 15 volts in either direction, so that's going to help too.

RS-232 requires at least a 6 volt swing to register as mark or a space. "Real" RS-232 operates on +/- 15V, though these TTL adaptors tend to use 3.3V to be safe when interfaced with random microcontrollers. RS-422 only needs a 0.4V difference from ground to register a mark or a space, resulting in lower rise and fall times.

In this thought-experiment we're getting the Internet over these wires. I gotta say: I don't really want to set up a PPP link. I've done it, a bunch of times, but the purer version of this is SLIP. It's not going to require me to set one end as active and one end as passive and negotiate and all that modern stuff. It's just going to take IP packets in one end and send serial packets out the other.

We can install net-tools on Debian to get the slattach command that you need to set a serial port to be used as a SLIP device.

Then we can fall at the first hurdle:

raspberrypi:~$ sudo slattach -L -s 2000000 /dev/ttyUSB0
slattach: tty_open: cannot set 2000000 bps!

You will be shocked (shocked!) to learn that the SLIP tools from Linux are a little old fashioned and don't use the latest interface for talking to the serial port. Worse still, slattach has its own conditional code around baud rates and so it can only support rates that its authors included. These top out at the traditional 115200 baud which was the maximum the 16550A UART supported.

Let's try setting the speed with stty. Reading the code, it looks like slattach will just ignore the speed altogether if we don't supply the -s flag.

raspberrypi:~$ stty -F /dev/ttyUSB0 speed 1000000
9600
raspberrypi:~$ stty -F /dev/ttyUSB0
speed 1000000 baud; line = 1;
intr = <undef>; quit = <undef>; erase = <undef>; kill = <undef>; eof = <undef>; start = <undef>; stop = <undef>; susp = <undef>; rprnt = <undef>;
werase = <undef>; lnext = <undef>; discard = <undef>; min = 1; time = 0;
ignbrk -brkint -icrnl -imaxbel
-opost -onlcr
-isig -icanon -iexten -echo -echoe -echok -echoctl -echoke

stty prints the old values when the new value is accepted. I experimented with a few, but 1000000 (a million baud) was the most I could get working with the adaptors I have on hand.

Let's establish networking and check it works. Serial stuff was so easy! Enough to make a man nostalgic.

raspberrypi:~$ ip link show dev sl0
6: sl0:  mtu 296 qdisc noop state DOWN mode DEFAULT group default qlen 10
    link/cslip
raspberrypi:~$ sudo ip link set up dev sl0
raspberrypi:~$ sudo ip addr add 10.99.0.2/24 dev sl0
raspberrypi:~$ ping 10.99.0.1
PING 10.99.0.1 (10.99.0.1) 56(84) bytes of data.
64 bytes from 10.99.0.1: icmp_seq=1 ttl=64 time=3.54 ms
64 bytes from 10.99.0.1: icmp_seq=2 ttl=64 time=2.88 ms
^C
--- 10.99.0.1 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1002ms
rtt min/avg/max/mdev = 2.875/3.208/3.541/0.333 ms

Ah, the latency sucks. Maybe not so nostalgic after all. But it works! Also check out that 296 byte MTU. (We can change it with ip link set mtu 1500 dev sl0 as long as we do it on both sides).

Let's check if it's actually fast.

t480x:~$ iperf3 --client 10.99.0.2
Connecting to host 10.99.0.2, port 5201
[  5] local 10.99.0.1 port 39142 connected to 10.99.0.2 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec   136 KBytes  1.11 Mbits/sec    4   6.43 KBytes
[  5]   1.00-2.00   sec   166 KBytes  1.36 Mbits/sec    4   5.72 KBytes
[  5]   2.00-3.00   sec  0.00 Bytes  0.00 bits/sec    5   5.48 KBytes
[  5]   3.00-4.00   sec  86.7 KBytes   711 Kbits/sec    0   6.91 KBytes
[  5]   4.00-5.00   sec  85.1 KBytes   697 Kbits/sec    0   8.34 KBytes
[  5]   5.00-6.00   sec  83.9 KBytes   687 Kbits/sec    6   7.62 KBytes
[  5]   6.00-7.00   sec  83.4 KBytes   683 Kbits/sec    2   6.43 KBytes
[  5]   7.00-8.00   sec  83.4 KBytes   683 Kbits/sec    0   7.86 KBytes
[  5]   8.00-9.00   sec  83.4 KBytes   683 Kbits/sec    2   7.15 KBytes
[  5]   9.00-10.00  sec  84.1 KBytes   689 Kbits/sec    2   5.96 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec   892 KBytes   731 Kbits/sec   25             sender
[  5]   0.00-10.06  sec   840 KBytes   684 Kbits/sec                  receiver

It's interesting that from the iperf3 client (in my case my laptop) it exceeds the link speed before averaging out. We must be filling up a buffer somewhere for the first few seconds. 1 megabaud does not come out to 1 megabit of TCP traffic. That 296 byte MTU means we have 40 bytes of IP overhead for a 256 byte IP packet. Still: this is an Internet connection I would have killed for in the 90's.

Maybe there was a future where serial was as ubiquitous as Ethernet for this kind of thing? To get past a few megabit we'd probably have needed clock recovery and NRZI or Manchester encoding to ensure that the connection didn't sit at DC zero or one too long and we fall out of sync. Arguably, to make serial fast it would stop being serial and become this whole new thing. Could that thing be delivered over a tty or a Windows COM port? Maybe! But I suppose that in a world where we have to deal with Ethernet anyway, maybe it's okay that Ethernet has displaced so many others.

Aaron Brady — March 11, 2024