ZeroTier’s root servers don’t do very much. They relay initial traffic between peers, help peers establish peer-to-peer connectivity, and act as a cache of identities and associated public keys. If you’ve typed “zerotier-cli listpeers” you’ve seen them. They’re called “planets” in the current version.
For four years two clusters of nodes affectionately named Alice and Bob (after the canonical names given to securely communicating parties in cryptography texts) have carried the entire ZeroTier network. When we say “for four years”, we mean it. They have been running the exact same binaries since they were brought online in November 2015. If it’s not broke, don’t fix it.
Since Alice and Bob came online in 2015 we’ve experienced continuous exponential growth. We are now adding more users per month than we added in our first two years. Around early-mid 2019 the old clusters were still working but were starting to approach their capacity limits. We started to research new configurations and hosting options.
Since 2015 the cloud hosting world has grown considerably. Prices have fallen and new options have appeared. One option that didn’t really exist (at a reasonable cost) in 2015 was infrastructure-grade bare-metal hosting. Today you can get fast bare metal hosts with 10gbps or higher network connections at major peering points for a few hundred dollars per month. For our new roots that’s exactly what we have done.
| root-lax-01 | Los Angeles | 185.180.13.82 | 2a02:6ea0:c815:: |
| root-mia-01 | Miami | 207.246.73.245 | 2001:19f0:9002:5cb:ec4:7aff:fe8f:69d9 |
| root-ams-01 | Amsterdam | 195.181.173.159 | 2a02:6ea0:c024:: |
| root-tok-01 | Tokyo | 147.75.92.2 | 2604:1380:3000:7100::1 |
These are hosted by three different hosting companies: datapacket.com, packet.com, and vultr.com. Spreading across multiple providers adds redundancy, especially given that each of these nodes is capable of servicing the entire global user base.
(If you are curious: we don’t use Amazon, Google, Microsoft, etc. for our roots because their bandwidth costs are at least 10X what we pay through bare metal hosting providers for the same class of network service.)
Overall the new roots are much faster and leave us with considerably more room to grow.
The old system did have one advantage though, at least for some users: its nodes were more geographically diverse. The new roots will unfortunately leave users in places like Africa, South America, and Australia with a bit higher latency.
This matters less than you might think. ZeroTier is a peer to peer system. Unless you’re behind a P2P-hostile firewall or NAT device contact with the roots is only required for initial connection setup. Once you have a peer to peer link your traffic should take the shortest available path between devices, bypassing the roots entirely.
ZeroTier 2.0 is under heavy development. 2.0 makes it easier to host your own independent root servers and also offers a new system for root location that will help us add new roots without burdening all hosts with the need to make announcements to all of them. Stay tuned for updates on 2.0 progress. ETA is currently late 2019.