Using Flow Rules To Direct Users to Services

The setup

I have a network with several services on it:

• Plex
• ZeroNSD
• Gitea

And I want to make sure a number of things are possible:

• All users can reach the above 3 services on their appropriate ports
• All users can resolve DNS.
• Users cannot see each other or interact with each other.

This is actually quite simple with flow rules in a way that is not affected by name or IP changes.

Identity

ZeroTier has the notion of universal identity which is a key pair that identifies your client, stored with the client is the private key, and the public key is used for interactions with external entities. With this identity you can authenticate yourself to networks; if you’ve used the product you’re probably familiar with what an identity is at its basics.

Flow Rules can be used to manage networks by identity by providing an authorization layer. The ztsrc and ztdest flow rule directives can be used to allow traffic to and from identities; something we’ll see used later on in this article.

Rule Evaluation

In ZeroTier Flow Rules, rules are evaluated to the end of the rule set unless a break, drop, or accept statement is received. This means:

break not ethertype ipv4;
accept;  

Will accept packets for ipv4 but will not accept ipv6.

Grouping

Flow Rules have a unique method of grouping themselves: simply left to right. What this means is that this will have a different effect than you think:

accept ztdest 1234567890 and dport 22 or dport 80;
drop;  

Parentheses aren’t allowed in the flow rules syntax, but if they were this would look something like:

accept ((ztdest 1234567890 and dport 22) or dport 80);
drop;  

The appropriate rule is:

accept ztdest 1234567890 and dport 22;
accept ztdest 1234567890 and dport 80;
drop;  

Our Rule Set

So, to get what we want assuming we want to have SSH access as well, we need to:

• Allow 22 and 53 over TCP and UDP on ZeroNSD
• Allow ports 22, 80, and 32400 over TCP on Plex. Also support multicast for DLNA operation.
• Allow ports 22 and 80 on TCP for Gitea.

So let’s go over what that would look like. Let’s assume these identities for the different hosts:

• deadbeef01is ZeroNSD, our DNS server.
• cafebead01is Plex, our media server.
• feedcafe01is Gitea, our code repository.

To get these identities, one can zerotier-cli info from the CLI on the host, or check inside ZeroTier Central:

Installing the Rule Set

Look for this dialog:

Then install this rule set in the control panel:

drop
  not ethertype ipv4
    and not ethertype ipv6
    and not ethertype arp;

drop not chr ipauth;

accept ztdest deadbeef01 and dport 22 and ipprotocol tcp;
accept ztdest deadbeef01 and dport 53;
accept ztsrc deadbeef01;

accept ztdest cafebead01 and dport 22 and ipprotocol tcp; 
accept ztdest cafebead01 and dport 80 and ipprotocol tcp;
accept ztdest cafebead01 and dport 32400 and ipprotocol tcp;
accept ztdest cafebead01 and chr multicast;
accept ztsrc cafebead01;

accept ztdest feedcafe01 and dport 80 and ipprotocol tcp;
accept ztdest feedcafe01 and dport 22 and ipprotocol tcp;
accept ztsrc feedcafe01;

accept ethertype arp;

drop;  

drop
  not ethertype ipv4
    and not ethertype ipv6
    and not ethertype arp;  

drop not chr ipauth;  

accept ztdest deadbeef01 and dport 22 and ipprotocol tcp;
accept ztdest deadbeef01 and dport 53;
accept ztsrc deadbeef01;  

drop;  

cap dns
  id 10
  accept ipprotocol udp;
  accept ipprotocol tcp;
  accept dport 53;
  ;

cap http
  id 11
  accept dport 80 or dport 443 and ipprotocol tcp;
  ;

cap htpc
  id 12
  accept dport 80 or dport 32400 and ipprotocol tcp;
  ;

cap admin
  id 42
  accept;

drop not chr ipauth;

# Let arp, ping to servers work.
accept ethertype arp or ipprotocol icmp4;

accept dport 53;

# accept;
break; # not drop, so we can use capabilities.  

cap dns
  id 10
  accept ipprotocol udp;
  accept ipprotocol tcp;
  accept dport 53;
  ;  

accept ipprotocol udp;
accept ipprotocol tcp;
accept dport 53;