The situation:
- Main network and private network (physically separate in this case)
- Main network has dhcp server
- Private network has dhcp server (server A, OPNsense in this case), which also acts as a router to main network
- Want to set up a separate system (server B) that we can ssh, rdp etc. into from main network to access private network
- Desire to use dhcp addressing for both interfaces on server B
A problem can arise with this setup if the system uses server A as it’s default gateway for whatever reason. The outside network only sees the route to server B via server A, and when you try to remote access server B from the main network (using the appropriate IP address for the main network interface) it fails, as it routes via server A and doesn’t go anywhere.
In theory this could be fixed by telling server A not to advertise itself as a gateway. In this case we want it to act as a gateway for everything on the private network except this one system, so we’d like to configure dhcpd on server A to exclude the option routers for this one case. For a static mapping entry on OPNsense the gateway option appears to imply you can type none here and it will unset the default gateway (in the same way that setting this at the main dhcp page for the interface does). This does not in fact happen – see the forum discussion here for example.
There are workarounds:
- Edit the dhcpd conf file manually. Example
- Unset the option for the server and then include it in all static entries (but this doesn’t cover dynamic entries)
Alternatively, we can change the behaviour of server B. As this is a one-off issue this may be preferable. We could just set up everything as static addresses, but there is an arguably more elegant way – use interface metrics.
In this case we have two Ethernet interfaces, both configured by /etc/network/interfaces and obtaining their addresses by dhcp. Both will have a metric of zero (running ip route shows this – if there’s no metric number then it is 0 by default). The default gateway is probably chosen by the first dhcp server to answer or some other semi-random condition. We can alter the metric of one (or both) interfaces to prioritise the one we want. If we have interfaces ens19 (main network) and ens18 (private network) for example, then an example is:
# This file describes the network interfaces available on your system # and how to activate them. For more information, see interfaces(5). source /etc/network/interfaces.d/* # The loopback network interface auto lo iface lo inet loopback # Main network allow-hotplug ens19 iface ens19 inet dhcp # Private network allow-hotplug ens18 iface ens18 inet dhcp metric 100
Adding the metric 100 line to ens18 deprioritises it (as ens19 should get the default metric 0). Restarting the networking with systemctl restart networking and then running ip route gives something like
default via 172.40.204.1 dev ens19
default via 192.168.60.1 dev ens18 metric 100
172.40.204.0/24 dev ens19 proto kernel scope link src 172.40.204.27
192.168.60.0/24 dev ens18 proto kernel scope link src 192.168.60.8
Accessing server B from the main network using 172.40.204.27 should then work properly. As a bonus if ens19 looses it’s connection then it should fall back to the private network gateway (although for our particular use case this isn’t important).