Category: Networking

Choosing preferred gateway in linux with two interfaces configured for dhcp

The situation:

  • Main network and private network (physically separate in this case)
  • Main network has dhcp server
  • Private network has dhcp server (server A, OPNsense in this case), which also acts as a router to main network
  • Want to set up a separate system (server B) that we can ssh, rdp etc. into from main network to access private network
    • Desire to use dhcp addressing for both interfaces on server B

A problem can arise with this setup if the system uses server A as it’s default gateway for whatever reason. The outside network only sees the route to server B via server A, and when you try to remote access server B from the main network (using the appropriate IP address for the main network interface) it fails, as it routes via server A and doesn’t go anywhere.

In theory this could be fixed by telling server A not to advertise itself as a gateway. In this case we want it to act as a gateway for everything on the private network except this one system, so we’d like to configure dhcpd on server A to exclude the option routers for this one case. For a static mapping entry on OPNsense the gateway option appears to imply you can type none here and it will unset the default gateway (in the same way that setting this at the main dhcp page for the interface does). This does not in fact happen – see the forum discussion here for example.

There are workarounds:

  • Edit the dhcpd conf file manually. Example
  • Unset the option for the server and then include it in all static entries (but this doesn’t cover dynamic entries)

Alternatively, we can change the behaviour of server B. As this is a one-off issue this may be preferable. We could just set up everything as static addresses, but there is an arguably more elegant way – use interface metrics.

In this case we have two Ethernet interfaces, both configured by /etc/network/interfaces and obtaining their addresses by dhcp. Both will have a metric of zero (running ip route shows this – if there’s no metric number then it is 0 by default). The default gateway is probably chosen by the first dhcp server to answer or some other semi-random condition. We can alter the metric of one (or both) interfaces to prioritise the one we want. If we have interfaces ens19 (main network) and ens18 (private network) for example, then an example is:

# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).

source /etc/network/interfaces.d/*

# The loopback network interface
auto lo
iface lo inet loopback

# Main network
allow-hotplug ens19
iface ens19 inet dhcp

# Private network
allow-hotplug ens18
iface ens18 inet dhcp
metric 100

Adding the metric 100 line to ens18 deprioritises it (as ens19 should get the default metric 0). Restarting the networking with systemctl restart networking and then running ip route gives something like

default via 172.40.204.1 dev ens19
default via 192.168.60.1 dev ens18 metric 100
172.40.204.0/24 dev ens19 proto kernel scope link src 172.40.204.27
192.168.60.0/24 dev ens18 proto kernel scope link src 192.168.60.8

Accessing server B from the main network using 172.40.204.27 should then work properly. As a bonus if ens19 looses it’s connection then it should fall back to the private network gateway (although for our particular use case this isn’t important).

Kea DHCP service failing to open network socket on boot

System – Kea IPv4 DHCP server (ver 2.2.0) running on Debian 12, serving addresses on second network interface (PCIe card).

On reboot the server starts, but warns that it could not open the socket on the prescribed interface, and so is not serving any addresses. Running

systemctl status kea-dhcp4-server.service

The error is


kea-dhcp4[582]: WARN DHCPSRV_OPEN_SOCKET_FAIL failed to open socket: the interface enp3s0 is not running
kea-dhcp4[582]: INFO DHCP4_OPEN_SOCKETS_FAILED maximum number of open service sockets attempts: 0, has been exhausted without success

Restarting the service afterwards works normally. This is a known problem. There are various suggestions to fix the ordering, but the simplest (albeit slightly inelegant) way to get it to work is to tell it to retry a bunch of times with the service-sockets-max-retries and service-sockets-retry-wait-time options. So the start of the configuration looks like this:

"Dhcp4": {
    "interfaces-config": {
        "interfaces": [ "enp3s0" ],
        "dhcp-socket-type": "raw",
        "service-sockets-max-retries": 100,
        "service-sockets-retry-wait-time": 5000
    },

This retries every 5 seconds up to 100 times, which should be enough to get things in order. Note that, annoyingly, there is no message logged when a retry succeeds.

Blocking or disabling autofs automounts with the -null map

Suppose you have a linux network setup with automounter maps that come from the network (via nis, sssd, LDAP etc.) and you want to block some of them acting on a particular system. In our case we have an automount map that acts on /opt and mounts various software packages from network shares. The problem with this is that you can’t then install your own stuff locally to /opt, which is what a lot of Debian/Ubuntu packages expect to be able to do.

It turns out there is a option in the automounter for this sort of situation. There is a built-in map called -null that blocks any further automounts to a particular mountpoint. In our case we want to block auto.opt, so we add a line to auto.master (somewhere before the bottom +auto.master line)

/opt  -null

Then restart the autofs service (if stuff was mounted on /opt then unmount it). Or reboot the system. You should find that you can put stuff in the local /opt.

To check the map is blocked you can also run

automount --dumpmaps

(also handy for checking what is actually meant to be mapped where).

Another way of doing this that leaves the system auto.master untouched is to create a file /etc/auto.master.d/opt.autofs (the first part of the name can be anything you want). Put the same contents in the file, e.g.

/opt  -null

Note that using this mechanism normally requires two files – one in /etc/auto.master.d/ and a map file that it refers to. In this case -null is a built-in map.

Unfortunately this option is not well documented. Places where it is referred to are:

There are also other built-in maps, e.g. -passwd, -hosts, -fedfs. Of these only the -hosts map is documented in the auto.master(5) man page.

-null is confirmed to work in CentOS 7, CentOS 8, Ubuntu 20.04, Debian 10.

Triggering redetection of network type in Server 2012

Had an issue where a Windows Server 2012 R2 system could not be accessed by RDP or remote management, as the network type had changed to Private (and thus the firewall wasn’t letting these connections through). File sharing was still working.

Found solution via SpiceWorks forum. Restart the Network Location Awareness service (needed to log on to system locally to do this). This triggered a redetection and the type wend back to Domain. RDP etc then worked again.

Setting Windows 10 web proxy per-user

There are a couple of GUI routes for setting the system web proxy for Windows 10 – the old control panel page (via Network and Internet – Network Options):

Windows 7 and later Control Panel system web proxy settings panel.

And the new settings style:

Windows 10 system Proxy settings new style.

Note that the new style does not warn you that you may not be allowed to set the proxy – you can change the settings, but if you select another panel and then go back to Proxy your settings will be gone.

The reason for this is often that the system is configured to set the proxy at the machine level, not per-user. On domain systems this can be changed using Group policy. On standalone systems this can be changed using a registry key, located under

HKLM\Software\Policies\Microsoft\Windows\CurrentVersion\internet Settings

There is a DWORD key here called ProxySettingsPerUser (if not, create it). 0 means the proxy is set at machine level, 1 enables per-user settings.

If you change this to 1 then you should immediately be able to change the proxy settings.

FDQNs in FlexNet license files

When trying to query FlexNet licenses using lmutil or similar from systems with a different DNS suffix, make sure that the license file server name contains the FDQN for the SERVER line. If not you can find that lmutils complains that the lmgrd process is not running, even though you can run the actual program with the appropriate license.

For example, with a line in the licence file like

SERVER servername 4eca3b4b8326 1055

running

lmutil lmstat -a -c 1055@servername.physics.gla.ac.uk

results in a HOST_NOT_FOUND error. Running the Client ANSLIC_ADMIN Utility gives the same error. However, ANSYS fires up as normal.

To fix this put the FDQN in the license file (refreshing the licence server afterwards!)

SERVER servername.physics.gla.ac.uk 4eca3b4b8326 1055

The lmutil queries should then work as normal.

Making nis authentication work with Ubuntu 16, Debian 8, Fedora 28 etc.

After updating anything to use systemd-235 NIS logins either don’t work at all (usually for GUI logins), or take a long time to login (console or ssh, sometimes). The culprit is a line in the systemd-logind.service:

IPAddressDeny=any

This sandboxes the service and doesn’t allow it to talk to the network. Unfortunately this affects nis lookups done via the glibc NSS API. See the links at https://github.com/systemd/systemd/pull/7343

The quick solution is to turn off the sandboxing, either by commenting out or changing the line in systemd-logind.service, or creating a drop-in snippet that overrides it. This can be done by creating a file /etc/systemd/system/systemd-logind.service.d/IPAddress_clear.conf with the contents:

[Service]
IPAddressDeny=

The file can be called anything you like (.conf).

Then restart things:

systemctl daemon-reload
systemctl restart systemd-logind.service

You can check that the drop-in is being loaded with

systemctl status systemd-logind.service

In the output you should see something like:

   Loaded: loaded (/lib/systemd/system/systemd-logind.service; static; vendor preset: enabled)
  Drop-In: /etc/systemd/system/systemd-logind.service.d
  └─IPAddress_clear.conf

The other test is to see if NIS logins work correctly, of course…

The slightly slower solution is to use nscd to cache the lookup requests, and apparently does so in a way that plays nicely with the sandboxing. The much slower solution is to switch to using sssd or similar and ditch NIS once and for all…

Note – this may also affect systemd-udevd.

Making static networking work in Ubuntu 18.04

Ubuntu 18.04 has switched to netplan for configuring the network interfaces. Netplan generates configurations for NetworkManager or systemd-networkd and effectively replaces ifupdown and the /etc/network/interfaces file.

In an install of Ubuntu Desktop, the default netplan configuration comes from /etc/netplan/01-network-manager-all.yaml which reads:

network:
  version: 2
  renderer: NetworkManager

This basically hands over all network control to NetworkManager. For a static setup we can change the configuration to:

network:
  version: 2
  renderer: networkd
  ethernets:
    enp0s25:
      dhcp4: no
      addresses: [aaa.aaa.aaa.aaa/24]
      gateway4: bbb.bbb.bbb.bbb
      nameservers:
        search: [example.co.uk]
        addresses: [xxx.xxx.xxx.xxx,yyy.yyy.yyy.yyy,zzz.zzz.zzz.zzz]

Where enp0s25 is my network interface in this case, the address has a netmask of 255.255.255.0 and the search is the default dns search domain(s) (note this can be vital for getting automounting to work if your setup just uses machine names and assumes the domain is the same).

Note that if you have a laptop you could put this in a file called, say 02_ethernet_interface.yaml and it should override the first configuration for that interface only. I think. Later configurations override earlier ones.

To test:

netplan try

This applies the configuration and then rolls it back in 120 seconds (by default). Press Enter to accept the new settings.

netplan apply

To apply the changes.

For a desktop I just deleted the first config file.

In theory you could probably use this to generate a configuration for NetworkManager (note that on a server you need to explicitly configure NetworkManager to bring up the interface on boot).

Connecting to NFS shares at boot using fstab in Debian 9 Stretch

Note – this fix in principle should work on most systemd distributions.

Problem – trying to get a Debian 9 system to mount an NFS share at boot. This was declared in /etc/fstab in the normal way, but kept failing on boot. However, once the system was up you could log in and do a mount -a, which would work fine. Reading around, it looks like a case of the system trying to mount before the network is up (and in this case the network should be reliable, as it’s an internal one between a VM and it’s host…)

Tried using the bg option first, which should mount in the background and come up eventually, but still got error on boot.

192.168.45.1:/export  /hostshare  nfs  bg,rw,soft  0   0

There is another option that in theory should help: _netdev. I haven’t tried this yet.

What does work is adding an option x-systemd.automount. This, unsurprisingly, tells systemd to try and mount the share on demand. So changing the line in fstab to read:

192.168.45.1:/export  /hostshare  nfs  rw,soft,x-systemd.automount  0   0

works. Booting the system gives no errors (on the console anyway). The share does not show as mounted until the local mountpoint is accessed, and then it works without complaint.

Footnotes

  1. The context for this is a VM running under VirtualBox. The VM is Debian 9, the host is Ubuntu 17.10. The VM has one network interface with the default NAT setup to talk to the outside world, and a second interface to talk to a host-only network. This allows you to SSH into the the guest from the host, and also allows this NFS setup. You can use the VirtualBox shared folder setup to transfer files, but I figured as both the host and guest were Linux NFS would be easier (and not require the Guest Extensions to be installed on the guest).
  2. Debian 9 wouldn’t successfully install on the laptop, but I needed it for an easy install of LALSuite (Debian is a reference system for this, Ubuntu isn’t and has dependency issues). Hence this rather complicated setup. Fortunately LALSuite is entirely command line based…
  3. Yes, Docker or similar would be more efficient. I’m not so familiar with it and it’s a bit of a pain to get it talking to the host filesystem. I’d argue that running a full VM is slightly more portable, although you’d need to change how the filesharing is set up on a Windows or Mac host.