TrueNAS and Windows clients – NTLMv2 issues

Situation – TrueNAS (or FreeNAS, or other Samba servers) serving a SMB share with NTLMv1 authentication disabled. A standalone Windows 10 system can connect to it, but a domain joined Win 10 system constantly claims wrong password.

The culprit here was a old group policy setting in the domain:

Network Security: LAN Manager authentication level

(found in Computer Configuration - Windows Settings - Security Settings - Local Policies - Security Options)

This was set to Send LM & NTLM - use NTLMv2 session security if negotiated, for backwards compatibility reasons with Win 2000 boxes and the like. This affects the registry key lmcompatibilitylevel (setting it to 1) under HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Lsa.

Unfortunately this is a bit misleading. According to this article:

Security Watch: The Most Misunderstood Windows Security Setting of All Time

This should negotiate better session security if possible, but does not actually send NTLMv2 requests or responses.

Thus trying to connect to a TrueNAS SMB share fails unless NTLMv1 Auth is explicitly enabled (in the service settings).

Ideally the group policy should be removed and the normal setting restored (NTLMv2 only). Or we can enable NTLMv1 on the share if it isn’t going to be a permanent setup.

Invalid self-signed certificates in FreeNAS 11

Had a problem with several FreeNAS 11 systems – when trying to enable https for the web interface I couldn’t get a working certificate.

  1. Generate internal CA.
  2. Generate internal certificate.
  3. Chrome complains the certificate is invalid and does not allow you to bypass it.
  4. Delete certificate and CA, repeat, same problem.

After mucking about with using upper and lower case for the certificate CN names etc, eventually got things to work by generating a second certificate with the same CA. Which makes me think it might be this problem:

FreeNAS-Generated Certificates: Buyer Beware

So either generating a couple of certificates (and using the second one) or changing the serial number of the CA to something bigger than 1 (and then generating a certificate) should work.

(Go to the CA tab, double-click the CA entry and scroll down to the bottom to view/edit the serial number).

Creating mirrored swap devices on FreeNAS 11

As per the previous post, I was having problems with the swap partitions on my main FreeNAS 11 server. All the swap partitions were there, but were not being used until manually mounted. It turns out that FreeNAS now has a new way of using these – it seems to set up mirror devices using gmirror and uses these for swap. There are a couple of gotchas here – the first is that if you do a gmirror list on a working system you will see:

Balance: prefer

and

Type: MANUAL

The command line to create the mirror is:

 # gmirror create -b prefer swap0 /dev/nvd0p1 /dev/nvd1p1

(in this case, I’m creating the first swap on my NVMe cache drives. Note that the standard 2 Gb partition 1 of type swap is already present on all the data drives).

Note that pretty much all the guides out there use gmirror label – this gives you a type: AUTOMATIC mirror. We don’t want this, presumably because FreeNAS has it’s own scanning system for these things.

Verify the mirror exists:

# gmirror list
Geom name: swap0
State: COMPLETE
Components: 2
Balance: prefer
Slice: 4096
Flags: NONE
GenID: 0
SyncID: 1
ID: 3774900941
Type: MANUAL
Providers:
1. Name: mirror/swap0
 Mediasize: 2147483648 (2.0G)
 Sectorsize: 512
 Mode: r0w0e0
Consumers:
1. Name: nvd0p1
 Mediasize: 2147483648 (2.0G)
 Sectorsize: 512
 Stripesize: 0
 Stripeoffset: 65536
 Mode: r1w1e1
 State: ACTIVE
 Priority: 0
 Flags: (null)
 GenID: 0
 SyncID: 1
 ID: 2013054055
2. Name: nvd1p1
 Mediasize: 2147483648 (2.0G)
 Sectorsize: 512
 Stripesize: 0
 Stripeoffset: 65536
 Mode: r1w1e1
 State: ACTIVE
 Priority: 1
 Flags: (null)
 GenID: 0
 SyncID: 1
 ID: 1984168260

Then test the detection logic:

midclt call disk.swaps_configure

This should return

["mirror/swap0"]

Looking at the logs, this also encrypts the swap, creating a device

/mirror/swap0.eli

It also appears to automount it, as it then appears when you do a swapinfo.

Note that the system log complains about the other swap devices I set up earlier, so it may be it’s only really happy with encrypted swap.

You can create more mirrors and run midclt call disk.swaps_configure again. After that swapinfo should show you all the details:

# swapinfo -h
Device 1K-blocks Used Avail Capacity
/dev/mirror/swap0.eli 2097152 632M 1.4G 31%
/dev/mirror/swap2.eli 2097152 25M 2.0G 1%
/dev/mirror/swap1.eli 2097152 25M 2.0G 1%
Total 6291456 682M 5.3G 11%

This setup should in theory automatically work on reboot. Unfortunately, as this hosts lots of VM disk images I don’t want to test this anytime soon…

FreeNAS kernel python3.6 was killed: out of swap space

Ran into an error with FreeNAS (version 11.1-RELEASE) where the system was complaining about running out of swap space (first noticed there was a problem when getting repeated emails about ‘Unauthorized system reboot’). The error on the console and in /var/log/messages is:

servername kernel: pid 26118 (python3.6), uid 0, was killed: out of swap space

Fortunately NFS was still working, and console login (using the second console) worked. Used this to enable ssh:

service sshd start

and then I could login remotely which is more civilised. The problem was there was no swap:

# swapinfo
Device 1K-blocks Used Avail Capacity

Checking the disks showed there were 2 Gb swap partitions on all the data drives (default configuration):

# gpart show
=> 40 286749400 mfisyspd0 GPT (137G)
 40 1024 1 bios-boot (512K)
 1064 286748368 2 freebsd-zfs (137G)
 286749432 8 - free - (4.0K)

=> 40 286749400 mfisyspd1 GPT (137G)
 40 1024 1 bios-boot (512K)
 1064 286748368 2 freebsd-zfs (137G)
 286749432 8 - free - (4.0K)

=> 40 7814037088 mfisyspd2 GPT (3.6T)
 40 88 - free - (44K)
 128 4194304 1 freebsd-swap (2.0G)
 4194432 7809842688 2 freebsd-zfs (3.6T)
 7814037120 8 - free - (4.0K)

=> 40 7814037088 mfisyspd3 GPT (3.6T)
 40 88 - free - (44K)
 128 4194304 1 freebsd-swap (2.0G)
 4194432 7809842688 2 freebsd-zfs (3.6T)
 7814037120 8 - free - (4.0K)

=> 40 7814037088 mfisyspd4 GPT (3.6T)
 40 88 - free - (44K)
 128 4194304 1 freebsd-swap (2.0G)
 4194432 7809842688 2 freebsd-zfs (3.6T)
 7814037120 8 - free - (4.0K)

=> 40 7814037088 mfisyspd5 GPT (3.6T)
 40 88 - free - (44K)
 128 4194304 1 freebsd-swap (2.0G)
 4194432 7809842688 2 freebsd-zfs (3.6T)
 7814037120 8 - free - (4.0K)

=> 40 234441568 nvd0 GPT (112G)
 40 88 - free - (44K)
 128 4194304 1 freebsd-swap (2.0G)
 4194432 230247168 2 freebsd-zfs (110G)
 234441600 8 - free - (4.0K)

=> 40 234441568 nvd1 GPT (112G)
 40 88 - free - (44K)
 128 4194304 1 freebsd-swap (2.0G)
 4194432 230247168 2 freebsd-zfs (110G)
 234441600 8 - free - (4.0K)

The swap could be remounted with

# swapon /dev/mfisyspd0p1

etc.

This works:

# swapinfo -h
Device 1K-blocks Used Avail Capacity
/dev/nvd0p1 2097152 516M 1.5G 25%
/dev/nvd1p1 2097152 77M 1.9G 4%
/dev/mfisyspd2p1 2097152 12M 2.0G 1%
/dev/mfisyspd3p1 2097152 11M 2.0G 1%
/dev/mfisyspd4p1 2097152 10M 2.0G 1%
/dev/mfisyspd5p1 2097152 10M 2.0G 1%
Total 12582912 636M 11G 5%

The question is – did these fall off at some point, or were they not mounted at boot (a couple of months ago?). Will experiment with a test system.