Disable SMB3 directory leases: they silently truncate recursive directory walks

[artem-from-ua]

Summary

On a share holding a large, heavily branched directory tree — thousands of directories spread across a deep hierarchy — recursive walks fail with EAGAIN / errno 35 ("Resource temporarily unavailable"). A small share will not show this at all; it takes a tree big enough for a single traversal to accumulate thousands of open directory handles before finishing.

The failure is quiet, which is what makes it dangerous. tcapsule repair-xattrs reports a fraction of the tree as if it were the whole thing and exits successfully; xattr -l fails on arbitrary healthy files, making them look like they have corrupt metadata; any recursive traversal simply stops early.

The cause is SMB3 directory leases, which Samba enables by default from 4.22 onward. Adding one line to the generated smb.conf fixes it:

[global]
    smb3 directory leases = no

Environment

• Tested against upstream a109b23 ("Clean up mdns handling for WAN and LAN")
• Server: Samba 4.24.3 on NetBSD, Apple Time Capsule Gen 8, 128 MB RAM
• Client: macOS 27.0 public beta (build 26A5421a, Darwin 27.0.0), SMB 3.1.1, mounted via Finder
• Link: Wi-Fi 802.11ac, 5 GHz, 80 MHz, max rate 1.3 Gbps. Client in the same room as the capsule; RTT ~2.5 ms

Measured effect

Full-share repair-xattrs --dry-run --include-hidden over a tree of 216K objects (183K files, 33K directories, counted on the device itself, excluding the Time Machine sparsebundle):

Server config	Path coverage	Wall clock	Paths/s	Open dir handles on the server
stock (leases on)	5.5% (11799)	10m	18.9	climbs until it hits real_max_open_files
smb3 directory leases = no	100% (216233)	2h55m	20.6	3–7, flat for the entire run

Throughput does not degrade. The longer wall-clock time is entirely a consequence of covering twenty times more of the tree — per path the scan is if anything marginally faster with the fix in place. The same holds on a smaller subtree measured separately: 15.2 paths/s stock against 15.6 and 16.0 on two consecutive runs with the fix. Whatever the directory cache was buying, a recursive walk was not benefiting from it, since each directory is visited once.

The working-set figure is the striking one, and it was sampled every five minutes for the duration of each run, by counting directory entries in the session smbd's file table on the device:

fstat -p <smbd_pid> | grep drwx | wc -l

(on the capsule itself that is sed -n '/drwx/p' and sed -n '$=', since it has neither grep nor wc). Over the ~3-hour run with leases disabled, every one of those samples returned between 3 and 7, with no upward trend at any point. On the stock configuration the same counter climbs steadily until it reaches real_max_open_files and the walk breaks.

The state also persists across walks. Running the same scan twice in a row against a directory tree with several thousand directories, without remounting in between, the second run covers dramatically less than the first — in the worst case it returns almost immediately having enumerated essentially nothing, and still exits successfully. Handles accumulated by the first traversal are still held when the second one starts, so it begins already saturated. With directory leases disabled both runs are identical.

What exactly makes a tree susceptible was not established. It reproduces reliably on a tree of a few thousand directories and not at all on one of a few dozen, but whether the deciding factor is the directory count, the nesting depth, the fan-out, or the ratio of directories to files was not tested.

Why this happens

A directory lease is the client's protocol-level permission to keep a directory handle open after the readdir completes, so that subsequent metadata requests can be served from its cache (MS-SMB2 §3.3.5.9.11). A recursive walk therefore accumulates open directory handles faster than they are released.

Samba caps concurrent handles per session at real_max_open_files, computed in source3/smbd/files.c, file_init_global():

int request_max = lp_max_open_files();
real_lim = set_maxfiles(request_max + MAX_OPEN_FUDGEFACTOR);   /* 40 */
real_max = real_lim - MAX_OPEN_FUDGEFACTOR;

Past that, smbXsrv_open_create() in source3/smbd/smbXsrv_open.c refuses:

if (table->local.num_opens >= table->local.max_opens) {
    return NT_STATUS_INSUFFICIENT_RESOURCES;
}

On this device max open files = 512, so real_max is 512 and the walk breaks once the session is holding that many open directories. The relationship was verified directly on a run configured with a higher limit: with real_max = 980, fstat -p <pid> | grep drwx showed 981 open directories at the moment of failure — the arithmetic matches exactly.

The failure is silent on both sides. On the server, this path logs nothing at any debug level — the familiar "Too many open files, unable to open more!" message lives on the EMFILE path in open.c and is emitted at DEBUG(0), so its absence is positive evidence that the kernel never returned EMFILE. On the client, NT_STATUS_INSUFFICIENT_RESOURCES maps to EAGAIN rather than EMFILE (nt2errno[] in Apple's kernel/netsmb/smb_subr.c), which is what makes errno 35 the fingerprint of this specific counter.

Raising the limit does not fix it. It moves the failure point further out — a run with max open files raised to 980 got roughly five times as far before breaking — but the client keeps accumulating regardless of how many handles are available, so the walk still truncates.

Why the fix belongs on the server

smb3 directory leases is a global option introduced in Samba 4.22 and enabled by default on non-clustered installs (MR 3842, 4.22.0 release notes). This build is 4.24.3, so it is active.

Setting it to no takes effect at NEGOTIATE — source3/smbd/smb2_negprot.c:

if (protocol >= PROTOCOL_SMB3_00 &&
    in_capabilities & SMB2_CAP_DIRECTORY_LEASING &&
    lp_smb3_directory_leases())
{
	capabilities |= SMB2_CAP_DIRECTORY_LEASING;
}

The capability simply never appears in the NEGOTIATE response, so no client can request a directory lease in the first place. Confirmed on the live mount: smbutil statshares -a reports FILE_LEASING_SUPPORTED TRUE and no DIR_LEASING_SUPPORTED line at all once the option is set.

There is a second line of defence on CREATE (source3/smbd/open.c), which quietly ignores a lease request rather than failing it:

if (fsp->fsp_flags.is_directory &&
    oplock_request == LEASE_OPLOCK &&
    !lp_smb3_directory_leases())
{
	DBG_NOTICE("Ignoring disabled DirectoryLease request on [%s]\n", fsp_str_dbg(fsp));
	oplock_request = NO_OPLOCK;
	lease = NULL;
}

Being a server-side setting, it covers every client at once — including the Time Machine and Spotlight mounts, which mount the same share separately under /Volumes/.timemachine/ and would otherwise need to be handled individually.

Known side effects

Two things change beyond leases themselves, and both should be recorded rather than discovered later.

strict rename stops being forced on. Enabling directory leases implicitly enables strict rename; disabling them returns it to its own default of no. Nothing stops working as a result — the parameter only controls whether renaming a directory that has open files below it is refused (Windows behaviour, yes) or allowed (POSIX behaviour, no, and Samba's default for many years). The change makes that operation more permissive, not less. It does not apply to a macOS client at all, since the documentation states renames are always allowed once the client requests UNIX extensions. no is also what Samba recommends, because yes forces a scan of the entire open-handle database on every directory rename — a real cost on a 128 MB device.

Directory metadata caching goes away — but the measured cost is nil.

The concern was that a directory lease is what lets a client answer repeated metadata requests for the same directory locally, so Finder-style browsing (returning to a directory already visited) might get slower. Measured on a 116-entry directory, three consecutive passes in each configuration:

	pass 1	pass 2	pass 3
ls -l, leases on	2054 ms	969 ms	855 ms
ls -l, leases off	1370 ms	887 ms	866 ms
stat all entries, leases on	223 ms	201 ms	205 ms
stat all entries, leases off	218 ms	196 ms	194 ms

The warm passes are indistinguishable — 887/866 ms without leases against 969/855 ms with them, and the stat figures are flat in both. Repeated access is still served from cache after the change, because the client's own directory cache is a separate mechanism that this option does not touch. (The cold first pass differs, but a single sample either way is not evidence of anything.)

The test also confirms the option takes effect as described: smbutil statshares -a reported DIR_LEASING_SUPPORTED TRUE in the stock configuration and omitted the line entirely once leases were disabled.

Happy to test further

This reproduces on my share every single time — with leases enabled a traversal has never once completed, and with them disabled it has never once failed. Nothing about it is intermittent or load-dependent, so it is cheap to test against.

If it would help, I can run the same measurements against other builds, other Samba settings, or other client configurations, and report coverage against a known object count along with the server-side working set. Say which combinations are of interest.

[jamesyc]

Fix the merge conflict and this can be checked in