Explicit Web Proxy NTLM / Kerberos Worker Exhaustion (HTTP 503)
Enterprise web users experience sudden, campus-wide HTTP 503 Service Unavailable errors when unauthenticated client connection storms saturate the FortiOS explicit proxy authentication worker pool, blocking communication with Active Directory domain controllers.
Technical Root Cause Analysis
When FortiGate acts as an Explicit Web Proxy configured for Single Sign-On (SSO) via Kerberos or NTLM, the wad daemon delegates user identity verification to a pool of authentication workers (wad_auth_worker). These workers exchange SPNEGO tokens and query Active Directory Domain Controllers via LDAP and SMB RPC.
In affected firmware versions, the authentication worker allocation is statically capped. When morning logon storms occur, or when non-interactive client applications (such as cloud synchronizers, software updaters, or browser background tabs) send thousands of unauthenticated requests without Kerberos tickets, each request consumes an auth worker thread while negotiating NTLM fallback. If any domain controller experiences latency over 150ms, worker threads queue indefinitely. Once all auth workers are occupied, the WAD proxy drops incoming connections immediately with HTTP/1.1 503 Service Unavailable: Authentication worker unavailable.
Affected Firmware & Blast Radius Matrix
| FortiOS Branch | Vulnerable Builds | Confirmed Clean Build | Status & Workaround |
|---|---|---|---|
| FortiOS 7.2 | 7.2.3 – 7.2.7 | 7.2.8+ | Increase auth-worker-num and bypass automated user-agents |
| FortiOS 7.4 | 7.4.0 – 7.4.2 | 7.4.3+ | Upgrade to 7.4.3+ for non-blocking async auth engine |
| FortiOS 7.0 | Not Affected | 7.0.x Branch Clean | Legacy WAD thread pool utilized separate queue per VDOM |
Platform Impact: Affects FortiGate 100F, 200F, 400F, and 600F appliances running Explicit Web Proxy with Kerberos/NTLM authentication serving more than 500 concurrent users.
Step 01: Free Verification CLI (Safe Read-Only)
Execute these diagnostic commands to verify whether your WAD authentication worker pool is exhausted and identify hung domain controller queries:
Diagnostic Commands
# 1. Inspect the active authentication worker queue and backlog status
diagnose wad auth list
# 2. View per-worker WAD memory and process health
diagnose test application wad 1000
# 3. Dump active proxy authentication statistics and error counters
diagnose test application wad 200
# 4. List authenticated proxy users and active session tokens
diagnose wad user listRemediation & Workaround Steps (Teaser Preview)
Follow the complete step-by-step remediation guide in the interactive Ferrite platform:
Frequently Asked Questions
When explicit web proxy is configured with Kerberos or NTLM authentication, WAD allocates a fixed pool of synchronous auth workers. During logon bursts or when background services (Windows Update, Office telemetry) send repeated unauthenticated requests, all worker threads block waiting on Active Directory responses. Once the worker queue fills, the proxy rejects all subsequent user HTTP requests with 503 Service Unavailable.
Restarting the auth worker subsystem via 'diagnose test application wad 201' flushes queued, hung authentication transactions and immediately restores proxy request processing without rebooting the FortiGate.
FortiOS 7.4.3 and 7.2.8 replace the synchronous worker pool with an asynchronous event-driven authentication engine capable of handling tens of thousands of concurrent authentication handshakes without thread starvation.