OpenSSL HollowByte Flaw Could Freeze Server Memory with 11-Byte TLS Requests

Eleven bytes will make an unpatched OpenSSL server set aside up to 131 KB of memory for a message that never arrives. On the glibc systems Okta tested, that memory is gone until the process restarts.

OpenSSL shipped the **HollowByte **fix in June with no CVE, no advisory, and no changelog entry pointing at it. Okta's Red Team, which reported the denial-of-service bug and named it, published the details on Thursday.

The fixed releases are OpenSSL 4.0.1, 3.6.3, 3.5.7, 3.4.6, and 3.0.21, all dated June 9. Every release on those branches before the fixed ones has it. Nothing in a normal patch pipeline will point you at them: there is no identifier for a scanner to match and no advisory to read.

The flaw is that OpenSSL took the attacker's word for it. Every TLS handshake message carries a 4-byte header, three bytes of which declare how long the body will be. Older versions grew the receive buffer to that declared size the moment the header landed, before a single byte of the body showed up, and before the handshake's own checks ran.

For an inbound ClientHello the ceiling is 131 KB. Then the worker thread blocks, waiting on a body that never comes. No authentication, no session, no key exchange.

The memory does not come back

On its own, that is a connection-exhaustion attack, and those are as old as Slowloris. What makes HollowByte stick is glibc. When the attacker drops the connection, OpenSSL frees the buffer, but glibc holds small and medium chunks for reuse rather than returning them to the kernel.

The attack varies the claimed size on every connection, and in Okta's tests, that was enough to stop the allocator from reusing what it freed. The heap fragments, resident set size climbs, and it stays climbed long after the attacker has gone.

In Okta's NGINX testing, a 1 GB server was OOM-killed with 547 MB of memory frozen in fragments. On a 16 GB server, HollowByte locked up 25% of system memory without ever crossing the connection ceiling, which is why the Red Team says "standard connection-limiting defenses won't stop it".

Those figures are Okta's own, and it published no exploit code alongside them. The Hacker News found no public proof-of-concept repository on GitHub as of July 18.

OpenSSL decided this wasn't a vulnerability

The pull request from Matt Caswell, who wrote the patch, puts it plainly: the security team chose to "handle this as a 'bug or hardening' only fix". OpenSSL's own security policy defines four severity tiers, Critical down to Low, and "bug or hardening" is not among them.

Even a Low issue earns a CVE, a changelog note, and an entry on the vulnerabilities page. HollowByte has none of the three. The Hacker News found no mention of the fix in the release notes or in all 23 entries of OpenSSL's 4.0.1 changelog.

OpenSSL has not said why. Here is the case for them: 131 KB per connection is small, every TLS server allocates memory per connection, and a bounded allocation is not a vulnerability. Okta's answer is that the memory never comes back.

The Hacker News has asked OpenSSL why HollowByte was triaged below Low, and whether the fix reached the extended-support 1.1.1 and 1.0.2 branches. It has also asked Okta whether the fragmentation survives allocators other than glibc. This story will be updated with any response.

The project's line is finer than it looks. In January, OpenSSL assigned CVE-2025-66199, rated Low, to a TLS 1.3 certificate-compression bug in which a peer-supplied length grew a heap buffer before validation, worth around 22 MiB per connection.

That one needed four things to line up: certificate compression compiled in, a compression algorithm available, the extension negotiated, and, on servers, client certificates requested. HollowByte needs none of them.

The same June 9 release assigned CVE-2026-34183, rated Moderate, to unbounded memory growth in the QUIC PATH_CHALLENGE handler. Both are memory-exhaustion DoS. Both got numbers.

The release also closed 18 CVEs, including a High-severity use-after-free in PKCS7_verify(), so anyone running one of those upstream builds has the fix without being told.

Downstream is worse. Red Hat's documented default is to backport rather than move the version, so a patched package still reports the version it was built from. What normally resolves that is the advisory and the OVAL feed, both keyed to CVE names. There is no CVE here to key on.

That leaves the package changelog or the maintainer: ask whether they rebased on the June 9 release or took the patch, which is pull request 30792 for master and 4.0, 30793 for 3.6, 3.5, and 3.4, and 30794 for 3.0.

If you build OpenSSL yourself, upgrade to the listed release and restart whatever loaded the old one.

The fix covers TLS only. Caswell wrote on the pull request that DTLS was left alone because doing it properly would have been far more invasive, and that the project decided not to bother with it for now. The Hacker News compared OpenSSL's source at the 3.6.2 and 3.6.3 tags and found the DTLS handshake file byte-identical across the fix. In 4.0.1, the newest release, that path still sizes its buffer from the length the peer declares.

OpenSSL has not classified that path or committed to fixing it. The release notes, the changelog, and the vulnerabilities page say nothing about it. The pull request does.