<?xml version="1.0" encoding="utf-8" standalone="yes"?><rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:blog="https://jonesrussell.github.io/blog/ns"><channel><title>Reliability on Web Developer Blog</title><link>https://jonesrussell.github.io/blog/tags/reliability/</link><description>Recent content in Reliability on Web Developer Blog</description><image><title>Web Developer Blog</title><url>https://jonesrussell.github.io/blog/images/og-default.png</url><link>https://jonesrussell.github.io/blog/images/og-default.png</link></image><generator>Hugo -- 0.165.0</generator><language>en-us</language><lastBuildDate>Sun, 06 Sep 2026 00:00:00 +0000</lastBuildDate><atom:link href="https://jonesrussell.github.io/blog/tags/reliability/feed.xml" rel="self" type="application/rss+xml"/><item><title>Fixing a silent truncation bug in Waaseyaa's StreamHttpClient</title><link>https://jonesrussell.github.io/blog/streamhttpclient-truncated-body-fail-closed/</link><pubDate>Sun, 06 Sep 2026 00:00:00 +0000</pubDate><guid>https://jonesrussell.github.io/blog/streamhttpclient-truncated-body-fail-closed/</guid><category>php</category><category>waaseyaa</category><blog:tag>php</blog:tag><blog:tag>waaseyaa</blog:tag><blog:tag>http</blog:tag><blog:tag>reliability</blog:tag><description>How Waaseyaa&amp;rsquo;s StreamHttpClient silently turned a truncated, over-limit response body into an HTTP 200, and the fail-closed fix that rejects incomplete bodies instead of guessing.</description><content:encoded><![CDATA[<p>Ahnii!</p>
<p><a href="https://github.com/waaseyaa/framework">Waaseyaa</a>&rsquo;s <code>packages/http-client</code> package wraps PHP streams behind a small <code>HttpClientInterface</code>, with <code>StreamHttpClient</code> as the production implementation. It caps how many bytes of a response body it will read, so a runaway or hostile endpoint can&rsquo;t exhaust worker memory. That cap had a bug: hitting it didn&rsquo;t fail the request. It silently handed back a truncated body as a successful <code>HttpResponse</code>. What follows walks through the bug, the fix, and the broader lesson about bounding a read without lying about what you actually read.</p>
<h2 id="the-bug-a-byte-ceiling-that-didnt-fail">The Bug: A Byte Ceiling That Didn&rsquo;t Fail</h2>
<p>The original <code>fetch()</code> method read the body with a single call:</p>
<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;-webkit-text-size-adjust:none;"><code class="language-php" data-lang="php"><span style="display:flex;"><span><span style="color:#75715e">// m4: cap the body so a runaway/hostile endpoint can&#39;t OOM the worker.
</span></span></span><span style="display:flex;"><span>$responseBody <span style="color:#f92672">=</span> <span style="color:#f92672">@</span><span style="color:#a6e22e">stream_get_contents</span>($handle, $this<span style="color:#f92672">-&gt;</span><span style="color:#a6e22e">maxResponseBytes</span>);
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span><span style="color:#66d9ef">if</span> ($responseBody <span style="color:#f92672">===</span> <span style="color:#66d9ef">false</span>) {
</span></span><span style="display:flex;"><span>    <span style="color:#66d9ef">throw</span> $this<span style="color:#f92672">-&gt;</span><span style="color:#a6e22e">transportFailure</span>($method, $url);
</span></span><span style="display:flex;"><span>}
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span><span style="color:#66d9ef">return</span> $responseBody;
</span></span></code></pre></div><p><code>stream_get_contents()</code> with a length argument stops reading once it hits that many bytes and returns whatever it has. It doesn&rsquo;t return <code>false</code> just because the stream had more data waiting — an endpoint returning a body larger than <code>maxResponseBytes</code> produced a <strong>200 OK</strong> with a truncated prefix instead of an error. The caller had no way to know the body was incomplete unless it happened to check the length itself.</p>
<h2 id="the-fix-fail-closed-not-silent">The Fix: Fail Closed, Not Silent</h2>
<p>The fix, tracked as <a href="https://github.com/waaseyaa/framework/blob/main/docs/change-records/FW-HTTP-STREAM-TRUNCATION-01.md">FW-HTTP-STREAM-TRUNCATION-01</a>, replaces the single <code>stream_get_contents()</code> call with a loop that reads in chunks and checks completeness against what the response actually declared:</p>
<ul>
<li><strong>Over-limit bodies throw.</strong> If the body would exceed <code>maxResponseBytes</code>, the client throws a typed <code>HttpRequestException</code> instead of returning a partial string.</li>
<li><strong><code>Content-Length</code> mismatches throw.</strong> If the connection closes (or times out) before the declared length is reached, that&rsquo;s a failure, not a short success.</li>
<li><strong>A declared length above the ceiling is rejected before reading starts</strong>, so memory stays bounded even for a body the client will never accept.</li>
<li><strong>Exact-limit bodies still succeed.</strong> A body whose size equals <code>maxResponseBytes</code> exactly, including chunked and connection-close bodies with no <code>Content-Length</code> at all, is a valid, complete response.</li>
</ul>
<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;-webkit-text-size-adjust:none;"><code class="language-php" data-lang="php"><span style="display:flex;"><span><span style="color:#66d9ef">private</span> <span style="color:#66d9ef">function</span> <span style="color:#a6e22e">readCompleteBody</span>($handle, <span style="color:#a6e22e">string</span> $method, <span style="color:#a6e22e">string</span> $url)<span style="color:#f92672">:</span> <span style="color:#a6e22e">string</span>
</span></span><span style="display:flex;"><span>{
</span></span><span style="display:flex;"><span>    $headers <span style="color:#f92672">=</span> <span style="color:#a6e22e">http_get_last_response_headers</span>() <span style="color:#f92672">??</span> [];
</span></span><span style="display:flex;"><span>    $status <span style="color:#f92672">=</span> $this<span style="color:#f92672">-&gt;</span><span style="color:#a6e22e">parseStatusCode</span>($headers);
</span></span><span style="display:flex;"><span>    <span style="color:#75715e">// HEAD and these status codes carry metadata, never a response body.
</span></span></span><span style="display:flex;"><span>    <span style="color:#66d9ef">if</span> (<span style="color:#a6e22e">strtoupper</span>($method) <span style="color:#f92672">===</span> <span style="color:#e6db74">&#39;HEAD&#39;</span> <span style="color:#f92672">||</span> ($status <span style="color:#f92672">&gt;=</span> <span style="color:#ae81ff">100</span> <span style="color:#f92672">&amp;&amp;</span> $status <span style="color:#f92672">&lt;</span> <span style="color:#ae81ff">200</span>) <span style="color:#f92672">||</span> $status <span style="color:#f92672">===</span> <span style="color:#ae81ff">204</span> <span style="color:#f92672">||</span> $status <span style="color:#f92672">===</span> <span style="color:#ae81ff">304</span>) {
</span></span><span style="display:flex;"><span>        <span style="color:#66d9ef">return</span> <span style="color:#e6db74">&#39;&#39;</span>;
</span></span><span style="display:flex;"><span>    }
</span></span><span style="display:flex;"><span>    $contentLength <span style="color:#f92672">=</span> $this<span style="color:#f92672">-&gt;</span><span style="color:#a6e22e">headerContentLength</span>($headers);
</span></span><span style="display:flex;"><span>    <span style="color:#66d9ef">if</span> ($contentLength <span style="color:#f92672">!==</span> <span style="color:#66d9ef">null</span> <span style="color:#f92672">&amp;&amp;</span> $contentLength <span style="color:#f92672">&gt;</span> $this<span style="color:#f92672">-&gt;</span><span style="color:#a6e22e">maxResponseBytes</span>) {
</span></span><span style="display:flex;"><span>        <span style="color:#66d9ef">throw</span> $this<span style="color:#f92672">-&gt;</span><span style="color:#a6e22e">boundedBodyFailure</span>($method, $url, <span style="color:#e6db74">&#39;HTTP response body exceeded the configured maximum&#39;</span>);
</span></span><span style="display:flex;"><span>    }
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span>    $body <span style="color:#f92672">=</span> <span style="color:#e6db74">&#39;&#39;</span>;
</span></span><span style="display:flex;"><span>    <span style="color:#66d9ef">while</span> (<span style="color:#f92672">!</span><span style="color:#a6e22e">feof</span>($handle)) {
</span></span><span style="display:flex;"><span>        $remaining <span style="color:#f92672">=</span> $this<span style="color:#f92672">-&gt;</span><span style="color:#a6e22e">maxResponseBytes</span> <span style="color:#f92672">-</span> <span style="color:#a6e22e">strlen</span>($body);
</span></span><span style="display:flex;"><span>        <span style="color:#66d9ef">if</span> ($contentLength <span style="color:#f92672">!==</span> <span style="color:#66d9ef">null</span> <span style="color:#f92672">&amp;&amp;</span> <span style="color:#a6e22e">strlen</span>($body) <span style="color:#f92672">===</span> $contentLength) {
</span></span><span style="display:flex;"><span>            <span style="color:#66d9ef">break</span>;
</span></span><span style="display:flex;"><span>        }
</span></span><span style="display:flex;"><span>        $readBytes <span style="color:#f92672">=</span> $contentLength <span style="color:#f92672">!==</span> <span style="color:#66d9ef">null</span> <span style="color:#f92672">?</span> $contentLength <span style="color:#f92672">-</span> <span style="color:#a6e22e">strlen</span>($body) <span style="color:#f92672">:</span> $remaining <span style="color:#f92672">+</span> <span style="color:#ae81ff">1</span>;
</span></span><span style="display:flex;"><span>        $chunk <span style="color:#f92672">=</span> <span style="color:#f92672">@</span><span style="color:#a6e22e">fread</span>($handle, <span style="color:#a6e22e">max</span>(<span style="color:#ae81ff">1</span>, <span style="color:#a6e22e">min</span>(<span style="color:#ae81ff">8192</span>, $readBytes)));
</span></span><span style="display:flex;"><span>        $meta <span style="color:#f92672">=</span> <span style="color:#a6e22e">stream_get_meta_data</span>($handle);
</span></span><span style="display:flex;"><span>        <span style="color:#66d9ef">if</span> ($meta[<span style="color:#e6db74">&#39;timed_out&#39;</span>] <span style="color:#f92672">===</span> <span style="color:#66d9ef">true</span>) {
</span></span><span style="display:flex;"><span>            <span style="color:#66d9ef">throw</span> $this<span style="color:#f92672">-&gt;</span><span style="color:#a6e22e">boundedBodyFailure</span>($method, $url, <span style="color:#e6db74">&#39;HTTP response body was incomplete&#39;</span>);
</span></span><span style="display:flex;"><span>        }
</span></span><span style="display:flex;"><span>        <span style="color:#66d9ef">if</span> ($chunk <span style="color:#f92672">===</span> <span style="color:#66d9ef">false</span>) {
</span></span><span style="display:flex;"><span>            <span style="color:#66d9ef">throw</span> $this<span style="color:#f92672">-&gt;</span><span style="color:#a6e22e">transportFailure</span>($method, $url);
</span></span><span style="display:flex;"><span>        }
</span></span><span style="display:flex;"><span>        <span style="color:#66d9ef">if</span> ($chunk <span style="color:#f92672">===</span> <span style="color:#e6db74">&#39;&#39;</span>) {
</span></span><span style="display:flex;"><span>            <span style="color:#66d9ef">break</span>;
</span></span><span style="display:flex;"><span>        }
</span></span><span style="display:flex;"><span>        $body <span style="color:#f92672">.=</span> $chunk;
</span></span><span style="display:flex;"><span>        <span style="color:#66d9ef">if</span> (<span style="color:#a6e22e">strlen</span>($body) <span style="color:#f92672">&gt;</span> $this<span style="color:#f92672">-&gt;</span><span style="color:#a6e22e">maxResponseBytes</span>) {
</span></span><span style="display:flex;"><span>            <span style="color:#66d9ef">throw</span> $this<span style="color:#f92672">-&gt;</span><span style="color:#a6e22e">boundedBodyFailure</span>($method, $url, <span style="color:#e6db74">&#39;HTTP response body exceeded the configured maximum&#39;</span>);
</span></span><span style="display:flex;"><span>        }
</span></span><span style="display:flex;"><span>    }
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span>    <span style="color:#66d9ef">if</span> ($contentLength <span style="color:#f92672">!==</span> <span style="color:#66d9ef">null</span> <span style="color:#f92672">&amp;&amp;</span> <span style="color:#a6e22e">strlen</span>($body) <span style="color:#f92672">!==</span> $contentLength) {
</span></span><span style="display:flex;"><span>        <span style="color:#66d9ef">throw</span> $this<span style="color:#f92672">-&gt;</span><span style="color:#a6e22e">boundedBodyFailure</span>($method, $url, <span style="color:#e6db74">&#39;HTTP response body was incomplete&#39;</span>);
</span></span><span style="display:flex;"><span>    }
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span>    <span style="color:#66d9ef">return</span> $body;
</span></span><span style="display:flex;"><span>}
</span></span></code></pre></div><p>Reading one extra byte beyond <code>maxResponseBytes</code> when there&rsquo;s no declared <code>Content-Length</code> is what turns &ldquo;we stopped because we&rsquo;re at the limit&rdquo; into &ldquo;we detected the body actually exceeds the limit.&rdquo; Without that extra byte, an exact-limit body and an over-limit body look identical.</p>
<p>A <code>stream_set_timeout()</code> call was also added before the read loop. A connection that stalls mid-body now times out and fails closed instead of hanging or silently truncating.</p>
<h2 id="where-the-framing-boundary-actually-sits">Where the Framing Boundary Actually Sits</h2>
<p>Part of getting this right is knowing which responses have no body at all, regardless of what their headers claim:</p>
<table>
	<thead>
			<tr>
					<th>Response</th>
					<th>Body</th>
			</tr>
	</thead>
	<tbody>
			<tr>
					<td><code>HEAD</code> request</td>
					<td>none</td>
			</tr>
			<tr>
					<td><code>1xx</code> status</td>
					<td>none</td>
			</tr>
			<tr>
					<td><code>204 No Content</code></td>
					<td>none</td>
			</tr>
			<tr>
					<td><code>304 Not Modified</code></td>
					<td>none</td>
			</tr>
			<tr>
					<td>Declared <code>Content-Length</code></td>
					<td>stops exactly at that length, doesn&rsquo;t wait for connection close</td>
			</tr>
			<tr>
					<td>No <code>Content-Length</code> (connection-close framing)</td>
					<td>EOF defines completion</td>
			</tr>
	</tbody>
</table>
<p>The <a href="https://github.com/waaseyaa/framework/blob/main/docs/change-records/FW-HTTP-STREAM-TRUNCATION-01.md">change record</a> is honest about the limits of this fix, too. PHP&rsquo;s HTTP stream wrapper dechunks <code>Transfer-Encoding: chunked</code> responses and strips the header before the client ever sees the stream, so the byte ceiling still applies to the decoded body. But <code>StreamHttpClient</code> can&rsquo;t independently verify a missing chunk terminator — that&rsquo;s a problem for PHP&rsquo;s stream layer, not this client. Scoping the fix to what the client can actually observe, and documenting what it can&rsquo;t, beats pretending it covers every framing edge case.</p>
<h2 id="verifying-it">Verifying It</h2>
<p><code>packages/http-client/tests/Unit/StreamHttpClientTransportTest.php</code> covers the cases that matter:</p>
<ul>
<li>Bodies below, at, and above the limit</li>
<li>Chunked transfer</li>
<li>Absent <code>Content-Length</code></li>
<li>Mismatched <code>Content-Length</code></li>
<li>A mid-body timeout</li>
<li>An over-limit <code>5xx</code></li>
<li>A complete <code>404</code></li>
</ul>
<p>The suite also gained a small raw HTTP test server (<code>tests/Support/RawHttpServer.php</code>), so these cases could be driven against real socket behavior instead of mocked streams.</p>
<h2 id="the-general-lesson">The General Lesson</h2>
<p>Bounding a read and detecting truncation are two different problems, and it&rsquo;s easy to solve only the first one. <code>stream_get_contents($handle, $limit)</code> does exactly what it says: it reads at most <code>$limit</code> bytes and returns them. It was never going to tell you whether that&rsquo;s <em>all</em> the bytes there were. Any time you cap a read for memory safety, whether that&rsquo;s an HTTP body, a file, or a queue message, ask what happens at the boundary: does hitting the cap look identical to a legitimate response that happens to be exactly that size? If your code can&rsquo;t tell those two cases apart, it&rsquo;s not bounding the read — it&rsquo;s lying about it when the answer would be inconvenient.</p>
<p>Baamaapii</p>
]]></content:encoded></item></channel></rss>