<?xml version="1.0" encoding="utf-8" standalone="yes"?><rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:blog="https://jonesrussell.github.io/blog/ns"><channel><title>Bash on Web Developer Blog</title><link>https://jonesrussell.github.io/blog/tags/bash/</link><description>Recent content in Bash on Web Developer Blog</description><image><title>Web Developer Blog</title><url>https://jonesrussell.github.io/blog/images/og-default.png</url><link>https://jonesrussell.github.io/blog/images/og-default.png</link></image><generator>Hugo -- 0.165.0</generator><language>en-us</language><lastBuildDate>Wed, 02 Sep 2026 00:00:00 +0000</lastBuildDate><atom:link href="https://jonesrussell.github.io/blog/tags/bash/feed.xml" rel="self" type="application/rss+xml"/><item><title>A missing npm ci silently starved the content-mining pipeline for a week</title><link>https://jonesrussell.github.io/blog/content-mining-pipeline-silent-failure/</link><pubDate>Wed, 02 Sep 2026 00:00:00 +0000</pubDate><guid>https://jonesrussell.github.io/blog/content-mining-pipeline-silent-failure/</guid><category>devops</category><blog:tag>github-actions</blog:tag><blog:tag>automation</blog:tag><blog:tag>bash</blog:tag><blog:tag>ci</blog:tag><description>A missing npm ci let schema validation crash on every mined candidate, filing zero content-queue issues for a week — the fix layers a loud fail-open guard over the actual root-cause patch.</description><content:encoded><![CDATA[<p>Ahnii!</p>
<p>Earlier posts covered <a href="/blog/automated-content-pipeline-github-actions/">building</a> and <a href="/blog/refining-content-pipeline-github-actions/">refining</a> the automated content pipeline: a scheduled job scans recent commits, groups them by theme, and files GitHub issues as raw material for future posts. Every mined candidate gets validated against a JSON schema before the workflow files an issue for it — that&rsquo;s the guardrail that keeps malformed seeds out of the queue. For about a week, that guardrail silently dropped every candidate instead of filing a single one.</p>
<h2 id="what-broke">What Broke</h2>
<p>The mining script, <code>scripts/mine-git-activity.sh</code>, builds a seed for each commit group and validates it with <code>schemas/validate.js</code>, which loads the <a href="https://ajv.js.org/">ajv</a> JSON Schema validator:</p>
<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;-webkit-text-size-adjust:none;"><code class="language-js" data-lang="js"><span style="display:flex;"><span><span style="color:#66d9ef">const</span> <span style="color:#a6e22e">Ajv</span> <span style="color:#f92672">=</span> <span style="color:#a6e22e">require</span>(<span style="color:#e6db74">&#39;ajv&#39;</span>);
</span></span><span style="display:flex;"><span><span style="color:#66d9ef">const</span> <span style="color:#a6e22e">addFormats</span> <span style="color:#f92672">=</span> <span style="color:#a6e22e">require</span>(<span style="color:#e6db74">&#39;ajv-formats&#39;</span>);
</span></span></code></pre></div><p><code>ajv</code> and <code>ajv-formats</code> are <code>devDependencies</code> in <code>package.json</code>, so they only exist after <code>npm ci</code> runs. The <code>content-mine.yml</code> workflow checked out the blog repo and ran the mining script directly — it never installed Node dependencies first. Every call to the validator hit <code>Cannot find module 'ajv'</code> and exited non-zero.</p>
<p>The script treated that crash the same as a real validation failure: reject the candidate, move on. No error surfaced anywhere. The workflow itself finished green, because the script&rsquo;s own exit code was still <code>0</code> — it just never filed anything. Net result: <strong>zero issues created, for about a week</strong>, and nothing in the logs said so unless you went looking.</p>
<h2 id="fix-1-fail-open-warn-loud">Fix 1: Fail Open, Warn Loud</h2>
<p>The first patch, in the blog repo, doesn&rsquo;t wait for CI to be fixed — it makes the script defend itself. Before validating anything, it checks whether the validator&rsquo;s dependencies actually resolve:</p>
<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;-webkit-text-size-adjust:none;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>VALIDATOR_OK<span style="color:#f92672">=</span><span style="color:#ae81ff">1</span>
</span></span><span style="display:flex;"><span><span style="color:#66d9ef">if</span> ! node -e <span style="color:#e6db74">&#34;require(&#39;ajv&#39;); require(&#39;ajv-formats&#39;)&#34;</span> &gt;/dev/null 2&gt;&amp;1; <span style="color:#66d9ef">then</span>
</span></span><span style="display:flex;"><span>  echo <span style="color:#e6db74">&#34;WARN: schema-validator deps (ajv) not installed — creating issues WITHOUT seed validation. Add &#39;npm ci&#39; to the mining workflow to restore validation.&#34;</span> &gt;&amp;<span style="color:#ae81ff">2</span>
</span></span><span style="display:flex;"><span>  VALIDATOR_OK<span style="color:#f92672">=</span><span style="color:#ae81ff">0</span>
</span></span><span style="display:flex;"><span><span style="color:#66d9ef">fi</span>
</span></span></code></pre></div><p>Downstream, the validation gate becomes an <em>or</em>: skip validation and file the issue anyway if the deps aren&rsquo;t there, instead of letting the crash masquerade as &ldquo;this candidate is invalid&rdquo;:</p>
<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;-webkit-text-size-adjust:none;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span><span style="color:#66d9ef">if</span> <span style="color:#f92672">[[</span> <span style="color:#e6db74">&#34;</span>$VALIDATOR_OK<span style="color:#e6db74">&#34;</span> <span style="color:#f92672">==</span> <span style="color:#e6db74">&#34;0&#34;</span> <span style="color:#f92672">]]</span> <span style="color:#f92672">||</span> $VALIDATOR mined-seed <span style="color:#e6db74">&#34;</span>$SEED_FILE<span style="color:#e6db74">&#34;</span> &gt; /dev/null 2&gt;&amp;1; <span style="color:#66d9ef">then</span>
</span></span><span style="display:flex;"><span>  <span style="color:#75715e"># ...build and create the issue...</span>
</span></span></code></pre></div><p>That&rsquo;s a deliberate trade: an unvalidated issue is a minor cleanup problem for a human curator. A silently starved queue is a week of lost content ideas nobody knew to chase. Given the choice, fail open and be loud about it.</p>
<h2 id="fix-2-actually-install-the-dependency">Fix 2: Actually Install the Dependency</h2>
<p>The warning fixes the symptom. The root cause was still the missing install step, patched separately in the <code>content-mine.yml</code> workflow (in <code>jonesrussell/jonesrussell</code>, the repo that owns the queue):</p>
<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;-webkit-text-size-adjust:none;"><code class="language-yaml" data-lang="yaml"><span style="display:flex;"><span>- <span style="color:#f92672">name</span>: <span style="color:#ae81ff">Setup Node.js</span>
</span></span><span style="display:flex;"><span>  <span style="color:#f92672">uses</span>: <span style="color:#ae81ff">actions/setup-node@v4</span>
</span></span><span style="display:flex;"><span>  <span style="color:#f92672">with</span>:
</span></span><span style="display:flex;"><span>    <span style="color:#f92672">node-version</span>: <span style="color:#e6db74">&#39;lts/*&#39;</span>
</span></span><span style="display:flex;"><span>    <span style="color:#f92672">cache</span>: <span style="color:#e6db74">&#39;npm&#39;</span>
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span>- <span style="color:#f92672">name</span>: <span style="color:#ae81ff">Install dependencies (ajv for seed validation)</span>
</span></span><span style="display:flex;"><span>  <span style="color:#f92672">run</span>: <span style="color:#ae81ff">npm ci</span>
</span></span></code></pre></div><p>With that in place, validation runs for real again — the fail-open guard in fix 1 becomes a safety net for the next unrelated dependency gap, not a permanent workaround for this one.</p>
<h2 id="what-changed-at-a-glance">What Changed, at a Glance</h2>
<table>
	<thead>
			<tr>
					<th></th>
					<th>Before</th>
					<th>After</th>
			</tr>
	</thead>
	<tbody>
			<tr>
					<td>Missing <code>ajv</code></td>
					<td>Validator crashes, treated as &ldquo;invalid,&rdquo; issue silently dropped</td>
					<td>Absence detected up front, logs a <code>WARN</code>, issue still filed</td>
			</tr>
			<tr>
					<td>Node deps in CI</td>
					<td>Never installed</td>
					<td><code>actions/setup-node</code> + <code>npm ci</code> before the script runs</td>
			</tr>
			<tr>
					<td>Failure visibility</td>
					<td>None — 0 issues/week, workflow exits <code>0</code></td>
					<td>Explicit warning line in job logs</td>
			</tr>
	</tbody>
</table>
<h2 id="the-lesson">The Lesson</h2>
<p>A script that &ldquo;succeeds&rdquo; by doing nothing is worse than one that fails loudly, because nothing pages you for it. The mining workflow&rsquo;s exit code was never wrong; it was just measuring the wrong thing. The mining job <a href="/blog/automated-content-pipeline-github-actions/">runs daily</a>, so this wasn&rsquo;t one bad run — it was seven straight silent failures before the empty queue was noticeable on its own. Exit code isn&rsquo;t the metric that would have caught it. Throughput is: issues filed per run, checked against zero. That&rsquo;s the check I was missing.</p>
<p>Baamaapii</p>
]]></content:encoded></item></channel></rss>