You Upgraded GitLab and Stayed Vulnerable: The Patch Wasn't in GitLab

On October 22, the Alpine Linux team upgraded their GitLab instance to 19.3.6. On October 26, they discovered it was still vulnerable to the bug they had just tried to patch. Same version number on the sign-in page. Same waiting exploit. The upgrade had shipped the label and left the fix behind.

That failure is not an Alpine problem. It's a structural one, and it probably lives in your stack right now. Most teams verify a patch by checking a version string. Version strings are claims about releases, not statements about the contents of a build. When the vulnerable code lives one layer below the thing you upgraded, the claim is true and the bug is still there.

What actually happened to Alpine

The bug was CVE-2026-97229, a critical (CVSS 10.0) flaw in how GitLab evaluates regular expressions in CI configuration. The root cause was inside Onigmo, the regular expression engine that ships with Ruby — the language GitLab is written in. This class of bug is a ReDoS, or regular expression denial of service: a crafted input makes the matcher take exponentially long, and the process that runs it falls over.

GitLab published fixed releases on October 12. The fix wasn't a change to GitLab's own code — it was a set of patches applied to the Ruby interpreter bundled inside GitLab's official packages and container images. Upstream Ruby only shipped the same fix on October 23, in Ruby 3.3.13, 3.4.12 and 4.0.9.

Alpine was running 19.3.4, and by their own account had skipped 19.3.5, the version that carried the fix, by mistake. On October 20 and 21, a newly registered account sent crafted CI configurations at the instance. GitLab worker processes crashed eight times; in four of those crashes the attacker got partway through the exploit and failed at the next step. They also downloaded Alpine's GitLab container image, which suggests they were tuning the payload against that exact build.

Alpine upgraded to 19.3.6 on October 22. Still vulnerable. Why? Because Alpine's image uses standard upstream Ruby rather than GitLab's bundled build. GitLab's patch never entered the picture. They were current on GitLab and unpatched on Onigmo at the same time.

The version number is a claim about a release, not a diff

Here is the mental model worth keeping. A released artifact is a bundle of many codebases: your application, its interpreter or runtime, a handful of system libraries, and whatever the base image adds. A patch enters that bundle at exactly one layer — whichever layer holds the vulnerable code.

If you consume the vendor's own build, the vendor applies the patch and hands you a coherent bundle. If you rebuild from upstream sources, or swap in a stock runtime, or let a distro repackage it, you break that chain. You keep the version and lose the diff.

This is not rare, and it is not vendor negligence. Distributions deliberately unpick vendor patches because they patch against their own library versions. Vendors patch their bundle instead of waiting for upstream because upstream takes months. Both are reasonable. The gap between them is where your audit goes to die.

The nastiest detail in the Alpine write-up is this: if you use their alpinelinux/gitlab container image, a 19.3.6 image pulled before October 27 is vulnerable, and a 19.3.6 image pulled after October 27 is not. Same tag, different bits. If your deployment tooling pins tags rather than digests, your record of what you run is a memory, not a fact.

Ask three questions about every serious CVE

You do not need to do this for everything. You need it for anything that parses untrusted input on an internet-facing path — which is most of the things that get CVSS 10.0. When one lands, answer these in order:

1. Which artifact contains the vulnerable code? Not which product is named in the advisory. Ask what file the fix touched. If the answer is "the regex engine," "the TLS library," "the image decoder" or "the template compiler," the patched layer is a dependency, not your app.
2. How did that artifact get into my build? Vendor package, distro package, upstream source, or something you compiled yourself. Each path has a different chance of carrying the fix.
3. How do I confirm the fix is present? If your answer is "the version looks right," you don't have an answer. You have a guess with good formatting.

Confirming presence, in order of reliability

• A behavior probe. A test that exercises the fixed path and fails on the old one. Strongest evidence, hardest to write — and for a ReDoS specifically, often flaky enough to be worse than nothing.
• An artifact digest. Pin container images by digest, not tag, and record the digest you deployed. This is what would have caught the Alpine case directly: two different digests wearing one tag.
• A patch inventory. For your five highest-exposure dependencies, write down whether you take vendor builds or upstream source, and where the patched layer sits relative to what you upgrade. Twenty minutes of work that survives contact with an incident.

The second bill: "we cannot rule it out"

Alpine's honest conclusion is the part most teams should read twice. They found no evidence the attacker ran code. They also couldn't rule it out: a successful exploit of this kind leaves no crash and no trace in web logs, background job logs from the attack day were already gone, the old container filesystem was gone, and nobody was logging outbound connections — not them, not their hosting provider.

So they treated the instance as compromised and rebuilt it from scratch. Every password reset. Secret keys replaced. GitLab closed to everyone but developers and build servers for days. Their release verification held up — aports commits were checked against a developer's clone and all 45 builders, packages rebuilt on every architecture, no signing keys exposed — but the rebuild happened anyway.

That is the real cost curve. Not the hours spent upgrading, but the days spent proving a negative you have no data for. Three days of egress logs would have been cheaper than a full rebuild. If your agents, your CI runners and your build hosts can reach the internet without a record, you have bought the same lottery ticket.

Where this advice costs you

Be honest about the trade-off, because "always use vendor builds" is not free. Vendor images couple you to their release cadence and their base OS, and often lag on everything else. Distro packages give you one clean patch stream and slower version adoption. Building from upstream gets you the newest code and quietly makes you the team that applies security patches to an interpreter. Pick per component, based on exposure rather than preference: internet-facing and parsing untrusted input gets the strict path; an internal batch job can wait.

The one thing that isn't optional is knowing which path each component is on. Alpine did almost everything right — they read the advisory, they upgraded within two days, they published a detailed public postmortem. They still spent a week offline because a version number told them something that wasn't true.

Next time an advisory lands, don't ask whether you're on the fixed version. Ask which layer holds the fix, and whether your build carries that layer's patches. If you can't answer in ten minutes, that's the finding.

Key Takeaways

• A version number describes a release, not the patch level of every codebase inside it. GitLab 19.3.6 on stock Ruby was current and vulnerable at once.
• Find the layer that holds the vulnerable code before you plan the upgrade. If the advisory names your product but the fix touches your runtime, upgrading the product is theater.
• Pin by digest, not tag. Two 19.3.6 images pulled on different days had different contents, and only one was safe.
• Keep a five-line patch inventory for your highest-exposure dependencies: what they are, how they got into your build, and whether you consume vendor builds or upstream source.
• Log outbound connections and keep the logs. The difference between "no evidence of compromise" and "we cannot rule it out" is measured in rebuild days.