We asked Claude (HAL9000): “Review and criticize the article at https://www.csoonline.com/article/4177700/cybersecurity-trends-in-sec-filings.html. Have SEC 10-K historically provided any real and useful assessment of cyber risk?”
Review of the Article
What the article does well
The piece is refreshingly transparent about its methodology. Derek Dye is upfront that this is a contributor blog post, not peer-reviewed research, and he openly acknowledges AI accuracy issues, the iteration required to get reliable outputs, and the 10–30% discrepancy rate between Gemini and Claude on the same data. He notes that the bulk of his 15–20 hours was spent on verification and refinement to ensure AI answers were correct — a candid admission many “AI-assisted research” pieces skip over. The methodology section, including the actual prompt and SQL code, adds credibility and reproducibility.
The finding that over 50% of companies didn’t clearly disclose who the CISO reports to is actually a substantive observation that the article buries without sufficiently emphasizing — that’s a significant governance transparency gap.
Weaknesses and criticisms
Surface-level findings dressed up as analysis. The core findings — CISO is the top role, Audit Committee oversees cyber, NIST is the dominant framework — are completely unsurprising and were already well-established from the prior Harvard and PwC studies. The article’s stated aim was to check for year-over-year change between 2024 and 2025, but the actual deltas reported are tiny and arguably within statistical noise (e.g., NIST references going from 113 to 118 companies). No test of statistical significance is applied, so we don’t know if any trend is real.
The “boilerplate” problem is raised but not pursued. The article itself notes that “the devil is in the details” on proactive testing disclosures, and that while it’s good to see such language, “it’s mostly boilerplate.” This is actually the most important critique one could level at Section 1.C — that companies are filing legally compliant but substantively empty disclosures — yet the article treats it as a parenthetical rather than a central finding.
The “no material impact” observation needs harder scrutiny. The article flags the curious pattern of companies acknowledging continuous sophisticated attacks while consistently disclosing no material adverse effect, specifically noting the VOLT/SALT TYPHOON telecom compromises as a tension point. This is the most genuinely interesting observation in the piece, but the author simply says he’ll “do further research” rather than digging in. It’s the one place where the article touches on the core credibility problem with these filings, then pivots away.
The AI methodology section dominates the back half. The article is split roughly 50/50 between findings and a description of the pipeline used to generate them. For a CSO Online audience, the “I learned Python and used Gemini-CLI” narrative is more self-congratulatory than analytically useful. A tighter piece would have moved this to a sidebar or appendix.
Sample is limited to the S&P top 200. These are the best-resourced, most legally scrutinized companies in the country. Conclusions drawn about them shouldn’t be generalized to public companies broadly, and the article doesn’t adequately flag this limitation.
The Bigger Question: Have 10-K Cyber Disclosures Historically Been Useful?
Honestly, the answer is mostly no — and this article inadvertently illustrates why.
The structural problem. SEC disclosures are drafted by legal and compliance teams whose primary goal is liability minimization, not investor education. Cybersecurity sections in particular are written to satisfy the regulatory checkbox while revealing as little operationally sensitive (or reputationally damaging) information as possible. The result is language that is technically accurate and substantively hollow.
The “materiality” escape hatch. The SEC’s 2023 rule requiring disclosure of “material” cybersecurity incidents created a standard that companies have interpreted very permissively. As the article notes, virtually every large company claims no material impact despite being under continuous attack. The definitional flexibility around “material” has functioned as a wide escape hatch. The SolarWinds and Salt Typhoon situations demonstrated that companies can be deeply compromised for extended periods without triggering material disclosure obligations.
Prior to the 2023 rule, disclosures were almost meaningless. Before the SEC’s formal cybersecurity disclosure requirements, companies mentioned cyber risk in boilerplate risk factor sections that were essentially identical year-to-year and gave no company-specific information. Academic studies found that these disclosures had near-zero informational content for investors attempting to assess relative cyber risk.
The governance data is marginally useful. The one area where 10-K disclosures arguably provide some signal is the governance questions: who owns cyber risk, does it reach the board, what frameworks are in use. This is what Dye’s analysis focuses on, and it’s fair to say this data is more useful than nothing. Investors and analysts can at least distinguish companies where cyber is a board-level conversation from those where it’s buried under the CIO. But even here, the mapping between disclosed governance structure and actual security posture is weak — a company can tick every governance box and still be poorly defended.
The material incident disclosure piece has more potential but needs teeth. The 2023 rules’ 4-day reporting requirement for material incidents is the most potentially useful element, but its effectiveness depends entirely on how “material” gets interpreted and enforced. The SEC’s action against SolarWinds and its CISO in 2023 (subsequently partially dismissed) showed both the ambition and the limits of that enforcement approach.
The verdict. These disclosures function better as a compliance artifact than as a genuine risk assessment tool. They tell you about governance structure and framework adoption, which is something. They tell you almost nothing about actual security posture, attack surface, incident history, or relative risk compared to peers. A sophisticated investor or analyst looking for real cyber risk signal would do better looking at breach databases, cyber insurance pricing, and independent security ratings services than at Section 1.C of a 10-K.
The article is a competent data-aggregation exercise that confirms what most security professionals already suspected, without pushing hard enough on the most uncomfortable conclusion lurking in its own findings: that the SEC’s disclosure framework may be generating a false sense of transparency.
