{ "qc_role": "adversarial QC B (demote by default; promote only with proof)", "qc_date": "2026-09-26", "method": "All pages fetched with curl on the research host (the research working directory) and converted to text; quotes string-matched against extracted text. IBM docs pages via WebFetch (IBM blocks curl). Local machine used only for reading.", "cell_rulings": [ { "slug": "ibm-guardium-quantum-safe", "cell": "C5", "researcher_score": 6, "qc_score": 0, "verdict": "lowered", "evidence": "arXiv 2608.04857v1 HTML: 'we ran CBOMkit-hyperion (the PQCA sonar-cryptography plugin, v1.6.1) on the Go-invocation subset of Cryben'; Table 2 'All Go CBOMkit 70 38 7 32 0.84 0.54 0.66'. The paper contains zero occurrences of 'IBM' or 'Guardium'. github.com/IBM/sonar-cryptography and github.com/IBM/cbomkit return 301 to github.com/cbomkit/*; GitHub API: cbomkit/cbomkit license Apache-2.0, owner 'cbomkit'. research.ibm.com/blog/quantum-safe-cbomkit: 'IBM Research has developed and open-sourced CBOMkit' and 'IBM is donating its CBOM toolset to the Linux Foundation'; the blog links CBOMkit to no Guardium, GCM or QSE product. The QSE 2.x overview (WebFetch) mentions no sonar-cryptography, CBOMkit, hyperion or SonarQube, and has no accuracy, precision, recall or false-positive statement. A web search for Guardium Cryptography Manager / Guardium Quantum Safe together with CBOMkit / sonar-cryptography returned no link. A third party, QCecuring's import docs (docs.qcecuring.com/cbom/platform/import-export), lists 'cbomkit-theia — IBM's open-source CBOM scanner' and 'IBM Quantum Safe Explorer' as SEPARATE sources.", "reason": "The re-score says the CBOMkit footing is 'consistent with C1 and C2', but that does not hold: the C1 rationale uses no CBOMkit evidence, and the C2 quote comes from the QSE overview (the cbomkit README adds nothing). CBOMkit therefore carries weight in C5 only. The metric measures an Apache-2.0 community tool that the paper credits to PQCA and that is now governed outside IBM. No IBM product doc (GQS 3.x, GCM 2.0.x, QSE 2.x) says it uses that engine. No metric or false-positive handling was found in the Guardium/QSE docs searched, so anchor 0 applies. What would restore 6 (and only 6, because n=70 < 100 and there is no dual annotation): an IBM doc stating that GQS, GCM or QSE uses the sonar-cryptography/CBOMkit-hyperion engine. Row total falls 6.21 -> 5.58 (106/19). Anchor 8, which the published 15 Aug correction used, fails on every axis whatever the scope." }, { "slug": "cbom-secure", "cell": "C1", "researcher_score": 10, "qc_score": 10, "verdict": "upheld", "evidence": "/cryptographic-discovery-inventory/: '20+ production sensors inventory cryptographic assets across cloud platforms, hardware security modules, KMIP servers, TLS endpoints, directory services, databases, file systems, source code, and binaries.' Datasheet PDF: 'Full NIST quantum-safe coverage: ML-KEM, ML-DSA, SLH-DSA (FIPS 203–205), plus Falcon. Readiness tracked across keys, certificates, and cipher suites — hybrid TLS (X25519MLKEM768) detected in production.' Posture page: 'capturing algorithm, key size, key state, and rotation metadata'.", "reason": "At least 7 surfaces carry per-surface mechanisms in the datasheet coverage table. Parameter depth and explicit PQC plus hybrid detection are documented. All three 10-conditions are met, so no demotion ground was found." }, { "slug": "cbom-secure", "cell": "C2", "researcher_score": 7, "qc_score": 7, "verdict": "upheld", "evidence": "/cryptographic-discovery-inventory/: 'Audit artifacts generated in CycloneDX 1.6 and 1.7.' Posture page: 'Exportable proof: the CycloneDX export and the tamper-proof audit trail give assessors current-state evidence and verifiable history in one package.' and 'What does it export? The full inventory in CycloneDX, plus dashboards, KPI reports, and a cryptographically verifiable audit trail of every asset change.'", "reason": "I searched to RAISE this cell. The export package does include a 'cryptographically verifiable' audit trail, but integrity is claimed for the change log, not for the CBOM artifact itself, and no signature or hash-chain mechanism is named. No public sample was found. The evidence sits between 6 and 8, so 7 holds. The same unnamed log earns full credit in C3 because C3's anchor asks for tamper-evident HISTORY, which is claimed directly, while C2's anchor asks for integrity ON THE EXPORTED ARTIFACT, which is not claimed." }, { "slug": "cbom-secure", "cell": "C3", "researcher_score": 8, "qc_score": 8, "verdict": "upheld", "evidence": "Functionalities page: 'Continuously monitor cryptographic posture and detect configuration changes, key reuse, and emerging risks over time.' Posture page: 'alerting via email and Microsoft Teams' and 'A tamper-proof audit trail records every asset change in a cryptographically verifiable log: what changed, when, and by whom.'", "reason": "Continuous monitoring, change alerts and tamper-evident history are all claimed, the history repeatedly across 2 pages. No detection latency is published anywhere in the 7 vendor pages or the datasheet I fetched, so 9 is not met." }, { "slug": "cbom-secure", "cell": "C4", "researcher_score": 7, "qc_score": 7, "verdict": "upheld", "evidence": "Datasheet: 'Four risk bands: Critical, High, Low, Safe. Scoring weighs algorithm strength, expiry, key reuse, cipher mode, IV/nonce handling, KDF parameters, quantum exposure.' /cryptographic-discovery-inventory/: 'Classify assets by sensitivity, NIST policy alignment, and quantum exposure.' Blog /cbom-inventory-to-intelligence/: 'Effective risk scoring across your CBOM should account for four distinct dimensions: Harvest Now, Decrypt Later (HNDL) exposure...'", "reason": "I searched to RAISE this cell. The HNDL passage is generic advice ('should account for') and never states that CBOM Secure's 0-100 score includes HNDL or data lifetime. Sensitivity classification is not data lifetime. The 8-anchor HNDL/lifetime factor is therefore not documented, and 7 (numeric multi-factor score plus bands, short of 8) holds." }, { "slug": "cbom-secure", "cell": "C5", "researcher_score": 4, "qc_score": 4, "verdict": "upheld", "evidence": "/cryptographic-discovery-inventory/: '...focuses remediation efforts on real-world exposure, eliminating false positives...'. The V1.1 'What Testing and Validation Evidence Backs V1.1?' section gives only outcome claims ('70 to 80 percent less time', 'more than 90 percent'), with no precision, recall or benchmark. The Limitations section says custom crypto 'may still need manual review'.", "reason": "False-positive handling is described but no metric was found. docs.encryptionconsulting.com returned HTTP 525 again on 2026-09-26, and Wayback has no snapshots, so the portal is unread and proves nothing either way." }, { "slug": "cbom-secure", "cell": "C6", "researcher_score": 5, "qc_score": 6, "verdict": "raised", "evidence": "/cryptographic-discovery-inventory/: 'Audit artifacts generated in CycloneDX 1.6 and 1.7.' and 'Drive phased remediation across mapped dependencies.' Functionalities page: 'Integrate with SIEM, GRC, and ticketing platforms to streamline remediation workflows.' and 'Plan migration with crypto agility scoring and dependency analysis. Replace weak, deprecated, and quantum vulnerable cryptography first.' V1.1 page: 'so analysts know what to fix'. On /integration/, the ServiceNow ('Automate Certificate Issue Tracking...') and Jira ('Raises and syncs Jira tickets...') entries are tagged 'Certsecure Manager', not CBOM Secure.", "reason": "The 6 anchor reads 'one-way ticket/EXPORT plus guidance'. A CycloneDX export plus concrete remediation guidance meets it on its face. The researcher gave QCecuring 6 on exactly this pattern (JSON export plus an 'Action Required' field), so leaving CBOM Secure at 5 was an inconsistency against the #2 product. The raise stops at 6 because the named Jira/ServiceNow ticketing belongs to a different product (CertSecure Manager) and no automated remediation is documented for CBOM Secure. Row total rises 7.37 -> 7.47 (142/19)." }, { "slug": "cbom-secure", "cell": "C7", "researcher_score": 8, "qc_score": 8, "verdict": "upheld", "evidence": "Posture page, a per-framework table: 'NIST IR 8547 Surfaces all quantum-vulnerable asymmetric cryptography for migration planning.' plus CNSA 2.0, SP 800-131A, FIPS 140-3, CMMC, PCI DSS 4.0 and FedRAMP rows. 'Reporting includes dashboards built from 29 widgets and 52 built-in KPIs'. Datasheet: 'role based dashboards' and 'OpenAPI documented endpoints for every dashboard widget and KPI'.", "reason": "I searched to RAISE this cell. No published sample report was found in any fetched page and no distinct executive-report artifact is documented, so 10 is not met and 8 holds." }, { "slug": "qcecuring-cbom", "cell": "C1", "researcher_score": 9, "qc_score": 9, "verdict": "upheld", "evidence": "docs.qcecuring.com/cbom: 'Sensors scan 11 source types' in a table covering Network (TLS Endpoint, SSH Endpoint), Filesystem, Source Code, Binary, Cloud (AWS, Azure Key Vault), Identity (AD/ADCS) and Windows Certificate Store. The risk table lists 'NONE | Quantum-safe | AES-256, SHA-256+, ML-KEM, ML-DSA'. /scanners/network example: 'Certificate: \"api.example.com\" (RSA-2048...)' and 'SSH-2 with key exchange curve25519-sha256'. No 'hybrid', 'key_share' or 'X25519MLKEM768' string appears in any of the 22 pages.", "reason": "Without double-counting KMS as both (e) and (f), the docs still show 7 surfaces with scanner pages: a, b, c, d, f, g (agent filesystem) and i (SSH). Each has algorithm plus key-size depth. PQC appears only as a classification row and hybrid detection is not documented, so the evidence sits between 8 and 10." }, { "slug": "qcecuring-cbom", "cell": "C2", "researcher_score": 6, "qc_score": 6, "verdict": "upheld", "evidence": "/platform/import-export: 'Exports your full cryptographic inventory as a CycloneDX v1.6 JSON document.' The export lists algorithmProperties, certificateProperties, relatedCryptoMaterialProperties, protocolProperties, dependencies and BOM-Link. The word 'tamper' appears only on /administration/licensing. The docs link to https://github.com/qcecuring, which has no public repos.", "reason": "A standard-schema export is documented, with no integrity mechanism on the CBOM and no public sample." }, { "slug": "qcecuring-cbom", "cell": "C3", "researcher_score": 7, "qc_score": 6, "verdict": "lowered", "evidence": "/platform/sensors: 'Schedule | Scan frequency (hourly, 6h, 12h, daily, weekly)'. /platform/compliance: 'Click Run Assessment on any standard...' then 'Trend Comparison | Delta vs. the previous assessment for the same standard | Direction indicator: IMPROVED, REGRESSED, or UNCHANGED'. /architecture: one table row, 'Alerts | Email notifications for policy violations and certificate expiry'. No tamper-evident history, audit log, hash chain or signature was found in the 22 docs pages.", "reason": "The diff comes from a user-triggered compliance assessment, not from scan-to-scan drift on the scheduled scans. Anchor 6 is therefore met only loosely. One of the two 8-conditions (tamper-evident history) is wholly absent, and the alert is a single architecture-table row. That is not 'clearly between' two anchors, so the rubric's lower-anchor default applies. Row total falls 6.68 -> 6.53 (124/19)." }, { "slug": "qcecuring-cbom", "cell": "C4", "researcher_score": 6, "qc_score": 6, "verdict": "upheld", "evidence": "docs /cbom: 'Every asset is classified automatically: Risk Level Meaning Examples CRITICAL Broken or deprecated ... HIGH Quantum-vulnerable RSA, ECDSA, ECDH, DH, DSA'. The vendor blog /blog/quantum-risk-scoring-methodology (10 Jun 2026) publishes 'Weight_AV = 0.25 ... Weight_RP = 0.20 ... Weight_HE = 0.15' and claims 'QCecuring CBOM integrates quantum risk scoring directly into the CBOM generation process'. The blog's example uses 'qrs:' CBOM properties, but grep finds 0 'qrs' occurrences in the documented export (/platform/import-export) or inventory page.", "reason": "The technical docs contradict the blog's integration claim: the export property list and asset panel have no score field. Categorical risk by algorithm plus key-size rules is anchor 6. SENSITIVITY: if a reviewer credits the blog, this cell jumps to 10 (formula, weights, HNDL and retention published). That is the single largest swing in this file, so it should be disclosed." }, { "slug": "qcecuring-cbom", "cell": "C5", "researcher_score": 4, "qc_score": 4, "verdict": "upheld", "evidence": "/scanners/filesystem: 'Encrypted PEM keys are detected but cannot be parsed without the passphrase (they still appear as assets with algorithm info)'. /platform/sensors: 'completed / partial / failed (with error count badge)'. 'precision' and 'false positive' each appear 0 times across the 22 pages.", "reason": "Parse limitations and error handling are described, with no metric. Anchor 4 is generous but defensible." }, { "slug": "qcecuring-cbom", "cell": "C6", "researcher_score": 6, "qc_score": 6, "verdict": "upheld", "evidence": "/platform/compliance: 'Action | Required remediation (e.g., “Migrate to ML-KEM”)' and 'Export the full assessment as JSON.' 'Jira', 'ServiceNow', 'ticket' and 'webhook' each appear 0 times in the 22 docs pages.", "reason": "Export plus guidance meets the 6 anchor as written. The same reading is applied to CBOM Secure C6 for consistency." }, { "slug": "qcecuring-cbom", "cell": "C7", "researcher_score": 7, "qc_score": 7, "verdict": "upheld", "evidence": "/platform/compliance: 'Built-in Standards | CNSA 2.0 — NSA Commercial National Security Algorithm Suite | NIST PQC — Post-Quantum Cryptography transition requirements | FIPS 140-3 — Approved algorithms and minimum key sizes'. /platform/dashboard: 'The Dashboard provides an executive summary of your cryptographic posture' and 'Reports | Generate and export reports' (no dedicated docs page).", "reason": "Compliance mapping, export and API are documented. The executive artifact is a dashboard, not a report, so the cell sits between 6 and 8." } ], "row_totals_after_qc": { "ibm-guardium-quantum-safe": { "researcher": 6.21, "qc": 5.58, "note": "Only C5 was reviewed. The total uses the researcher's other six cells." }, "cbom-secure": { "researcher": 7.37, "qc": 7.47 }, "qcecuring-cbom": { "researcher": 6.68, "qc": 6.53 } }, "ibm_paper": { "exists": true, "authors": "Christian Näther (XITASO GmbH, Augsburg) and Eduard Hirsch (University of Applied Sciences Amberg-Weiden). Title: 'Hidden Ciphers and Where to Find Them: Static Discovery and Assessment of Cryptographic Assets in Software', arXiv:2608.04857v1 [cs.CR], submitted 5 Aug 2026, CC BY 4.0.", "what_tested": "Mainly the authors' own scanner Crypsy (rule repository Crypistry) on their synthetic benchmark Cryben (197 ground-truth occurrences, Go/Ruby/nginx; Crypsy P 0.87 R 0.66 F1 0.75) and on a 10-service real-world system with only partial manual ground truth. As a side comparison it tested 'CBOMkit-hyperion (the PQCA sonar-cryptography plugin, v1.6.1)' on the Go-invocation subset only. No IBM Guardium, Guardium Cryptography Manager or Quantum Safe Explorer product was tested, and the string 'IBM' appears 0 times in the paper.", "is_ibm_product": "No. CBOMkit / sonar-cryptography is an Apache-2.0 open-source project that IBM Research originated ('IBM Research has developed and open-sourced CBOMkit'; 'IBM is donating its CBOM toolset to the Linux Foundation'). github.com/IBM/cbomkit and github.com/IBM/sonar-cryptography now 301 to the independent 'cbomkit' GitHub org, and the paper credits it to PQCA. No IBM product documentation links it to Guardium Quantum Safe, GCM or QSE. Crediting the result to 'IBM Guardium + Quantum Safe' does not hold.", "n": "70 Go-invocation ground-truth occurrences for CBOMkit (TP 38, FP 7, FN 32 -> P 0.84, R 0.54, F1 0.66). On the 55-occurrence Go-stdlib subset: P 0.80, R 0.65, F1 0.72. Every n is well under 100 and far under the 1000 that anchors 8-10 require.", "ground_truth": "The synthetic corpus Cryben. The paper says: 'Its ground truth is a CycloneDX v1.7 CBOM constructed independently of Crypsy' and 'Cryben was built independently of Crypsy's current rule set'. It was built by the authors of the competing tool Crypsy, which makes it independent of CBOMkit by construction. It is not independent of the paper's own authors, and no dual annotation or inter-annotator agreement is stated. The CBOMkit comparison scores both tools with the authors' own evaluator.", "peer_reviewed_status": "Accepted after peer review at ICICS 2026 (28th International Conference on Information and Communications Security, Fukui, Japan, 27-30 Oct 2026, Springer LNCS). This is corroborated beyond the arXiv comment: the official program (sulab-sever.u-aizu.ac.jp/icics2026/program.html) lists it in 'Session 15: Security Assessment and Software Vulnerability Analysis · Regular papers', slot 11:00-11:15. The claim that 'accepted at ICICS 2026' appears only in the arXiv comments is FALSE. The conference has not yet taken place and the proceedings are unpublished. What was read is the arXiv v1 preprint, and the camera-ready version may differ. On 15 Aug 2026 'peer-reviewed' could have been checked only through the arXiv comment and the program listing.", "correction_description_accurate": "Partially. The word 'peer-reviewed' holds (accepted paper, confirmed by the program). 'Independently constructed ground truth' holds relative to CBOMkit, not relative to the benchmark's authors. The material implications behind raising IBM C5 to 8 fail: the tool tested is not a Guardium product but a Linux Foundation / PQCA open-source tool; n=70, not >=1000; there is no dual annotation; the result is unflattering (recall 0.54); and the paper is primarily a benchmark of the authors' own tool, with CBOMkit as a baseline. At most it supports anchor 6 for CBOMkit itself, and 0 for the IBM Guardium + Quantum Safe row unless IBM documents that its products use that engine.", "evidence_urls": [ "https://arxiv.org/abs/2608.04857", "https://arxiv.org/html/2608.04857", "https://sulab-sever.u-aizu.ac.jp/icics2026/program.html", "https://sulab-sever.u-aizu.ac.jp/icics2026/", "https://research.ibm.com/blog/quantum-safe-cbomkit", "https://github.com/cbomkit/cbomkit", "https://github.com/cbomkit/sonar-cryptography", "https://github.com/IBM/sonar-cryptography (301 -> cbomkit/sonar-cryptography)", "https://www.ibm.com/docs/en/quantum-safe/quantum-safe-explorer/2.x?topic=quantum-safe-explorer-overview" ] }, "additional_findings": [ "The prompt's premise that ICICS acceptance 'appears only in the arXiv comments' is wrong. The ICICS 2026 program lists the paper as a regular paper.", "The IBM re-score says its CBOMkit footing is 'consistent with C1 and C2', but that does not hold: C1 uses no CBOMkit evidence, and the C2 quote comes from QSE docs. CBOMkit carries weight in C5 only.", "CBOM Secure C6 was inconsistent with QCecuring C6: the same export-plus-guidance pattern had been scored 5 and 6. It is fixed in CBOM Secure's favour.", "The CBOM Secure evidence base is mostly vendor blog posts on encryptionconsulting.com (the posture page is 'Posted by Surbhi Singh June 17, 2026'). The rubric bars only third-party blogs, so they stay admissible, but the product's technical docs portal (docs.encryptionconsulting.com) is unreachable (HTTP 525, no Wayback copy).", "The QCecuring docs (22 pages, Starlight site) read as genuine product documentation, with config schemas, field defaults and example outputs. Caveats: the pages are undated, the documented 'git clone https://github.com/qcecuring/cbom.git' target returns 404, and the GitHub org has no public repos. Nothing can be exercised without a vendor licence.", "The QCecuring blog's integrated HNDL scoring claim conflicts with its own docs (0 'qrs:' properties in the documented export). If a reviewer credits the blog, C4 rises from 6 to 10 (+0.47 on the total)." ], "overall_verdict": "Three cell changes. (1) IBM C5 lowered from 6 to 0 (row 6.21 -> 5.58): the 'peer-reviewed benchmark' is real and was accepted at ICICS 2026, but it measured an Apache-2.0 PQCA/Linux Foundation tool (CBOMkit-hyperion v1.6.1) on n=70 with single-source ground truth, and no IBM doc ties that engine to Guardium, GCM or QSE. The published 15 Aug rise to 8 has no support under any scope. (2) CBOM Secure C6 raised from 5 to 6 (row 7.37 -> 7.47) for consistency with the export-plus-guidance anchor. Its other six cells are upheld after an attempt to raise each. (3) QCecuring C3 lowered from 7 to 6 (row 6.68 -> 6.53): the delta is on-demand only and there is no tamper-evident history. Its docs portal is genuine product documentation and the rest of its rise stands." }