# Datasheet — Safeguarding Institutional-Failure Network Dataset (v1.0) Inference Institute · Safeguarding domain. Following Gebru et al., *Datasheets for Datasets* (2021). Companion to the paper *Structure of Failure*. **AI-involvement disclosure is in §8 — it is part of the release, not a footnote.** --- ## 1. Motivation - **Why was the dataset created?** To make the lessons inside public statutory reviews and inquiries into child sexual abuse and related deaths *reproducibly comparable*. It is the coded, anonymised layer behind a pipeline that represents each case as a network and compares cases for recurring structural failure patterns. - **What task does it support?** Cross-case, graph-theoretic analysis of institutional-failure structure (centrality, community detection, similarity, clustering, motif mining) — **record-integrity analysis of the institutional record**. - **Out of scope, by design.** The dataset does **not** support, and must not be used for, scoring the credibility of any complainant or judging/predicting individuals. It contains no complainant data. - **Who created it?** Daniel Fat, Inference Institute. ## 2. Composition - **What do instances represent?** One instance = one public review/inquiry, coded into a controlled-vocabulary structural record (agencies involved, ordered decisions, risk indicators, and the record-integrity failures the review *itself* identified). - **How many instances?** 11 cases (reference corpus): 6 IICSA investigation reports, 2 council-commissioned independent inquiries, 1 Serious Case Review, 1 Child Safeguarding Practice Review, 1 Domestic Homicide Review. Jurisdiction: 10 England & Wales, 1 England. - **Is it a sample?** Yes — a *purposive* sample across review types to demonstrate and evaluate the method. It is **not** a representative or complete sample; no rate or population claim is supportable from it. - **What does each instance consist of?** `case_id` (anonymised), `source_type`, `jurisdiction`, `year`, public `source_ref` (title, repository, URL, licence), `agencies[]`, `risk_indicators[]`, ordered `decisions[]` (decision type + owner agency), `integrity_flags[]`, and an aggregate `scale_band`. Controlled vocabularies: 21 agencies, 15 decision types, 18 risk indicators, 15 record-integrity flags (schema v1.0). - **Is there a label / target?** No predictive target. The `integrity_flags` are descriptive codings of failures the source reviews state. - **Any confidential or identifying data?** No. Only a coded structural layer with anonymised case identifiers and public source citations. Institution names are **not** stored in the coded layer; victim-identifying content never enters it. - **Known errors / noise / redundancy?** Coding of interpretive categories (decision types, integrity flags) is lower-agreement than factual ones (see §5/§8). Six coded items were flagged by the hallucination audit as inferential and retained, marked. ## 3. Collection process - **How was the data acquired?** Public reports were retrieved from their official repositories (IICSA via the National Archives; NSPCC National Case Review Repository; council and Home Office sites) in June 2026. A provenance manifest records source, URL, retrieval date, licence, and a SHA-256 hash per document. - **Raw vs derived.** Raw report text is held in a **git-ignored** working area and is **not** published. Only the coded, anonymised layer, the pipeline code, and the provenance manifest are released. - **Ethics.** Public-domain, already-anonymised material only; complainant anonymity (Sexual Offences (Amendment) Act 1992) and reporting restrictions respected; no re-identification. Deployment on non-public records would require ethics-board and data-controller sign-off. ## 4. Preprocessing / cleaning / labelling - Each report was reduced to an institutional-process summary (no graphic or victim-identifying narrative), then coded into the controlled vocabulary by an LLM-assisted, human-validated process (see §8). A schema validator rejects any out-of-vocabulary token. The cleaned coded layer is `cases.json`; a flat binary indicator matrix (`cases_indicators.csv`) is derived for analysis. ## 5. Validation - **Inter-rater reliability** (human-validated pass A vs an independent second coding, pass B): Cohen's κ = 0.93 (agencies), 0.91 (risk indicators), 0.76 (integrity flags), 0.70 (decision types); mean κ = 0.83; percent agreement 0.89–0.97. Agreement is almost perfect on factual categories and substantial on interpretive ones. - **Hallucination / error audit:** 97.2% of coded items (206/212) had direct lexical support in the source text; 6 inferential codings were surfaced and retained, marked. - **Caveat:** the second coder is an independent *model* pass, not a team of trained human coders. Human inter-rater reliability is **required future work**, not claimed here. ## 6. Uses - **Suitable uses:** methodological research on reproducible statutory-review synthesis; cross-case structural analysis; teaching examples of record-integrity analysis; prevention-hypothesis generation (as causal-diagram hypotheses, not effects). - **Unsuitable uses (do not):** estimating prevalence or rates; judging individuals; scoring complainant credibility; any operational decision about a specific person or case. The corpus is small and purposive. ## 7. Distribution & maintenance - **Licence:** CC-BY 4.0 for the coded dataset. - **Where:** the coded dataset, indicator matrix, code, and this datasheet ship with the paper on inference.institute. - **Versioning:** schema v1.0 / dataset v1.0. Changes are versioned; this datasheet is updated with the data. - **Maintainer:** Daniel Fat, Inference Institute. ## 8. AI-involvement disclosure - **Where AI was used.** A language model performs the first-pass extraction: proposing a controlled-vocabulary coding (agencies, decisions, risk indicators, integrity flags) from each report's institutional-process summary, through grammar-constrained structured output (Pydantic-AI) over an OpenAI-compatible interface. A local embedding model produces a content-based semantic view of the cases. - **Models and controls.** Model and version are pinned, temperature is 0, prompts are versioned, and prompts/responses are logged. The model **proposes**; a human validator and a schema validator **dispose** — out-of-vocabulary tokens are rejected. We run two tiers over the same interface: (a) a strong reference model, used to produce the published, human-validated coding and the independent second coding; and (b) a small, fully-local, private model — `liquid/lfm2.5-1.2b` (1.2B parameters) served by LM Studio — run as an on-device baseline, with the chat model capped at three concurrent requests and kept resident (no reload per call). Embeddings use `text-embedding-nomic-embed-text-v1.5` (local, up to ten concurrent requests). - **How AI involvement was validated.** Inter-rater agreement (§5), a lexical hallucination audit (§5), and a separate fully-local baseline are reported in full. The local 1.2B model reaches moderate agreement on the concrete agencies category (Cohen's κ ≈ 0.60) but low agreement on the interpretive/ordered categories (κ ≈ 0.07–0.23; mean ≈ 0.27), against the strong reference's mean κ ≈ 0.83. Per-field decomposition was necessary to elicit usable output from the small model. - **Residual limitations.** Reliability decreases as categories become interpretive; a small local model is far weaker than a strong one and is not, on its own, fit to finalise the coding (hence the released dataset is human-validated, not model-final); automated agreement is not a substitute for trained-human reliability; the hallucination audit is a conservative lexical proxy. These are stated plainly — that transparency is the point. ## 9. Scaled model-coded corpus (a second, separate release) Alongside the 11-case human-validated reference dataset, we release a larger, **model-coded** corpus to demonstrate that the pipeline scales and to provide a reusable starting point. - **What it is.** 61 public statutory reviews and inquiry reports compiled from 22 publishers (28 Child Safeguarding Practice Reviews, 11 Domestic Homicide Reviews, 9 Serious Case Reviews, 8 IICSA investigation volumes, 4 council inquiries, 1 Northern Ireland review), 2014–2026. Reports were fetched from their official sources (provenance manifest: URL, retrieval date, SHA-256, licence) and coded with the local model (`liquid/lfm2.5-1.2b`, per-field, temperature 0). - **What is published.** Only the **derived structural codes + provenance links** (`safeguarding-reviews-scaled-dataset.json`). The report **text is not published** — per-document copyright applies (kept in the git-ignored raw area). - **Quality — read before use.** This corpus is **AI-coded and NOT human-validated**. The agency coding is sound at scale (police, children's social care, local-authority leadership, and the safeguarding partnership are the recurrent core, reproducing the human-coded result). But the **interpretive coding is unreliable**: the small model tags "information not shared / siloed working" in ~98% of reviews (an artefact) and under-codes the oversight and ignored-warning patterns the validated core shows. **Do not** use the model-coded record-integrity flags or motif/syndrome shares from this corpus as findings; use the agency and source-type structure, or re-code with human validation or a stronger model. - **Licence:** CC-BY 4.0 for the derived codes. Provenance links resolve to the original publishers' reports under their own terms.