---
name: replicate
description: >-
  Independently reproduce the experiments in The Blind Machine paper and confirm
  its published numbers, using only the open Blind CLI and public data — no hosted
  service, no credentials, no private records. Use this whenever someone wants to
  replicate, reproduce, re-run, verify, or check the results, experiments, or
  benchmarks of The Blind Machine (blindmachine.org/papers/the-blind-machine):
  "does this paper reproduce?", "rerun their experiments", "confirm the
  allele-frequency / differencing / FST / LD numbers", or "check the
  encrypted-vs-cleartext exactness". Runs the synthetic BFV evaluation and the four
  real-genome studies (E5-E8) and diffs the outputs against the committed results.
---

# Replicate The Blind Machine experiments

You are an AI agent asked to **independently replicate** the experiments in the
paper *The Blind Machine: Governed Computation on Encrypted Data with
Certificate-Bound, Reproducible Verification* and confirm that the numbers
reproduce. The point of this skill is that the whole trust surface is open: the
same signed applications the platform runs are on your machine, so you can prove
the results yourself rather than take them on faith. Everything runs on open tools
and public data — you never need the hosted service, credentials, or any private
record.

Work in a scratch directory and end with a short PASS / FAIL / SKIP summary.

## What you are proving

Two empirical claims, both checkable end to end:

1. **Exactness** — for six BFV applications on synthetic cohorts and for four
   studies on real public human genomes, the value decrypted from encrypted
   execution equals a plaintext oracle *exactly*.
2. **Release governance** — a minimum-cohort floor alone does not stop a
   K-versus-K+1 differencing attack; cohort freeze and a query budget do.

## Prerequisites

- Python 3.11+ and [`uv`](https://docs.astral.sh/uv/) — the toolchain the
  application bundles seal their environments with.
- `git`, `bash`, and (only for the real-genome studies E5-E8) `bcftools`.
- Network on first run: the bundles download TenSEAL once, and E5-E8 fetch bounded
  public intervals from the IGSR/1000 Genomes FTP. ~2 GB free disk.

If a prerequisite is missing, mark the affected steps **SKIP** rather than FAIL —
a tool you could not install is not a failed replication.

## Procedure

### 1. Get the code

The open-source Blind CLI is the whole trust surface; the six curated applications
are open at https://blindmachine.org/applications/. Obtain the project (CLI at
https://github.com/blindmachine/blind-cli, plus the `applications/` and
`docs/paper/experiments/` trees the paper documents) and change into it.

### 2. Reproduce the synthetic evaluation (paper Section 10) — no network, no real data

```bash
cd docs/paper/experiments
bash run_all.sh              # seals 6 app envs, drives real BFV, asserts invariants (~90s)
python3 verify.py            # re-check machine-independent invariants only
```

Expect `RESULT: PASS`. Confirm the six ciphertext sizes and `max_error == 0`
against **`references/expected-results.md`** (the exact byte counts are
deterministic, so a mismatch is a real regression, not noise).

### 3. Reproduce the real-genome studies (paper Section 11) — public IGSR data

```bash
bash e5_real_human_dna_igsr.sh     # allele-frequency panel
bash e6_public_af_fst_panel.sh     # cross-population differentiation
bash e7_beacon_release_policy.sh   # differencing attack + release policy
bash e8_public_ld_window.sh        # draft encrypted-product LD window
python3 summarize_public_real_dna.py
```

Confirm each study's headline numbers against **`references/expected-results.md`**.
Each script fetches only the bounded interval it needs, runs the signed
applications locally under real BFV, checks the decrypted output against a
cleartext oracle, and writes aggregate-only results (individual genotype vectors
stay under an ignored `work/` directory).

### 4. (Optional) verify a computation you did not run

```bash
uv run blind certificates verify <certificate.json>   # re-check schema + binding
uv run blind applications install <name>              # verify digest + signature
uv run blind explain <name>                           # human-readable account
```

### 5. Report

Report **PASS** only if `run_all.sh` printed `RESULT: PASS`, `verify.py` exited
zero, all six synthetic ciphertext sizes matched with `max_error` 0, and the E5-E8
headline numbers matched. Otherwise report **FAIL** with the specific script,
expected value, and observed value. Mark anything you could not run **SKIP**. Do
not invent numbers you did not observe.

If a step fails, consult **`references/troubleshooting.md`** before concluding —
most failures are environment issues (missing `bcftools`, no network, a cold
TenSEAL build) that resolve to SKIP, not a real regression.

## Reference files

- `references/expected-results.md` — every expected value to check (synthetic
  ciphertext taxonomy + E5-E8 headline numbers). Read it before reporting.
- `references/troubleshooting.md` — failure modes and whether each is SKIP or FAIL.