Allele Frequency with Variance

# `allele_frequency_with_variance` — Blind Machine multiplicative protocol> tenseal-BFV, **multiplication-supporting params** (depth 1). The server SQUARES> an encrypted value (ciphertext × ciphertext) to derive the second moment, so it> exercises exactly **one** BFV multiplicative level. Same published coordinate> definition and same contributor payload as the flagship> (`allele_frequency_count`) — which makes it the controlled "money comparison"> row: hold the inputs fixed, add one multiplicative level, measure the premium.> See `docs/protocol_catalog.md` §5. The additive **client-precompute benchmark> variant** (client pre-squares, server only sums) is documented in> `BENCHMARK.md`; it is not a separate registry protocol.## What it computesEach contributor holds an alt-allele **dosage vector** `g ∈ {0,1,2}^L` over afixed, published coordinate definition (ordered variants `(chrom,pos,ref,alt)`),identical to the flagship; missing calls encoded as 0. The cohort aggregatereleased is **two** per-coordinate integer vectors — the first and second moments:```sum_g[j]   = Σ_i g_i[j]              (integer, exact — additive path)sum_g2[j]  = Σ_i g_i[j]²             (integer, exact — server squares under encryption)mean[j]     = sum_g[j] / N                          (mean dosage; derived post-decrypt)variance[j] = sum_g2[j] / N − (sum_g[j] / N)²       (population variance E[g²]−E[g]²)frequency[j]= mean[j] / 2                            (2 alleles per diploid coordinate)```**The square is server-side.** The client sends only one ciphertext of `g`. Theserver computes `Σ_i enc(g_ij)²` under encryption — squaring **each contributorfirst, then summing**, because `(Σ g)² ≠ Σ g²`. That per-contributor square isthe ct × ct multiply; it needs relinearization keys in the public context. Thisis the whole point versus the additive benchmark variant, where the client wouldalso encrypt `g²` and the server would only add (`BENCHMARK.md`, `docs/spec.md`).**Exactness:** BFV is exact in `Z_t`. The largest value is `max sum_g2 = 4N`(`g² ≤ 4`), and the plaintext modulus `t = 786433` exceeds it for `N` up to ~196k.`tolerance: 0` — both encrypted integer vectors equal the cleartext moments**bit-for-bit**. `mean`/`variance`/`frequency` are real-valued derivations of thetwo exact integer aggregates.**Append-1 sentinel:** encryption appends a trailing `1` slot to everycontribution, so **both** result vectors' last slot decrypts to **exactly N**(sum path: `Σ 1 = N`; square path: `Σ 1² = N`). `50_decode.py` cross-checks thatthe two sentinels agree. It is an integrity/corruption check, **not a MAC** — itsays nothing about whether contributors are distinct or genuine (see `SECURITY.md`).## Crypto approach — why multiplication-supporting params| parameter | value | why ||-----------|-------|-----|| `poly_modulus_degree` | **16384** (fixed, all levels) | multiplication-supporting ring; 16384 slots ≫ L+1. The depth-1 noise floor (~200 bits) can't fit under the 152/118 caps at n=8192, so 8192 is out for 192/256 — N stays 16384 across all three levels (vary the chain, fix N). The larger ring is the dominant cost vs the flagship's 8192. || `coeff_mod_bit_sizes` | **selected by `--security`** (see table below) | this is the ONLY knob security moves; depth-1 needs ≥2 interior 40/60-bit primes between the two 60-bit special primes. || `plain_modulus` | **786433** (fixed, all levels) | 20-bit NTT batching prime, `≡ 1 (mod 32768)` — **required** at n=16384 (the flagship's `1032193` is invalid here). Exact for `max sum_g2 = 4N`. A function of the value envelope + depth, not of security. || relinearization keys | **yes** | ct × ct raises ciphertext degree to 3; TenSEAL relinearizes back to 2 using relin keys, which `00_keygen.py` generates (a secret key exists at context creation) and **retains through `make_context_public()`**. || Galois keys | **no** | the square is element-wise per slot; there is no rotation, so no Galois keys are generated. |### `--security {128,192,256}` — the coeff-modulus chain`00_keygen.py --security L` (default **128**) selects `coeff_mod_bit_sizes` fromthe authoritative per-level table; `poly_modulus_degree` and `plain_modulus` arefixed. The benchmark's `security` column is the achieved level computed from`(N, Σ coeff_mod_bit_sizes)` against the HomomorphicEncryption.org caps atN=16384 (256 ≤ 237, 192 = 238–305, 128 = 306–438), and **achieved == requested**for every row (verified bit-exact against the cleartext oracle, TenSEAL 0.3.16):| `--security` | `coeff_mod_bit_sizes` | Σ bits | achieved | ciphertext cost ||--------------|-----------------------|--------|----------|-----------------|| `128` (default) | `[60, 60, 60, 60, 60, 60]` | 360 | **128** | largest || `192` | `[60, 60, 60, 60]` | 240 | **192** | middle || `256` | `[60, 40, 40, 60]` | 200 | **256** | smallest |**Intentional inversion:** at fixed N, security level == the q-band and *smaller*Σ ⇒ *more* secure, so the **128-bit** cell uses a **larger** coeff modulus (bigger,slower ciphertexts) than the **256-bit** cell. This is correct RLWE behaviour, nota bug — the depth-1 noise floor for this payload already sits in the 256 band, socertifying 128/192 spends *surplus* modulus. (`[60,40,40,60]=200` technically meetsall three targets; we publish cap-tracking chains so the `security` column reads adistinct, honest 128/192/256.) The chain flows unchanged into every downstreamstage — they all `ts.context_from(...)`, so `10`–`50` stay security-agnostic.**§3 escape hatch (not in the default table):** a *quantized-trait* /oversized-cohort deployment whose grown `t` breaches the 256 cap at N=16384 movesto **N=32768** via the explicit overrides — e.g.`--security 256 --poly-modulus-degree 32768 --plain-modulus 537133057--coeff-mod-bit-sizes 60 50 50 50 60`. This is a per-deployment override, not thedefault binary payload.## Stage lifecycle & I/O contractThe author's logic lives in three pure-function files, grouped by role: `server.py`(`compute`, the only server-side function), `local_project_owner.py`(`keygen`/`decrypt`/`decode`), and `local_data_owner.py` (`encode`/`encrypt`) —these are what sibling `tests/` import. The six numbered files are materialized into `signed/` at run time and are**kit-owned shims** (thin argparse wrappers; do not edit) that map each stage's CLI(`python NN_*.py --help`) onto those functions, keeping the lifecycle visiblewithout opening a subdirectory.| stage | runs | trust in → out | I/O ||-------|------|----------------|-----|| `00_keygen.py` | local (researcher) | — → PRIVATE + PUBLIC context | `--out-dir DIR` → `secret_context.tenseal` (never upload), `public_context.tenseal` (uploadable; **relin keys retained**) || `10_encode.py` | local (data owner) | RAW → ENCODED | `--raw raw.json --length L --out encoded.json` (validate {0,1,2}, null→0, pad to L) — verbatim flagship || `20_encrypt.py` | local (data owner) | ENCODED → ENCRYPTED | `--context public_context.tenseal --encoded encoded.json --out cipher.bin` (appends sentinel, BFV-encrypts a SINGLE ciphertext) — verbatim flagship || `30_compute_encrypted.py` | **SERVER** | ENCRYPTED → ENCRYPTED × 2 | `--context public_context.tenseal --inputs c0.bin c1.bin … --out result.bin` (**squares under encryption**; packs `sum_g` + `sum_g2` into ONE deterministic container; **no secret key present**) || `40_decrypt.py` | local (researcher) | ENCRYPTED × 2 → PRIVATE | `--context secret_context.tenseal --result result.bin --out plain.json` (unpacks both moments, each length L+1) || `50_decode.py` | local (researcher) | PRIVATE → RELEASED | `--plain plain.json --length L --out result.json` (splits both sentinels→N, sum_g, sum_g2, mean, variance, frequency) |**`--out` on the compute stage is a single FILE.** Although this protocol emitstwo result ciphertexts (`sum_g`, `sum_g2`), `30_compute_encrypted.py` packs theminto ONE self-describing, deterministic binary container written at the `--out`FILE path — magic `BMCT1\n` (Blind Machine multi-CipherText container v1), auint8 count then, in fixed `MOMENT_ORDER = (sum, sumsq)`, each moment as alength-prefixed name + length-prefixed raw ciphertext (`pack_results` /`unpack_results`). This is the SAME container format`genotype_phenotype_covariance` uses (each bundle carries its own verbatim copy —bundles are self-contained). The single-file output is what the hosted workercontent-addresses (one opaque `result.bin`, SHA-256'd), so the flag convention(`--context` / `--inputs` / `--out`) matches the flagship's exactly. Fixed fieldorder + length prefixes + no timestamps/maps make the packed bytes deterministic,giving verify-by-re-execution.Inter-stage formats: contexts and ciphertexts are TenSEAL's raw serialized bytes(binary); raw/encoded are JSON int lists; `plain.json` is`{"sum": [L+1 ints], "sumsq": [L+1 ints]}`; the released result is JSON with`n_contributors`, `sum_g`, `sum_g2`, `mean`, `variance`, `allele_frequency`.`server.py`'s `compute` is written **once** against an abstract evaluator `E`(`zero`/`add`/**`mul`**), so `docs/simulation_mode.md`'s cleartext correctnessoracle swaps a `PlaintextEvaluator` for the same `compute` and cannot drift fromthis encrypted path. Determinism gives verify-by-re-execution: the same orderedciphertexts in → bit-identical result digests out (compute is deterministic;encryption is not).## Run the full loop by hand```bashcd protocols/allele_frequency_with_varianceD=/tmp/afv && mkdir -p "$D"R() { (cd signed && uv --project env run python "$@"); }R 00_keygen.py --out-dir "$D"          # add --security {128,192,256} (default 128)for i in 00 01 02 03; do  R 10_encode.py  --raw ../tests/vectors/contributor_$i.json --length 16 --out "$D/enc_$i.json"  R 20_encrypt.py --context "$D/public_context.tenseal" --encoded "$D/enc_$i.json" --out "$D/c_$i.bin"doneR 30_compute_encrypted.py --context "$D/public_context.tenseal" \  --inputs "$D/c_00.bin" "$D/c_01.bin" "$D/c_02.bin" "$D/c_03.bin" --out "$D/result.bin"R 40_decrypt.py --context "$D/secret_context.tenseal" \  --result "$D/result.bin" --out "$D/plain.json"R 50_decode.py  --plain "$D/plain.json" --length 16 --out "$D/result.json"cat "$D/result.json"```## Test (local-loop equivalence)```bashuv --project signed/env run --group dev python -m pytest tests/```Proves keygen → encode → encrypt (≥3 synthetic contributors) → compute (**serversquares**) → decrypt → decode equals the cleartext first- and second-momentoracle **exactly** (both `sum_g` and `sum_g2`), and that the sentinel decrypts to**exactly N** in both paths (including that dropping one upload yields N−1). A**parametrized case runs the full loop at each `--security` level (128, 192,256)** and asserts bit-exact moments + sentinel==N at every level, plus that theshipped chain lands in the requested q-band (achieved == requested). One testguards the mandatory square-then-sum (`Σ g² ≠ (Σ g)²`); one runs the **additiveclient-precompute benchmark variant** and asserts a bit-identical `sum_g2`. Skipswith a clear reason only if TenSEAL cannot be imported.## Coordinate definition & synthetic dataFor the synthetic v1 demo the `L=1000` coordinate list is generateddeterministically from `manifest.yml`'s `input.coordinates.seed` — the **sameseed as the flagship**, which is what makes protocol 5 the controlledmultiplicative comparison. The invariant that matters is not a separatecoordinate file — it is that every contributor encodes against the **same**published definition and that definition is folded into the bundle SHA-256. Alldata here is synthetic integer vectors; no real genomic data is used anywhere.