# Development log ## 2026-10-14: Dataset preparation ### Scope Build training data first. Do not start SFT or GRPO yet. The intended model returns a risk level from 1 to 5 and reasons from the owner's fixed list. It supports human compliance review, not an automatic finding of fraud. The owner supplied `input.json`, `output.json`, `label.json`, `events.json`, and `reason-codes.json`. These are commented examples and Markdown tables, not parseable JSON Schemas. Preserve them as supplied. Modal credential environment variables are present; their values have not been printed. ### Research - [Coinbase, part 3, October 8, 2026](https://www.coinbase.com/blog/part-three-owning-intelligence-series): Qwen3.5-9B, LoRA, GRPO through SkyRL, deterministic rewards against historical fraud outcomes, and class-balanced rewards. The article does not describe an SFT stage. Training used one RTX PRO 6000 96GB; L40S was used for serving. Its proprietary results are not evidence that our synthetic dataset will work on Hiive. - [Qwen3.5-4B](https://huggingface.co/Qwen/Qwen3.5-4B): public Apache-2.0 checkpoint. Resolved revision `851bf6e806efd8d0a36b00ddf55e13ccb7b8cd0a`. This is a verified 4B option, not a claim that no newer model exists. - [RBA](https://zenodo.org/records/6782156): CC BY 4.0, synthesized login records based on real behavior. Added bank changes or password resets must be marked synthetic, not attributed to this source. - [SAML-D](https://www.kaggle.com/datasets/berkanoztas/synthetic-transaction-monitoring-dataset-aml): synthetic AML transactions; CC BY-NC-SA 4.0. The owner clarified that this is noncommercial research, so include it with attribution and applicable license terms. - [BAF](https://www.kaggle.com/datasets/sgpjesus/bank-account-fraud-dataset-neurips-2022): synthetic bank application records; CC BY-NC-SA 4.0. Include for noncommercial research. Account application features do not establish private-share ownership or accreditation fraud. - [SEC LR-25393](https://www.sec.gov/enforcement-litigation/litigation-releases/lr-25393): allegations of selling pre-IPO shares not owned, commingling, and share deficits. Use only as an anonymized scenario inspiration. Do not turn allegations into adjudicated findings or present invented account events as SEC facts. ### Dataset decisions Start with a small candidate-data pilot using generated scenarios and a cited SEC ownership playbook. Do not claim that IBM AML, RBA, NFT, or phishing records have been ingested before adapters and downloads exist. Code owns event ordering, arithmetic, scenario evidence, and proposed targets. The LLM maps the evidence packet to the supplied input contract. Reject changes to structured evidence. Keep full raw generations and rejection reasons for audit. The pilot has no source listing notes; discard generated notes, so invented text never becomes evidence. Targets are proposed synthetic supervision, not confirmed historical fraud. Risk levels are an explicit pilot policy. `label.json` has no risk-level field, so keep the exact output target separately rather than altering the supplied label structure. The source metadata and targets must not appear in the model input. Future splits must group source cases, connected accounts, and generated scenario families before expansion. The pilot is train-only and cannot establish generalization, calibrated fraud probabilities, or real-world recall. ### Remaining sources and coverage Verify IBM AMLworld's dataset license and access. Select and verify concrete NFT wash-trading and phishing datasets. Implement record adapters before claiming source coverage. Accreditation inconsistency is included in the synthetic pilot; a new account, unusual price, shared IP, or missing document alone is not proof of fraud. Insider-timing examples need news that was observable at review time; never leak future news into the input. ### Owner clarifications and teacher selection The owner clarified that this is a research project, not an intended Hiive deployment. SAML-D and BAF belong in the research source collection. Their noncommercial restrictions are not a reason to exclude them here. The owner requested the latest DeepSeek through their OpenRouter key. The live OpenRouter `/api/v1/models` catalog lists `deepseek/deepseek-v4.1-flash` as the newest non-batch DeepSeek entry. Use this exact ID for generation, with temperature 0, maximum 4,096 output tokens, reasoning disabled, JSON-object responses, and no provider fallbacks. Record the requested and returned model, response ID, provider, usage, cost, prompt hash, and timestamp per call. The existing key variable is available and was used without printing it. Qwen3.5-4B remains the proposed student. Modal L40S training is deferred. A preliminary Qwen generation attempt was stopped after the owner selected DeepSeek. Its Modal image built successfully. One earlier launch rejected whitespace in the token ID; trimming in memory resolved authentication. A container import failed because a local-only module was imported remotely; moving that import fixed it. Both temporary Modal apps were stopped. No training occurred, and no Qwen generations were used in the dataset. Modal warm-up/build charges were not measured. ### Source acquisition `prepare_sources.py` downloads SAML-D and the BAF archive containing all six variants, then samples SAML-D and BAF Base. Stratified reservoir sampling scans the complete CSV, preserves raw source fields and original labels, and samples each class without replacement. Each source manifest records authors, URLs, license links, archive hashes, original class counts, and sample counts. Command: `.venv/bin/python prepare_sources.py --per-class 100 --output data/source-samples-001` | Source | Full-file negatives | Full-file positives | Sample | | -------- | ------------------: | ------------------: | --------------------------: | | SAML-D | 9,494,979 | 9,873 | 100 negative + 100 positive | | BAF Base | 988,971 | 11,029 | 100 negative + 100 positive | Result: 400 raw source records in `data/source-samples-001/source_records.jsonl`, with a separate `manifest.json`. These are source records, not schema-adapted share-review training examples. Sampling is balanced for development and does not preserve source prevalence. BAF's person age and application timing must not be mislabeled as platform account age. SAML-D monetary currencies must not be silently treated as USD. ### Generation runs and validation 1. `.venv/bin/python generate_data.py --per-scenario 2 --output data/deepseek-pilot-001`: 22 attempts, 14 accepted, 8 rejected. Inspection found that all eight rejections were only a missing final period in a neutral listing note, not altered structured facts. Reported API cost: $0.00719625048. Retain this original audit unchanged. 2. Remove the generated-note requirement. Discard a model-generated note when no source note exists, while continuing to require exact preservation of every other input field and every source note that does exist. Raw responses remain available for inspection. 3. `.venv/bin/python generate_data.py --per-scenario 2 --seed 43 --output data/deepseek-pilot-002`: 22 attempts, 22 accepted, 0 rejected; 22.63 seconds. All returned model IDs are `deepseek/deepseek-v4.1-flash`. Reported API cost: $0.00651376290. Total reported OpenRouter cost for these two runs: $0.01371001338, excluding Modal charges. Latest pilot distribution: level 1 = 2, level 2 = 4, level 3 = 8, level 4 = 8, level 5 = 0. Eleven scenario families include benign bank changes, a verified founder's large order, and several suspicious combinations. These are proposed synthetic labels, not adjudicated fraud or calibrated probabilities. No arbitrary level-5 examples were added. `.venv/bin/python -m unittest -v`: six tests passed. Checks cover evidence changes, quarantine behavior, the 24-hour boundary, arithmetic, future events, input-label leakage, fixed reason codes, reproducible cases, benign controls, and preservation of raw sampled labels/rows. The latest candidate JSONL, run report, source manifest, and plot are copied to `.amp/in/artifacts/` for review. The plot was inspected: all counts match the report, category labels are readable, and its title distinguishes candidate data from model performance. Ruff checks passed after formatting. All 22 saved cases revalidated, and all six report file checksums matched. Data and source archives are ignored by Git. Code changes remain local and uncommitted. Source adapters, other source downloads, leakage-safe held-out splits, reviewed labels, and SFT/GRPO exports remain unfinished. The 22-case pilot proves only that the mapping and validation path runs, not that a fraud detector works. ## 2026-10-14: Free-model 500-case quality batch The owner authorized the proposed 500-case checkpoint using the suggested free OpenRouter model. No 10,000-case expansion or training was authorized. The intended mixture is 250 suspicious cases, 125 ordinary cases, and 125 benign-but-unusual cases. ### Model and safeguards Selected `dots-studio/dots-3-note-preview:free`, listed as 280B total/16B active. The live catalog and AtlasCloud endpoint advertised structured output and zero prompt/completion pricing. The account initially showed 1,000 free requests remaining. OpenRouter documents a shared free-model quota and a 20-request/minute limit. `quality_batch.py` uses five cases/request, four workers, and a global four-second start interval. Paid DeepSeek calls do not count against that free-model quota. Every final-run response is saved with requested/returned model IDs, provider, response ID, usage, finish reason, timestamp, case IDs, and prompt hash. Catalog pricing is checked; provider maximum prices are zero and fallback is disabled. A missing cost field does not count as a reported zero. Pending markers prevent a blind POST retry after an unknown outcome. Repairs apply only to rejected text after a saved successful response. ### Source adaptation and labels Backgrounds: 200 preserved SAML-D rows, 200 preserved BAF Base rows, 50 variants referencing the SEC LR-25393 ownership playbook, and 50 generated roots. Raw source records, original labels, IDs, archive hashes, and licenses remain in audit metadata. Original laundering/application labels never enter the teacher input or determine a private-share target. A regression test flips every original source label and verifies that the review inputs and targets do not change. The SAML-D data card calls `Amount` a transaction amount but does not identify its denomination. It defines `Payment_currency` and `Received_currency` as sender/receiver account currencies. The adapter therefore preserves all three values, marks the amount's currency unspecified, and never silently assigns USD or invents an exchange rate. Name matching is unknown, not fabricated as true or false. Relative event times are synthetic. BAF contributes five unchanged telemetry fields in a hypothetical report, not observed identity checks, account tenure, or actual uploaded documents. Code generates the hypothetical share-review context, event chronology, computed summaries, evidence pointers, and proposed labels. Synthetic documents and attack events are explicitly marked in audit metadata. There are 25 level-5 cases with a cap-table/commitment contradiction: the listing exceeds the owner's uncommitted shares. These documents are generated evidence, not SEC findings or verified records from actual owners. ### Failed verdict experiments Preserve all three experiments: | Run | Attempted | Validated | Rejected | Findings | | ----------------- | --------: | --------: | -------: | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `quality-500-001` | 50 | 5 | 45 | Repeated case IDs, notes containing numbers/claims, and one truncated response. Extra rows reused supplied IDs rather than introducing unknown IDs. | | `quality-500-002` | 50 | 45 | 5 | System instructions and constrained output shape improved formatting. Risk agreement was 28/45; exact reason-set agreement was 23/45. Invalid repeated reason codes rejected a five-case batch. | | `quality-500-003` | 50 | 35 | 15 | Additional reason-array constraints did not solve semantic errors or duplicated codes. Risk agreement was 22/35; exact reason-set agreement was 20/35. | The free model sometimes missed explicit phishing, account-takeover signals, or commitments, and invented unsupported circular-flow reasons. Its verdicts are not suitable as a teacher's ground-truth labels. Do not replace code targets with these outputs or present agreement with a synthetic policy as fraud accuracy. The final path asks Dots3 only for neutral transaction-intent notes. It sees anonymous case IDs and order types, not original source labels or target risk levels. Any injected verdict is rejected. Structured evidence and target labels are unchanged. The model is helping with text, not labeling or reconstructing source networks. ### Final generation and repair 1. `.venv/bin/python quality_batch.py --limit 50 --output data/quality-500-004`: 50/50 passed the initial structural/text checks, with 46 distinct notes. 2. Continue with `--limit 500`: 100 cached/new batch responses; 487 accepted and 13 rejected for short notes. 3. Inspect the generated text. Seventeen otherwise accepted notes introduced unsupported goods, unused possessions, or real-estate/rental context. Tighten validation and regenerate these 17 plus the 13 short notes with explicit private-share context. Original responses and the pre-audit candidate/report snapshots remain intact. 4. `.venv/bin/python quality_batch.py --limit 500 --repair --output data/quality-500-004`: six repair requests, 30 repaired notes, 500 accepted, 0 remaining rejections. No structured facts or labels changed. There are 348 distinct notes. | Distribution | Count | | ------------------------- | ------------------------: | | Suspicious | 250 | | Ordinary | 125 | | Benign but unusual | 125 | | Risk 1 / 2 / 3 / 4 / 5 | 125 / 125 / 80 / 145 / 25 | | Train / validation / test | 430 / 41 / 29 | Suspicious families: ownership 50, double selling 25, takeover 40, linked-account trading 30, third-party/front-person concern 25, rapid-cancellation/price concern 30, phishing 25, accreditation inconsistency 25. Benign-unusual families have 25 cases each: verified large founder order, same-name bank change, shared office IP, authorized third-party funding, and documented duplicate-bid correction. All 106 final-run responses returned the exact selected free model and explicitly reported cost 0. The three failed verdict checkpoints also reported $0. The quota endpoint after generation showed 132 used and 868 remaining out of 1,000; this API counter is reported as observed rather than equated with the local response-file count. No paid fallback or Modal job ran in this phase. ### Quality findings and next gate All 500 inputs and verdicts revalidated. Comparing each accepted input with its saved plan after removing the generated note showed exact evidence preservation and unchanged targets. All reason-evidence pointers resolved. Independently checking every level-5 cap-table calculation verified insufficient uncommitted shares in all 25 cases. There were 500 unique IDs, 500 distinct structural fingerprints, no detected source-group crossing, and no pending requests. Fingerprint uniqueness does not establish diverse scenarios. SAML-D and the shared SEC playbook are train-only. BAF months 0–5 are train, month 6 validation, month 7 test. Generated roots are grouped separately. BAF has no person ID, so person-level separation is not established. Scenario templates appear across splits; the holdouts do not measure unseen-family or real-world generalization. All 125 ordinary cases are seller listings. Add ordinary buyer bids before expansion. Evidence and documents remain template-heavy, and the source adapters are shallow: there are no reconstructed SAML-D networks or actual RBA login sequences. IBM, NFT, and phishing datasets remain un-ingested. Insider timing and circular/many-to-one flows remain uncovered. Review the label policy and add stronger controls/source adapters before moving to 10,000 cases or SFT/GRPO. The 500 cases are candidate synthetic supervision, not a production fraud detector or confirmed outcomes. Final checks: `.venv/bin/python -m unittest -v` passed all eight tests. `uvx ruff check dataset.py generate_data.py quality_batch.py prepare_sources.py test_dataset.py` reported `All checks passed!`. The kept tests cover original-label independence, the exact mixture, ambiguous currency preservation, temporal source splits, every generated level-5 contradiction, and rejection of verdict injection and unsupported rental context. Review exports in `.amp/in/artifacts/`: `quality-500-candidates.jsonl`, `quality-500-report.json`, `quality-500-mixture.png`, `quality-500-distribution.png`, and `quality-500-label-review.jsonl`. The review queue contains 24 validated checkpoint cases where the teacher's risk level or reason set disagreed with the proposed target. Disagreement does not establish which label is correct. The two final plots were inspected and their counts matched the report; titles explicitly distinguish candidate data from model performance and confirmed fraud. Export checksums are saved in `quality-500-export-checksums.json`. Code, documentation, and data remain local and uncommitted. No push, merge, publication, or training occurred. ## 2026-10-14: DeepSeek versus Dots3 diagnostic The owner requested DeepSeek tests on the two previously cited failures to check whether the problem was model capability. Used `deepseek/deepseek-v4.1-flash`, still the newest non-batch DeepSeek catalog entry, and `dots-studio/dots-3-note-preview:free`. Original case IDs: phishing `c_59fcb82fddf056ab`, double selling `c_38d3d0c92f220120`. Evidence, policy targets, and the 500-case dataset were not changed. Reconstructed the two original five-case prompts and verified their exact SHA-256 hashes against the saved checkpoint. A first reconstruction check failed because sorted JSONL keys changed serialization order; rebuilding the deterministic inputs restored the original ordering, with input equality checked against the saved plan. No inference requests occurred before the prompt hash verification succeeded. Sixteen new requests cover two cases, two models, and four conditions: original five-case batch plus notes; single-case verdict-only; single-case verdict-only with the codebook explicitly in the prompt under strict JSON schema; and the same explicit-codebook messages under JSON-object mode. Temperature 0, reasoning disabled, provider fallback disabled. The selected case's target and original verdict were never supplied to either model. Complete request bodies and responses are retained in `data/verdict-comparison-001`. | Setup | DeepSeek phishing | Dots3 phishing | DeepSeek double selling | Dots3 double selling | | ------------------------------------------------ | --------------------------------------------------------------- | -------------------------------------------------------------------------------- | ------------------------------------------------------------------------------- | ---------------------------------------------------------------------- | | Original batch plus notes | Risk 4; off-platform plus unsupported shared-IP reason | Risk 1; no concern | Risk 5; shares already committed; another case made the full batch invalid | Risk 1; no concern | | Single verdict, codebook only in API schema | Risk 4; unsupported and repeated reasons, rejected by validator | Risk 4; repeated circular-flow reason, rejected by validator | Risk 5; correct commitment reason plus unsupported holding/owner-change reasons | Risk 1; no concern | | Single verdict, explicit codebook, strict schema | Risk 4; exactly off-platform and phishing-language reasons | Risk 4; exactly off-platform and phishing-language reasons | Risk 5; exactly shares-already-committed reason | Risk 5; correct commitment reason plus false no-ownership-proof reason | | Same explicit-codebook prompt, JSON object | Risk 4; off-platform plus unsupported third-party-funds reason | Risk 4; correct phishing reasons plus unsupported third-party-funds/rush reasons | Risk 5; exactly shares-already-committed reason | Risk 5; commitment plus unsupported missing-proof/holding reasons | The clearer single-case strict-schema setup recovered the expected risk levels for both models and both cases. DeepSeek also matched the complete proposed reason sets in that setup. Dots3's extra missing-ownership-proof reason contradicted the input's verified ownership status. These observations do not support a categorical claim that Dots3 cannot understand either case. They show setup sensitivity and continuing reason-selection errors. A small test of selected failures does not establish a model ranking, general fraud accuracy, or readiness to label a large corpus. The explicit-codebook prompt also added an instruction to select supported reasons without repetition, so the codebook and that instruction are not isolated. Strict-schema DeepSeek calls used OpenInference, whereas JSON-object calls routed to Relace. This confounds provider with output mode; do not attribute the mode-control differences solely to decoding. Dots3 used AtlasCloud throughout. Provider fallback being disabled does not pin the initial provider route. All 16 responses returned the exact requested model. Matched mode-control request payloads differed only in response format. No pending requests remained. Total reported API cost: $0.0013033662, approximately 0.13 US cents; Dots3 reported $0. No automatic retries, training, dataset relabeling, or deployment occurred. Review exports are `verdict-comparison-report.json` and `verdict-comparison-requests-responses.zip` in `.amp/in/artifacts/`. The scratch comparison script was removed after execution; request payloads preserve the exact tested conditions. ## 2026-10-14: GLM-5.3 diagnostic The owner asked about GLM-5.3 in the model-comparison discussion. Verified `z-ai/glm-5.3` in the live OpenRouter catalog, with structured-output support. Catalog pricing at inspection was $0.079/million input tokens and $4.80/million output tokens; actual provider prices differ. Used the base model, not Prime or Flash. Tested the same two selected failures in the original five-case-plus-notes setup and the single-case verdict-only setup, without the extra explicit-codebook hints. All request messages and response schemas were checked for exact equality with the corresponding saved DeepSeek requests. The initial matched request returned HTTP 400. Inspected endpoint capabilities and pinned a structured-output-capable DeepInfra route; its captured rejection explained that reasoning is mandatory and cannot be disabled. Enabled reasoning, excluded reasoning text from returned content, and increased the output allowance to 8,000 tokens. No case facts or prompts changed. This is not a compute-matched comparison against the reasoning-disabled DeepSeek/Dots3 runs. The two HTTP rejections did not return usage/cost data. | Case | Original batch verdict | Single-case verdict | | -------------- | ---------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------- | | Phishing | Risk 4; `OFF_PLATFORM_REQUEST`, unsupported `CIRCULAR_FLOW` and `MANY_TO_ONE_FLOW` | Risk 4; `OFF_PLATFORM_REQUEST`, unsupported `CIRCULAR_FLOW` and `FUNDS_FROM_THIRD_PARTY` | | Double selling | Risk 5; `SHARES_ALREADY_COMMITTED`, unsupported `DOCUMENT_NAME_MISMATCH` | Risk 5; `SHARES_ALREADY_COMMITTED`, unsupported `LISTING_ABOVE_TYPICAL_HOLDING` | GLM recovered all four focal risk levels, but none of the four complete reason sets matched the proposed policy. The phishing input had no circular payment network or third-party payer evidence. The double-selling listing was only 0.153 times the high end of typical employee holdings, so the oversized-holding reason was directly inconsistent with the computed input; its document event indicated name matching, not a name mismatch. The cases remain synthetic, and matching risk levels is not measured real-world fraud accuracy. All four successful responses returned exactly `z-ai/glm-5.3`, through DeepInfra, and ended normally. Both batches contained exactly their five supplied case IDs and fixed-code verdicts; the four focal outputs validated structurally despite their semantic reason errors. Total reported cost for successful responses: $0.0339811875, about 3.4 US cents, including reasoning tokens. There are no pending outcomes. These selected examples favor GLM over the original Dots3 outputs for risk detection, but do not show that it is a better labeler than DeepSeek or safe for automatic reason labeling. Raw requests/responses and results remain in `data/glm53-comparison-001`. Review exports are `glm53-comparison-report.json` and `glm53-requests-responses.zip` in `.amp/in/artifacts/`. No dataset labels, generator code, training configuration, or remote settings were changed. The diagnostic log remains local and uncommitted. ## 2026-10-14: DeepSeek rerun with audit and correction The owner authorized paid DeepSeek inference on the same 500 cases, a separate violation-finding pass, and correction of violating outputs. `review_batch.py` preserves the existing `quality-500-004` inputs, notes, source metadata, and splits. It generates verdicts with literal evidence citations, audits each structurally valid output in a separate request, and allows two correction rounds with re-audits. Acceptance also requires agreement with the saved synthetic-policy target. That answer is not supplied to either model; disagreement is reported without an expected verdict. Unresolved cases are quarantined, not relabeled to force agreement. Requests use `deepseek/deepseek-v4.1-flash`, pinned to OpenInference, temperature 0, reasoning disabled, and strict JSON schemas. Full request bodies and responses are cached with hashes; pending or rejected HTTP request markers block automatic resubmission. No secrets are saved. The reported-cost submission stop is $2; concurrent in-flight requests can exceed that threshold. The first 20-case setup (`data/deepseek-reviewed-500-001`) accepted 0/20 after 120 responses, costing $0.03631510944. Inspection found a checker/prompt mismatch: valid citations frequently started with `/input/`, while the checker accepted only paths relative to the unwrapped item. The auditor also frequently put explanations of correct decisions in its violations array. This is a failed setup checkpoint, not a model accuracy estimate. Retained all responses and did not silently reinterpret its audit lists. The revised checker accepts either path root, rejects invalid array indices and partial numeric quotations, and requires evidence for exactly the selected reasons. The prompt clarifies that the audit array contains only actual defects. It also distinguishes source-background BAF telemetry from platform device links and unknown SAML-D payer matching from an explicitly mismatched payer. Case evidence and the labeling policy were unchanged. The revised 20-case checkpoint (`data/deepseek-reviewed-500-002`) accepted 10/20: seven first-pass accepts, three recovered by correction, and ten quarantines. Thirteen cases needed another attempt. Ninety responses cost $0.01968454512. The known phishing case cleared on the first pass. The known double-selling case stayed quarantined: the auditor incorrectly called the explicit share-commitment contradiction Level 4, a later audit disagreed, and the final generator added an unsupported oversized-holding reason despite a ratio of 0.153. These audits are separate calls to the same model, not independent reviewers. Started the full 500-case run with the revised prompts, reusing checkpoint responses. Six workers were used for checkpoints, sixteen for the full continuation. No training, remote publication, or pushes are authorized or running. Validation before the full continuation: nine unit tests passed, including repair versus quarantine on fabricated citations, auditor objections, and synthetic-reference disagreements. Scratch checks passed for response-cache reuse, stale-prompt rejection, unknown-outcome protection, both citation roots, exact numeric quotes, and invalid array indices. Scratch checks made no inference requests. The full run finished: 500 reviewed, 294 admitted, 206 quarantined. Of 259 cases sent through correction, 53 eventually cleared; 241 cleared immediately. All 1,990 requests returned the specified model/provider. One auditor response hit the output cap and produced invalid JSON; that case was retried through the bounded correction process and remained quarantined. The other 1,989 responses ended normally. There are no HTTP-error or unknown-outcome markers. Main-run reported cost was $0.41036667444, including the revised checkpoint; adding the discarded setup checkpoint gives $0.44668178388 for this task. Earlier comparison experiments are not included in that total. The initial complete verdict matched the saved synthetic reference in 432/500 cases. Only seven initially nonmatching verdicts later cleared. The remaining 46 correction-stage accepts began with matching labels but had citation or auditor objections. Conversely, 145 quarantines began with matching labels. These are reference-agreement/admission counts, not fraud accuracy; the audit can create false alarms and sometimes regress a correct draft. For takeover case `c_4877e39d941e2b4c`, the first audit treated same_name_as_account=false as not mismatched and pushed a correct Level 4 draft down to Level 3. Another audit caught that mistake, and the final Level 4 output cleared. For double-selling case `c_15528a5a18d1bf25`, the audit correctly removed LISTING_ABOVE_TYPICAL_HOLDING because its ratio was 0.807, retaining Level 5 and SHARES_ALREADY_COMMITTED. Admitted mixture: 173 suspicious, 91 ordinary, 30 benign-unusual. Admitted risk levels 1–5: 91/30/62/89/22. Splits: 254 train, 25 validation, 15 test. Source backgrounds: 114 SAML-D, 115 BAF, 33 SEC playbook, 32 generated. Categories: 121 none, 50 ownership, 22 double selling, 24 accreditation, 16 takeover, 23 phishing, 19 price manipulation, 19 front person. All 30 linked-account-trading cases stayed quarantined, and 95/125 benign-unusual controls were excluded. Do not treat this selected subset as the planned balanced corpus. Inspected linked-account example `c_8aaa139abb76c144`. Its reference is Level 4 with SHARED_BANK, TRADES_WITH_LINKED_ACCOUNT, and REPEATED_TRADES_SAME_PARTIES. The drafts repeatedly selected Level 2 without TRADES_WITH_LINKED_ACCOUNT. The auditor also questioned whether the bank-sharing account and repeated counterparty were demonstrably the same account: input exposes aggregate link counts rather than identity-resolved links. This is an evidence-linkage ambiguity worth resolving alongside reviewer errors, not grounds to silently replace all 30 labels or alter the fixed inputs in this rerun. Full-batch verification passed: all 500 input hashes and original source/split metadata preserved; candidate/quarantine IDs partition the original 500 exactly; every admitted final citation, reason set, risk level, auditor result, and synthetic-reference gate passes; all 1,990 unique response IDs map to the saved attempt histories and request hashes. Offline replay with the inference function blocked regenerated the results without new API requests. Verification details are in `data/deepseek-reviewed-500-002/verification.json`. Review exports under `.amp/in/artifacts/` include admitted candidates, quarantined source cases and attempt trails, report, verification, examples, all raw requests/responses in a ZIP, and checksums. Inspected both final plots: `deepseek-review-summary.png` shows first-pass/correction/quarantine counts and mixture attrition; `deepseek-reviewed-distribution.png` shows explicitly labeled admitted risk/category counts. Neither claims measured fraud accuracy. The shared plotting function now prints bar counts and makes accepted-only scope explicit. All changes and exports remain local and uncommitted. No SFT, GRPO, Modal job, push, or publication occurred. The 206 quarantines and reviewer reliability remain unresolved; the 500-case rerun itself is finished. ## 2026-10-14: GPT-6 Luna through the connected Droid CLI The owner requested the latest available Haiku or Luna through the installed Droid connection, the same 500-case task, and graphs. Inspected Droid 0.241.1's model list: Haiku 4.5 and GPT-6 Luna are available. Selected the exact `gpt-6-luna` ID, with its default medium reasoning. A one-response smoke test confirmed authentication and model selection; it reported 69 Factory credits. No account settings were changed. Added a Droid transport to the existing review runner. It reuses exactly the existing case inputs, core generation/audit messages, policy, schemas, evidence checker, reference gate, and two-correction limit. Each generation and audit uses a fresh CLI session in an empty directory outside the repository. Builtin skills are disabled and every discovered tool ID is passed to the removal flag. Droid leaves ToolSearch available in its tool-permission listing, and its initialization event prints a broad static tool list, so removal flags alone are not proof of no access. The runner additionally requires exactly one response turn, rejects tool events, and checks the reported model and reasoning effort. No model is authorized to use tools or access saved labels. The initial `data/droid-luna-reviewed-500-001` calibration stopped after five completed cases and 20 responses because the stream guard incorrectly rejected `reasoning` events as tool activity. Corrected the parser to accept reasoning events while rejecting actual tool calls. Initial outputs also demonstrated literal citation-format mismatches: enclosing quotation marks around text and compact JSON for objects, whereas the unchanged checker requires raw text and Python's default JSON spacing. Retained those streams and spent 4,920 credits, without interpreting formatting rejection as an intelligence failure. Added explicit citation-format instructions to the Droid-only system appendix, without adding facts or expected labels. This is a workflow comparison, not a prompt-, decoding-, provider-, or compute-matched model benchmark. Luna has Factory's agent system prompt, medium reasoning, CLI-default decoding, and locally enforced JSON schemas; DeepSeek had reasoning disabled and API-enforced schemas. The core policy/messages and literal checker remain unchanged. New results are isolated in `data/droid-luna-reviewed-500-002`; DeepSeek files are not overwritten. The revised 20-case checkpoint cleared 19: 18 immediately, one after correction, and one quarantine. Its 42 sessions reported 10,390 credits. The known phishing and double-selling examples cleared. The quarantined takeover case repeatedly placed evidence inside the verdict instead of at the proposal root, failing the schema on all three attempts. Started the full 500-case continuation with twelve concurrent CLI processes, reusing the checkpoint. Factory reports credits and tokens, not a dollar charge; report dollar cost as unavailable, never as zero/free. Ten unit tests passed before the continuation, including a stream regression test for reasoning events versus tool calls and wrong-model rejection. Ruff passed after fixing an explicit subprocess check flag. Raw requests, streams, model IDs, session IDs, correction trails, and usage are retained locally. No training, pushes, or remote publication are part of this task. Full run finished: 480 admitted, 20 quarantined. Of 109 cases requiring another attempt, 89 cleared; 391 cleared immediately. Initial schema validity was 419/500, and 411/419 schema-valid initial verdicts matched the synthetic reference. That conditional 98.1% excludes 81 malformed proposals and must not be compared without qualification to DeepSeek's 432/500 policy matches with 500/500 schema-valid proposals. Luna's final admission was 96.0%, versus DeepSeek's 58.8%, but admission is not real fraud accuracy or an isolated model-capability measurement. Admitted mixture: 235 suspicious, 125 ordinary, 120 benign-unusual. Risk levels 1–5: 125/120/79/131/25. Splits: 415 train, 37 validation, 28 test. Categories: 245 none, 49 ownership, 25 double selling, 25 accreditation, 25 phishing, 29 price manipulation, 29 takeover, 28 linked-account trading, 25 front person. Coverage improvements include 120/125 benign-unusual controls versus DeepSeek's 30/125, and 28/30 linked-account cases versus 0/30. This preserves the research corpus's balance much better, without establishing fraud truth. Quarantines: 11 takeover, five none, one ownership, one price manipulation, two linked-account trading. The last proposal was schema-invalid in 17 of the 20; three parsed proposals failed other checks. A read-only diagnostic extracted only risk/reason fields from those 17 malformed responses: 14 matched the reference, one differed, and two could not be extracted. No cases were accepted or rewritten by that diagnostic; all 20 remain quarantined under the same rules. The main run's 1,163 sessions consumed 290,450 reported Factory credits, including the revised checkpoint. Calibration added 4,920 and smoke testing 69, for 295,439 credits total in this task. No dollar cost was exposed. Reported token fields: input 3,489; output 352,215; cache-read input 4,786; cache-created input 4,385,235; thinking 228,144. These are provider-reported fields, not a claim that their sum measures a matched compute budget. Full verification passed: accepted/quarantine IDs partition the original 500; every original input hash and source/split metadata item is unchanged; all 480 admitted cases have literal evidence, fixed-code verdicts, clean audits, auditor agreement, and reference agreement. All 1,163 session IDs map to attempt histories, report the selected model/effort, have one response turn, and show zero tool events. All 500 initial core message arrays exactly match their saved DeepSeek counterparts; the Droid-only appendix still differs. DeepSeek's saved report is unchanged. A first offline replay attempt blocked even the harmless version lookup because subprocess.check_output calls subprocess.run; supplying the previously captured version metadata let the full replay complete with all subprocess execution blocked. No inference was repeated during validation. Saved comparison JSON, admitted candidates, quarantines, audits, verification, and request/result ZIP under `.amp/in/artifacts/droid-luna-*`. Exported results omit reasoning-event text; full internal streams remain in the run directory. Saved and inspected workflow-comparison, category-coverage, and admitted-distribution plots. Initial comparison captures had a headline/bar overlap and a legend over the final coverage row; corrected both layouts and inspected only the final captures for delivery. The inline comparison view also passed preview at 720/360 CSS pixels in light/dark themes; inspected all four screenshots, with no script errors or clipping. The same-500 Luna rerun is finished; the 20 unresolved cases are not training candidates. Code/data stay local and uncommitted. No SFT, GRPO, Modal job, push, merge, or publication occurred. ## 2026-10-14: 2,000-case Luna expansion The owner authorized 2,000 additional cases in four 500-case batches and requested an HTML progress bar rather than scheduled wakeups. Added a supervised review job with a live portal dashboard. It polls saved case-audit counts every five seconds; no schedule was created. The dashboard contains no source rows or model responses, only aggregate counts and job status. Desktop and 360px Chromium renders were inspected; the initial progressbar accessibility value matched the backend, with no horizontal overflow. Prepared 1,000 suspicious, 500 ordinary, and 500 benign-unusual inputs. Source backgrounds: 800 new SAML-D rows, 800 new BAF rows, 200 synthetic SEC-playbook variants, and 200 new generated roots. Full-file reservoir sampling used the preserved archives with seed 20261015. All 1,600 adapted source rows are unique and absent from the pilot; source fields and licenses remain recorded. Across pilot plus expansion, 2,500 IDs are unique and the existing source-group/month split checks pass. SEC variants are still inspired by one release, not 200 separate cases. Added ordinary buyer bids, missing-proof cases below/at/above the 1.5 ratio boundary, varied repeated-deal and bid/cancellation histories, bank-change times, and founder account ages/order sizes. Neutral listing notes are code-written; no new external evidence is invented as source fact. The scenario families remain template-based and shared across splits. This expansion is not a new fraud benchmark. The Droid CLI has no structured-output flag. Added an opt-in explicit JSON-envelope illustration to reduce the observed nested-evidence errors, without relaxing Pydantic, evidence, audit, or reference-consistency checks. Old pilot cache settings remain unchanged. First checkpoint is 50 cases; each completed checkpoint below 85% admission stops the next batch for inspection. All calls retain hash-checked caches and unknown-outcome protection. Factory credits will be reported; USD billing is unavailable. No training or publishing is authorized in this step. Validation before launch: Ruff passed; all 10 unit tests passed, with the existing generator test extended to cover ordinary buyers, ownership policy on both sides of the holding boundary, and disjoint generated roots across batches. All 2,000 prepared inputs passed source/duplicate/split validation. Run started under the supervised `luna-expansion` service, with raw responses and logs under `data/luna-expansion-2000-001`. Expansion batch 1, checkpoint 50: 50 accepted, 0 quarantined; 49 initial schema-valid; 26696 cumulative batch credits. Verified actual browser auto-refresh without reloading: the displayed count advanced from 164 to 167 over 6.5 seconds, and the accessible progressbar value was 167. The live API reconciled reviewed = accepted + quarantined and matched the batch totals. Inspected nonzero desktop and narrow screenshots; no clipping or overflow. The original 480 admitted pilot cases remain unchanged and excluded from the new progress total. Expansion batch 1, checkpoint 500: 499 accepted, 1 quarantined; 492 initial schema-valid; 282272 cumulative batch credits. Expansion batch 2, checkpoint 500: 497 accepted, 3 quarantined; 484 initial schema-valid; 279500 cumulative batch credits. Expansion batch 3, checkpoint 500: 499 accepted, 1 quarantined; 494 initial schema-valid; 283788 cumulative batch credits. Expansion batch 4, checkpoint 500: 494 accepted, 6 quarantined; 475 initial schema-valid; 297161 cumulative batch credits. Expansion finished: 1989/2000 admitted; 11 quarantined. Report and plots exported. No training or publication. ## 2026-10-14: first Qwen SFT baseline The owner authorized an SFT run on their Modal account and requested the loss graph in this thread. Prepared run `qwen35-4b-sft-001` using the pinned Qwen/Qwen3.5-4B revision 851bf6e806efd8d0a36b00ddf55e13ccb7b8cd0a. The source files contain 2,469 accepted examples: 2,199 train, 146 validation, and 124 test. Duplicate/source/group checks passed across the combined corpus. No quarantines are included. The existing splits still share scenario templates, so this is a synthetic-policy baseline, not independent fraud evaluation. A new challenge set has not been created. Tokenized with the checkpoint's chat template, thinking disabled. Only the assistant JSON and end-of-message token receive loss; the entire prompt is masked with -100. All target tokenizations round-trip exactly, every input stays intact, and no sample is truncated. Mean total length is 1,717.1 tokens; maximum is 2,957. Transformers 5.19 returns a BatchEncoding by default from apply_chat_template; specifying return_dict=False fixed the local preflight before any GPU launch. Installed missing Jinja2 locally. Verified the checkpoint tokenizer's special-token IDs directly against tokenizer.json; generation will stop on its actual im_end token, not the inconsistent EOS ID in text_config. Configuration: one L40S, BF16 frozen base, language-model-only LoRA rank 16/alpha 32/dropout 0.05, microbatch 1, accumulation 16, two epochs, learning rate 1e-4 with 5% warmup and cosine decay, gradient checkpointing. Validation runs at step zero and every 25 optimizer steps. Checkpoints are selected by validation loss; test examples are reserved for post-training evaluation. The vision encoder receives no adapters. Dependencies are pinned in sft_modal.py; GPU packages are installed in Modal, not in this orb. Modal launch succeeded: app ap-OSoEkCKf5LmDbstfyN6OEZ, function call fc-01M4Y753J7057QZ8W9S6PM48HZ, launched at 22:05:02 UTC. Durable outputs go to the private hiive-research-sft Modal volume. Automatic retries are disabled; existing remote run directories and local call IDs block duplicate launches. Credentials are trimmed only in memory and never logged or embedded in requests. Image build took 84.34 seconds. The detached function was confirmed running; no training result or loss graph is claimed yet. No model or dataset is being publicly published, and no GRPO is included. Run 001 failed before Trainer initialization or optimizer updates: Transformers 5.19 removed the warmup_ratio keyword. The model and adapters had loaded successfully on an NVIDIA L40S; environment.json records 32,464,896 trainable parameters out of 4,571,730,432 total (0.7101%). Preserved its call ID and environment records. It incurred setup/GPU time, but Modal billing was not measured. No loss curve or trained checkpoint came from this attempt. Replaced warmup_ratio=0.05 with the current warmup_steps=0.05 API; verified that the fractional value yields 14 warmup steps for 276 optimizer steps. Checked every TrainingArguments and Trainer keyword against the pinned installed version before relaunch. New run qwen35-4b-sft-002 launched at 22:11:47 UTC, app ap-CSl5w8MXs8OBMkQp0jhonh, call fc-01M4Y7HECC2VPXET66RD71VMDR. Same data and hyperparameters; no old outputs overwritten. Its initial validation forward passes are running successfully on the real GPU. Run 002 reached step 25 of 276. Validation loss was 0.4646064937 before training and 0.0140127763 at step 25; the five-step training average at step 25 was 0.0132306874. These measure assistant-token cross-entropy on synthetic-policy targets, not observed fraud accuracy. No final test result is available yet. Real forward passes, gradients, and optimizer updates work on the L40S. All 10 local unit tests and Ruff checks passed. Started the supervised CPU-only qwen-sft-graph service on port 8766. It reads committed Modal metrics and refreshes the page every 30 seconds without launching another GPU job. The page exposes aggregate progress and the loss plot, not dataset records. Desktop, narrow, and dark narrow renders were captured and inspected. The collector exports metrics and loss plots under .amp/in/artifacts and records the final report in this log when first collected. Training remains detached on Modal; the adapter and checkpoints remain private in hiive-research-sft. Code and documentation are local and uncommitted. No GRPO, push, or public publication has occurred. The owner requested an embedded live dark chart instead of a static image. Forced the page and Matplotlib plot to matching dark backgrounds and restarted only the CPU graph service, leaving GPU training untouched. In Chromium, the page advanced from step 45 to step 50 and fetched a new plot URL after its 30-second interval without navigation or reload. Inspected 720px and 360px captures; metadata fits without horizontal overflow, though plot labels are small at 360px. Ruff lint and formatting checks passed. The live portal is embedded in the response rather than substituting a saved screenshot. SFT qwen35-4b-sft-002 finished. Best checkpoint: /runs/qwen35-4b-sft-002/checkpoints/checkpoint-150. Final validation: {"eval_loss": 0.0031906324438750744, "eval_model_preparation_time": 0.0235, "eval_runtime": 33.1673, "eval_samples_per_second": 4.402, "eval_steps_per_second": 4.402, "epoch": 2.0}. Synthetic holdout results: {"cases": 124, "schema_valid": 124, "schema_valid_rate": 1.0, "risk_exact_rate": 1.0, "full_verdict_exact_rate": 1.0, "reason_micro_precision": 1.0, "reason_micro_recall": 1.0, "reason_micro_f1": 1.0, "benign_cases": 64, "false_alarms_on_benign": 0, "invalid_outputs": 0}. Adapter and raw outputs remain private in Modal volume hiive-research-sft. No real-world fraud accuracy is established. ## 2026-10-15: repository documentation The owner requested a minimal README in ASD-STE100-style language and authorized a push. Replaced the experiment history in the README with the task, data counts, training settings, final results, and test limits. Copied the final dark loss plot, metrics, and run report into docs so repository links do not depend on the orb. The full history remains in this log. Raw datasets, model weights, API responses, and credentials remain outside the commit. Inspected the plot; all 10 tests, Ruff, and staged whitespace checks passed. No GRPO was run. ## 2026-10-17: binary triage decision model The owner asked for a Clef-style decision model on Qwen/Qwen3.5-4B that sorts cases into two categories: fine, or needs a deeper look. Clef and Clef-flash are Cloudflare decision models that return a probability for every allowed answer instead of generated text. Mapping: risk levels 1–2 are `ok`, levels 3–5 are `review`. This matches the policy's "worth a look" boundary at level 3 and the SFT report's benign definition. Splits are 2,199 / 146 / 124 cases, with 49.5%, 55.5%, and 48.4% review. `decision_modal.py` uses the pinned revision, the same LoRA configuration, and the same accepted cases and splits as SFT run 002. The prompt is a short triage instruction, the policy, and the grounding rules, without the verdict JSON schema. `ok` and `review` are verified single tokens (547 and 18828). Loss is cross-entropy over those two logits at the generation position, computed with `logits_to_keep=1`. Microbatch is 1, so padding never moves the answer position. Trainer's `loss_is_scaled_for_ga` is set to False, so the per-microbatch mean is divided by the 16 accumulation steps. The two-step smoke run started near ln 2, which confirms that scaling. The owner provided a new Modal token, set as the active CLI profile without printing it. The shell's older `MODAL_TOKEN_*` variables override profiles, so Modal commands unset them. The two-step L40S smoke run passed training, scoring, adapter switching, and the SFT latency comparison. Run `qwen35-4b-decision-001` launched at 00:55 UTC. Before training, the base model reached test AUROC 0.941, but at 0.5 it flagged 41 of 64 ok cases. Only 62% of next-token probability fell on the two answers. Validation loss went from 0.568 to 0.0 at step 25, about 14 minutes into the run, and stayed at 0.0 through step 125. Training was about 26 seconds per optimizer step. The planned 276 steps were far more than needed. At step 145, the run stopped because the Modal account ran out of credits. Checkpoints 25 and 125 were retained. After the owner added credits, an evaluation-only function loaded checkpoint 25. It is the earliest checkpoint with the lowest validation loss, and the function verified data hashes against the remote manifest. It saved checkpoint 25 as the run adapter and scored validation and test. It then timed the SFT 002 adapter on the same 124 test prompts. No retraining occurred; the job took 307 seconds. | Result | Value | | ---------------------------------------- | --------------------------------------------------------------- | | Validation | 146/146; 81 review, 65 ok | | Test | 124/124; 60 review, 64 ok; AUROC 1.0 | | Mean probability mass on the two answers | 0.9997 | | Decision model median / p95 latency | 194 / 375 ms | | Generative SFT median / p95 latency | 1,535 / 2,553 ms; 19.0 generated tokens; 124/124 binary-correct | | Peak GPU memory | 10.3 GB | The first threshold rule took the lowest validation P(review) among review cases. Saturated probabilities made that exactly 1.0, which would be fragile for new cases. The rule is now min(0.5, that value), so the cutoff can only move down to protect recall. The saved report was recomputed from the saved predictions; test results did not change. Test P(review) is at most 1.1e-11 on ok cases and 1.0 on review cases. These are saturated probabilities on synthetic-policy labels whose scenario families also appear in training. They do not show calibration or performance on unseen case types. The logs showed Qwen3.5's linear-attention kernels falling back to reference PyTorch code. Adding `flash-linear-attention==0.5.2` cut median test latency from 194 to 128 ms. P(review) changed by at most 2.8e-12, and no decision changed. Training forward plus backward took 1.81 seconds per case over eight cases, with no measured improvement. `causal-conv1d` still falls back; it ships only as a source build. Future runs use early stopping with patience 2, so a run like this one would stop at step 75. Ruff passed and all 11 unit tests passed, including a new test for label mapping, AUROC, the confusion matrix, threshold capping, and calibration. Exported `docs/decision-model.png`, `docs/decision-report.json`, `docs/decision-metrics.jsonl`, and `docs/decision-benchmark.json`. The plot was inspected after fixing zero-loss display and legend placement. The adapter remains private in `hiive-research-sft/qwen35-4b-decision-001/adapter`. No GRPO, push, or publication occurred. ## 2026-10-17: interactive cascade replay Read the owner's Factory Droid discussion and saved decision and SFT reports. Built a static replay in `web/`. After the owner rejected the dense dashboard, replaced it with a light neumorphic node graph. A pulsing trace follows each route. Clicking a node opens the case, decision probability, SFT reasons, evidence packet, or review queue. The graph makes room for the detail panel on wide screens; narrow screens use a vertical layout. Escape closes details and returns focus. Reduced-motion mode removes pulses and replay waits. `web/build_cases.py` joins the 124 saved test predictions from both models to their synthetic input records. The UI offers 14 representative case families. It includes account facts, event details, links, summaries, documents, and messages. These are recorded outputs, not live inference. The 128 ms and 1,535 ms waits use measured median model latency; travel animation adds time. No Modal job or model API call was made. The demo auto-clears P(review) below 0.02 when answer-token mass is at least 0.95 and no unusual-input or conflict override applies. It sends a random 2% of eligible cases to SFT for audit. Conflict overrides or disagreement between the binary decision and SFT's level-3 boundary request a senior handoff. These are illustrative routing rules, not calibrated production thresholds. Stage 3 does not invoke a model. Overrides do not change saved predictions. Validation: `node web/routing.test.mjs` passed 11 routing checks; Ruff and JavaScript syntax checks passed. Browser checks covered all 14 selector cases, auto-clear without SFT, forced audit, unusual-input bypass, escalation, descending review priority, expanded evidence, Escape focus return, reduced motion, and unavailable case data. Inspected desktop and 390px Chromium screenshots. Fixed wires crossing labels in the narrow layout and a detail panel covering downstream nodes. Recorded and inspected the moving-pointer replay in `.amp/in/artifacts/trace-replay.mp4`. Served only `web/` with the supervised `trace-demo` service on port 8770. Website changes remain local and uncommitted. ### Cascade playback controls Slowed each signal leg from 800 to 1,500 ms. Added an All cases selection that runs all 124 records sequentially. Pause finishes the current case; resume continues at the next record. Destination counts accumulate, and each case ID has only one current destination. Human review groups cases by priority, highest first. Groups and individual cases expand to show reasons and source evidence. Open groups stay open while the batch runs. Auto-clear and escalation have separate case lists. Reset clears the results and starts the selection again. Added light and dark themes. The initial theme follows the system setting; an explicit choice persists across reloads. The speed selector ranges from 0.01× (100× slower) to 100×. It changes travel, visual model waits, and between-case gaps immediately, including during a run. It does not change model predictions or measured latency values. Reduced-motion mode bypasses replay waits and movement. The website still uses saved outputs, not live Modal inference. Browser validation with random audits disabled placed all 124 unique records: 64 auto-cleared and 60 in human review. Priority groups contained 6 level-5, 28 level-4, and 26 level-3 cases. Pause/resume produced the same totals with no duplicate IDs. Forced-audit playback queued all 124, including 36 level-2 and 28 level-1 cases. Forced-conflict playback escalated all 124 and left human review empty. Rerouting one case between clear, audit, and escalation updated counts without duplication. Expanded groups and case evidence stayed open through resume. Tested system-theme detection, saved theme choice, both themes, and the 390px layout. At 0.01×, a one-second probe moved the signal 2.07346 pixels; the expected distance was 2.07365 pixels for that path. Changing to 100× completed the same case without restarting it. A separate probe slowed an active SFT wait and then accelerated it to completion. A full 100× batch retained the 64/60 routing split. JavaScript syntax, Prettier, and all 11 routing checks passed. Inspected screenshots and moving-pointer recordings in `trace-batch-flow.mp4` and `trace-speed-flow.mp4`. The first speed recording overloaded the encoder; a new 1280×800, 2× capture succeeded and was inspected. Updated files remain local and uncommitted; no GPU job, commit, or push was made. ## 2026-10-26: Elixir runtime review hook Added a standalone OTP application in `elixir/`. `HiiveRisk.enqueue_review/2` accepts a caller-built snapshot. Its Ecto.Multi variant can join an order transaction in the same repository. Snapshot insertion and Oban job insertion commit together. The unique intake key is the source event's `case_id` plus `policy_version`. Repeated intake reuses the case and job; changed evidence or routing options under the same key return an idempotency conflict. Atom and string JSON keys normalize to the stored JSON representation. Separate Oban queues limit decision calls to four and SFT calls to one. Model calls run outside database transactions. Row locks protect result updates; decision results and downstream SFT jobs commit together. Invalid responses and transport errors retry up to five attempts, then leave a visible failed case. Discarded jobs can be retried after repair. Failed calls never auto-clear. Case assessments include the saved snapshot, scores, verdict, reasons, and error code. PostgreSQL stores the priority-sorted human worklist and escalation records. PubSub announces committed updates, including failures, but is not the source of truth. The routing policy matches the replay's research rules: auto-clear below P(review) 0.02 with answer-token mass at least 0.95; sample 2% of eligible cases for SFT audit; route unusual or conflicting cases to SFT; escalate conflicts or decision/SFT binary disagreement. No frontier model, order approval, or money movement is implemented. Callers still compute summaries. The Elixir boundary uses JSON schemas generated from the Python models and validates event chronology and cross-field structure. Added a configurable HTTP model adapter and documented its contract in README. Both endpoints accept a bare snapshot. Decision must return `p_review` and full-vocabulary `answer_mass`; SFT must return the existing verdict schema. Optional bearer authentication, a 30-second receive timeout, and disabled redirects are included. Oban owns retries. The current Modal SDK method does not expose this HTTP contract or answer mass. Compatible endpoints or a replacement behaviour module remain required. No live model was called and the static website remains a recorded replay. Installed Elixir/OTP and PostgreSQL in the orb for validation. Used Elixir 1.18.4 / OTP 25 and an isolated PostgreSQL 15 instance on loopback port 55432, managed by the `risk-postgres` orb service. The first dependency set exposed old-package advisories and a minimum-Elixir mismatch. Updated the new application's dependencies and used Elixir 1.18.4. The final lockfile passes `mix hex.audit` with no retired or security-advisory packages. Verification: `mix test` passed 17 tests against real PostgreSQL. Covered contracts, transaction rollback, idempotency, atom-key normalization, queued stage transitions, priority order, PubSub, routing boundaries, audit and unusual-input overrides, escalation, invalid responses, failure exhaustion, and repair/retry for both stages. Real loopback HTTP tests checked JSON requests, bearer headers, missing endpoints, status errors, and refused redirects. An independent concurrent-intake probe used separate database connections: 20 identical submissions produced one review ID and one durable job. Probe rows were removed. `mix format --check-formatted` passed. The existing Python suite passed all 11 tests; `node web/routing.test.mjs` passed all 11 checks. Changes remain local and uncommitted. No GPU job, deployment, or push occurred. ### Start both trained models Added `start_models.py`. It starts a temporary Modal app with one warm L40S, loads the saved decision and SFT adapters on a shared frozen Qwen base, and reads both system prompts from the saved run manifests. It reuses `decision_modal.load_base`, `answer_probabilities`, and the existing inference image. No training code is called. One container handles one request at a time so adapter changes cannot race. The script binds a private loopback HTTP gateway before allocating GPU time, then loads both adapters before serving requests. POST `/decision` returns normalized decision scores and full-vocabulary answer-token mass. POST `/sft` generates at most 160 tokens with greedy decoding and validates the verdict. Both paths use the training chat template with thinking disabled and reject prompt lengths over 8,192 tokens without truncating evidence. The gateway validates snapshots, requires a bearer token when configured, caps request bodies at 1 MiB, and returns sanitized errors. It prints environment export commands for the Elixir client after model loading. GET `/health` reports gateway readiness, not prediction quality. The script holds one GPU warm until stopped; README explains charges, startup, configuration, and orb service shutdown. Validation: real loopback HTTP tests with a stubbed Modal backend covered both response contracts, health, loopback binding, authentication, malformed inputs, unknown routes, invalid scores/verdicts, and sanitized backend exceptions. The full Python suite passed all 13 tests. Ruff checks and formatting checks passed for both new files. These tests do not verify GPU inference or remote startup. `python start_models.py --help` passed without contacting Modal. No GPU was launched, no permanent deployment was created, and no commit or push was made for this step. ### Five trace design studies Reviewed public pages from Reelfolio, Inspora, 60fps, Mobbin, Landbook, Refero, Brand Guidelines, and Backgrounds Supply. Inspected rendered references from Inspora, 60fps, Landbook, and Backgrounds Supply, plus the existing replay. Used Painter to explore a five-panel comparison board before implementation. No paid library access or third-party background assets were used. Added five separately addressable prototypes under `web/prototypes/?view=...`. Porcelain uses a raised ceramic circuit; Orbit places the decision at the center of a radial map; Dossier pairs a paper case cover with a numbered stage record; Frame uses one large active stage and an inspection filmstrip; Current uses broad colored routing paths and a case inlet. Each study includes reference links and a short design rationale. The original website remains unchanged. All studies share the existing 124 saved cases and `routeCase` helper. Controls support individual or sequential all-case playback, immediate pause, replay of the unfinished case on resume, 0.01× to 100× speed, both themes, and audit, unusual-input, or conflict overrides. Stage buttons open recorded scores, reasons, source events, and full case JSON. Outcome queues group completed cases by priority and retain one destination per case ID. These prototypes use explicit audit controls rather than random sampling so layouts can be compared with the same routes. Stage 3 remains a handoff; no model calls occur. Browser verification replayed all 124 cases through each of the five layouts. Every layout produced 64 auto-clear cases and 60 human-review cases: priority 5 had 6, priority 4 had 28, and priority 3 had 26. Verified clear skips SFT, audits and unusual inputs visit SFT, conflicts escalate, and rerouting does not duplicate a case across queues. Pause/resume retained the same full-batch result. During a 0.01× trace, the signal moved about 2 pixels between probes, then completed after changing to 100×. Inspected default and alternate themes and 390px Chromium layouts; the narrow layouts had no horizontal overflow. Removed the body theme fade after intermediate colors reduced text contrast. Verified saved reasons, evidence expansion, modal Escape behavior, and design notes. Prettier, JavaScript syntax, and all 11 existing routing checks passed. Saved and inspected rendered captures and an Orbit moving-pointer interaction under `.amp/in/artifacts/prototypes/`. The first recorder attached to a blank restored tab; discarded that take and reopened the correct page before recording. The final clip shows playback, evidence expansion, and theme switching. The existing supervised `trace-demo` service serves the prototypes. No GPU, deployment, commit, or push was made for this design step. ### Operations dashboard Added `web/dashboard/`, a review-operations view of the same 124 recorded cases. The trace explains one case at a time; the dashboard shows system state. A KPI strip reports processed, auto-cleared, open human review by priority, escalations, and median time to route. The routing graph keeps the connection flow: cases stream through it concurrently, edge weight grows with traffic, nodes show worker load and the SFT backlog, and selecting a stage filters the review queue. An intake feed, a log-scale P(review) strip plot with the 2% threshold, SFT risk-level and reason-code charts, and a priority-sorted queue table follow. Rows open a case drawer with the route, both model outputs, reasons, facts, signals, account timeline, text evidence, and the raw packet. The replay is a discrete-event simulation at measured median latencies: one arrival every 0.7 s, four decision workers at 128 ms, and one SFT worker at 1,535 ms, matching the Elixir queue limits. The single SFT worker is slower than the forwarded arrival rate, so a small backlog forms; cleared cases route in about 1.9 s and reviewed cases in about 8–11 s. Routing calls the shared `routeCase`. The 2% audit sample is chosen by case-ID hash so runs are repeatable; it selects one case, giving 63 auto-clear and 61 human-review cases. SFT outputs are shown only for cases the router sends to SFT. Reviewer decisions (clear, confirm, escalate) stay in the browser tab and are not saved or sent. `?end` renders the finished state and `?paused` loads without playing. Verification: browser checks at 1440, 1100, and 390 px in both themes, with no horizontal page overflow. Checked playback, pause, reset, skip to end, speed changes, node filtering, the empty escalation view, drawer open and Escape close with focus return, and a reviewer action updating the queue status and open count. No console errors. `node web/routing.test.mjs` passed all 11 checks and Prettier passed. The original trace gained a Dashboard link and is otherwise unchanged. No GPU, deployment, commit, or push was made. ### Dashboard restyle Restyled `web/dashboard/` after the public design language of clarity.com, the platform this risk model is meant for. That means a warm cream background with near-black ink, deep teal accents, and brown and green for risk and clear states. Display type is a light serif with tight tracking over a plain sans, with mono for IDs and timings. Buttons are square, nav links carry a thin left rule, and sections are divided by barcode stripes. Stats sit in large serif numbers in hairline columns with footnotes and a "Last updated" line. The live routing graph and intake feed sit on a solid teal band. Panels are flat and bordered with italic caveats, and the page closes with a numbered "How it works" section. Clarity's fonts are licensed, so the page uses Newsreader, Inter, and IBM Plex Mono from Google Fonts. Plex Mono was chosen over DM Mono because DM Mono's slashed zero read as Ø. The page keeps the `trace.` wordmark, uses none of Clarity's logo, name, or slogans in the design, and shows a notice bar and footer saying the project is independent, not affiliated with or endorsed by Clarity, and uses synthetic cases. The page structure changed, but the simulation, routing, and data did not. The JS changes cover the new play-button label and icon, the nav section observer, and the footer clock. The P(review) threshold label moved below the axis so it no longer overlaps the cluster counts, and sections gained a scroll margin so the sticky header does not cover titles after a nav jump. Fixed a keyboard bug found while testing: pressing Enter on a queue row or feed item opened the drawer, moved focus to Close, and then the same key press closed the drawer. Row keydown now calls `preventDefault`, and Space also opens a case. Verification: browser checks at 1440, 1100, and 390 px in light and dark themes, with no horizontal page overflow (the graph, nav, and queue table scroll inside their own containers on narrow screens). Checked nav jumps, opening the drawer by keyboard with focus on Close, a reviewer action updating the open count and note, and Escape returning focus to the row. No console errors. `node web/routing.test.mjs` passed all 11 checks and Prettier passed. No GPU, deployment, commit, or push was made. ### Dashboard copy pass Cut the dashboard copy down to labels and numbers. Removed the eyebrows and slogan headlines that repeated the lede (section headings are now just "Live routing", "Review queue", and "How it works"), the stats section title, static helper lines under the stats, the feed rate, the graph hint (now a tooltip on each stage), the chart subtitles, and the long policy prose in "How it works". Stat labels, node status lines, and chart labels are shorter. The stats footnotes and the calibration caveat now sit in one collapsed "About this data" disclosure. The footer keeps a one-line disclaimer and a developer credit (Akki Singh, tcombinator.dev); the notice bar still states non-affiliation. The hero intro paragraph was removed, leaving the headline and replay controls. The case drawer opens on the essentials: status, route, both model outputs, reasons, and reviewer actions. Facts and signals, the account timeline, text evidence, and the raw packet are collapsed sections. The drawer re-renders as the replay advances, so open sections are tracked outside the DOM and stay open. Answer-token mass moved from the score box into the facts, and reason codes moved into tooltips. Verification: checked 1440 px light and 390 px dark with no horizontal page overflow, the "About this data" toggle, and a drawer section staying open through 3 s of 10× playback. No console errors. `node web/routing.test.mjs` passed all 11 checks and Prettier passed. No commit or push was made. ### About page Added `web/dashboard/about.html`, with `about.css` and `about.js`, a technical write-up in the dashboard's design. It has six sections. Data covers why the cases are synthetic: no public private-share fraud dataset exists. Cases are seeded from SAML-D and BAF rows downloaded from Kaggle, rebuilt in the project schema, labelled by code, written by GPT-6 Luna, and gated by JSON validation (strict Pydantic models, event chronology, recomputed summaries, order fields, fixed reason codes, literal evidence, and reference agreement). SFT model and Decision model give settings, results, and the base-versus-trained comparison. Runtime shows the Ecto.Multi intake, the `risk_decision` and `risk_sft` Oban queues, and the PostgreSQL worklist. Serving covers the model gateway, and Limits states what the results do not show. Loss charts are inline SVG drawn from points copied from `docs/training-metrics.jsonl` and `docs/decision-metrics.jsonl`, marking the selected checkpoints and the step where credits ran out. Teacher selection and source licences are collapsed sections. The dashboard nav gained an About link, and "How it works" links to the page. Adding a non-hash nav link would have thrown in the section observer, which passed every `href` to `querySelector`. The observer now only reads `#` links. On both pages the last nav item now activates at the page end; the final section was too short to reach the observer band. Verification: checked the About page at 1440 px light and dark and 390 px, with no horizontal overflow, readable charts, and a working back link on narrow screens. Also checked the dashboard's nav links, the link to About, and active-section highlighting while scrolling on both pages. No console errors. `node web/routing.test.mjs` passed all 11 checks and Prettier passed. No commit or push was made. Owner revisions: the dashboard headline is now "Fraud detection." The About page links the Amp repository from the hero and footer. The Data section was rewritten in three steps from the owner's text. Step 1 builds cases from SAML-D fields, BAF telemetry, SEC-inspired scenarios, and generated controls, with code-assigned labels; source labels did not become private-share labels, and the proposed IBM AMLworld, RBA, NFT, and phishing sources were not ingested. Step 2 has Luna propose and separately audit each verdict, with code checks and acceptance requiring agreement with the saved policy label. Step 3 covers the 500 plus 2,000 expansion and the final counts. The JSON validation checks are now a collapsed list in step 2. The licences note and the Limits section were removed because this is a test dashboard, not a public site, and Serving replaced Limits in the nav. The section observer now skips updates at the page end, so the last nav item stays active there. Rechecked after bypassing the browser cache: the last item activates at the end of both pages, with no console errors. Prettier and `node web/routing.test.mjs` passed. ### Remove the single-case trace At the owner's request, deleted the original single-case trace page: `web/index.html`, `web/app.js`, `web/style.css`, and the design studies in `web/prototypes/`. The dashboard is now the only web view. Kept `web/routing.mjs`, `web/routing.test.mjs`, `web/cases.json`, and `web/build_cases.py`, which the dashboard and tests still use. Removed the dashboard's "Single-case trace" header link and updated the README line about the static site. The Amp repository link moved from the About hero into the header nav of both pages as "Source code on Amp ↗", in teal and bold. On narrow screens it is the first nav item, so it stays visible in the scrolling nav. The About page's section observer now only reads `#` links. A narrow-screen rule that hid the last header action was removed; it had hidden the trace link and would now have hidden the theme toggle. Verification: checked both headers at 1440 and 390 px with no horizontal overflow, the theme toggle visible on phones, and no console errors. `node web/routing.test.mjs` passed all 11 checks and Prettier passed for `web/dashboard/`. README.md already failed Prettier before this change, so it was not reformatted. No commit or push was made. ### Base versus SFT evaluation on the About page The owner ran the untuned Qwen3.5-4B and the SFT adapter on the same 124 held-out cases, with identical prompts, greedy decoding, a 160-token cap, and thinking off, on one Modal L40S. The results were supplied in conversation; no evaluation file is in the repository yet. Base versus SFT: valid output contract 87.9% vs 100%, exact risk (JSON only) 71.8% vs 100%, exact risk with a valid contract 62.1% vs 100%, exact full verdict 40.3% vs 100%, reason-code F1 65.4% vs 100%, review recall 66.7% vs 100%. Full-verdict accuracy rose 59.7 points and risk-level accuracy 28.2 points. In one example, a case whose stated finances contradicted the accreditation claim got level 1 with `NO_CONCERN` from the base model and level 3 with `ACCREDITATION_INCONSISTENT` from SFT. Added this as section 01, Evaluation, at the top of the About page: three headline numbers, a paired bar chart in HTML and CSS, the run settings, and the example. The other sections were renumbered 02–06, and Evaluation is first in the nav. Verified at 1440 px light and dark and at 390 px, with no horizontal overflow and no console errors. Prettier passed. No commit or push was made. ### Simpler headers At the owner's request, removed the notice bar from both pages and the dashboard's Overview, Live routing, Review queue, and How it works nav links. The dashboard header now has only About and "Source code on Amp ↗". Also removed the code those links needed: the review-queue count badge, the dashboard's section observer and its scroll handler, and the `.notice` and `.nav-count` styles. The About page nav is unchanged. Verification: checked the dashboard at 1440 px and the About page at 1440 and 390 px, with no horizontal overflow and no console errors. Replay still updates the stats. `node web/routing.test.mjs` passed all 11 checks and Prettier passed for `web/dashboard/`. No commit or push was made.