Default Study Methodology

Version 1.0 · Pre-registered · Published before any results exist · sentinel-agent.dev

A score means nothing until someone checks whether it was right. This document specifies, in advance and in full, how Sentinel Oracle will check its own. It is published before the first Default Study is compiled, precisely so the rules cannot be adjusted after the fact to fit a convenient result.

This is the same discipline a credit rating agency is judged by: not the sophistication of the model, but the durability of a public record showing whether its calls held up. Section 7 of the Methodology commits to publishing an annual Default Study beginning July 2027. This document is the fixed rulebook that study will follow.

1. Why this is published before any results

A methodology written after the outcomes are known can be quietly shaped to produce a flattering number — which bucket boundary to use, which incidents to count, which time window to apply. None of that is available here: these rules are fixed now, while Sentinel Oracle has no completed study to protect. If the first Default Study shows poor calibration, that result gets published under this same methodology, unchanged. Rigor that only applies when it is convenient is not rigor.

2. Scope: three separate studies, never blended

Sentinel Oracle makes three structurally different predictions. Mixing their outcomes into a single headline accuracy figure would hide which one is actually working.

Each is reported on its own, with its own sample size, its own calibration table, and its own stated limitations. A strong buyer-direction result does not get to borrow credibility from a weak evaluator result, or vice versa.

3. Study population and observation window

A score is eligible for a given study period only if two conditions both hold at compile time:

Scores issued during a publicly acknowledged Sentinel Oracle infrastructure incident (logged separately, never silently) are included but flagged, not excluded — removing them would let an outage conveniently erase inconvenient data.

4. What counts as a default: ground truth per direction

Buyer direction. A default is any Default Registry entry of type non_payment or replay_abuse, attributed to the scored wallet, with a created_at timestamp falling within the observation window after the specific attestation's issued_at.

Seller/counterparty direction. A default is a Default Registry entry of type non_delivery or quality_fraud, windowed the same way.

Evaluator verdicts — a structurally weaker signal, stated plainly. Once a signed verdict is used to call complete() or reject() on-chain, Sentinel Oracle has no further visibility into what happens to the released or withheld funds. A reject verdict is scored as confirmed if the same provider accrues an independent Default Registry incident within the window; a complete verdict is scored as contradicted under the same rule. This cannot capture disputes that never reach the Default Registry. The evaluator study will state this limitation in its own text every time it is published, not once in a footnote.

No other event counts as a default. A wallet going inactive, a dispute with no verified incident behind it, or a subjective complaint with no Default Registry entry are all explicitly not defaults under this methodology.

5. Cohorting: score deciles

Every eligible score is placed into one of ten buckets by its value at issuance: 0–10, 10–20, …, 90–100. The study reports the default rate inside each bucket. This table — not a single blended accuracy number — is the actual claim under test: a 90+ score should show a measurably lower default rate than a 40–50 score. If it does not, that is reported exactly as plainly as if it does.

6. Scores that change mid-window

A wallet can be scored more than once across a study period, and the score can move between issuances. The rule: each signed attestation is evaluated independently, against outcomes only in its own observation window, using the score it actually carried at issuance. A later, improved attestation does not retroactively reclassify an earlier one's prediction. This is what stops a bad early call from being quietly laundered by a subsequent good one — exactly the failure mode a methodology written after the fact would be tempted to paper over.

7. Censoring and incomplete data

A score whose 90-day (or 30-day, for evaluator verdicts) window has not yet elapsed by compile time is excluded from that period's study and carried into the next one — it is never assigned a default/non-default label on partial information. Every published study reports the number of records excluded this way, so the sample actually used cannot be quietly narrowed to whatever set of records happens to look best.

8. What gets published

9. Honesty about small samples

Sentinel Oracle is young; early studies will have thin buckets. Any decile with fewer than 30 eligible scores is reported as a raw count, not a rate, and marked insufficient data rather than implying a precision the sample cannot support. A confident-looking percentage built on six data points is worse than no number at all — it borrows credibility the sample has not earned.

10. Publish regardless of outcome

This methodology is a pre-commitment to publish every study it produces, on schedule, whether the calibration is strong, weak, or absent. A study that only sees daylight when it is flattering is not a Default Study — it is marketing wearing a Default Study's name.

11. Independent recomputability

Every input this methodology uses is already public: signed attestations with hash-anchored, immutable issued_at timestamps, and the append-only Default Registry. Anyone can pull both data sources independently and recompute the same study from this document alone, without taking Sentinel Oracle's word for the result. Alongside each published study, the underlying dataset (wallet addresses are already public on-chain; nothing here is private data) will be made available for exactly this purpose.

12. Amendments

Any future change to this methodology ships as a new version number with a public changelog entry, and applies only prospectively — it never reinterprets a study period already compiled under a prior version. A methodology that can be rewritten to reclassify the past is not a methodology; it is a moving target.


Addendum A: Foundational Cohort Selection Rule (Phase 1)

Sections 1–12 fix the rules for judging a score once it exists. This addendum fixes something that comes before that: which wallets get scored at all, in the buyer direction, ahead of having meaningful real-payer volume of Sentinel Oracle's own. Left unstated, that question invites exactly the kind of after-the-fact, favorable selection this whole document exists to rule out.

A.1 The rule

A wallet enters the buyer-direction foundational cohort the first time a USDC transferWithAuthorization call — EIP-3009, method ID 0xe3ee160e or 0xcf092995, the on-chain fingerprint of an x402 gasless payment — is observed in the USDC contract's own transaction history on Base, to any recipient, excluding Sentinel Oracle's documented internal test wallets (published in the project's engineering log, never a real independent counterparty).

The wallet is scored once, via the same /v1/attestation buyer-direction pipeline any caller uses, and enters the cohort on that date. It is not re-scored daily simply because it keeps transacting; Section 6 already governs what happens when a wallet is scored again later.

A.2 Why this rule and not a hand-picked list

It makes no selection a person could bias. The wallet does not need to have ever paid Sentinel Oracle, or be discoverable any way Sentinel Oracle could curate for — it is read directly off a public token contract every participant in the ecosystem already shares. Nobody at Sentinel Oracle chooses who gets scored; the order wallets enter the cohort is simply the order they transacted.

A.3 Scope, stated honestly

This is Phase 1: Base only, discovered from the USDC contract's own call history, address-agnostic across recipients. It is the one discovery source that can be claimed with full confidence today. Extending discovery to other chains or other stablecoins is a documented future expansion, not silently assumed to already be covered.

A.4 Cadence

An automated job runs daily, discovering new eligible wallets and scoring a bounded number of them per run (published as COHORT_MAX_NEW_PER_RUN in the implementation; currently 6). The cap exists because Blockscout's unauthenticated API — the data source for this discovery — enforces its own request-rate limit, shared between discovering candidates and scoring each one. The cohort grows steadily across many runs rather than all at once; there is no deadline pressure to relax that cap given the first Default Study is not due until July 2027.


Default Study Methodology v1.0. This document governs the first Default Study, due July 2027, committed to in the Methodology, Section 7. Sentinel Oracle is live at sentinel-agent.dev.