Announcing Morphy, MouseCat's System One Model, and FraudBench, a New Fraud Benchmark
Today we are announcing Morphy v0.1.0, MouseCat’s System One model, and MouseCat FraudBench, our first proprietary fraud benchmark. On the first FraudBench dataset, Morphy’s predictions were better than all six of the off-the-shelf models we tested, with about 10% less error than TypeSafe’s Jev and 7% less than Cloudflare’s Clef.
Morphy: Morphy is a System One model. System One models answer structured questions and return a probability for each answer within milliseconds, much faster than a “system two” model like Claude. Morphy returns the probability of risks like fraud, scams and account takeovers, based on information like a user’s profile, payments, sessions and product activity. Like Claude, Morphy can understand a wide variety of unstructured content, limiting the work needed to “featurize” information before providing it to Morphy. Also, because it answers in milliseconds, Morphy is suitable to run on every transaction event, constantly scanning for signs of high-risk behavior. A fraud team can use that probability to approve/block/friction a transaction or to flag it for review. For example, a team can hold a withdrawal for review when the probability passes a threshold the team sets.
FraudBench: FraudBench’s first dataset is a proprietary set of 1,600 labeled cases built from real production deposit and withdrawal activity. It has transaction-level fraud labels based on a chargeback, a stolen-card decline or an account block. We ran Morphy, six off-the-shelf System One models, and a classic statistical fraud model as a baseline on this dataset and compared their performance.
First FraudBench dataset
The first MouseCat FraudBench dataset has 1,600 labeled cases from opted-in customers’ production deposit and withdrawal activity over the past year. Each case consists of a user and a point in time at which their transactions are being evaluated (one of the deposit or withdrawal events’ timestamps). The case contains information recorded at or before the request: the user’s profile, their deposits with payment details and any decline codes, their withdrawal requests, their activity on the platform, their sessions with device and location, and product events. The label is never included in the case.
A user’s transaction is labeled fraudulent if it has one of three signals: a chargeback or dispute on a deposit, a failed deposit that the card issuer declined as a stolen, lost, closed or blocked card, or a manual fraud intervention (i.e., withdrawal block, payment method flag or manual ban on the account). A user’s transaction is labeled legitimate if they have none of these signals and no weaker fraud signal, such as an issuer fraud decline or a fraud rejection from the payment processor. That gives 126 fraud cases and 1,474 legitimate cases, a fraud rate of 7.9%. All of the transactions are beyond the p99 chargeback maturity window.
Questions and model inputs
Every model, including Morphy, received the account’s activity at the time of the transaction and was asked the same 40 yes/no questions, such as “Does it look like the user is card testing?” Each model returns a probability of yes for each question.1
For each off-the-shelf model, the answers are then passed to a statistical model on top, which produces one predicted probability of fraud. The statistical model is a generic machine learning algorithm. We evaluated Morphy separately on the same cases. Forecasts were fit on earlier data and scored on later data.
| Color | Model | Description |
|---|---|---|
| TypeSafe Jev 1.13.0 | The original System One model released by TypeSafe | |
| TypeSafe Jev 1.14.0 | Newer Jev version, released October 14, 2026 | |
| Cloudflare Clef | Larger model in Cloudflare’s open-weight decision family | |
| Cloudflare Clef-flash | Smaller Clef variant designed for latency-sensitive decisions | |
| Strands Decider | From Amazon’s Strands Labs | |
| OpenAI Decisions API | Based on OpenAI’s gpt-6-luna | |
| Statistical fraud model | Reference forecast | |
| Morphy | MouseCat’s System One model |
Evaluation
We score each forecast with the Brier score, which measures how far its predicted probabilities are from what happened. A lower score is better. A forecast gets a low Brier score if its probabilities match the observed fraud rates and separate fraud from legitimate transactions. We report each forecast’s error relative to the statistical model, so 10% means 10% less error than the statistical model and zero means the same error.2 Each result comes with a 90% interval. An interval that contains zero means the forecast’s error cannot be told apart from the statistical model’s on this dataset.
Results

On the first FraudBench dataset, the six off-the-shelf System One models perform about the same as the statistical model. Cloudflare’s Clef and OpenAI’s Decisions API produce the best off-the-shelf estimates, reducing Brier error by 3.4% and 1.7% relative to the statistical model. Clef-flash is the weakest, with 6.1% more error, and both Jev versions are close to zero. For all six, the 90% interval contains zero, so these minor differences in performance are indistinguishable from noise. Morphy reduces Brier error by 10% relative to the statistical model. Its 90% interval lies entirely above zero. Compared directly with the off-the-shelf models, its Brier error is about 10% lower than either Jev version and 7% lower than Clef.
In plain terms, when Morphy gives a group of transactions a 20% chance of fraud, the share that turns out to be fraud is closer to 20% than it is for the other models, and Morphy gives fraud cases higher probabilities relative to legitimate ones.
Latency and cost
We measured latency and cost per case.3
| Model | Mean | p50 | p90 | p99 | Cost per 1,000 cases |
|---|---|---|---|---|---|
| TypeSafe Jev 1.13.0 | 92 ms | 87 ms | 116 ms | 159 ms | $0.19 |
| TypeSafe Jev 1.14.0 | 89 ms | 86 ms | 111 ms | 155 ms | $0.19 |
| Cloudflare Clef | 778 ms | 681 ms | 973 ms | 3,407 ms | $1.25 |
| Cloudflare Clef-flash | 653 ms | 548 ms | 1,046 ms | 2,672 ms | $0.20 |
| Strands Decider | 334 ms | 287 ms | 400 ms | 1,152 ms | $1.15 |
| OpenAI Decisions API | 364 ms | 308 ms | 637 ms | 966 ms | $0.88 |
| Morphy | 104 ms | 95 ms | 123 ms | 306 ms | $0.31 |
Jev is the fastest. Morphy is about 10 milliseconds slower than Jev at the median and the 90th percentile, and about twice as slow at the 99th percentile. Jev and Clef-flash are the cheapest at about 20 cents per 1,000 cases. Morphy costs 31 cents.
Clef-flash costs about the same as Jev but is about six times slower at the median, at about 550 milliseconds. Clef is the slowest at the median, at about 700 milliseconds, and the most expensive, at $1.25 per 1,000 cases. OpenAI’s Decisions API and Strands Decider each take about 300 milliseconds at the median, and cost 88 cents and $1.15 per 1,000 cases.


Brier score decomposition
Morphy’s predictions score best both because they are closer to the observed fraud rates than any other forecast’s, and do a better job separating fraud from legitimate transactions. We found this by splitting the Brier score into its components.4 Miscalibration (MCB) measures how far the predicted probabilities are from the observed fraud rates, and lower is better. Discrimination (DSC) measures how well a forecast separates fraud from legitimate transactions, and higher is better.

The statistical model has the worst calibration but better discrimination than any of the off-the-shelf decision models. Among the off-the-shelf models, Clef has the best discrimination and Clef-flash, the smaller model in the same family, has the worst. The two Jev models perform similarly.
Relative to the statistical model, Morphy adds 0.0015 of discrimination and removes 0.0026 of miscalibration, both in Brier-score units. Its discrimination is higher than the statistical model’s and higher than every off-the-shelf model’s. Its calibration error is the lowest of the eight forecasts, slightly below Clef and OpenAI’s Decisions API.
Closing
The dataset in this post is the first in FraudBench. We plan to add more datasets and to report results for new System One models as they are released.
Morphy is already being used by several MouseCat customers.
Reach out to us for early access:
- Performance on individual questions differed between models. In one of the exploratory questions while setting up the evaluation, we asked whether the latest payment occurred at night in the user’s location. When given a timestamp in UTC, Jev’s AUC was 0.77 compared with Clef-flash’s 0.56. When local time was supplied explicitly, both scores were 1.00. ↩
- The Brier score is the mean squared error of the predicted probabilities. The reported value is Brier skill: 1 − Brier score / Brier score of the baseline statistical model. Intervals are 90% paired bootstrap intervals from 10,000 resamples of case families, with the same resampled cases used for every forecast. ↩
- Latency is per case: the time from sending one request with the case and all 40 questions to receiving the full response, measured from a single client over the public internet, one case at a time on an already-open connection. Cost is the average input tokens each provider reported per case, multiplied by its list price per million input tokens: $0.042 for Jev, $0.24 for Clef, $0.038 for Clef-flash and $0.10 for OpenAI’s Decisions API. None of the four charges for output tokens. Strands Decider has no hosted API, so it was hosted on a dedicated NVIDIA B200 GPU on Modal, and its cost is the GPU’s hourly price at 50% utilization. ↩
- There’s a long history of Brier score decompositions into reliability and resolution components. We used the modern CORP decomposition from Dimitriadis et al. and their implementation in the {reliabilitydiag} package. ↩