MouseCat Trends: Finding Emerging Fraud Before the Labels Arrive

Finding what is changing

Let’s return to the consumer marketplace. Trend Detection compares the most recent orders with a large collection of historical orders to identify emerging patterns of behavior. First, it assembles a rich set of features about each transaction in both periods: payment details, account attributes, order history, delivery information, and so on.

It then creates groups of transactions by iteratively building conjunctions of statements about individual features. For example, it might first select orders from accounts at most a week old, then narrow that group to orders paid with cards from ‘Bank X’, then limit further to orders from accounts that had already made at least three paid purchases in the previous 24 hours.

There are far too many potential combinations of conditions to check them all. For example, with 100 features available, there are roughly 20 trillion ways to choose up to ten distinct features, and that’s even before considering all of the possible conditions that could be applied to each one. So the tricky part is deciding which combinations to prioritize. Trend Detection probes candidate combinations while optimizing along three axes:

• Growth. Groups of events whose share of traffic is increasing. Raw percentage growth can push tiny fluctuations to the top of the search results simply because they start from almost nothing (or in some cases literally nothing). MouseCat’s growth metric accounts for statistical uncertainty, helping better-supported increases stand out while still allowing new patterns with no historical matches to surface.
• Coherence. Events that resemble one another. MouseCat computes a similarity score by comparing attributes across events. Scores range from 0 to 100; higher scores indicate that events are more alike.
• Diversity. Groups of events that are far from other groups, so the results cover different patterns rather than many versions of the same one.

In the stylized example below, we look at six candidate fraud trends explored during search to see how growth, coherence, and diversity shape the results.

Open diagram at full size ↗

Candidate A

• account_age_days <= 7
• AND card_issuer = "Bank_X"
• AND paid_orders_prior_24h >= 3

Orders

0 → 2

Adjusted growth

0.1×

Within-group similarity

88

Not retained · Low growth

Candidate B

• account_age_days <= 7
• AND paid_orders_prior_30d = 0
• AND shipping_speed = "standard"

Orders

1,000 → 4,000

Adjusted growth

3.6×

Within-group similarity

25

Not retained · Low similarity

Candidate C

• card_issuer = "Bank_X"
• AND card_first_seen_hours <= 24
• AND parcel_forwarder = true

Orders

1 → 21

Adjusted growth

2.3×

Within-group similarity

86

Not retained · Near-duplicate of F

Candidate D

• loyalty_offer = true
• AND account_age_days >= 180
• AND shipping_speed = "standard"

Orders

100 → 400

Adjusted growth

2.9×

Within-group similarity

92

Retained

Candidate E

• account_age_days >= 180
• AND paid_orders_prior_30d >= 5
• AND paid_order_addresses_30d >= 3

Orders

80 → 1,000

Adjusted growth

9.4×

Within-group similarity

84

Retained

Candidate F

• card_issuer = "Bank_X"
• AND account_age_days <= 7
• AND parcel_forwarder = true

Orders

0 → 20

Adjusted growth

3.3×

Within-group similarity

89

Retained

Between-group similarity

A–B

35

A–C

48

A–D

10

A–E

15

A–F

47

B–C

40

B–D

18

B–E

22

B–F

42

C–D

18

C–E

11

C–F

88

D–E

32

D–F

19

E–F

12

Let’s break down what we’re looking at here. Candidate A only captures two transactions in the newest slice of traffic, which doesn’t represent enough growth to warrant our attention. B has grown a lot across the two time periods, but its orders are too dissimilar to form a coherent fraud pattern. C shows promising growth and coherence, but it covers a lot of the same orders as F, just with weaker growth and less coherence. So F is preferred between the two. D, E, and F are all retained because each is growing, each contains similar orders, and each is sufficiently distinct from the others. All, some, or none of these three remaining patterns could be fraud. The search has given us distinct, growing cohorts of similar activity; we now need to assess whether they actually represent fraud.

Preparing candidates for assessment

To make that assessment, we first need an informative description of the behavior each candidate’s cases share. While we could just use the conditions that found the group during the search, those conditions often leave out details that explain why those transactions resemble one another, and why they may be fraud.

We therefore add a redescription step that builds a fuller definition of each candidate from the characteristics that distinguish its events from the wider traffic. Redescription can add conditions that show how the group’s events stand out from the crowd, remove conditions that are common in the wider traffic, and adjust thresholds to express the shared behavior more clearly. The revised group definition must still retain most of the events in the original group. Growth and coherence are then recomputed for each revised group.

After redescription, the candidates become easier to interpret: D captures returning customers using familiar cards and delivery addresses; E includes a burst of purchases using familiar cards but new delivery addresses; F highlights first-time purchases and expedited shipping. We didn’t know any of these details before. But these fuller definitions give the next stage more meaningful patterns to assess.

Triaging the candidates

With these fuller group definitions in hand, an agent triages each pattern with a preliminary fraud assessment. It knows that each group was discovered in unlabeled customer traffic, rather than a collection of confirmed fraud. Its job is to weigh fraud and legitimate explanations together, reason about the combination of conditions that defines the group, and return an interpretation with p_fraud: its estimated probability that most cases in the pattern are fraudulent.

We see that D fits a known loyalty promotion and is filtered out. E and F each have plausible fraud explanations, so both proceed to the investigation stage.

This triage stage filters out approximately 90% of candidate patterns. That concentrates the deeper, more expensive investigations on the patterns that warrant them.

Investigating cases to generate synthetic fraud labels

Triage has given us plausible fraud explanations for E and F, but we still need to get to the bottom of whether they’re actually fraud or not. To assess that, MouseCat has its Investigator Agent review a sample of cases from each candidate fraud trend. For each sample case, the agent develops hypotheses for fraud and legitimate activity, and then tests them across the customer’s connected data sources. It returns a recommended decision and a calibrated fraud probability, supported by an auditable evidence log.

Trend Detection uses these decisions and probabilities as ‘synthetic’ labels–before chargebacks arrive–in order to make its final evaluation of whether the group of events is an emerging fraud trend or not. Here is how the investigations shake out for E and F:

E’s sampled cases receive low fraud probabilities and recommendations to allow the orders to pass through, so E is filtered out. F’s sampled cases receive high fraud probabilities and recommendations to block, so F is published as an emerging fraud trend.

Finding the attack in time to act

The first day’s activity can often be enough to surface a rising fraud wave before any chargebacks have arrived.

The next stage is taking each emerging fraud trend and refining it into a deployable detection rule that has been backtested to assess its fraud coverage and its impact on legitimate users. MouseCat has a dedicated agent for that process, which we will discuss in an upcoming blog post.

If your team is discovering new fraud waves through customer reports or chargebacks, talk to MouseCat about finding them sooner.