How a Neural Network Learned Its Own Fraud Rules: A Neuro-Symbolic AI Experiment

systems inject rules written by humans. But what if a neural network could discover those rules itself?

On this experiment, I extend a hybrid neural network with a differentiable rule-learning module that mechanically extracts IF-THEN fraud rules during training. On the Kaggle Credit Card Fraud dataset (0.17% fraud rate), the model learned interpretable rules comparable to:

IF V14 < −1.5σ AND V4 > +0.5σ → Fraud

where σ denotes the feature standard deviation after normalization.

The rule learner achieved ROC-AUC 0.933 ± 0.029, while maintaining 99.3% fidelity to the neural network’s predictions.

Most interestingly, the model independently rediscovered V14 — a feature long known by analysts to correlate strongly with fraud — without being told to search for it.

Full code: github.com/Emmimal/neuro-symbolic-ai-fraud-pytorch

What the Model Discovered

Before the architecture, the loss function, or any training details — here’s what got here out the opposite end.

After as much as 80 epochs of coaching (with early stopping, most seeds converged between epochs 56–78), the rule learner produced these within the two seeds where rules emerged clearly:

Seed 42 — cleanest rule (5 conditions, conf=0.95)

Learned Fraud Rule — Seed 42 · Rules were never hand-coded

IF   V14 < −1.5σ
AND V4  > +0.5σ
AND V12 < −0.9σ
AND V11 > +0.5σ
AND V10 < −0.8σ

THEN FRAUD

Seed 7 — complementary rule (8 conditions, conf=0.74)

Learned Fraud Rule — Seed 7 · Rules were never hand-coded

IF   V14 < −1.6σ
AND V12 < −1.3σ
AND V4  > +0.3σ
AND V11 > +0.5σ
AND V10 < −1.0σ
AND V3  < −0.8σ
AND V17 < −1.5σ
AND V16 < −1.0σ

THEN FRAUD

In each cases, low values of V14 sit at the guts of the logic — a striking convergence given zero prior guidance.

The model was never told which feature mattered.

Yet it independently rediscovered the identical feature human analysts have identified for years.

A neural network discovering its own fraud rules is strictly the promise of neuro-symbolic AI: combining statistical learning with human-readable logic. The remainder of this text explains how — and why the gradient kept finding V14 even when told nothing about it.

From Injected Rules to Learned Rules — Why It Matters

Every fraud model has a call boundary. Fraud teams, nonetheless, operate using rules. The gap between them, between what the model learned and what analysts can read, audit, and defend to a regulator — is where compliance teams live and die.

In my previous article on this series, I encoded two analyst rules directly into the loss function: and . That approach worked. The hybrid model matched the pure neural net’s detection performance while remaining interpretable.

But there was an obvious limitation I left unaddressed. I wrote those rules. I selected those two features because they made intuitive sense to me. Hand-coded rules encode what you already know, they're a very good solution when fraud patterns are stable and domain knowledge is deep. They're a poor solution when fraud patterns are shifting, when crucial features are anonymized (as they're on this dataset), or whenever you want the model to surface signals you haven’t thought to search for.

The natural next query:

This pattern extends beyond fraud. Medical diagnosis systems need rules that doctors can confirm before acting. Cybersecurity models need rules that engineers can audit. Anti-money laundering systems operate under regulatory frameworks requiring explainable decisions. In any domain combining rare events, domain expertise, and compliance requirements, the power to extract auditable IF-THEN rules from a trained neural network is directly helpful.

Architecturally, the change is surprisingly easy. You aren't replacing the MLP, you're adding a second path that learns to precise the MLP’s decisions as human-readable symbolic rules. The MLP trains normally. The rule module learns to agree with it, in symbolic form. That's the subject of this text: differentiable rule induction in ~250 lines of PyTorch, with no prior knowledge of which features matter.

“You aren't replacing the neural network. You're teaching it to elucidate itself.”

The Architecture: Three Learnable Pieces

The architecture keeps a typical neural network intact, but adds a second path that learns symbolic rules explaining the network’s decisions. The 2 paths run in parallel from the identical input and their outputs are combined by a learnable weight α:

The MLP path is an identical to the previous article: three fully connected layers with batch normalization. The rule path is recent. Alpha is a learnable scalar that the model uses to weight the 2 paths, it starts at 0.5 and is trained by gradient descent like all other parameter. After training, α converged to roughly 0.88 on average across seeds (range: 0.80–0.94). The model learned to weight the neural path at roughly 88% and the rule path at 12% on average. The principles aren't replacing the MLP, they're a structured symbolic summary of what the MLP learned.

1. Learnable Discretizer

Rules need binary inputs — . Neural networks need continuous, differentiable operations. The soft sigmoid threshold bridges each.

For every feature and every learnable threshold :

$$b_{f,t} = sigma!left(frac{x_f – theta_{f,t}}{tau}right)$$

Where:

• $x_f$​ is the worth of feature *f* for this transaction
• $theta_{f,t}$t​ is a learnable threshold, initialized randomly, trained by backpropagation
• $tau$ is temperature — high early in training (exploratory), low later (crisp)
• $b_{f,t}$ is the soft binary output: “is feature *f* above threshold *t*?”

The model learns three thresholds per feature, giving it three “cuts” per dimension. Each threshold is independent — the model can spread them across the feature’s range or concentrate them around probably the most discriminative cutpoint.

At τ=5.0 (epoch 0): the sigmoid is sort of flat. Every feature value produces a gradient. The model explores freely. At τ=0.1 (epoch 79): the sigmoid is almost a step function. Thresholds have committed. The boundaries are readable as human conditions.

class LearnableDiscretizer(nn.Module):
    def __init__(self, n_features, n_thresholds=3):
        super().__init__()
        # One learnable threshold per (feature × bin)
        self.thresholds = nn.Parameter(
            torch.randn(n_features, n_thresholds) * 0.5
        )
        self.n_thresholds = n_thresholds

    def forward(self, x, temperature=1.0):
        # x: [B, F] → output: [B, F * n_thresholds] soft binary features
        x_exp = x.unsqueeze(-1)               # [B, F, 1]
        t_exp = self.thresholds.unsqueeze(0)  # [1, F, T]
        soft_bits = torch.sigmoid(
            (x_exp - t_exp) / temperature
        )
        return soft_bits.view(x.size(0), -1)  # [B, F*T]

2. Rule Learner Layer

Each rule is a weighted combination of binarized features, passed through a sigmoid:$$text{rule}_r(x) = sigma!left(frac{sum_i w_{r,i} cdot b_i}{tau}right)$$

The sign of every weight has a direct interpretation after tanh squashing:

• $w > +0.5$ → feature have to be HIGH for this rule to fireside
• $w < -0.5$ → feature have to be LOW for this rule to fireside
• $|w| < 0.5$ → feature is irrelevant to this rule

Rule extraction follows directly: threshold absolutely the weight values after training to discover which features each rule uses. That is how IF-THEN statements emerge from continuous parameters — by reading the load matrix.

class RuleLearner(nn.Module):
    def __init__(self, n_bits, n_rules=4):
        super().__init__()
        # w_{r,i}: which binarized features matter for every rule
        self.rule_weights = nn.Parameter(
            torch.randn(n_rules, n_bits) * 0.1
        )
        # confidence: relative importance of every rule
        self.rule_confidence = nn.Parameter(torch.ones(n_rules))

    def forward(self, bits, temperature=1.0):
        w = torch.tanh(self.rule_weights)        # bounded in (-1, 1)
        logits = bits @ w.T                       # [B, R]
        rule_acts = torch.sigmoid(logits / temperature)  # [B, R]
        conf = torch.softmax(self.rule_confidence, dim=0)
        fraud_prob = (rule_acts * conf.unsqueeze(0)).sum(dim=1, keepdim=True)
        return fraud_prob, rule_acts

3. Temperature Annealing

The temperature follows an exponential decay schedule:$$tau(t) = tau_{text{start}} cdot left(frac{tau_{text{end}}}{tau_{text{start}}}right)^{t/T}$$

With τ_start=5.0, τ_end=0.1, T=80 epochs:

def get_temperature(epoch, total_epochs, tau_start=5.0, tau_end=0.1):
    progress = epoch / max(total_epochs - 1, 1)
    return tau_start * (tau_end / tau_start) ** progress

Without annealing, the model stays soft and rules never crystallize into anything a fraud analyst can read or a compliance team can log off on. Annealing is what converts a continuous optimization right into a symbolic output.

Before the loss function — a fast note on where this concept comes from, and what makes this implementation different from prior work.

Standing on the Shoulders of ∂ILP, NeuRules, and FINRule

It's price situating this work in the prevailing literature not as a full survey, but to make clear what ideas are borrowed and what's recent.

Differentiable Inductive Logic Programming introduced the core concept that inductive logic programming traditionally a combinatorial search problem — may be reformulated as a differentiable program trained with gradient descent. The important thing insight used here is using soft logical operators that allow gradients to flow through rule-like structures. Nonetheless, ∂ILP requires predefined rule templates and background knowledge declarations, which makes it harder to integrate into standard deep learning pipelines.

Recent work applying differentiable rules to fraud detection comparable to FINRule — shows that rule-learning approaches can perform well even on highly imbalanced financial datasets. These studies exhibit that learned rules can match hand-crafted detection logic while adapting more easily to recent fraud patterns.

Other systems comparable to RIFF and Neuro-Symbolic Rule Lists introduce decision-tree-style differentiable rules and emphasize sparsity to take care of interpretability. The L1 regularization utilized in this implementation follows the identical principle: encouraging rules to depend on only a couple of conditions reasonably than all available features.

The implementation in this text combines these ideas differentiable discretization plus conjunction learning — but reduces them to roughly 250 lines of dependency-free PyTorch. No template language. No background knowledge declarations. The goal is a minimal rule-learning module that may be dropped into a typical training loop.

Three-Part Loss: Detection + Consistency + Sparsity

The total training objective:

$$mathcal{L}_{text{total}} = mathcal{L}_{text{BCE}} + lambda_c cdot mathcal{L}_{text{consistency}} + lambda_s cdot mathcal{L}_{text{sparsity}} + lambda_{text{conf}} cdot mathcal{L}_{text{confidence}}$$

L_BCE — Weighted Binary Cross-Entropy

An identical to the previous article. pos_weight = count(y=0) / count(y=1) ≈ 578. One labeled fraud sample generates 578× the gradient of a non-fraud sample. This term is unchanged the rule path adds no complexity to the core detection objective.

L_consistency — The Recent Term

Rules should agree with the MLP where the MLP is confident. Operationally: MSE between rule_prob and mlp_prob, masked to predictions where the MLP is either clearly fraud (>0.7) or clearly non-fraud (<0.3):

confident_mask = (mlp_prob > 0.7) | (mlp_prob < 0.3)
if confident_mask.sum() > 0:
    consist_loss = F.mse_loss(
        rule_prob.squeeze()[confident_mask],
        mlp_prob.squeeze()[confident_mask].detach()  # ← critical
    )

The .detach() is critical: we're teaching the foundations to follow the MLP, not the opposite way around. The MLP stays the first learner. The uncertain region (0.3–0.7) is deliberately excluded that's where rules might catch something the MLP misses.

L_sparsity — Keep Rules Easy

L1 penalty on the raw (pre-tanh) rule weights: mean(|W_rules|). Without this, rules absorb all 30 features and turn out to be unreadable. With λ_s=0.25, the optimizer pushes irrelevant features toward zero while leaving genuinely useful features — V14, V4, V12 — at |w| ≈ 0.5–0.8 after tanh squashing.

L_confidence — Kill Noise Rules

A small L1 penalty on the boldness logits (λ_conf=0.01) drives low-confidence rules toward zero weight within the output combination, effectively eliminating them. Without this, multiple technically energetic but meaningless rules appear with confidence 0.02–0.04 that obscure the true signal.

Final hyperparameters: λ_c=0.3, λ_s=0.25, n_rules=4, λ_conf=0.01.

With the machinery in place here's what it produced.

Results: Does Rule Learning Work — and What Did It Find?

Experimental Setup

• Dataset: Kaggle Credit Card Fraud, 284,807 transactions, 0.173% fraud rate
• Split: 70/15/15 stratified by class label, 5 random seeds [42, 0, 7, 123, 2024]
• Threshold: F1-maximizing on validation set, applied symmetrically to check set
• Same evaluation protocol as Article 1

Detection Performance

The rule learner sits barely below the pure neural baseline on all three detection metrics, roughly 1.5 F1 points on average. The tradeoff is explainability. The per-seed breakdown shows the total picture:

In seeds 42 and 0, the rule learner exceeds the pure neural baseline on F1. In seed 2024, it exceeds on ROC-AUC. The performance variance across seeds is the honest picture of what gradient-based rule induction produces on a 0.17% imbalanced dataset.

Rule Quality — The Recent Contribution

Three metrics, Each answers a unique query a compliance officer would ask.

Rule Fidelity — can I trust this rule set to represent the model’s actual decisions?

def rule_fidelity(mlp_probs, rule_probs, threshold=0.5):
    mlp_preds  = (mlp_probs  > threshold).astype(int)
    rule_preds = (rule_probs > threshold).astype(int)
    return (mlp_preds == rule_preds).mean()

Rule Coverage — what fraction of actual fraud does at the least one rule catch?

def rule_coverage(rule_acts, y_true, threshold=0.5):
    any_rule_fired = (rule_acts > threshold).any(axis=1)
    return any_rule_fired[y_true == 1].mean()

Rule Simplicity — what number of unique feature conditions per rule, after deduplication?

def rule_simplicity(rule_weights_numpy, weight_threshold=0.50):
    # Divide by n_thresholds (=3) to get unique features,
    # the meaningful readability metric. Goal: < 8.
    active = (np.abs(rule_weights_numpy) > weight_threshold).sum(axis=1)
    unique_features = np.ceil(energetic / 3.0)
    unique_features = unique_features[unique_features > 0]
    return float(unique_features.mean()) if len(unique_features) > 0 else 0.0

This highlights an actual tension in differentiable rule learning: strong sparsity regularization produces clean rules after they appear, but may cause the symbolic path to go dark in some initializations. Reporting mean ± std across seeds reasonably than cherry-picking the perfect seed is important precisely for this reason variance.

Fidelity at 0.993 signifies that in seeds where rules are energetic, they agree with the MLP on 99.3% of binary decisions — the consistency loss working exactly as designed.

The Extracted Rules — What the Gradient Found

Each rules are shown in full at the highest of this text. The short version: seed 42 produced a decent 5-condition rule (conf=0.95), seed 7 a broader 8-condition rule (conf=0.74). In each, V14 < −1.5σ (or −1.6σ) appears because the leading condition.

The cross-seed feature evaluation confirms the pattern across all five seeds:

Even with only two seeds producing visible rules, V14 ranked first or second in each — a statistically striking convergence given zero prior feature guidance. The model didn't must be told what to search for.

What the Model Found — and Why It Makes Sense

V14 is one in all 28 PCA components extracted from anonymized bank card transaction data. Exactly what it represents isn't public knowledge — that's the point of the anonymization. What multiple independent analyses have established is that V14 has the best absolute correlation with the fraud label of any feature within the dataset.

Why did the rule learner find it? The mechanism is the consistency loss. By training rules to agree with the MLP’s confident predictions, the rule learner is reading the MLP’s internal representations and translating them into symbolic form. The MLP had already learned from the labels that V14 was essential. The consistency loss transferred that signal into the rule weight matrix. Temperature annealing then hardened that weight right into a crisp threshold condition.

That is the basic difference between Rule Injection (Article 1) and Rule Learning (this text). Rule injection encodes what you already know. Rule learning discovers what you don’t. On this experiment, the invention was V14 — a signal the gradient found independently, without being told to search for it.

Across five seeds, readable rules emerged in two — consistently highlighting V14. That could be a powerful demonstration that gradient descent can rediscover domain-critical signals without being told to search for them.

A compliance team can now read Rule 1, confirm that V14 < −1.5σ makes domain sense, and log off on it — without opening a single weight matrix. That's what neuro-symbolic rule learning is for.

4 Things to Watch Before Deploying This

• Annealing speed is your most sensitive hyperparameter Too fast: rules crystallize before the MLP has learned anything — you get crisp nonsense. Too slow: τ never falls low enough and rules stay soft. Treat τ_end as the primary parameter to tune on a brand new dataset.
• n_rules sets your interpretability budget Above 8–10 rules, you might have a lookup table, not an auditable rule set. Below 4, it's possible you'll miss tail fraud patterns. The sweet spot for compliance use is 4–8 rules.
• The consistency threshold assumes a calibrated MLP In case your base MLP is poorly calibrated — common on severely imbalanced data — the mask fires too rarely. Run a calibration plot on validation outputs. Consider Platt scaling if calibration is poor.
• Learned rules need auditing after every retrain Unlike frozen hand-coded rules, learned rules update at any time when the model retrains. The compliance team cannot log off once and walk away — the sign-off must occur every retrain cycle.

Rule Injection vs. Rule Learning — When to Use Which

The rule learner adds roughly 200 lines of code and a hyperparameter sweep. It isn't free. On very small datasets, the consistency loss may not accumulate enough signal to learn meaningful rules — validate fidelity before treating extracted rules as authoritative. The approach is a tool, not an answer.

One honest commentary from the five-seed experiment: in 3 of 5 seeds, strong sparsity pressure drove all rule weights below the extraction threshold. The model converged to the best detection answer but expressed it purely through the MLP path. This variance is real. Single-seed results would give a misleadingly clean picture — which is why multi-seed evaluation is non-negotiable for any paper or article making claims about learned rule behavior.

The following query on this series is whether or not these extracted rules can flag concept drift — detecting when fraud patterns have shifted enough that the foundations need updating before model performance degrades. When V14’s importance drops within the rule weights while detection metrics hold regular, the fraud distribution could also be changing. That early warning signal is the topic of the following article.

Disclosure

This text is predicated on independent experiments using publicly available data (Kaggle Credit Card Fraud dataset, CC-0 Public Domain) and open-source tools (PyTorch, scikit-learn). No proprietary datasets, company resources, or confidential information were used. The outcomes and code are fully reproducible as described, and the GitHub repository comprises the whole implementation. The views and conclusions expressed listed below are my very own and don't represent any employer or organization.

References

[1] Evans, R., & Grefenstette, E. (2018). Learning Explanatory Rules from Noisy Data. JAIR, 61, 1–64. https://arxiv.org/abs/1711.04574

[2] Wolfson, B., & Acar, E. (2024). Differentiable Inductive Logic Programming for Fraud Detection. arXiv preprint arXiv:2410.21928. https://arxiv.org/abs/2410.21928

[3] Martins, J. L., Bravo, J., Gomes, A. S., Soares, C., & Bizarro, P. (2024). RIFF: Inducing Rules for Fraud Detection from Decision Trees. RuleML+RR 2024. arXiv:2408.12989. https://arxiv.org/abs/2408.12989

[4] Xu, S., Walter, N. P., & Vreeken, J. (2024). Neuro-Symbolic Rule Lists. arXiv preprint arXiv:2411.06428. https://arxiv.org/abs/2411.06428

[5] Kusters, R., Kim, Y., Collery, M., de Sainte Marie, C., & Gupta, S. (2022). Differentiable Rule Induction with Learned Relational Features. arXiv preprint arXiv:2201.06515. https://arxiv.org/abs/2201.06515

[6] Dal Pozzolo, A. et al. (2015). Calibrating Probability with Undersampling for Unbalanced Classification. IEEE SSCI. Dataset: https://www.kaggle.com/datasets/mlg-ulb/creditcardfraud (CC-0)

[7] Alexander, E. P. (2026). Hybrid Neuro-Symbolic Fraud Detection. Towards Data Science. https://towardsdatascience.com/hybrid-neuro-symbolic-fraud-detection-guiding-neural-networks-with-domain-rules/

[8] Liu, F. T., Ting, K. M., & Zhou, Z.-H. (2008). Isolation Forest. In 2008 Eighth IEEE International Conference on Data Mining (ICDM), pp. 413–422. IEEE. https://doi.org/10.1109/ICDM.2008.17

[9] Paszke, A. et al. (2019). PyTorch. NeurIPS 32. https://pytorch.org

[10] Pedregosa, F. et al. (2011). Scikit-learn: Machine Learning in Python. JMLR, 12, 2825–2830. https://scikit-learn.org