All write-ups

MSc dissertation: explainable detection of AI-written phishing

Submitted, result pending

My MSc dissertation at Liverpool John Moores University: detecting phishing emails written by AI, and measuring whether the explanations of each detector are true to what it actually does.

My role
Author: experiment design, code, evaluation and writing
Year
2026
Built with
Python · scikit-learn · XGBoost · LightGBM · PyTorch · Transformers · SHAP · LIME · Integrated Gradients

The problem

Phishing written by language models reads differently from the old templates, and a detector nobody can explain is hard to trust or audit. I asked two questions: which kind of model holds up on AI-written email, and are the explanations faithful to what the model really does?

How it is built

Three families of models compared on the same splits: classical models (logistic regression, naive Bayes, SVM, decision tree, random forest), gradient-boosted ensembles (XGBoost, LightGBM) and transformers (BERT, DistilBERT, RoBERTa), with cross-corpus tests between older and AI-generated email. Explanations from SHAP, LIME and Integrated Gradients were scored for faithfulness, stability and plausibility against a random-attribution baseline. The protocol was registered before the experiments ran, the cue lexicon was committed before any results, a positive-control test suite guards the explainers, and the plausibility measure was checked against human raters and an LLM judge. Everything ran on a local RTX 3090.

Decisions that mattered

  • Registered before running

    Protocols, the lexicon and the hypotheses were committed before any results existed, so nothing could be picked after the fact.

  • Explanations were measured

    A method that scores like random attribution is reported as such. When LIME scored like random, the cause was a bug in my LIME adapter, an inverted tokenizer argument. I disclosed it, fixed it and ran it again. Corrected LIME was faithful but the least stable method.

  • The simple model won

    On AI-generated email a tuned linear TF-IDF model beat RoBERTa on ROC-AUC (0.9993 against 0.9976), and SHAP explained it faithfully, 64.5 times faster than corrected LIME.

Code from the repo

src/xai/metrics.pyFaithfulness, measured against a random baseline
def faithfulness(champ, texts, attrs, preds, seed: int = 42) -> dict:
    rng = np.random.default_rng(seed)
    preds = np.asarray(preds)
    p_orig = _p_pred(champ.proba(texts), preds)1
 
    variants: list[str] = []
    meta: list[tuple] = []   # (doc_i, kind, k, is_random)
    for i, (text, attr, pred) in enumerate(zip(texts, attrs, preds)):
        nw = len(textops.word_spans(text))
        top = textops.top_k_positions(attr, int(pred), min(20, nw))
        randperm = rng.permutation(nw).tolist()2
        for k in K_AOPC:
            variants.append(textops.remove_words(text, top[:k])); meta.append((i, "comp", k, False))3
            variants.append(textops.remove_words(text, randperm[:k])); meta.append((i, "comp", k, True))
        for k in K_REPORT:
            variants.append(textops.keep_only_words(text, top[:k])); meta.append((i, "suff", k, False))
            variants.append(textops.keep_only_words(text, randperm[:k])); meta.append((i, "suff", k, True))
 
    P = _p_pred(champ.proba(variants), preds[[m[0] for m in meta]])4

  1. Line 4. The probability of the predicted class on the original document, the reference for every comparison.

  2. Line 11. The random baseline: the same removals in a random word order, through the identical pipeline.

  3. Line 13. Comprehensiveness: remove the top-k words the explanation points to. The more the probability drops, the more faithful the explanation.

  4. Line 19. Every variant of every document goes through the GPU in one batch.

Where it is now

Submitted in August 2026; the result is pending. Every table and figure can be reproduced from committed scripts and fixed seeds. I'm now turning it into a conference paper with my supervisor.

Next write-upFitEscala