Lab 3: Build a phishing email classifier

AI for Cyber Defenders · lesson 3 of 8 · 120 min

“URGENT: your account will be suspended.” You have seen messages like this. Today you teach a computer to spot them by learning from examples, not from rules you write by hand.

From rules to learning

Writing a rule for every phishing trick is impossible; attackers change wording daily. Machine learning flips the job around: you give the computer labeled examples (this one is phishing, this one is fine) and it learns which patterns go with each label. This is called supervised learning, and the task is classification: putting each email into a class.

Turning words into numbers

Models only understand numbers. TF-IDF (term frequency–inverse document frequency) turns each email into a list of numbers, one per word:

Term frequency: words used more in this email count more.
Inverse document frequency: words that appear in almost every email (“the”, “please”) count less, because they do not help tell emails apart.

We also use ngram_range=(1, 2), which adds two-word phrases like “verify your” as features.

Logistic regression and fair testing

Logistic regression learns one weight per word. Positive weights push toward “phish”, negative toward “legit”. It adds them up and outputs a probability from 0 to 1. It is simple, fast, and you can read its weights, which matters in security: you need to explain why an email was flagged.

To test honestly, we hide 25% of emails in a test set that the model never sees during training. The vectorizer also learns its vocabulary from training emails only. If test emails leak into training, scores look great and mean nothing.

The lab code

The code generates about 200 emails from templates: staff meetings, church-van schedules, lunch orders, and many phishing styles (fake password resets, payroll scams, gift-card requests). Links point to reserved .test and .example domains. Open a new notebook at colab.research.google.com, paste the code into one cell, and press Shift+Enter. Running locally instead? Save it as a .py file and run python lab3.py.

# Lab 3: Build a phishing email classifier (TF-IDF + Logistic Regression)
import numpy as np
import pandas as pd
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split
from sklearn.metrics import classification_report, confusion_matrix

rng = np.random.default_rng(seed=3)

names = ["Aisha", "Darnell", "Keisha", "Marcus", "Jada", "Terrence", "Imani", "Andre"]
legit_templates = [
    "Hi {name}, the staff meeting moved to {day} at {time} in the main branch conference room.",
    "Reminder: the youth financial literacy workshop is {day}. Please bring sign-in sheets.",
    "{name}, can you review the Q2 branch schedule before {day}? Thanks.",
    "The community fair booth needs two volunteers on {day}. Reply if you can help.",
    "Lunch order for {day}: tacos from the corner spot. Let me know your choice by {time}.",
    "Updated holiday hours are posted on the intranet. No action needed.",
    "{name}, great job on the member survey results. Let's discuss {day}.",
    "IT maintenance window {day} at {time}. Printers may be offline for 20 minutes.",
    "New hire orientation for the teller team is {day}. Agenda attached.",
    "Please submit your timesheet by {day} at {time}. Thanks for your hard work.",
    "Volunteer sign-up form for the back-to-school drive: https://forms.example.org/drive",
    "{name}, please verify your new desk phone extension with HR by {day}.",
    "Reset reminder: the shared drive password policy changes {day}. Details on the intranet.",
]
phish_templates = [
    "URGENT: your account will be suspended. Verify your password now at {url}",
    "{name}, unusual sign-in detected. Confirm your identity immediately: {url}",
    "Your mailbox is full. Click here to keep receiving email: {url}",
    "Payroll update required today. Login to avoid missing your direct deposit: {url}",
    "You have a pending wire transfer. Act now and verify account details at {url}",
    "Final notice: invoice overdue. Download the secure document here {url}",
    "Congratulations {name}! You won a gift card. Claim within 24 hours: {url}",
    "Security alert: reset your password immediately or lose access {url}",
    "CEO request: buy gift cards urgently and send the codes. Keep this confidential.",
    "Your package could not be delivered. Update payment info at {url}",
    "Hi {name}, are you at your desk? I need a quick favor before {time}. Reply ASAP.",
    "{name}, please review the attached document and sign by {day}.",
]
days = ["Monday", "Tuesday", "Wednesday", "Thursday", "Friday"]
times = ["9am", "10:30am", "noon", "2pm", "4pm"]
urls = ["http://verify-account.example.test/login", "http://203.0.113.9/secure",
        "http://mail-support.example.net.test/reset", "http://bit-ly.example/x7k2"]

def make(template):
    return template.format(name=rng.choice(names), day=rng.choice(days),
                           time=rng.choice(times), url=rng.choice(urls))

emails = [(make(rng.choice(legit_templates)), 0) for _ in range(110)] + \
         [(make(rng.choice(phish_templates)), 1) for _ in range(90)]
df = pd.DataFrame(emails, columns=["text", "is_phish"])
# Add a little "noise": a few legit emails that sound urgent
df.loc[len(df)] = ["URGENT: the vault audit moved to Friday, please confirm you can attend.", 0]
df.loc[len(df)] = ["Please verify the attendance list for Saturday's event.", 0]
print("Emails:", len(df), "| phishing share:", round(df["is_phish"].mean(), 2))

X_train, X_test, y_train, y_test = train_test_split(
    df["text"], df["is_phish"], test_size=0.25, random_state=42, stratify=df["is_phish"])

vectorizer = TfidfVectorizer(lowercase=True, ngram_range=(1, 2), min_df=1)
X_train_vec = vectorizer.fit_transform(X_train)   # learn vocabulary on TRAIN only
X_test_vec = vectorizer.transform(X_test)
print("Vocabulary size:", len(vectorizer.vocabulary_))

model = LogisticRegression(max_iter=1000)
model.fit(X_train_vec, y_train)
pred = model.predict(X_test_vec)

print("\n== Test results ==")
print(confusion_matrix(y_test, pred))
print(classification_report(y_test, pred, target_names=["legit", "phish"]))

# Which words push toward "phish"?
words = np.array(vectorizer.get_feature_names_out())
coefs = model.coef_[0]
print("Top phishing words:", ", ".join(words[np.argsort(coefs)[-10:]][::-1]))
print("Top legit words:   ", ", ".join(words[np.argsort(coefs)[:10]]))

# Try brand-new emails the model has never seen
new_emails = [
    "Hey Jada, choir rehearsal is Thursday at 6pm, the church van leaves at 5:30.",
    "Your account is locked. Verify your password at http://login-check.example.test now",
    "Reminder: please confirm the budget numbers before Friday's meeting.",
    "Immediate action required: confirm payroll details to receive your paycheck",
]
for text, p in zip(new_emails, model.predict_proba(vectorizer.transform(new_emails))[:, 1]):
    print(f"{p:5.2f}  {text}")

What you should see

Test accuracy near 98%: one or two mistakes out of 51. Top phishing words include “http”, “account” and link fragments. Top legit words are everyday ones like “thanks” and day names. On the new emails, the choir message scores low (about 0.3), the locked-account message scores high (about 0.8), and the payroll message lands near 0.5: the model is unsure because it has no link.

Warning sign: near-perfect scores usually mean the data is too easy. Our emails come from a few templates. Real phishing is messier, so real models score lower. Always be suspicious of a perfect result.

Try it

Run the code and write down your accuracy and the top five phishing words.
Write five new emails of your own, three legit and two phishing, and score them with the model. Which one fooled it?
Add three new phishing templates with no links and no urgent words (for example, a fake “invoice attached” note). Retrain. Did recall for phish drop?
Change test_size to 0.5. What happens to the scores, and why?
Explain to a friend in three sentences why the payroll email scored about 0.5.

Check yourself

Why must the vectorizer learn its vocabulary from the training set only?

Otherwise information from the test set leaks into training, and the test score no longer shows how the model handles truly new emails.

What does TF-IDF do to a word like “the”?

It gives it a low weight, because it appears in nearly every email and does not help separate phishing from legit.

The model scores 100% on synthetic data. Should the SOC deploy it?

No. Synthetic templates are easier than real mail. Test it on realistic, varied emails before trusting it.

Key words

Supervised learning — learning from examples that already have correct labels
Classification — predicting which category something belongs to
TF-IDF — a way to turn text into numbers that favors distinctive words
Logistic regression — a simple model that outputs a probability from weighted features
Train/test split — holding back data the model never sees, to test it fairly

Previous lesson: Lab 2: Rule-based detection — catch brute-force logins
Back to AI for Cyber Defenders
Next lesson: Lab 4: Measuring detectors — precision, recall and alert fatigue