Menu
Book an info sessionInfo session

Lab 2: Rule-based detection — catch brute-force logins

AI for Cyber Defenders · lesson 2 of 8 · 120 min

Before anyone trains a model, a good defender writes a rule. If 60 wrong passwords hit one account in six minutes, you do not need AI to know something is off. Rules are the baseline every AI detector has to beat.

Two classic login attacks

  • Brute force — one source tries many passwords against one account, fast.
  • Password spraying — one source tries one common password against many accounts, slowly, to stay under per-account lockouts.

Both leave the same fingerprint in logs: lots of failed logins from one IP address. The difference is the shape: many tries on one user versus a few tries across many users.

Thresholds and time windows

A detection rule is an if-then statement over log data. Two dials control it:

  • Threshold — how many suspicious events before we alert (for example, 10 failures).
  • Time window — how close together they must be (for example, within 5 minutes).

A rolling window slides along the timeline. At each event, it looks back 5 minutes and counts failures from the same IP. pandas does this with .rolling('5min') after grouping by IP.

Set the threshold too low and normal people trip it: an employee who forgets her password after vacation fails four times. That alert is a false positive. Set it too high and a careful attacker slips under it: a false negative. Tuning rules is a constant trade-off.

The lab code

The code builds a day of logins with three “stories” mixed in: a forgetful employee, a brute-force attack from 198.51.100.77, and a spray from 203.0.113.45. We kept a truth column so you can grade the rules. Real logs never come with answers. Open a new notebook at colab.research.google.com, paste the code into one cell, and press Shift+Enter. Running locally instead? Save it as a .py file and run python lab2.py.

# Lab 2: Rule-based detection - brute-force and password spraying
import numpy as np
import pandas as pd

rng = np.random.default_rng(seed=21)
users = [f"user{i:02d}" for i in range(1, 41)]
office_ips = [f"192.0.2.{i}" for i in range(10, 60)]
start = pd.Timestamp("2026-03-09 08:00")

rows = []
# 1) Normal traffic: 1,500 logins over 10 hours, about 5% typos
for _ in range(1500):
    ts = start + pd.Timedelta(seconds=int(rng.integers(0, 10 * 3600)))
    rows.append({"timestamp": ts, "user": rng.choice(users),
                 "src_ip": rng.choice(office_ips),
                 "success": rng.random() > 0.05, "truth": "normal"})

# 2) Forgetful employee: 4 fails then success - NOT an attack
t0 = start + pd.Timedelta(hours=2)
for i in range(4):
    rows.append({"timestamp": t0 + pd.Timedelta(seconds=20 * i), "user": "user07",
                 "src_ip": "192.0.2.33", "success": False, "truth": "forgetful"})
rows.append({"timestamp": t0 + pd.Timedelta(seconds=100), "user": "user07",
             "src_ip": "192.0.2.33", "success": True, "truth": "forgetful"})

# 3) Brute force: one outside IP hammers one account, 60 tries in ~6 minutes
t1 = start + pd.Timedelta(hours=5)
for i in range(60):
    rows.append({"timestamp": t1 + pd.Timedelta(seconds=6 * i), "user": "user12",
                 "src_ip": "198.51.100.77", "success": False, "truth": "brute_force"})

# 4) Password spray: one IP tries 1 password on 25 different accounts, slowly
t2 = start + pd.Timedelta(hours=7)
for i, u in enumerate(rng.choice(users, size=25, replace=False)):
    rows.append({"timestamp": t2 + pd.Timedelta(seconds=15 * i), "user": u,
                 "src_ip": "203.0.113.45", "success": False, "truth": "spray"})

log = pd.DataFrame(rows).sort_values("timestamp").reset_index(drop=True)
log["failed"] = (~log["success"]).astype(int)
print("Total events:", len(log))
print(log["truth"].value_counts(), "\n")

# ---- Rule A: brute force = >= 10 failures from one IP within 5 minutes ----
FAIL_THRESHOLD = 10
WINDOW = "5min"
per_ip = log.set_index("timestamp").groupby("src_ip")["failed"].rolling(WINDOW).sum()
per_ip = per_ip.reset_index().rename(columns={"failed": "fails_in_window"})
hits_a = per_ip[per_ip["fails_in_window"] >= FAIL_THRESHOLD]
print(f"Rule A (>= {FAIL_THRESHOLD} fails / {WINDOW} per IP) fired for:")
print(hits_a.groupby("src_ip")["fails_in_window"].max(), "\n")

# ---- Rule B: spray = one IP failing on >= 8 different users within 10 minutes ----
USER_THRESHOLD = 8
fails = log[log["failed"] == 1]
spray_hits = []
for ip, grp in fails.groupby("src_ip"):
    grp = grp.sort_values("timestamp")
    for ts in grp["timestamp"]:
        window = grp[(grp["timestamp"] > ts - pd.Timedelta("10min")) & (grp["timestamp"] <= ts)]
        n_users = window["user"].nunique()
        if n_users >= USER_THRESHOLD:
            spray_hits.append((ip, n_users))
spray = pd.DataFrame(spray_hits, columns=["src_ip", "distinct_users"])
print(f"Rule B (>= {USER_THRESHOLD} distinct users / 10min per IP) fired for:")
print(spray.groupby("src_ip")["distinct_users"].max() if len(spray) else "nothing", "\n")

# ---- How did the rules do? ----
flagged_ips = set(hits_a["src_ip"]) | set(spray["src_ip"])
attack_ips = set(log.loc[log["truth"].isin(["brute_force", "spray"]), "src_ip"])
print("Flagged IPs:     ", sorted(flagged_ips))
print("Real attack IPs: ", sorted(attack_ips))
print("Missed attacks:  ", sorted(attack_ips - flagged_ips))
print("False alarms:    ", sorted(flagged_ips - attack_ips))

# Experiment: what if the threshold is too low?
for th in [3, 5, 10, 20]:
    ips = set(per_ip.loc[per_ip["fails_in_window"] >= th, "src_ip"])
    print(f"threshold={th:>2}: {len(ips)} IPs flagged, false alarms={len(ips - attack_ips)}")

What you should see

Rule A fires on both attack IPs (about 50 failures in a window for the brute force, 20 for the spray). Rule B fires only on the spray IP, with 25 distinct users. Missed attacks and false alarms are both empty. The threshold experiment at the end is the key lesson: at a threshold of 3, a third IP appears. That is the forgetful employee at 192.0.2.33, a false alarm. At 5 or more, it disappears.

Rules are fast, cheap and easy to explain. When an AI model is proposed, ask: does it catch something these rules miss, without burying us in false alarms?

Try it

  1. Run the code and confirm both attacks are caught with zero false alarms.
  2. Change WINDOW to "1min". Does Rule A still catch the spray? Why or why not?
  3. Make the spray slower: change 15 * i to 60 * i seconds. Which rule still catches it?
  4. Add a Rule C that alerts when an IP fails several times and then succeeds. Who does it flag? Is that good or bad?
  5. Write a one-paragraph rule description a teammate could follow: what it detects, the threshold, the window, and one known false positive.

Check yourself

What is the difference between brute force and password spraying?

Brute force tries many passwords on one account quickly. Spraying tries a few passwords across many accounts, often slowly, to avoid lockouts.

Why did the threshold of 3 create a false alarm?

The forgetful employee failed four times in a few minutes. A low threshold cannot tell an honest mistake from an attack.

Why build rules before machine learning?

Rules are a simple, explainable baseline. If a model cannot beat them, it is not worth the extra complexity.

Key words

  • Detection rule — an if-then check over log data that raises an alert
  • Threshold — the count that must be reached before a rule alerts
  • Rolling window — a time span that slides along the log, counting recent events
  • False positive — an alert on something harmless
  • False negative — a real attack that produced no alert

Previous lesson: Lab 1: Python and Colab for security data
Back to AI for Cyber Defenders
Next lesson: Lab 3: Build a phishing email classifier