Network protection with
Fortress & Hawk

Two small Perl daemons that keep production servers standing under real-world attack traffic.

Marian (HackMan) Marinov

mm@yuhu.biz

Agenda

1. The problem — what a sysadmin does at 3am
2. Fortress — a TCP connection watchdog
   • Attacks it catches
   • Connection states it looks at
   • Blocking backends
3. Hawk — a bruteforce log analyzer
   • Design: tail -F instead of Perl I/O (and why)
   • Supported services
   • Blocking + web interface
4. When to use which

The 3am problem

Load spikes. Site is unresponsive. What does a sysadmin do?

ss -ntp | awk '{print $5}' | cut -d: -f1 | sort | uniq -c | sort -rn | head

• Slow on a busy box
• Manual: type, read, decide, iptables -I ..., repeat
• By the time you're done, the attacker moved on — or you're still typing

Both tools automate exactly this reflex. Different data sources, same goal: identify a bad IP and block it in less than a second.

Part 1 — Fortress

Automatic TCP connection monitoring and blocking

Fortress: what it is

• A single-file Perl daemon (~250 lines)
• Reads /proc/net/tcp every second
• Reads /proc/loadavg every 5 seconds
• Decides — based on state, count, and current load — whether an IP is abusing the server
• Calls an external shell script to do the actual blocking

Fortress: attacks it detects

Fortress: what it does not try to do

• No packet capture (no libpcap, no BPF)
• No deep packet inspection
• No UDP yet — on the TODO list
• No log parsing — that's Hawk's job

It solves one problem:
too many concurrent TCP sockets from one source.

That's it. That's the whole product.

Fortress: the connections it looks at

Linux exposes every TCP socket in /proc/net/tcp:

 sl  local_address  rem_address    st  tx_queue rx_queue tr ...
  0: 00000000:0016  00000000:0000  0A  00000000:00000000 ...

Column st is the TCP state as a hex byte. Fortress only cares about two of them:

TIME_WAIT, CLOSE_WAIT, FIN_WAIT_*, LISTEN — skipped. Those don't consume the resources we're protecting.

Linux kernel network documentation

Fortress: SYN_RECV vs ESTABLISHED

Fortress: adaptive thresholds

high_load = 5
low_conns  = 50      # ESTABLISHED cap when load < 5
high_conns = 20      # ESTABLISHED cap when load >= 5

low_syn_recv_conns  = 50
high_syn_recv_conns = 25

Fortress: what is a Patricia Trie?

PATRICIA = Practical Algorithm To Retrieve Information Coded In Alphanumeric — Morrison, 1968.
Same data structure BSD has used for the kernel routing table since forever.

Prefixes stored:
  10.0.0.0/8
  10.1.0.0/16
  192.168.0.0/16

           (root)
          /      \
       0 /        \ 1
        /          \
    10.0.0.0/8   192.168.0.0/16
        |
     [bits 8..15]      <-- compressed edge
        |
    10.1.0.0/16

Lookup 10.1.2.3:
  root -> 0 branch (bit 0 = 0)
       -> match 10.0.0.0/8          (remember)
       -> follow compressed edge
       -> match 10.1.0.0/16         (remember, deeper)
       -> return 10.1.0.0/16

Fortress: whitelisting with the Patricia Trie

Fortress must never block Googlebot, Cloudflare, or the office IP.

• Excludes come from plain text files, one IP or CIDR per line
• Loaded into a Net::Patricia trie at startup
• ~5000 prefixes today (Google alone: ~3900)
• Lookup is O(prefix length) — effectively constant per check

my $excludes = Net::Patricia->new;
$excludes->add_string('66.249.64.0/19', 'google');
...
next if ($excludes->match_string($ip));

Local IPs auto-detected from ip -4 a l at startup — you can never accidentally block yourself.

Fortress: in one picture

Fortress: what's next (TODO)

• Signal handler for stop — persist the current blocked-IP list before exiting
• Configurable per-range limits (/24, /23, /22) to catch more distributed attacks that fly under the per-IP threshold
• Parse and check /proc/net/udp — UDP support
• Debian and Ubuntu packages (currently only RPM/CentOS ship)

Patches welcome — github.com/hackman/Fortress

Part 2 — Hawk

Log-driven bruteforce detection and IP blocking

Hawk: what it is

• A Perl daemon that reads authentication logs
• Recognises failed-login patterns for many services
• Counts failures per IP over a time window
• Blocks offenders via iptables / ipset / custom script
• Has a web UI (hawk-web.pl) for attack history
• SQLite / MySQL / PostgreSQL backend

Supported out of the box:
sshd, dovecot (POP3/IMAP), courier, pure-ftpd, proftpd, postfix, exim, cPanel, DirectAdmin, WHM

Hawk: the design choice we're here to talk about

Hawk does not read log files from Perl.

It opens a pipe to GNU tail:

my $log_list = "/usr/bin/sudo /usr/bin/tail -s 1.00 -F "
             . "--max-unchanged-stats=30 $monitor_list |";

$tail_pid = open LOGS, $log_list or die "open $log_list failed: $!\n";

while (<LOGS>) {
    # parse the line
}

That's the entire log ingestion layer. One open, one while. Rotation, truncation, multi-file follow — all handled by tail -F.

Hawk: why tail -F and not AnyEvent?

Hawk was written when AnyEvent was brand new.

At the time, benchmarking on real production mail servers showed:

• AnyEvent::Handle watching 6 log files → measurable CPU overhead per file, dominated by event loop bookkeeping
• tail -F file1 file2 ... | perl → the kernel does the epoll, tail does the buffering, Perl just reads a pipe

The C-implemented tail was cheaper than the Perl event loop it was supposed to replace. On a shared hosting box running dozens of services, that difference mattered.

Simplicity as a feature — one fewer moving part, one fewer CPAN dep, behaviour identical to what any sysadmin already understands.

Hawk: the trade-off, honestly

Hawk: parsing — one regex per service

Each service has a tiny parser returning (ip, attempts, service_id, user):

sub postfix_broot {
    # Dec 30 09:04:16 host postfix/smtpd[14147]: warning:
    #   unknown[46.148.40.150]: SASL LOGIN authentication failed: ...
    if ($_ =~ /\[([0-9]+\.[0-9]+\.[0-9]+\.[0-9]+)\]: /) {
        return ($1, 1, 7, 'unknown');
    }
}

The main loop tries each enabled service in turn. First parser that matches wins. Adding a new service = one sub + one if in the dispatcher.

Hawk: counting and blocking

broot_time     = 180    # sliding window (seconds)
broot_number   = 10     # failures in window -> "bruteforce"
max_attempts   = 2      # bruteforce records -> actual block
block_expire   = 86400  # unblock after 24h

Two counters:

1. Per-IP failure count inside broot_time — resets periodically.
2. Bruteforce record count — how many times the IP has crossed the failure threshold. When it hits max_attempts, the IP is added to iptables/ipset.

Everything is also written to SQLite/MySQL/PgSQL for the web UI to query later.

Hawk: the web interface

hawk-web.pl — a small CGI/FastCGI script that reads the same DB and shows:

• Recent failed logins per service
• Top attacking IPs
• Currently blocked list
• Per-user attack breakdown (which accounts are being probed)

Not fancy. Useful. Sysadmins tend to like it more than command-line iptables -L.

Hawk: in one picture

Fortress or Hawk?

Run both. They cover different layers and don't conflict — Fortress kills the flood, Hawk kills the patient guesser.

Takeaways

• Perl is great for this kind of glue: fast enough, expressive regexes, tiny memory footprint, easy to modify at 3am.
• Trust the kernel: /proc/net/tcp and tail -F are cheaper than any library you can layer on top.
• Adaptive thresholds (load-aware) beat static ones.
• Whitelisting is not optional — a Patricia trie makes it free.
• The simple design outlives the fancy one.

Links

Fortress:
https://github.com/hackman/Fortress

Hawk:
https://github.com/hackman/Hawk-IDS-IPS

Both GPLv2. Patches welcome.

Questions?

Marian Marinov