evo-suricata-custom-exfil
Generates and validates Suricata 7.0 rules for detecting data exfiltration hidden in HTTP telemetry traffic, using sticky buffers, exact URI matching, header inspection, and PCRE body pattern matching
SKILL.mdtasks-evolved/suricata-custom-exfil/environment/skills/evo-suricata-custom-exfil/SKILL.md
GitHubevo-suricata-custom-exfil
Description
Generates Suricata 7.0 rules for detecting data exfiltration hidden in HTTP telemetry traffic. Handles all five detection conditions with proper sticky buffer syntax, URL-decode transforms, parameter boundary awareness, and PCRE anchoring to minimize false positives and false negatives.
Domain Knowledge
Suricata 7.0 Rule Structure
- Header:
alert http any any -> any any— usealert httpto leverage Application Layer Protocol Identification (ALPI); avoids port-dependent matching. - Use
any any -> any anywhen both source and destination may be internal (e.g., 10.x to 10.x). - Options in parentheses, semicolon-delimited.
- Must include
sid,rev, and ideallyflow:established,to_server.
Sticky Buffers (Suricata 7.0)
Sticky buffers are context switches: once declared, all subsequent content, pcre, bsize,
depth, offset, distance, within keywords apply to that buffer until a new sticky buffer
is declared. This is the modern 7.0 approach — legacy trailing modifiers like http_method are
deprecated.
http.method;— matches HTTP verb (POST, GET, etc.)http.uri;— normalized URI including query string and leading/http.uri.raw;— unnormalized URI as on the wirehttp.header;— all request headers concatenated, CRLF-separatedhttp.request_header;— individual header, name and value joined by|3a 20|(:)http.request_body;— HTTP request body (POST/PUT/PATCH payload)
Critical Rules & Edge Cases
Sticky buffer ordering: Buffer keyword BEFORE content match. Correct:
http.method; content:"POST";Wrong:content:"POST"; http.method;Exact URI matching:
http.uriincludes query string. Usestartswith; endswith;to prevent substring matches (e.g.,/telemetry/v2/report/extra).Header matching options:
http.header;withcontent:"X-TLM-Mode"; content:"exfil";— works but risks cross-header false positives.http.request_header;withcontent:"X-TLM-Mode|3a 20|exfil";— isolates a single header, more precise. The|3a 20|is the colon-space separator Suricata normalizes.
http.request_bodyis NOT URL-decoded by default. If the POST body usesapplication/x-www-form-urlencoded, characters like=become%3Dand+replaces spaces. Theurl_decodetransform must be applied after the sticky buffer declaration to decode before matching. However, if the body is raw/JSON/binary,url_decodeis harmless (it only transforms%HHsequences and+).PCRE relative flag
R: Whenpcrefollows acontentmatch within the same sticky buffer, theRflag makes the PCRE start matching from the byte after the content match. This is essential for validating the value after a parameter name likeblob=.Parameter boundary awareness: To ensure
blob=is a top-level parameter (not a substring likexblob=), use a separatepcrewithoutRto check(?:^|&)blob=.Hex value exact length: For
sig=with exactly 64 hex chars, usepcre:"/^[0-9a-fA-F]{64}(?:[^0-9a-fA-F]|$)/R"— the lookahead ensures no 65th hex char.Base64 charset:
[A-Za-z0-9+/=]. In PCRE inside Suricata rules, forward slash must be escaped:[A-Za-z0-9+\\/=].-k nonefor PCAP testing: Disables checksum validation. PCAPs from tcpdump often have invalid checksums due to TCP Offload Engines; without this flag, packets are dropped.flow:established,to_server: Ensures TCP handshake completed and traffic flows client→server. Mandatory for production rules to prevent spoofed-IP false positives.
Multiple Approaches for Robustness
The skill generates multiple rule variants and tests them against training PCAPs to find the best-performing rule. This handles edge cases where:
- The body may or may not be URL-encoded
- Header casing may vary
- Parameters may appear in any order in the body
Usage
import sys
sys.path.insert(0, '/app/environment/skills/evo-suricata-custom-exfil/scripts')
from generate_rule import write_rules_file, test_rule_with_suricata, auto_select_best_rule
# Auto-select best rule variant by testing against training PCAPs
best_rule = auto_select_best_rule()
# Or manually generate and write rule
rule = write_rules_file('/root/local.rules')
# Test against pcaps
pos_count, _ = test_rule_with_suricata('/root/pcaps/train_pos.pcap')
neg_count, _ = test_rule_with_suricata('/root/pcaps/train_neg.pcap', log_dir='/tmp/suri-neg')
Files
scripts/generate_rule.py- Rule generation, variant testing, and validation utilities