Config Drift — Event-Driven Ansible (Stage 4)

Prerequisites

What the rulebook does

Kafka topic linux-audit-events
  → filter: event.body.message contains sshd_config_change and type=SYSCALL
  → throttle: once per host IP per 20 seconds
  → run_job_template: LINUX | SSHD Configuration Remediation
      config_drift_target_ip from event.body.host.ip[0]

The throttle is intentionally longer than a remediation job so remediation writes do not immediately re-trigger EDA on the same host.

Rulebook file: extensions/eda/rulebooks/config_drift_kafka.yml

Step 1 — Sync the EDA project

The setup job creates the Ansible Product Demos EDA project automatically (mirroring the Controller project’s SCM URL and branch) and syncs it. You do not need to create the EDA project manually.

If you prefer to verify first:

1. In AAP, open Automation Decisions → Projects.
2. Confirm Ansible Product Demos exists and contains extensions/eda/rulebooks/config_drift_kafka.yml.

Step 2 — Activate the rulebook

1. Run **APD

   Single demo setup** with category infrastructure (creates job templates).

2. Run **Infrastructure

   Config Drift - Full Setup** (recommended), or **Infrastructure

   Setup Rulebook for Kafka Queue - Config Drift & Remediation** alone.

The setup job reads aws_kafka from inventory and creates (or updates) EDA activation config_drift_kafka with broker host, topic, and rulebook path.

Step 3 — Verify activation is running

1. Automation Decisions → Rulebook activations
2. Open config_drift_kafka — status should be Running
3. Check activation logs for Kafka consumer connection to PUBLIC_IP:9095 (external listener)

Step 4 — End-to-end test

Or manually on a worker, drift sshd_config:

sudo sed -i 's/^#\?PermitRootLogin.*/PermitRootLogin yes/' /etc/ssh/sshd_config
grep PermitRootLogin /etc/ssh/sshd_config

1. Watch Automation Decisions activation logs — a matching event should appear within seconds.

2. Watch Automation Execution — **LINUX

   SSHD Configuration Remediation** should launch with config_drift_target_ip set to the worker private IP.

3. Confirm remediation on the worker:

   grep PermitRootLogin /etc/ssh/sshd_config
   

   Expected: PermitRootLogin no

Event shape EDA matches on

The rulebook reads Filebeat JSON from event.body. The message field contains the raw audit line; host.ip[0] is the worker private IP passed to remediation.

{
  "body": {
    "message": "type=SYSCALL ... key=\"sshd_config_change\" ... SYSCALL=rename ...",
    "host": { "ip": ["10.0.1.244"], "hostname": "ip-10-0-1-244..." }
  }
}

The rulebook condition requires both sshd_config_change and type=SYSCALL to ignore unrelated audit noise and reduce duplicate firings from companion records (PATH, PROCTITLE).

Troubleshooting

Recover after port 9095 was blocked or backlog crashed activation

1. Confirm TCP 9095 is open on aws-test-sg (nc -zv aws_kafka_PUBLIC_IP 9095 from your laptop).
2. On the Kafka host, skip queued audit noise before restarting EDA:

sudo podman exec kafka /opt/kafka/bin/kafka-consumer-groups.sh \
  --bootstrap-server 127.0.0.1:9094 \
  --group apd-config-drift \
  --topic linux-audit-events \
  --reset-offsets --to-latest --execute

1. Automation Decisions → disable and re-enable config_drift_kafka (or re-run Setup Rulebook for Kafka Queue - Config Drift & Remediation).
2. Drift from PermitRootLogin no to yes — not yes when already yes (no audit event).

Next step

See other Infrastructure demos

Browse all Infrastructure demos →