telemetry-lab

Small portfolio prototypes for telemetry analytics, monitoring, and detection-oriented signal processing.

Latest milestone: v0.6.0 — fourth demo and config-change investigation.

Demos

• telemetry-window-demo
• ai-assisted-detection-demo
• rule-evaluation-and-dedup-demo
• config-change-investigation-demo

Demo	Input	Deterministic core	LLM role	Main artifacts	Guardrails / non-goals
telemetry-window-demo	JSONL / CSV events	Windows Features Alert thresholds	None	features.csv alerts.csv summary.json 3 PNG plots	MVP only No realtime No case management
ai-assisted-detection-demo	JSONL auth / web / process	Normalize Rules Grouping ATT&CK mapping	JSON-only case drafting	rule_hits.json case_bundles.json case_summaries.json case_report.md audit_traces.jsonl	Human verification required No autonomous response No final verdict
rule-evaluation-and-dedup-demo	JSON raw rule hits	Scope resolution Cooldown grouping Suppression reasoning	None	rule_hits_before_dedup.json rule_hits_after_dedup.json dedup_explanations.json dedup_report.md	No realtime No dashboard No AI stage
config-change-investigation-demo	JSONL config changes Policy denials Follow-on events	Normalize Risky-change rules Bounded correlation	None	change_events_normalized.json investigation_hits.json investigation_summary.json investigation_report.md	No realtime No dashboard No AI stage

What This Repo Is

telemetry-lab is a small portfolio repository for telemetry analytics and constrained detection-oriented workflows. It is organized as four local, file-based demos that are reproducible from committed sample data and intentionally scoped for public review rather than production use.

telemetry-window-demo

telemetry-window-demo turns timestamped event streams into sliding-window feature tables, cooldown-reduced rule-based alerts, PNG timeline plots, and machine-readable run summaries.

ai-assisted-detection-demo

ai-assisted-detection-demo uses deterministic normalization, detection, case grouping, and ATT&CK mapping, then limits the LLM to JSON-only case summarization. Human verification is required, there is no autonomous response, and the demo does not produce a final incident verdict.

rule-evaluation-and-dedup-demo

rule-evaluation-and-dedup-demo starts from raw rule hits and makes cooldown behavior legible. It shows which hits were kept, which were suppressed, how scope was resolved, and why repeated hits collapsed into fewer retained alerts.

config-change-investigation-demo

config-change-investigation-demo follows risky configuration changes into bounded follow-on evidence such as policy denials and service signals. It stays deterministic, file-based, and review-oriented, with no added AI stage.

Quick Run

python -m pip install -e .
python -m telemetry_window_demo.cli run --config configs/default.yaml

Other demo entrypoints:

• python -m telemetry_window_demo.cli run-ai-demo
• python -m telemetry_window_demo.cli run-rule-dedup-demo
• python -m telemetry_window_demo.cli run-config-change-demo

That command reads data/raw/sample_events.jsonl and regenerates:

• data/processed/features.csv
• data/processed/alerts.csv
• data/processed/summary.json
• data/processed/event_count_timeline.png
• data/processed/error_rate_timeline.png
• data/processed/alerts_timeline.png

With the bundled default sample, the current repo state produces:

• 41 normalized events
• 24 windows
• 12 alerts after a 60 second cooldown

Why it is worth a quick look:

• it shows a full telemetry path from raw events to operator-facing outputs
• the sample inputs and outputs are reproducible in-repo
• a second bundled scenario gives a slightly richer walkthrough without changing the basic CLI flow

Demo Variants

Default sample:

• config: configs/default.yaml
• input: data/raw/sample_events.jsonl
• outputs: data/processed/
• current summary: 41 events, 24 windows, 12 alerts, summary.json included

Richer sample:

• config: configs/richer_sample.yaml
• input: data/raw/richer_sample_events.jsonl
• outputs: data/processed/richer_sample/
• current summary: 28 events, 24 windows, 8 alerts, summary.json included

Input Support

Runtime input support:

• .jsonl
• .csv

Required fields for both formats on every row or record:

• timestamp
• event_type
• source
• target
• status

Cooldown behavior:

• repeated alerts are keyed by (rule_name, scope)
• scope prefers the first available entity-like field in this order: entity, source, target, host
• when no entity-like field is present, cooldown falls back to per-rule_name behavior

Repo Guide

• demos/rule-evaluation-and-dedup-demo/README.md explains the third demo and links its committed before/after dedup artifacts
• demos/config-change-investigation-demo/README.md explains the config-change investigation demo and its committed artifacts
• docs/sample-output.md summarizes the committed sample artifacts
• docs/roadmap.md sketches the next demo directions
• data/processed/summary.json captures the default run in machine-readable form
• data/processed/richer_sample/summary.json captures the richer scenario pack
• tests/ keeps regression coverage close to the CLI behavior and windowing logic

Next Demo Directions

• strengthen JSONL and CSV validation so ingestion failures are clearer
• keep reducing repeated alert noise while preserving simple rule-based behavior
• keep sample-output docs and public repo presentation aligned with the checked-in demo state

Scope

This repository is a portfolio prototype, not a production monitoring system.

Limitations

• No real-time ingestion
• No streaming state management
• No alert routing or case management
• No dashboard or service deployment
• Sample-data driven only