Your monitoring stack can tell you the pod is dying. 01Agents brings it back.

An open-source Kubernetes incident resolution engine. It unifies real-time cluster triage (L1), secondary telemetry reconstruction (L2), and deep kernel eBPF & network packet intelligence (L3) via a 100% pluggable, unlimited-extensibility plugin architecture—diagnosing root causes from container state down to raw network packets and kernel scheduler throttling.

01. 404 Resilient
Postmortem Log Retrieval
Queries SigNoz, Loki & telemetry plugins when deleted pods return 404 from K8s API.
02. Zero Code Changes
Non-Invasive Deployment
No SDK or code instrumenting required. Plugs into current OTel & logs.
03. HITL Guardrails
Operator Approval Gate
High-risk cluster remedies pause for human approval before execution.
04. GitOps Verified
ArgoCD & FluxCD Sync
Verifies declarative manifest state and rollback snapshots automatically.
05. eBPF Kernel Truth
Wire-Level Packet Telemetry
Traces TCP drops, DNS stalls & CFS throttling with sub-millisecond kernel accuracy.
PLUGGABLE TELEMETRY & GITOPS ECOSYSTEM

Pluggable Observability & GitOps Stack

Zero application code modifications required. 01Agents plug dynamically into your cluster's existing telemetry collectors, log streams, metric stores, GitOps engines, and kernel eBPF probes via the Plugin Marketplace.

11+ Pre-Built Reference Plugins · Unlimited Custom Backends
SigNoz(L2)
Grafana Loki(WebSocket)
Grafana Alloy(Telemetry)
Prometheus(Real-Time)
ELK Stack(Elasticsearch)
Datadog(Enterprise)
ArgoCD(Declarative)
FluxCD(Kubernetes)
SigNoz(L2)
Grafana Loki(WebSocket)
Grafana Alloy(Telemetry)
Prometheus(Real-Time)
ELK Stack(Elasticsearch)
Datadog(Enterprise)
ArgoCD(Declarative)
FluxCD(Kubernetes)
Plugin

SigNoz

L2 Telemetry & ClickHouse

Native integration with SigNoz v5 for historical ClickHouse log queries & distributed APM tracing.

ClickHouseAPM TracingL2 Reconstruction
Plugin

Grafana Loki

WebSocket Live Logs & Pod Storage

Direct Loki Tail WS proxy & PVC volume mounting to retrieve logs even from terminated & deleted pods.

Live WS ProxyPostmortem LogsPVC Storage
Plugin

Grafana Alloy

Telemetry Collector Pipeline

Next-gen OpenTelemetry pipeline & agent for streaming node-level metrics and container logs.

OTel CollectorMetric StreamingUnified Agent
Plugin

Prometheus

Real-Time K8s Metrics

PromQL metric parsing for instant CPU, memory, OOMKilled, and crash-loop backoff detection.

PromQL QueriesAlertmanagerNode Exporters
Plugin

ELK Stack

Elasticsearch & Kibana

Seamless search & indexed log analysis via Elasticsearch REST APIs and Kibana dashboard audit sync.

ElasticsearchLogstashKibana Audit
Plugin

Datadog

Enterprise Cloud Telemetry

Bi-directional Datadog monitor webhook intake & automated incident remediation triggers.

Cloud APMMonitor WebhooksEnterprise Sync
Plugin

ArgoCD

Declarative GitOps Delivery

Automated GitOps state verification, commit rollback, and sync status checks upon incident resolution.

GitOps SyncAuto-RollbackK8s Declarative
Plugin

FluxCD

Kubernetes GitOps Operator

Native reconciliation with Flux Kustomization and HelmRelease objects for safe cluster state restoration.

HelmRelease SyncFlux KustomizeContinuous Delivery
ARCHITECTURE SPECIFICATION

Why Standard K8s Diagnostic Tools Fail on Pod Termination

When a pod enters CrashLoopBackOff or gets evicted due to OOMKilled, Kubernetes eventually removes the pod resource. Running kubectl logs or querying the API server at that point returns:

Error from server (NotFound): pods "auth-service-7f9b" not found

01Agents solves this with a 3-tier multi-agent triad—each tier escalating to the next when standard diagnostics hit their limits:

Level-1 (L1) In-Cluster Real-Time Triage AgentMonitors real-time Kubernetes events, node health, and container state transitions. Evaluates immediate low-risk remedies and handles active resource constraints. Runs unlimited custom SKILL.md runbooks—no hardcoded failure codes.
Level-2 (L2) Secondary Telemetry Reconstruction EngineWhen live APIs return 404, L2 initiates a lookback query across pluggable plugin backends (SigNoz ClickHouse, Grafana Loki WS proxy, Prometheus PromQL, ELK, Datadog, and any custom backend) to reconstruct full pre-failure log streams and memory curves.
Level-3 (L3) eBPF Kernel & Network Packet Intelligence Agent NEWWhen application logs and APM data are inconclusive, L3 deploys in-kernel eBPF probes to trace raw TCP packet drops, socket state machines, DNS latency spikes, CoreDNS NXDOMAIN cascades, and Linux CFS scheduler throttling—uncovering invisible incidents that neither K8s events nor telemetry backends can see.
incident-triage-trace.log
L1 → L2 → L3 Pipeline
17:04:12 [L1 Monitor] Event Received
Reason: OOMKilled | Pod: auth-api-9d2a | Namespace: production
17:04:13 [L1 Monitor] Pod Evicted (404 API)
K8s API: Pod deleted. Initiating L2 Secondary Telemetry lookup...
17:04:14 [L2 Telemetry Agent] Querying ClickHouse & Loki
SigNoz ClickHouse lookup window: [-15m]. Fetched 1,420 historical log entries.
17:04:15 [L2 RCA Agent] Logs Inconclusive — Escalating to L3
Log analysis: no explicit heap error. Escalating to eBPF kernel wire-level probes.
17:04:16 [L3 eBPF Engine] Kernel Tracepoint Triggered
net:net_dev_queue — 142 dropped TCP packets | SYN Retransmit: 3120ms | CoreDNS P99: 480ms
17:04:17 [L3 CFS Scheduler Agent] Throttle Detected
cgroup CFS quota exhausted: 128ms throttled | Burst period CPU peak: 98.4%
17:04:18 [L2 Remediation Agent] Root Cause: L3 Wire Evidence
CFS throttle caused latency cascade → DNS timeout → OOMKilled. Increase CPU limit to 2000m + patch coreDNS timeout.
17:04:19 [HITL Safety Gate] Approval Awaited
High-risk patch queued. Awaiting Slack callback from operator @sre-oncall.
17:04:24 [HITL Safety Gate] Approved & Executed
Operator APPROVED. Patch applied with GitOps rollback snapshot. RCA Confidence: 0.97
Active Trace LogeBPF + L1 + L2L3 eBPF: 142 Drops Detected
AGENT CAPABILITIES MATRIX

Domain-Specific Intelligence Across L1 Triage, L2 Secondary Telemetry & L3 eBPF Kernel Wire Analysis

Not limited to fixed failure codes or static agents. 01Agents features a dynamic SKILL.md runbook engine and a pluggable subagent registry—enabling teams to introduce unlimited custom skills, GitOps workflows, and kernel diagnostic probes without rebuilding Docker images or restarting pods.

Level-3 eBPF Kernel & Network Packet IntelligenceNEW
L3 eBPF Socket Agent

TCP Socket & Packet Flow Inspector

Traces TCP state transitions, connection resets (RST), round-trip time (RTT), and silent packet drops at the Linux network stack via eBPF kprobes and XDP hooks.

Kernel eBPF Probes · Wire TelemetryActive
L3 DNS & Protocol Agent

CoreDNS & Wire-Protocol Analyzer

Sniffs raw socket traffic on UDP/TCP port 53 and HTTP/gRPC headers to catch NXDOMAIN cascades, DNS query timeouts, and microservice inter-pod delays—without any proxy.

Zero-Proxy Packet SniffingActive
L3 Scheduler Agent

Kernel CFS Scheduler & Resource Contention

Detects Linux Completely Fair Scheduler (CFS) CPU quota throttling, memory allocation latency, and dirty page flushes invisible to Prometheus 30-second scrape intervals.

cgroup v2 · Linux SchedulerActive
L3 Root Cause Agent

Ghost Incident & Invisible Failure Correlator

Cross-correlates kernel-space anomalies with application APM traces to diagnose incidents where pods appear healthy but transactions fail silently with no container crash.

Invisible Incident DetectionActive
eBPF Ecosystem:Native 01Agent eBPF DaemonSetCilium / HubblePixie Auto-TracingInspektor GadgetFalco LSMCO-RE BPF Probes
Level-2 Secondary Telemetry Engine
L2 Telemetry Plugin Agent

Pluggable Telemetry Reconstruction

Plugs seamlessly into your telemetry plugins (SigNoz, Grafana Loki, Prometheus, ELK, Datadog, and any custom backend) to pull pre-failure logs, metric trends, and APM traces after pod deletion.

Pluggable Plugin · Custom BackendsActive
L2 RCA Agent

Multi-Agent Root Cause Analysis

Executes a 4-agent LangGraph workflow to cross-correlate log stacktraces, OOM events, Git commit history, and eBPF wire evidence via installed plugins.

LangGraph WorkflowActive
L2 Safety Agent

Human-in-the-Loop Safeguards

Auto-applies low-risk remedies while queuing high-risk cluster state mutations for SRE confirmation via Slack or API plugins.

HITL GatekeeperActive
Level-1 Real-Time Pod Failure SkillsUnlimited Custom Skills via SKILL.md
CrashLoopBackOffL1 Event Skill

CrashLoop Triage

Diagnoses container crash loops, missing env variables, panics, and unhandled exception stack traces.

Real-Time Triage
OOMKilledL1 Event Skill

Memory Exhaustion

Traces heap spikes, memory leaks, container memory limit breaches, and recommends scaled limits.

Resource Limits
ImagePullBackOffL1 Event Skill

Registry & Image Audit

Verifies container image availability, registry authentication tokens, and secret volume mounts.

Image & Credentials
CreateContainerErrorL1 Event Skill

Runtime Diagnostics

Identifies container runtime failures, cgroup misconfigurations, and volume mount permission errors.

Runtime Failure
FailedSchedulingL1 Event Skill

Scheduling Taints & Affinities

Analyzes node taint mismatches, unschedulable pod requirements, and node resource shortfalls.

Scheduler Analysis
NonZeroExitCodeL1 Event Skill

Process Exit Analysis

Correlates process exit codes (e.g. exit 137, 127) with binary startup scripts and entrypoint parameters.

Exit Code Tracing
NodePressureL1 Event Skill

Node Resource Pressure

Detects disk, memory, and PID pressure conditions triggering pod eviction and node cordoning.

Node Health
PodEvictedL1 Event Skill

Eviction Root Cause

Traces eviction history, priority class conflicts, resource quota exhaustion, and disruption budgets.

Eviction Analysis
Custom SkillCustom SKILL.md

Bring Your Own SKILL.md

Define any failure scenario as a declarative SKILL.md runbook. 01Agents hot-loads it instantly—no rebuild, no pod restart required.

Unlimited Custom Skills
Infinite Extensibility
Introduce Any Skill or Agent in Minutes
Create a SKILL.md runbook with YAML triggers, required tools, and remediation steps. 01Agents hot-loads it at runtime via the dynamic SkillRegistry—no Docker rebuild, no pod restart, no core code changes.
Write Your SKILL.md
EXTENSIBLE ARCHITECTURE & MARKETPLACE

Pluggable Telemetry Ecosystem

Services like SigNoz, Prometheus, ELK, Datadog and Grafana Loki, ArgoCD, Cilium Hubble, and Pixie are not hardcoded. They are decoupled plugins that users can install, uninstall, or replace with their own custom plugins via the Plugin Marketplace. No limit on the number of plugins.

01Agents Plugin Marketplace

Browse, install, and configure plugins for L1 triage, L2 telemetry, L3 eBPF, and custom GitOps backends.

Installed: 5 · Unlimited Available
Create Plugin SDK
APM & Tracesv1.4.0

SigNoz Telemetry Plugin

Extract historical ClickHouse traces & APM metrics pre-and-post pod eviction.

Author: 01Agents Core482
Metricsv2.1.0

Prometheus Metrics Plugin

PromQL metric parsing & threshold analysis for CPU, RAM, and OOMKilled events.

Author: 01Agents Core590
Log Managementv1.0.3

ELK Stack Plugin

Seamless Elasticsearch query engine and Kibana dashboard audit synchronizer.

Author: 01Agents Core310
Cloud Telemetryv1.2.1

Datadog Integration Plugin

Webhook intake and incident context enrichment from Datadog monitors.

Author: Community245
Live Log Streamv1.8.2

Grafana Loki WS Plugin

Loki WebSocket Tail proxy for persistent live log ingestion and PVC snapshotting.

Author: 01Agents Core612
GitOps & Deliveryv2.0.1

ArgoCD GitOps Plugin

Automated GitOps state verification, commit rollback, and sync status checks upon incident resolution.

Author: 01Agents Core534
GitOps & Deliveryv1.3.0

FluxCD Reconciliation Plugin

Native reconciliation with Flux Kustomization and HelmRelease objects for safe cluster state restoration.

Author: 01Agents Core298
eBPF & Kernelv0.9.0

Cilium Hubble PluginL3

Query Hubble flow metrics, packet drop reasons, and per-pod network policies via gRPC for L3 wire-level diagnostics.

Author: 01Agents Core187
eBPF & Kernelv0.7.2

Pixie Auto-Tracing PluginL3

Auto-traces HTTP, gRPC, PostgreSQL, and MySQL inter-pod protocols at the socket level without sidecars or code changes.

Author: Community143
Metricsv1.0.0

VictoriaMetrics Plugin

High-performance time-series queries via VictoriaMetrics MetricsQL for large-scale metric reconstruction.

Author: Community119
Log Managementv0.8.1

OpenSearch Log Plugin

Full-text log search and aggregation via OpenSearch REST APIs for postmortem log reconstruction.

Author: Community96
Custom SDKSDK v0.6

Bring Your Own PluginBYO

Build & plug your proprietary internal observability, ticketing, or ITSM tool using the Python/TypeScript Plugin SDK.

Author: You / Internal SRE∞
Zero Lock-In Architecture

Switch observability backends anytime without changing agent core code.

BYO Custom Plugin SDK

Write custom telemetry handlers in Python or TypeScript with a standard interface.

One-Click Hot-Swapping

Enable or disable plugins live without restarting 01Agent pods.

eBPF Kernel Plugins

Cilium Hubble, Pixie, and Inspektor Gadget plugins for L3 wire-level diagnostics.

l2-telemetry-audit-#4821.json
// L1→L2→L3 eBPF Multi-Tier Decision Entry
{
"agent_tier": "L3_eBPF_→_L2_LangGraph",
"telemetry_backend": "SigNoz_V5_ClickHouse",
"lookback_window": "-15m",
"logs_reconstructed": 1420,
"ebpf_tcp_drops": 142,
"dns_p99_ms": 480,
"kernel_cfs_throttled_ms": 128,
"rca_confidence": 0.97,
"hitl_approval": "OPERATOR_APPROVED",
"rollback_available": true
}
Observability & Integrations

Enterprise-Grade Observability & Instant Incident Auditability

Every L1 triage decision, L2 secondary telemetry analysis, and L3 eBPF kernel wire finding is fully logged, exportable, and queryable. 01Agents integrate natively into your existing observability and GitOps stack without modifying application code.

Decision history is queryable via API, exportable in JSON or CSV, and structured for postmortem review. When something needs to be explained — the exact evidence, eBPF wire telemetry, and RCA path are ready for audit.

Full Decision Provenance
Complete L1/L2 audit trails for postmortem analysis.
Instant Multi-Format Export
Export structured evidence in JSON, CSV, or query via API.
HITL Safety Controls
Human approval guardrails for critical cluster actions.
GitOps Verified Rollbacks
Automated rollback verification via ArgoCD & FluxCD.
TECHNICAL GUARANTEES

Engineered for Production Kubernetes Clusters

Designed from the ground up to solve postmortem log loss, surface invisible kernel-level incidents, prevent accidental cluster mutations, and integrate natively into existing DevOps tooling—with unlimited extensibility for custom skills and backends.

Telemetry PersistenceSigNoz / Loki API

Postmortem Log Reconstruction

Queries ClickHouse (SigNoz) & Loki WebSocket streams to retrieve logs after pods are deleted or terminated.

Production Tested
Safety & GovernanceHuman-in-the-Loop

Deterministic HITL Approval Gates

Classifies remediation actions by risk level. High-risk mutations (scaling, config patches) require operator approval.

Production Tested
Agent ArchitectureLangGraph Engine

LangGraph 4-Agent Workflow

Orchestrates dedicated agents for Triage, Telemetry Querying, Root Cause Analysis (RCA), and Remediation Generation.

Production Tested
State ConsistencyArgoCD & FluxCD

GitOps Rollback Verification

Verifies application declarative state with ArgoCD and FluxCD before marking any incident resolved.

Production Tested
Non-Invasive SetupOperator / Agent

Zero Application Code Instrumentation

Operates entirely at the cluster control plane & telemetry backend layer without requiring app SDK changes.

Production Tested
Audit ProvenanceJSON / CSV Audit API

Exportable Incident Decision Records

Outputs structured JSON/CSV decision traces for postmortems, compliance reviews, and team handoffs.

Production Tested
eBPF Kernel IntelligenceeBPF / CO-RE Probes

Sub-Millisecond Wire-Level Telemetry

Level-3 eBPF probes trace TCP socket state machines, DNS query latency, packet drops, and CFS scheduler throttling invisible to standard K8s observability.

Production Validated · L3 eBPF
Unlimited ExtensibilityDynamic SkillRegistry

Infinite SKILL.md Hot-Loading

Custom failure remediation skills are introduced as declarative SKILL.md runbooks and loaded just-in-time by the SkillRegistry—zero rebuild, zero pod restart.

Production Validated · L3 eBPF
WORKFLOW SPECIFICATION

Five Steps from Incident to Verified Fix

How 01Agents integrates into your deployment pipeline and handles pod failures from detection through L3 kernel wire analysis to verified GitOps remediation.

01

Cluster Event Monitoring & Ingestion

Deploys non-invasively into your Kubernetes cluster to listen for real-time pod events, OOMKilled signals, and state transitions via native Kafka event streams. L1 triage skills are fully customizable via SKILL.md runbooks—not limited to any fixed set of failure codes.

02

Pluggable Telemetry Backend Query

When a pod terminates and returns 404, 01Agents automatically initiates a lookback query across installed telemetry plugins (SigNoz, Grafana Loki, Prometheus, ELK Stack, Datadog, or any custom backend). The plugin architecture is 100% open—you can introduce unlimited custom observability backends via plugin.json and the MCP standard.

03

L3 eBPF Kernel Escalation (When Logs Are Inconclusive)

NEW · L3 eBPF

If application logs, APM traces, and K8s events don't fully explain the incident, 01Agents escalates to Level-3. The L3 eBPF agent deploys in-kernel probes to trace raw TCP packet drops, DNS query latency, CoreDNS NXDOMAIN cascades, Linux CFS scheduler throttling, and socket buffer overflows—surfacing invisible failures that standard observability completely misses.

04

Multi-Agent Root Cause Analysis

The L2 LangGraph engine cross-correlates pre-failure logs, metric memory spikes, container configuration history, and L3 eBPF wire evidence to pinpoint the exact root cause with a measurable RCA confidence score.

05

Human-in-the-Loop Safeguards & GitOps Sync

Low-risk remedies execute automatically, while high-risk cluster changes pause for SRE confirmation before verifying manifest restoration with ArgoCD or FluxCD. Every decision is logged with full audit provenance, exportable as JSON or CSV.

OPEN SOURCE · APACHE 2.0 LICENSE

Start Resolving Kubernetes Incidents Automatically

Install the 01Agents operator on your cluster, enable your preferred telemetry plugins (SigNoz, Loki, Prometheus, ELK), and stop losing logs when pods are deleted.

Maintained by BerryBytes & 01Cloud Open Source Community