diff --git a/README.md b/README.md
index fdc09cd..9dc2659 100644
--- a/README.md
+++ b/README.md
@@ -17,21 +17,66 @@ Runs a Vector pipeline as a supervised transform stage in a DFE deployment.
The config engine and the process manager run Vector inside the pod, the bridge moves records on the direct transport, and the metrics merge serves Vector's metrics beside the supervisor's.
```mermaid
+---
+config:
+ theme: base
+ look: neo
+ fontFamily: Inter, -apple-system, Segoe UI, Roboto, Helvetica, Arial, sans-serif
+ flowchart:
+ curve: linear
+ nodeSpacing: 40
+ rankSpacing: 64
+ padding: 18
+ wrappingWidth: 280
+ htmlLabels: true
+ elk:
+ preset: modelOrder
+ themeVariables:
+ fontSize: 15px
+ lineColor: "#64748B"
+ primaryTextColor: "#0F172A"
+ clusterBorder: "#0072B2"
+ edgeLabelBackground: "#FFFFFF"
+ themeCSS: >-
+ p{margin:0}
+ .node[data-look] rect,.node[data-look] path,.node[data-look] polygon,.node[data-look] .outer-path{filter:drop-shadow(0 1px 2px rgba(15,23,42,.10)) drop-shadow(0 6px 14px rgba(15,23,42,.10))}
+ .node rect{rx:10px;ry:10px}
+ .cluster rect{rx:16px;ry:16px}
+ .node .label div{min-width:150px}
+ .node.decision .label div{min-width:0}
+ .cluster[data-look] rect{filter:none}
+ .nodeLabel{line-height:1.35}
+ .nodeLabel b{font-weight:600;font-size:15px;letter-spacing:-0.01em}
+ .nodeLabel span{font-size:12.5px;font-weight:400;color:#334155}
+ .cluster-label span.nodeLabel{display:inline-block;background:#FFFFFF;color:#0B2A3F;border:1.5px solid #0072B2;border-radius:999px;padding:2px 12px;font-size:11.5px;font-weight:700;letter-spacing:.09em;text-transform:uppercase}
+ .flowchart-link{stroke-width:1.75px}
+ .labelBkg{background:transparent!important}
+ span.edgeLabel{display:inline-block;background:#FFFFFF!important;color:#1E293B;border:1px solid #94A3B8;border-radius:999px;padding:2px 10px;font-size:12px;font-weight:500}
+ span.edgeLabel p{background:transparent!important}
+ .label-icon{width:20px;height:20px;vertical-align:-4px;margin-right:6px}
+ .ingest .label-icon{color:#0E8A7D}
+ .xform .label-icon{color:#7B4FB0}
+ .load .label-icon{color:#0072B2}
+ .bus .label-icon{color:#A8517F}
+ .store .label-icon{color:#00785A}
+ .ui .label-icon{color:#D9480F}
+ .ext .label-icon{color:#64748B}
+---
flowchart TB
accTitle: L3 Components -- dfe-transform-vector
accDescr: The previous stage sends records into the pod and the config and transform files configure it. Inside, the config engine writes the Vector config that the process manager starts Vector on, and on the direct transport the bridge intake hands records to Vector and the bridge return takes them back. Vector's metrics are merged into the supervisor's and served on the ops port. Records leave for the next stage and metrics go to the collector.
- prev["Previous stage"]
- files[("Config files")]
+ prev["Previous stage"]
+ files[("Config files")]
subgraph pod["dfe-transform-vector"]
direction LR
- cfg[["Config engine"]]
- proc["Process manager"]
- intake[["Bridge intake"]]
- vec[["Vector"]]
- ret[["Bridge return"]]
- merge["Metrics merge"]
+ cfg[["Config engine"]]
+ proc["Process manager"]
+ intake[["Bridge intake"]]
+ vec[["Vector"]]
+ ret[["Bridge return"]]
+ merge["Metrics merge"]
cfg --> proc
proc --> vec
intake --> vec
@@ -39,21 +84,28 @@ flowchart TB
vec --> merge
end
- next["Next stage"]
- collector["Metrics collector"]
+ next["Next stage"]
+ collector["Metrics collector"]
prev -->|records| pod
files -->|config| pod
pod -->|records| next
pod -->|metrics| collector
-
- classDef default fill:#F8FAFC,stroke:#64748B,stroke-width:1px,color:#0F172A
- classDef store fill:#DDF3EC,stroke:#00785A,stroke-width:1px,color:#0B3327
- classDef ext fill:#EEF0F2,stroke:#6B7280,stroke-width:1px,stroke-dasharray:4 3,color:#1F2937
- classDef group fill:none,stroke:#64748B,stroke-width:1px,stroke-dasharray:2 4
- class prev,next,collector ext
- class files store
- class pod group
+ classDef ext fill:#F4F6F8,stroke:#94A3B8,stroke-width:1.5px,stroke-dasharray:5 4,color:#0F172A
+ classDef ingest fill:#FFFFFF,stroke:#0E8A7D,stroke-width:3px,color:#0F172A
+ classDef xform fill:#FFFFFF,stroke:#7B4FB0,stroke-width:3px,color:#0F172A
+ classDef load fill:#FFFFFF,stroke:#0072B2,stroke-width:3px,color:#0F172A
+ classDef bus fill:#FFFFFF,stroke:#A8517F,stroke-width:3px,color:#0F172A
+ classDef store fill:#FFFFFF,stroke:#00785A,stroke-width:3px,color:#0F172A
+ classDef ui fill:#FFFFFF,stroke:#D9480F,stroke-width:3px,color:#0F172A
+ classDef decision fill:#FFFFFF,stroke:#9A6700,stroke-width:3px,color:#0F172A
+ classDef error fill:#FFFFFF,stroke:#C2410C,stroke-width:3px,color:#0F172A
+ classDef control fill:#FFFFFF,stroke:#334155,stroke-width:3px,color:#0F172A
+ classDef part fill:#FFFFFF,stroke:#94A3B8,stroke-width:1.5px,color:#0F172A
+ class prev,next,collector ext
+ class files store
+ class cfg,proc,intake,vec,ret,merge part
+ style pod fill:none,stroke:#0072B2,stroke-width:1.5px,stroke-dasharray:6 4
```
*Figure 1. L3 Components -- dfe-transform-vector. Dashed boxes are external.* Up: top of dfe-transform-vector. Down: [L4 Detail -- Config assembly](docs/architecture.md#config-assembly), [L4 Detail -- Direct transport bridge](docs/architecture.md#where-the-transports-differ), [L4 Detail -- Vector lifecycle](docs/internals.md#the-vector-subprocess).
diff --git a/docs/architecture.md b/docs/architecture.md
index 682aefc..b28cdf6 100644
--- a/docs/architecture.md
+++ b/docs/architecture.md
@@ -38,23 +38,78 @@ The pod's parts are in [Figure 1 of the README](../README.md). The config engine
The supervisor loads and validates the config, writes the secret files and the source, then wires the transforms, writes the sink and runs `vector validate` before Vector reads the directory.
```mermaid
+---
+config:
+ theme: base
+ look: neo
+ fontFamily: Inter, -apple-system, Segoe UI, Roboto, Helvetica, Arial, sans-serif
+ flowchart:
+ curve: linear
+ nodeSpacing: 40
+ rankSpacing: 64
+ padding: 18
+ wrappingWidth: 280
+ htmlLabels: true
+ elk:
+ preset: modelOrder
+ themeVariables:
+ fontSize: 15px
+ lineColor: "#64748B"
+ primaryTextColor: "#0F172A"
+ clusterBorder: "#0072B2"
+ edgeLabelBackground: "#FFFFFF"
+ themeCSS: >-
+ p{margin:0}
+ .node[data-look] rect,.node[data-look] path,.node[data-look] polygon,.node[data-look] .outer-path{filter:drop-shadow(0 1px 2px rgba(15,23,42,.10)) drop-shadow(0 6px 14px rgba(15,23,42,.10))}
+ .node rect{rx:10px;ry:10px}
+ .cluster rect{rx:16px;ry:16px}
+ .node .label div{min-width:150px}
+ .node.decision .label div{min-width:0}
+ .cluster[data-look] rect{filter:none}
+ .nodeLabel{line-height:1.35}
+ .nodeLabel b{font-weight:600;font-size:15px;letter-spacing:-0.01em}
+ .nodeLabel span{font-size:12.5px;font-weight:400;color:#334155}
+ .cluster-label span.nodeLabel{display:inline-block;background:#FFFFFF;color:#0B2A3F;border:1.5px solid #0072B2;border-radius:999px;padding:2px 12px;font-size:11.5px;font-weight:700;letter-spacing:.09em;text-transform:uppercase}
+ .flowchart-link{stroke-width:1.75px}
+ .labelBkg{background:transparent!important}
+ span.edgeLabel{display:inline-block;background:#FFFFFF!important;color:#1E293B;border:1px solid #94A3B8;border-radius:999px;padding:2px 10px;font-size:12px;font-weight:500}
+ span.edgeLabel p{background:transparent!important}
+ .label-icon{width:20px;height:20px;vertical-align:-4px;margin-right:6px}
+ .ingest .label-icon{color:#0E8A7D}
+ .xform .label-icon{color:#7B4FB0}
+ .load .label-icon{color:#0072B2}
+ .bus .label-icon{color:#A8517F}
+ .store .label-icon{color:#00785A}
+ .ui .label-icon{color:#D9480F}
+ .ext .label-icon{color:#64748B}
+---
flowchart TB
accTitle: L4 Detail -- Config assembly
accDescr: The config is loaded through its cascade and validated, the secret files are written and the source is generated. The transform files are read and wired into a checked graph, the sink is generated, and vector validate gates the directory before Vector is started or sent a SIGHUP.
subgraph s1["1 Prepare"]
direction LR
- load["Load cascade"] --> check["Validate config"] --> secrets["Write secret files"] --> source["Generate source"]
+ load["Load cascade"] --> check["Validate config"] --> secrets["Write secret files"] --> source["Generate source"]
end
subgraph s2["2 Wire and gate"]
direction LR
- read["Read transforms"] --> wire["Wire and check"] --> sink["Generate sink"] --> gate["vector validate"]
+ read["Read transforms"] --> wire["Wire and check"] --> sink["Generate sink"] --> gate["vector validate"]
end
s1 --> s2
-
- classDef default fill:#F8FAFC,stroke:#64748B,stroke-width:1px,color:#0F172A
- classDef group fill:none,stroke:#64748B,stroke-width:1px,stroke-dasharray:2 4
- class s1,s2 group
+ classDef ext fill:#F4F6F8,stroke:#94A3B8,stroke-width:1.5px,stroke-dasharray:5 4,color:#0F172A
+ classDef ingest fill:#FFFFFF,stroke:#0E8A7D,stroke-width:3px,color:#0F172A
+ classDef xform fill:#FFFFFF,stroke:#7B4FB0,stroke-width:3px,color:#0F172A
+ classDef load fill:#FFFFFF,stroke:#0072B2,stroke-width:3px,color:#0F172A
+ classDef bus fill:#FFFFFF,stroke:#A8517F,stroke-width:3px,color:#0F172A
+ classDef store fill:#FFFFFF,stroke:#00785A,stroke-width:3px,color:#0F172A
+ classDef ui fill:#FFFFFF,stroke:#D9480F,stroke-width:3px,color:#0F172A
+ classDef decision fill:#FFFFFF,stroke:#9A6700,stroke-width:3px,color:#0F172A
+ classDef error fill:#FFFFFF,stroke:#C2410C,stroke-width:3px,color:#0F172A
+ classDef control fill:#FFFFFF,stroke:#334155,stroke-width:3px,color:#0F172A
+ classDef part fill:#FFFFFF,stroke:#94A3B8,stroke-width:1.5px,color:#0F172A
+ class load,check,secrets,source,read,wire,sink,gate part
+ style s1 fill:none,stroke:#0072B2,stroke-width:1.5px,stroke-dasharray:6 4
+ style s2 fill:none,stroke:#0072B2,stroke-width:1.5px,stroke-dasharray:6 4
```
*Figure 2. L4 Detail -- Config assembly. Read each stage left to right, then down.* Up: [L3 Components -- dfe-transform-vector](../README.md).
@@ -104,28 +159,81 @@ Vector is redistributed unmodified under MPL-2.0, which has two obligations -- s
On direct a record enters at the intake listener, crosses Vector over loopback and leaves from the return listener, which sends a record over the next stage's size ceiling to the DLQ.
```mermaid
+---
+config:
+ theme: base
+ look: neo
+ fontFamily: Inter, -apple-system, Segoe UI, Roboto, Helvetica, Arial, sans-serif
+ flowchart:
+ curve: linear
+ nodeSpacing: 40
+ rankSpacing: 64
+ padding: 18
+ wrappingWidth: 280
+ htmlLabels: true
+ elk:
+ preset: modelOrder
+ themeVariables:
+ fontSize: 15px
+ lineColor: "#64748B"
+ primaryTextColor: "#0F172A"
+ clusterBorder: "#0072B2"
+ edgeLabelBackground: "#FFFFFF"
+ themeCSS: >-
+ p{margin:0}
+ .node[data-look] rect,.node[data-look] path,.node[data-look] polygon,.node[data-look] .outer-path{filter:drop-shadow(0 1px 2px rgba(15,23,42,.10)) drop-shadow(0 6px 14px rgba(15,23,42,.10))}
+ .node rect{rx:10px;ry:10px}
+ .cluster rect{rx:16px;ry:16px}
+ .node .label div{min-width:150px}
+ .node.decision .label div{min-width:0}
+ .cluster[data-look] rect{filter:none}
+ .nodeLabel{line-height:1.35}
+ .nodeLabel b{font-weight:600;font-size:15px;letter-spacing:-0.01em}
+ .nodeLabel span{font-size:12.5px;font-weight:400;color:#334155}
+ .cluster-label span.nodeLabel{display:inline-block;background:#FFFFFF;color:#0B2A3F;border:1.5px solid #0072B2;border-radius:999px;padding:2px 12px;font-size:11.5px;font-weight:700;letter-spacing:.09em;text-transform:uppercase}
+ .flowchart-link{stroke-width:1.75px}
+ .labelBkg{background:transparent!important}
+ span.edgeLabel{display:inline-block;background:#FFFFFF!important;color:#1E293B;border:1px solid #94A3B8;border-radius:999px;padding:2px 10px;font-size:12px;font-weight:500}
+ span.edgeLabel p{background:transparent!important}
+ .label-icon{width:20px;height:20px;vertical-align:-4px;margin-right:6px}
+ .ingest .label-icon{color:#0E8A7D}
+ .xform .label-icon{color:#7B4FB0}
+ .load .label-icon{color:#0072B2}
+ .bus .label-icon{color:#A8517F}
+ .store .label-icon{color:#00785A}
+ .ui .label-icon{color:#D9480F}
+ .ext .label-icon{color:#64748B}
+---
flowchart LR
accTitle: L4 Detail -- Direct transport bridge
accDescr: The previous stage pushes records to the intake listener, which hands them to Vector over loopback. Vector's output comes back to the return listener, which pushes it to the next stage, or sends a record over the size ceiling to the DLQ.
- prev["Previous
stage"]
- intake["Intake
listener"]
- vec["Vector"]
- ret["Return
listener"]
- next["Next
stage"]
- dlq[("DLQ")]
+ prev["Previous
stage"]
+ intake["Intake
listener"]
+ vec["Vector"]
+ ret["Return
listener"]
+ next["Next
stage"]
+ dlq[("DLQ")]
prev -->|Push| intake
intake --> vec
vec --> ret
ret -->|Push| next
ret -.->|too large| dlq
-
- classDef default fill:#F8FAFC,stroke:#64748B,stroke-width:1px,color:#0F172A
- classDef error fill:#FBE3D6,stroke:#C2410C,stroke-width:1px,color:#4A1A05
- classDef ext fill:#EEF0F2,stroke:#6B7280,stroke-width:1px,stroke-dasharray:4 3,color:#1F2937
- class prev,next ext
- class dlq error
+ classDef ext fill:#F4F6F8,stroke:#94A3B8,stroke-width:1.5px,stroke-dasharray:5 4,color:#0F172A
+ classDef ingest fill:#FFFFFF,stroke:#0E8A7D,stroke-width:3px,color:#0F172A
+ classDef xform fill:#FFFFFF,stroke:#7B4FB0,stroke-width:3px,color:#0F172A
+ classDef load fill:#FFFFFF,stroke:#0072B2,stroke-width:3px,color:#0F172A
+ classDef bus fill:#FFFFFF,stroke:#A8517F,stroke-width:3px,color:#0F172A
+ classDef store fill:#FFFFFF,stroke:#00785A,stroke-width:3px,color:#0F172A
+ classDef ui fill:#FFFFFF,stroke:#D9480F,stroke-width:3px,color:#0F172A
+ classDef decision fill:#FFFFFF,stroke:#9A6700,stroke-width:3px,color:#0F172A
+ classDef error fill:#FFFFFF,stroke:#C2410C,stroke-width:3px,color:#0F172A
+ classDef control fill:#FFFFFF,stroke:#334155,stroke-width:3px,color:#0F172A
+ classDef part fill:#FFFFFF,stroke:#94A3B8,stroke-width:1.5px,color:#0F172A
+ class prev,next ext
+ class intake,vec,ret part
+ class dlq bus
```
*Figure 3. L4 Detail -- Direct transport bridge. The dotted arrow is the dead letter path.* Up: [L3 Components -- dfe-transform-vector](../README.md).
diff --git a/docs/internals.md b/docs/internals.md
index dc21496..aa526c5 100644
--- a/docs/internals.md
+++ b/docs/internals.md
@@ -94,6 +94,21 @@ In a chart deployment an edit to the config or a file set rolls the pods anyway,
Vector is `Running` once it is alive 500 ms after spawn, and an exit nobody asked for is a crash, followed by a restart after the backoff.
```mermaid
+---
+config:
+ theme: base
+ look: neo
+ fontFamily: Inter, -apple-system, Segoe UI, Roboto, Helvetica, Arial, sans-serif
+ themeVariables:
+ fontSize: 15px
+ lineColor: "#64748B"
+ primaryTextColor: "#0F172A"
+ clusterBorder: "#0072B2"
+ edgeLabelBackground: "#FFFFFF"
+ primaryColor: "#FFFFFF"
+ primaryBorderColor: "#94A3B8"
+ tertiaryColor: "#F4F6F8"
+---
stateDiagram-v2
direction LR
accTitle: L4 Detail -- Vector lifecycle
@@ -110,7 +125,7 @@ stateDiagram-v2
Crashed --> Starting: backoff
Running --> ShuttingDown: SIGTERM
ShuttingDown --> [*]
- classDef error fill:#FBE3D6,stroke:#C2410C,stroke-width:1px,color:#4A1A05
+ classDef error fill:#FFFFFF,stroke:#C2410C,stroke-width:3px,color:#0F172A
class Crashed error
```