Skip to content

feat(node-server): detailed flag evaluation trace (RFC)#246

Draft
typotter wants to merge 3 commits into
mainfrom
typo/detailed-flag-evaluation
Draft

feat(node-server): detailed flag evaluation trace (RFC)#246
typotter wants to merge 3 commits into
mainfrom
typo/detailed-flag-evaluation

Conversation

@typotter
Copy link
Copy Markdown
Collaborator

@typotter typotter commented Apr 15, 2026

Summary

Implements the detailed flag evaluation trace RFC for the @datadog/openfeature-node-server package.

Every flag evaluation now produces a DDFlagEvaluationDetails object with a full allocation waterfall trace, surfacing why each allocation was skipped or matched. The trace is available at the OpenFeature boundary via an opt-in includeEvaluationTrace provider option.

What's new

  • DDFlagEvaluationDetails<T> — internal evaluation result with per-allocation outcome records, matched rule index, config metadata, and a structured outcomeCode enum
  • DDFlagEvaluationDetailsBuilder — records BEFORE_START_TIME, AFTER_END_TIME, RULES_MISMATCH, TRAFFIC_MISS, MISSING_VARIATION, and UNEVALUATED outcomes as the waterfall executes
  • toResolutionDetails() — converts the internal type to ResolutionDetails; serializes the full trace into flagMetadata.ddEvaluationTrace when includeEvaluationTrace: true
  • findMatchingRuleIndex() in rules.ts — returns the 0-indexed matched rule position for trace capture
  • Pre-waterfall error paths (PROVIDER_NOT_READY, FLAG_NOT_FOUND, TARGETING_KEY_MISSING) now carry configFetchedAt and environmentName when available

Changes

  • src/configuration/flagEvaluationDetails.ts — new file (types, builder, toResolutionDetails)
  • src/configuration/evaluateForSubject.ts — returns DDFlagEvaluationDetails, records per-allocation outcomes
  • src/configuration/evaluation.ts — returns DDFlagEvaluationDetails, threads config metadata
  • src/rules/rules.ts — adds findMatchingRuleIndex
  • src/provider.ts — adds includeEvaluationTrace option, calls toResolutionDetails
  • test/flagEvaluationDetails.spec.ts — 37 tests (32 original + 5 covering full trace JSON content)
  • test/demo.spec.ts — interactive demo (not a CI test; see below)
  • docs/rfc-detailed-flag-evaluation.md — design document

Running the demo

demo.spec.ts exercises three evaluation scenarios through the real OpenFeature client and prints the EvaluationDetails object and the full parsed ddEvaluationTrace for each:

cd packages/node-server
npx jest --testPathPatterns=demo --verbose --silent=false

Scenarios covered:

  • US userMATCH at allocation us-beta via rule index 1, two prior allocations skipped (AFTER_END_TIME, BEFORE_START_TIME), one unevaluated
  • GB user — falls through AFTER_END_TIME, BEFORE_START_TIME, RULES_MISMATCH to default-off (MATCH)
  • Flag not foundFLAG_NOT_FOUND with null match fields and empty allocation arrays

Test plan

  • npm test passes (280 tests, all suites green)
  • Verify includeEvaluationTrace: false (default) produces no ddEvaluationTrace in flagMetadata
  • Verify includeEvaluationTrace: true produces a parseable JSON trace with correct outcomeCode, allocation lists, and config metadata
  • Run the demo (npx jest --testPathPatterns=demo --verbose --silent=false) and inspect the printed trace
  • Review open questions in the RFC (§ Open Questions) and decide whether any require follow-on work before merge

Implements the detailed flag evaluation trace RFC. Every evaluation now
produces a DDFlagEvaluationDetails with per-allocation outcomes
(BEFORE_START_TIME, AFTER_END_TIME, RULES_MISMATCH, TRAFFIC_MISS,
MISSING_VARIATION, UNEVALUATED) and a matched allocation record.

The provider converts this to ResolutionDetails via toResolutionDetails().
An opt-in includeEvaluationTrace provider option serializes the full trace
into flagMetadata.ddEvaluationTrace for observability use cases.

Pre-waterfall error paths (PROVIDER_NOT_READY, FLAG_NOT_FOUND, etc.) are
handled with empty allocation lists per PRE_WATERFALL_CODES. Rules matching
now exposes findMatchingRuleIndex so the matched rule position is captured
in the trace.
Adds the design document that specifies DDFlagEvaluationDetails, the
AllocationOutcomeCode enum, DDFlagEvaluationDetailsBuilder, and the
toResolutionDetails() OpenFeature boundary conversion implemented in
the preceding commit.
Adds 5 unit tests covering the full ddEvaluationTrace JSON content
(all top-level fields, waterfall with AFTER_END_TIME/BEFORE_START_TIME/
RULES_MISMATCH, DEFAULT with all unmatched, FLAG_NOT_FOUND with empty
arrays, early MATCH with unevaluated allocations).

Adds demo.spec.ts showing EvaluationDetails returned from the OpenFeature
client with the full ddEvaluationTrace for three scenarios: US user (MATCH
via rule index 1), GB user (RULES_MISMATCH → default-off), flag not found.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant