Skip to content

[FUNK-2635] Expose reusable prometheus integration - #242

Draft
Ashwithabg wants to merge 3 commits into
mainfrom
feat/prometheus-start-exporter
Draft

Ashwithabg wants to merge 3 commits into
mainfrom
feat/prometheus-start-exporter

Conversation

@Ashwithabg

Copy link
Copy Markdown
  • StartExporter serves /metrics and tags the engine

Description:

What

Adds prometheus.StartExporter, which wires a stats engine to a Prometheus /metrics endpoint in one call:

exp, err := prometheus.StartExporter(prometheus.ExporterConfig{
    Addr:    ":9102",
    AppName: "my-service",
})
if err != nil { ... }
defer exp.Shutdown(ctx)

It:
- adds an app name tag (appName by default, configurable with AppNameTag) to the engine's tags, so a collector can name the service that owns the series
- registers a Handler on the engine (stats.DefaultEngine unless Engine is set), next to any handler already registered, such as Datadog
- binds the listener before returning, so a bad or taken address is an error and not a log line from a goroutine
- serves /metrics on Addr, with a graceful Shutdown(ctx) that is safe to call twice

Handler, MetricTimeout and ReadHeaderTimeout can be set through the config. A caller-supplied Handler wins over MetricTimeout.

Why

Each service that exposes stats through a Prometheus scrape (functions-service first, then funk-router, funk-job-creator and control-plane) re-implements the same setup: tag the engine, register the handler, run an HTTP server, shut it down. The copies differ in small ways, such as logging a listen failure instead of returning it. This puts it in one place.

Notes

- The app name tag is added to engine.Tags, so call StartExporter before deriving engines with WithTags or WithPrefix. Engines derived earlier do not inherit it. This is documented on the function and in the README.
- Histogram buckets are not part of the config. Keep using Engine.SetBuckets.
- Additive only. No existing API changes.

Test

go test -race ./prometheus passes. The new tests cover:
- the app name tag and engine tags in the scrape output
- a custom tag name
- an existing handler still receiving measures
- a missing Addr and an address already in use
- calling Shutdown twice

I haven't run this in a deployed service. Once released, functions-service #256 will be the first consumer.

HISTORY.md and the README have entries for it.

🤖 Generated with Claude Code

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@Ashwithabg

Copy link
Copy Markdown
Author

Review Summary

Automated review (bug-hunter, language-pattern-reviewer, test-analyzer). No bugs found: the Exporter.err handoff, Shutdown with an expired context, double Shutdown and the failure paths all check out, and go test -race -count=20 ./prometheus passes. 4 findings, all high: one test gap and three API/lifecycle points.

Recommendation: COMMENT (this line is a reading aid; no review status was set)

🟠 High Findings (4)

Caller-supplied Handler and MetricTimeout are not tested — prometheus/exporter_test.go:1

No test sets ExporterConfig.Handler or MetricTimeout, so the tests would still pass if StartExporter ignored both. Package coverage (93.3%) hides this. Suggestion: add a test that passes h := &Handler{...} and asserts e.Handler == h and that a measure shows up at /metrics with status 200, and a test with MetricTimeout set and no Handler that checks the value reaches the handler. The default-engine path, the Serve error path and ignored Shutdown errors are lower-priority gaps.
medium confidence

engine.Tags is written without synchronization — prometheus/exporter.go:83

StartExporter assigns the engine's Tags field with no lock, and the engine reads it on every measurement. If it runs after the engine is in concurrent use, that is a data race. The doc comment only warns about derived engines. A second call on the same engine is not a problem: SortTags dedupes the repeated name. Suggestion: state in the doc comment that it must be called before the engine is used concurrently, or tag a derived engine and return it.
medium confidence

A background server failure is only visible through Shutdown — prometheus/exporter.go:113

A fatal Serve error is stored in e.err and only returned by Shutdown. Nothing is logged and there is no way to wait on or poll the failure, so a service can silently stop serving metrics. Suggestion: add Done() <-chan struct{} and Err() error, or an optional error callback in ExporterConfig.
medium confidence

Only ReadHeaderTimeout is set on the server — prometheus/exporter.go:106

No WriteTimeout or IdleTimeout, so a slow /metrics reader or an idle keep-alive connection can hold a goroutine open. After an expired Shutdown context, active connections are left open and there is no Close path. Suggestion: add conservative defaults (for example 30s write, 60s idle) and expose them in the config, and close the server when Shutdown returns because the context expired.
medium confidence

📋 Agents Dispatched

Ran: bug-hunter, language-pattern-reviewer, test-analyzer
Skipped: security-reviewer (no input handling), dependency-scanner (no manifest changes), docs-validator (README and HISTORY updated)
Complexity tier: standard


Reviewed by review-toolkit

@Ashwithabg Ashwithabg left a comment

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Inline comments from review-toolkit agents.

Comment thread prometheus/exporter_test.go Resolved
Comment thread prometheus/exporter.go Resolved
Comment thread prometheus/exporter.go Resolved
Comment thread prometheus/exporter.go Resolved
Add Done and Err so a server that dies in the background is observable, default
WriteTimeout and IdleTimeout, close open connections when Shutdown's context
expires, and document that the engine writes are not synchronized. Tests cover
a caller-supplied Handler, MetricTimeout, the default engine, repeated calls,
Done/Err, a serve failure and a forced close.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@Ashwithabg
Ashwithabg requested a balanced review from Copilot November 18, 2026 04:38

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Warning

Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.

Copilot review overview

5 open issues
What changed in this PR

Adds a reusable prometheus.StartExporter helper to expose /metrics on a dedicated listener while tagging the stats engine with an owning-service label, reducing duplicated setup across services.

Changes:

  • Introduces ExporterConfig/Exporter and StartExporter to bind a listener, register a handler on a stats engine, and serve /metrics with graceful shutdown.
  • Adds Go tests covering tagging behavior, error cases, handler interplay, shutdown semantics, and connection closing on shutdown timeout.
  • Updates README and HISTORY with usage and release notes.
File Description
prometheus/​exporter.go Implements StartExporter, Exporter, and config-driven HTTP server setup + shutdown semantics
prometheus/​exporter_test.go Adds test coverage for exporter behavior (tags, handlers, timeouts, failures, shutdown)
README.md Documents StartExporter usage and ordering constraints with derived engines
HISTORY.md Adds Unreleased entry describing the new API and behavior

Review effort: Lite

Comment thread prometheus/exporter.go
Comment on lines +165 to +173
err := e.server.Shutdown(ctx)
if err != nil {
e.server.Close() // ctx expired with scrapes in flight: do not leave them open
}
<-e.done
if err == nil {
err = e.err
}
return err
engine.Add("requests", 3)
engine.Flush()

res, err := http.Get("http://" + e.Addr().String() + "/metrics")
t.Fatal(err)
}
defer res.Body.Close()
body, _ := io.ReadAll(res.Body)
func scrapeBody(t *testing.T, e *Exporter) string {
t.Helper()

res, err := http.Get("http://" + e.Addr().String() + "/metrics")
Comment on lines +281 to +291
// Hold a request open: send the headers of a request but never finish it, so it stays active.
conn, err := net.Dial("tcp", e.Addr().String())
if err != nil {
t.Fatal(err)
}
defer conn.Close()
if _, err := conn.Write([]byte("GET /metrics HTTP/1.1\r\nHost: x\r\n")); err != nil {
t.Fatal(err)
}
time.Sleep(50 * time.Millisecond)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants