[FEATURE] Complete the OData scaffolding: add `SkipPushdown` + named `odata_next_link` pagination algorithm

[jeffreyaven]

Summary

any-sdk has five of the OData v4 push-down configs wired (SelectPushdown, FilterPushdown, OrderByPushdown, TopPushdown, CountPushdown in internal/anysdk/query_param_pushdown.go). Two more are missing:

1. SkipPushdown — needed for SQL OFFSET N -> OData $skip=N.
2. A named OData-aware pagination algorithm — for following @odata.nextLink response cursors automatically. The generic Pagination config in pkg/streaming/pagination.go could be pointed at the right JSON path today, but only one named algorithm is registered (PaginationAlgorithmPageNumber = "page_number"), so providers can't declare their intent symbolically.

Both are blockers for a real Microsoft Graph / Entra ID provider, and both are blockers for the downstream stackql work that consumes them (the OData query-option wiring in stackql/, filed separately).

Why both belong in one PR

The two changes touch the same conceptual surface area (the OData feature parity in any-sdk) but no files overlap:

• SkipPushdown lives in internal/anysdk/query_param_pushdown.go alongside the existing five pushdowns.
• The named pagination algorithm lives in pkg/streaming/pagination.go (constant) + a small handler.

Shipping them together means stackql's downstream PR has a single any-sdk version to depend on, and gives reviewers one PR to evaluate the "complete OData v1 surface" against.

Scope

1. SkipPushdown struct

Add alongside TopPushdown:

// internal/anysdk/query_param_pushdown.go
type SkipPushdown struct {
    Dialect   string `json:"dialect,omitempty"   yaml:"dialect,omitempty"`
    ParamName string `json:"paramName,omitempty" yaml:"paramName,omitempty"`
    MaxValue  int64  `json:"maxValue,omitempty"  yaml:"maxValue,omitempty"`
}

func (s *SkipPushdown) GetParamName() string {
    if s == nil {
        return ""
    }
    if s.ParamName != "" {
        return s.ParamName
    }
    if s.Dialect == "odata" {
        return "$skip"
    }
    return ""
}

Add Skip *SkipPushdown field to QueryParamPushdown and the corresponding GetSkip() accessor. Update the inheritance walker in operation_store.go (around line 447) — it already does a uniform deep-merge across all QueryParamPushdown fields, so adding Skip is mechanical.

2. Named odata_next_link pagination algorithm

Add the constant:

// pkg/streaming/pagination.go
const (
    PaginationAlgorithmPageNumber   = "page_number"
    PaginationAlgorithmODataNextLink = "odata_next_link"  // NEW
)

Add a handler that, on each iteration:

1. Reads $.['@odata.nextLink'] from the decoded response body.
2. If present and non-empty:
   • The value is a complete URL (OData servers may include $skiptoken, server-side opaque state, etc.). The handler MUST use this URL verbatim for the next request — do NOT re-render from the original operation template, do NOT re-merge query params.
   • Detach the URL from the response body before passing rows downstream (so it doesn't appear in the projected row stream).
3. If absent, empty, or the response indicates an error: terminate.

3. Spec-side YAML

Documented in docs/provider_spec.md (lines ~462-490 already cover the other pushdowns). Add:

x-stackQL-config:
  queryParamPushdown:
    skip:
      dialect: odata        # auto-resolves paramName to $skip
      # OR
      paramName: custom_skip
      maxValue: 10000

serviceConfig:
  pagination:
    algorithm: odata_next_link
    # No request/response token semantic needed - the algorithm has
    # a hardcoded knowledge of `@odata.nextLink`. Future variants
    # (cosmos `nextLink`, Graph `@odata.nextLink` with different JSON
    # path) can be parametrised via responseToken.key if needed.

Test cases

Unit tests (any-sdk)

Add to internal/anysdk/registry_test.go alongside the existing pushdown tests (around lines 656-740):

func TestQueryParamPushdown_Skip(t *testing.T) {
    // Load a fixture that declares queryParamPushdown.skip:
    //   { dialect: odata }
    // Assert GetSkip().GetParamName() returns "$skip".
    // Assert GetSkip() returns nil when not declared.
}

func TestQueryParamPushdown_Skip_CustomParamName(t *testing.T) {
    // Fixture: queryParamPushdown.skip: { paramName: custom_skip }
    // Assert GetSkip().GetParamName() returns "custom_skip".
}

func TestQueryParamPushdown_Skip_InheritsFromService(t *testing.T) {
    // Method has no skip config; service declares one.
    // Assert the inheritance walker resolves to the service's config.
}

Unit tests (pagination)

Add to pkg/streaming/pagination_test.go:

func TestPaginationAlgorithm_ODataNextLink_FollowsLink(t *testing.T) {
    // Mock two response bodies:
    //   Page 1: { value: [...], "@odata.nextLink": "https://api.example/People?$skiptoken=abc" }
    //   Page 2: { value: [...] }
    // Assert the iterator issues exactly 2 requests, the second to the URL from page 1.
}

func TestPaginationAlgorithm_ODataNextLink_TerminatesOnAbsence(t *testing.T) {
    // Single response with no @odata.nextLink. Assert exactly 1 request.
}

func TestPaginationAlgorithm_ODataNextLink_TerminatesOnEmpty(t *testing.T) {
    // Response with "@odata.nextLink": "". Assert exactly 1 request (no follow).
}

func TestPaginationAlgorithm_ODataNextLink_UsesURLVerbatim(t *testing.T) {
    // Page 1 emits nextLink with $skiptoken=xyz and a $top=N that differs
    // from the original request. Assert page 2's request URL contains
    // the exact $skiptoken and the modified $top, not the original.
}

Trippin fixture extension

Extend test/registry/unsigned-src/odata_trippin/v00.00.00000/services/main.yaml to declare both new features. This fixture is what the downstream stackql PR will copy into stackql/test/registry/src/odata_trippin/ for end-to-end robot tests:

x-stackQL-config:
  queryParamPushdown:
    # ... existing select/filter/orderby/top/count ...
    skip:
      dialect: odata
serviceConfig:
  pagination:
    algorithm: odata_next_link

Why this matters

• Unblocks the stackql OData wiring PR. The downstream PR can't ship $skip push-down or @odata.nextLink pagination without these any-sdk additions. Without them, the OData v1 surface remains incomplete (no offset pagination, no auto-paging through multi-page results).
• Closes the gap between any-sdk's stated OData coverage and reality. query_param_pushdown.go and docs/provider_spec.md give the appearance of full OData support; SkipPushdown's absence is a surprising omission and @odata.nextLink only being reachable through generic responseToken.key configuration is a leaky abstraction.
• Small surface area. ~50 LOC for SkipPushdown + tests. ~80 LOC for the named pagination algorithm + tests. No new external dependencies. No API breakage.

Out of scope (separate any-sdk issues / future work)

• ExpandPushdown for OData $expand. Semantically heavy — $expand=relatedEntity returns nested entity payloads, and the right SQL surface (JSON column vs. auto-flattened rows) is a design decision worth its own discussion. Real Entra ID users?$expand=manager is the canonical motivator. Defer.
• SearchPushdown for OData $search. Free-text search has no good SQL analog without an FTS planner. Defer indefinitely.
• OData function-call push-down beyond what FilterPushdown.supportedOperators already covers. OData has tolower, toupper, length, year, month, day, etc. — useful but a large surface. Defer.
• $value (raw scalar return) and $batch (multi-request batching). Niche; defer.

File references

• internal/anysdk/query_param_pushdown.go — add SkipPushdown struct + field.
• internal/anysdk/operation_store.go line 447 — inheritance walker (mechanical update).
• internal/anysdk/registry_test.go lines 656-740 — pattern for new TestQueryParamPushdown_Skip* tests.
• pkg/streaming/pagination.go line 22 — algorithm constants; add PaginationAlgorithmODataNextLink.
• pkg/streaming/pagination_test.go — pattern for new TestPaginationAlgorithm_ODataNextLink_* tests.
• docs/provider_spec.md lines ~462-490 — add YAML documentation for both new features.
• test/registry/unsigned-src/odata_trippin/v00.00.00000/services/main.yaml — extend with skip + algorithm: odata_next_link declarations.

Related

• Downstream stackql issue: Wire OData and GraphQL push-down through the SQL planner with Trippin + Trevorblades robot tests. That issue depends on this one being merged + published.
• The Cloudflare provider's GraphQL operations currently expose limit as a WHERE-clause parameter as a workaround for absent push-down. The downstream stackql issue removes that workaround. This any-sdk PR is a prerequisite for shipping the complete OData feature set in that downstream PR.