Skip to content

RDSC-4603 Update RDI public documentation#3262

Merged
andy-stark-redis merged 14 commits into
redis:mainfrom
stoyanr:RDSC-4603-update-rdi-docs
May 11, 2026
Merged

RDSC-4603 Update RDI public documentation#3262
andy-stark-redis merged 14 commits into
redis:mainfrom
stoyanr:RDSC-4603-update-rdi-docs

Conversation

@stoyanr
Copy link
Copy Markdown
Contributor

@stoyanr stoyanr commented May 5, 2026

See https://redislabs.atlassian.net/browse/RDSC-4603 and https://github.com/RedisLabs/redis-data-integration/pull/2565


Note

Low Risk
Documentation-only changes, but they introduce new guidance around processor selection, configuration, and migration that could mislead users if inaccurate.

Overview
Adds end-to-end documentation for the Preview Flink stream processor: new classic-vs-flink comparison page, a migration guide, and architecture/config docs explaining per-pipeline selection via processors.type (defaulting to classic).

Updates pipeline configuration and transformation references to document Flink-specific behavior and limitations (notably hash/json outputs only), and introduces expression/lookup caching docs via a new cache block plus examples.

Refreshes installation/upgrade, FAQ, release notes, and observability docs to reflect Flink-specific images and Prometheus scraping (JobManager/TaskManager metrics vs rdi-metrics-exporter), and clarifies “when to use RDI” throughput/data-size assumptions for classic vs Flink.

Reviewed by Cursor Bugbot for commit b69be3e. Bugbot is set up for automated code reviews on this repo. Configure here.

@jit-ci
Copy link
Copy Markdown

jit-ci Bot commented May 5, 2026

Hi, I’m Jit, a friendly security platform designed to help developers build secure applications from day zero with an MVS (Minimal viable security) mindset.

In case there are security findings, they will be communicated to you as a comment inside the PR.

Hope you’ll enjoy using Jit.

Questions? Comments? Want to learn more? Get in touch with us.

@stoyanr stoyanr marked this pull request as draft May 5, 2026 13:23
@stoyanr stoyanr marked this pull request as ready for review May 5, 2026 14:57
Copy link
Copy Markdown
Contributor

@andy-stark-redis andy-stark-redis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A couple of minor suggestions but otherwise LGTM.

Comment thread content/integrate/redis-data-integration/data-pipelines/pipeline-config.md Outdated
Comment thread content/integrate/redis-data-integration/installation/upgrade.md Outdated

The classic processor silently ignores `processors.advanced`, so keeping
both top-level properties and their `processors.advanced` equivalents lets
you switch back without further edits.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
you switch back without further edits.
you switch back without further edits. Likewise, the Flink processor silently ignores the settings that only apply under classic.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I proposed a slightly different sentence, hopefully still ok.

@andy-stark-redis andy-stark-redis merged commit a645dca into redis:main May 11, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants