Add OADP toolset for managing Velero backups and restores

[shubham-pampattiwar]

Summary

Adds OADP (OpenShift API for Data Protection) toolset to the kubernetes-mcp-server, enabling AI agents to create, monitor, and manage data protection workflows with domain-specific intelligence.

Fixes https://redhat.atlassian.net/browse/OADP-7194

Tools (8 consolidated action-based tools)

Following the kubevirt pattern, each tool uses an action parameter to select the operation:

Tool	Actions	Description
oadp_backup	list, get, create, delete, status	Manage Velero backups (auto-discovers BSL, detects backup method from DPA)
oadp_restore	list, get, create, delete, status	Manage Velero restores (validates backup phase before restore)
oadp_schedule	list, get, create, update, delete, pause	Manage backup schedules (pause action with paused=true/false)
oadp_storage_location	list, get, create, update, delete	Manage BSL and VSL storage locations
oadp_dpa	list, get, create, update, delete	Manage DataProtectionApplication configuration
oadp_repository	list, get, delete	Manage backup repositories
oadp_data_mover	list, get, cancel	Manage DataUpload/DataDownload for CSI snapshots
oadp_data_protection_test	list, get, create, delete	Manage data protection tests

Prompt

Prompt	Description
oadp-troubleshoot	Curated diagnostic workflow: gathers DPA, BSL, backup/restore status, Velero pod health, logs, and events

Domain Intelligence (beyond generic CRUD)

• BSL auto-discovery — backup create auto-finds the default/available BSL when none specified
• Backup method detection — reads DPA config, auto-sets defaultVolumesToFsBackup when NodeAgent is enabled
• DeleteBackupRequest pattern — delete action creates a DeleteBackupRequest CRD (proper Velero cleanup) instead of direct CR deletion which orphans data in object storage
• Namespace validation — warns if includedNamespaces don't exist before backup creation
• Restore context — checks backup phase/contents before restore, warns on Failed/InProgress backups
• Pre-flight checks — advisory warnings on create operations (DPA exists, BSL available)

Example Usage

// Create a backup (BSL auto-discovered, backup method auto-detected)
{"name": "oadp_backup", "arguments": {"action": "create", "name": "my-backup", "includedNamespaces": ["app-ns"]}}

// Get backup status
{"name": "oadp_backup", "arguments": {"action": "status", "name": "my-backup"}}

// Create a scheduled backup
{"name": "oadp_schedule", "arguments": {"action": "create", "name": "daily-backup", "schedule": "0 3 * * *", "includedNamespaces": ["app-ns"]}}

// Pause a schedule
{"name": "oadp_schedule", "arguments": {"action": "pause", "name": "daily-backup", "paused": true}}

Test plan

• Unit tests pass (go test -v ./pkg/oadp/... ./pkg/toolsets/oadp/...)
• Build passes with no lint errors (make build && make lint)
• Tested with mcp-inspector against live OpenShift cluster with OADP installed
• 11/11 evals pass (mcpchecker, core-only vs core+oadp comparison)
• TestDeleteBackupCreatesDeleteBackupRequest proves proper Velero deletion pattern

Changes in this PR

• Add OADP toolset with 8 consolidated tools using action parameter pattern
• Add oadp-troubleshoot prompt for guided diagnostics (same pattern as kubevirt vm-troubleshoot)
• Add domain intelligence: BSL auto-discovery, backup method detection, namespace validation, restore context, pre-flight checks
• Add 11 eval tasks including auto-BSL, pause-schedule, and troubleshoot scenarios
• Add core-only eval config for comparison testing
• Add comprehensive unit tests including DeleteBackupRequest verification
• Update README, docs/OADP.md, and docs/configuration.md

Summary by CodeRabbit

• New Features

  • Added comprehensive OADP (OpenShift API for Data Protection) toolset including eight management tools: backup creation/deletion/status checks, restore operations, backup schedules, data protection applications, storage location configuration, data mover operations, backup repositories, and data protection tests.
  • Added OADP troubleshooting prompt with diagnostic capabilities.

• Documentation

  • Added complete OADP toolset documentation with configuration guides, prerequisites, use-case examples, and security considerations.
  • Updated toolset registry and configuration documentation.

• Tests

  • Added evaluation tasks for OADP workflows including backup/restore/schedule creation, status checks, and troubleshooting scenarios.

[matzew]

/assign matzew

[weshayutin]

woot! thanks @shubham-pampattiwar

[Cali0707]

/assign @Cali0707

[Cali0707]

@shubham-pampattiwar thanks for contributing a new toolset! A few thoughts:

Provides 90 tools covering all 23 CRDs shipped by OADP

Our experience with the core toolset and others so far is that in general we want to try to minimize the number of tools in the server, so that we don't overwhelm the model's context. E.g. cursor will return a warning to users when somewhere in the realm of 20-60 tools are loaded (and this is providing 90!)

IMO we should be working to drastically reduce the number of tools provided by this toolset. A few things that can likely help:

1. For any CRUD operations on CRs, there is the resources_<create|update|delete|list> tools from the core toolset, that can likely replace all your CR CRUD tools. Maybe we need to provide the model with more context on how to use your CRs specifically, but that is something we should look for in evals
2. For operations that involve more than a simple CRUD operation against the kube API server, or which needs to use verbs not exposed through the generic resource tools (I'm thinking something around these ones: oadp_data_upload_list/get/cancel, oadp_data_download_list/get/cancel, specifically the cancel bit), let's try to make the verbs part of the parameters of the tool.

For example, in the kubevirt toolset they made the action to be done to a VM a parameter of the tool, rather than having one tool for each action:

openshift-mcp-server/pkg/toolsets/kubevirt/vm/lifecycle/tool.go

Lines 40 to 44 in f13d475

	"action": {
	Type: "string",
	Enum: []any{string(ActionStart), string(ActionStop), string(ActionRestart)},
	Description: "The lifecycle action to perform: 'start' (changes runStrategy to Always), 'stop' (changes runStrategy to Halted), or 'restart' (stops then starts the VM)",
	},

Happy to discuss further on how to reduce the count of the tools while still ensuring agents can take all the actions they need to!

[shubham-pampattiwar]

@shubham-pampattiwar thanks for contributing a new toolset! A few thoughts:

Provides 90 tools covering all 23 CRDs shipped by OADP

Our experience with the core toolset and others so far is that in general we want to try to minimize the number of tools in the server, so that we don't overwhelm the model's context. E.g. cursor will return a warning to users when somewhere in the realm of 20-60 tools are loaded (and this is providing 90!)

IMO we should be working to drastically reduce the number of tools provided by this toolset. A few things that can likely help:

1. For any CRUD operations on CRs, there is the resources_<create|update|delete|list> tools from the core toolset, that can likely replace all your CR CRUD tools. Maybe we need to provide the model with more context on how to use your CRs specifically, but that is something we should look for in evals
2. For operations that involve more than a simple CRUD operation against the kube API server, or which needs to use verbs not exposed through the generic resource tools (I'm thinking something around these ones: oadp_data_upload_list/get/cancel, oadp_data_download_list/get/cancel, specifically the cancel bit), let's try to make the verbs part of the parameters of the tool.

For example, in the kubevirt toolset they made the action to be done to a VM a parameter of the tool, rather than having one tool for each action:

openshift-mcp-server/pkg/toolsets/kubevirt/vm/lifecycle/tool.go

Lines 40 to 44 in f13d475

	"action": {
	Type: "string",
	Enum: []any{string(ActionStart), string(ActionStop), string(ActionRestart)},
	Description: "The lifecycle action to perform: 'start' (changes runStrategy to Always), 'stop' (changes runStrategy to Halted), or 'restart' (stops then starts the VM)",
	},

Happy to discuss further on how to reduce the count of the tools while still ensuring agents can take all the actions they need to!

Hey @Cali0707, thanks for the feedback! Makes sense - 90 tools is definitely too many.

Here's my plan to reduce to ~9 tools using the action parameter pattern like kubevirt:

┌───────────────────────┬─────────────────────────────┬──────────────────────────────────────────┐                                                                       
│         Tool          │          Resources          │                 Actions                  │                                                                       
├───────────────────────┼─────────────────────────────┼──────────────────────────────────────────┤                                                                       
│ oadp_backup           │ Backup                      │ list, get, create, delete, logs          │                                                                       
├───────────────────────┼─────────────────────────────┼──────────────────────────────────────────┤                                                                       
│ oadp_restore          │ Restore                     │ list, get, create, delete, logs          │                                                                       
├───────────────────────┼─────────────────────────────┼──────────────────────────────────────────┤                                                                       
│ oadp_schedule         │ Schedule                    │ list, get, create, update, delete, pause │                                                                       
├───────────────────────┼─────────────────────────────┼──────────────────────────────────────────┤                                                                       
│ oadp_dpa              │ DataProtectionApplication   │ list, get, create, update, delete        │                                                                       
├───────────────────────┼─────────────────────────────┼──────────────────────────────────────────┤                                                                       
│ oadp_storage_location │ BSL + VSL (type param)      │ list, get, create, update, delete        │                                                                       
├───────────────────────┼─────────────────────────────┼──────────────────────────────────────────┤                                                                       
│ oadp_data_mover       │ DataUpload + DataDownload   │ list, get, cancel                        │                                                                       
├───────────────────────┼─────────────────────────────┼──────────────────────────────────────────┤                                                                       
│ oadp_repository       │ BackupRepository            │ list, get, delete                        │                                                                       
├───────────────────────┼─────────────────────────────┼──────────────────────────────────────────┤                                                                       
│ oadp_non_admin        │ All NonAdmin* resources     │ list, get, create, delete, approve       │                                                                       
├───────────────────────┼─────────────────────────────┼──────────────────────────────────────────┤                                                                       
│ oadp_vm_restore       │ VM discovery + file restore │ list, get, create, delete                │                                                                       
└───────────────────────┴─────────────────────────────┴──────────────────────────────────────────┘                                                                       

Less common CRDs (DeleteBackupRequest, DownloadRequest, ServerStatusRequest, PodVolumeBackup/Restore) can use generic resources_* tools.

Let me know if this approach works!

[matzew]

Reducing the number of tools sounds like a good idea, as well leveraging core tools (e.g. resources_*) for generic operations seem good.

A good idea is to also look at providing evals, like other toolsets (kubevirt for instance)

We use mcpchecker, which now has also quickstarts for get going!

https://github.com/mcpchecker

[openshift-ci]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: shubham-pampattiwar
Once this PR has been reviewed and has the lgtm label, please ask for approval from cali0707. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[shubham-pampattiwar]

Addressed review feedback - refactored from 90 individual tools to 8 consolidated tools using the action parameter pattern (like kubevirt).

Changes:

• oadp_backup - actions: list, get, create, delete, logs
• oadp_restore - actions: list, get, create, delete, logs
• oadp_schedule - actions: list, get, create, delete, pause, unpause
• oadp_storage_location - actions: list, get
• oadp_dpa - actions: list, get
• oadp_repository - actions: list, get
• oadp_data_mover - actions: list, get
• oadp_data_protection_test - actions: list, get, create, delete

Also added 8 eval tasks - all passing (8/8 tasks, 24/24 assertions).

Ready for re-review @Cali0707 @matzew

[shubham-pampattiwar]

Gentle ping @Cali0707 @matzew - any feedback on the refactored approach?

[Cali0707]

@shubham-pampattiwar the toolset looks much better now! I left a few comments around the code, but overall this is looking great

[shubham-pampattiwar]

Addressed review comments:

Removed unused files:

• pkg/oadp/nonadmin.go, vmrestore.go, cloudstorage.go, downloadrequest.go, podvolume.go, serverstatus.go, deletebackuprequest.go (and tests)

Code changes:

• Renamed logs action to status for backup/restore tools (returns status info, not actual logs - avoids confusing the agent)
• Using api.OptionalString() helper instead of manual param parsing
• Fixed parseLabelSelector to return error on non-equality selectors (e.g., !=, in, notin)
• Fixed comment: 8 tools instead of 10

Rebased on main to resolve merge conflict with netedge toolset.

All tests passing.

[shubham-pampattiwar]

Ready for re-review @Cali0707 @matzew - addressed all comments.

[shubham-pampattiwar]

Hi @Cali0707 @matzew - friendly follow-up on this PR. All review comments have been addressed. Would appreciate a re-review when you get a chance. Thanks!

[Cali0707]

@shubham-pampattiwar I'm curious - did you run the evals with just the core toolset enabled vs. this new toolset? Most of the tools seem to be CRUD tools, and on frontier models they can often succeed with the generic resource tools for many domains (but not all, which is why evals are so useful)

[coderabbitai]

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

• @coderabbitai resume to resume automatic reviews.
• @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

• ▶️ Resume reviews
• 🔍 Trigger review

📝 Walkthrough

Walkthrough

Adds a new OADP toolset: dynamic-client helpers, server tools and a troubleshooting prompt for Velero/OADP resources; preflight discovery; many eval tasks and scripts; documentation, README registration, and comprehensive unit tests.

Changes

Cohort / File(s)	Summary
Repo config & docs \.gitignore, README.md, docs/OADP.md, docs/README.md, docs/configuration.md	Added .notes/ to .gitignore; introduced OADP docs, README/toolset entries, and configuration docs.
Eval manifests & configs evals/claude-code/eval-oadp-core-only.yaml, evals/claude-code/eval.yaml, evals/openai-agent/eval.yaml	New Eval and taskSet entries referencing ../tasks/oadp/*/*.yaml with Kubernetes tool assertions and tool-call limits.
Eval tasks & scripts evals/tasks/oadp/...	Added many OADP evaluation Tasks and their setup.sh/verify.sh/cleanup.sh scripts (backups, restores, schedules, storage locations, data mover, status/list/pause/troubleshoot scenarios).
Module & README registration internal/tools/update-readme/main.go, pkg/mcp/modules.go	Blank imports added so the OADP toolset is registered for module init and README generation.
OADP core package pkg/oadp/resources.go, pkg/oadp/*.go, pkg/oadp/*_test.go	New package exposing GVRs/constants and dynamic-client helpers for backups/restores/schedules/BSL/VSL/DPA/DPT/backup-repo/datauploads/datadownloads, plus unit tests using a fake dynamic client.
Toolset implementations pkg/toolsets/oadp/*.go, pkg/toolsets/oadp/*_test.go	Implemented server tools and prompt: oadp_backup, oadp_restore, oadp_schedule, oadp_storage_location, oadp_dpa, oadp_data_mover, oadp_repository, oadp_data_protection_test, preflight discovery, oadp-troubleshoot prompt, toolset registration and tests.

Sequence Diagram(s)

sequenceDiagram
    participant Client
    participant MCPServer as "MCP Server (oadp tool)"
    participant K8s as "Kubernetes API (dynamic client)"
    participant Controller as "OADP/Velero Controller"

    Client->>MCPServer: Request (e.g., create backup with params)
    MCPServer->>K8s: discoverBackupDefaults (list DPA, list BSL)
    K8s-->>MCPServer: DPA/BSL results
    MCPServer->>K8s: Create Backup (BackupGVR) unstructured
    K8s-->>MCPServer: Created Backup object
    Controller->>K8s: Process Backup (async) -> update status
    K8s-->>MCPServer: Backup status updated
    MCPServer-->>Client: Return created object + preflight warnings/status

Loading

Estimated code review effort

🎯 5 (Critical) | ⏱️ ~120 minutes

"I hopped through code with curious paws,
Added backups, restores, and clever claws.
Schedules and prompts, tests all in line—
A rabbit's small hop, now OADP shines! 🐇"

🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 64.29% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (2 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title 'Add OADP toolset for managing Velero backups and restores' clearly summarizes the main change: introducing a new OADP toolset with capabilities to manage Velero backups and restores.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[shubham-pampattiwar]

Rebased on main, PTAL @Cali0707 @matzew, Thank you !

[Cali0707]

@shubham-pampattiwar do the evals need any specific set up on the cluster they run on? Is there any install steps for OADP that we need to do?

If so, can you add a make target for those?

[shubham-pampattiwar]

@Cali0707 yes, the evals need Velero + object storage on the cluster. I've added make targets for this:

Quick setup (Kind cluster + OADP):

make local-env-setup-oadp

Or on an existing cluster:

make oadp-install

This installs Velero with MinIO as the S3 backend in the openshift-adp namespace, applies the OADP CRDs (DPA, DataProtectionTest) from the oadp-operator repo, and creates a sample DPA resource. The BSL will be in Available state once the install completes.

Running the evals:

# Start server (note: --read-only=false is needed since the downstream default hides mutating tools)
./kubernetes-mcp-server --toolsets core,config,oadp --port 8080 --read-only=false

# Run OADP evals
mcpchecker check evals/claude-code/eval.yaml --run "oadp" --verbose

Other targets: make oadp-status, make oadp-uninstall

I tested end-to-end on a Kind cluster with mcpchecker — 11/11 tasks passed, 33/33 assertions passed.

[matzew]

PR has a merge conflict! Please address and fix

[matzew]

In pkg/oadp/backup.go, restore.go, and oadp_troubleshoot.go, the code uses _, _, _ to discard the (value, found, error) returns from unstructured.NestedString/NestedInt64.

We should not ignore those potential errors?

[Cali0707]

@shubham-pampattiwar when I run make oadp-install it times out on the minio deployment and I see:

kubectl get pods -n openshift-adp              
NAME                     READY   STATUS              RESTARTS   AGE
minio-765896f7b5-4h6sg   0/1     ImageInspectError   0          5m1s

More detailed error is:

containerStatuses:
    - image: minio/minio:latest
      imageID: ""
      lastState: {}
      name: minio
      ready: false
      restartCount: 0
      started: false
      state:
        waiting:
          message: 'Failed to inspect image "": rpc error: code = Unknown desc = short
            name mode is enforcing, but image name minio/minio:latest returns ambiguous
            list'
          reason: ImageInspectError

[shubham-pampattiwar]

In pkg/oadp/backup.go, restore.go, and oadp_troubleshoot.go, the code uses _, _, _ to discard the (value, found, error) returns from unstructured.NestedString/NestedInt64.

We should not ignore those potential errors?

@matzew Good catch, fixed. GetBackupStatus and GetRestoreStatus now return errors instead of silently using zero values. The troubleshoot prompt handlers default gracefully since they're building diagnostic output.

[shubham-pampattiwar]

@shubham-pampattiwar when I run make oadp-install it times out on the minio deployment and I see:

kubectl get pods -n openshift-adp              
NAME                     READY   STATUS              RESTARTS   AGE
minio-765896f7b5-4h6sg   0/1     ImageInspectError   0          5m1s

More detailed error is:

containerStatuses:
    - image: minio/minio:latest
      imageID: ""
      lastState: {}
      name: minio
      ready: false
      restartCount: 0
      started: false
      state:
        waiting:
          message: 'Failed to inspect image "": rpc error: code = Unknown desc = short
            name mode is enforcing, but image name minio/minio:latest returns ambiguous
            list'
          reason: ImageInspectError

@Cali0707 Fixed the MinIO images now use fully qualified names (docker.io/minio/minio:latest, docker.io/minio/mc:latest) to avoid CRI-O short-name ambiguity. Should work on your cluster now.

[Cali0707]

@shubham-pampattiwar do you have any red hat/ quay registry images for minio? @mvinkler has run into issues with docker images in our QE environment

[shubham-pampattiwar]

@shubham-pampattiwar do you have any red hat/ quay registry images for minio? @mvinkler has run into issues with docker images in our QE environment

@Cali0707 Switched to quay.io/minio/minio and quay.io/minio/mc images. Also aligned versions, Velero 1.16.2 + AWS plugin 1.12.2 to match OADP 1.5, and pinned CRDs to the oadp-1.5 branch. Should work in your QE environment now.

[openshift-ci]

@shubham-pampattiwar: all tests passed!

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

[matzew]

would this just work with the core toolset that we have? like crud against your resources - using the core toolset ?

[matzew]

Here too - would this just work w/ the tools from the core toolset ?

[matzew]

It would be interesting to check if the evals actually pass, using the core tools (E.g. we have CRUD operations for resources). So wondering if we really need to add the toolset - looks like, e.g. via pkg/oadp/schedule.go, there are a lot of CRUD operations on the new toolset here