Skip to content

Add model package support#2369

Open
xiaoyu-work wants to merge 13 commits intomainfrom
xiaoyu/context_binary
Open

Add model package support#2369
xiaoyu-work wants to merge 13 commits intomainfrom
xiaoyu/context_binary

Conversation

@xiaoyu-work
Copy link
Copy Markdown
Collaborator

@xiaoyu-work xiaoyu-work commented Mar 24, 2026

This pull request introduces support for a new "Model Package" feature, which enables handling and packaging multiple model variants (e.g., for different hardware targets or deployment configurations) as a single logical unit. It adds a new ModelPackageModelHandler, a corresponding CLI command, and extends the pass system to recognize and process model packages. Several parts of the codebase are updated to support this new functionality, including improvements to file handling and documentation.

Model Package Support

  • Added ModelPackageModelHandler in olive/model/handler/model_package.py, allowing the management of multiple deployment variants of the same model as a single package. This handler enforces that all variants are of the same type and provides utilities for serialization and variant iteration.
  • Registered ModelPackageModelHandler in the model handler registry and __init__.py to make it available throughout Olive. [1] [2]
  • Added a new ModelPackage pass to the configuration and documentation, supporting the creation and management of model packages. [1] [2]
  • Added a new CLI command (generate-model-package) for model package operations and updated documentation to describe its usage. [1] [2] [3]

Pass System Enhancements

  • Updated the Pass base class to add _accepts_model_package_model and _skip_additional_files_carry_forward flags, enabling passes to control how model packages are processed and how files are handled.
  • Modified the pass execution logic to automatically run passes independently on each model package variant unless the pass opts in to handle the package as a whole. The output is then reassembled as a new ModelPackageModelHandler. [1] [2]
  • Improved file carry-forward logic to support directories and to allow passes to skip this step if needed. [1] [2]
  • Updated the EPContextBinaryGenerator pass to optionally generate context binaries for multiple provider options and return a model package when a list of options is provided. [1] [2] [3]

Model and File Handling Improvements

  • Enhanced save_model in olive/cache.py to properly save model packages, copying only relevant subdirectories and updating paths and manifest information accordingly.
  • Updated output directory logic in engine.py to better distinguish between file and directory outputs, ensuring correct artifact placement for model packages. [1] [2]

Miscellaneous

  • Adjusted node exclusion pattern for matmul quantization in optimize.py.

These changes collectively enable Olive to support advanced deployment workflows where a single model package can encapsulate multiple optimized variants, streamlining model management and deployment for heterogeneous environments.

ORT reference: microsoft/onnxruntime#27786

@xiaoyu-work xiaoyu-work force-pushed the xiaoyu/context_binary branch 2 times, most recently from 48f1507 to 3749c93 Compare March 27, 2026 03:50
@xiaoyu-work xiaoyu-work marked this pull request as ready for review March 27, 2026 03:58
Copilot AI review requested due to automatic review settings March 27, 2026 03:58
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds first-class “multi-target” support to Olive so a single workflow can produce and package multiple context-binary variants (e.g., multiple QNN SoCs or multiple OpenVINO versions) under one logical model.

Changes:

  • Introduces MultiTargetModelHandler and pass-level auto-dispatch for passes that don’t natively accept multi-target models.
  • Extends context-binary generation passes (QNN + OpenVINO encapsulation) to emit multi-target outputs and persist relevant metadata in model_attributes.
  • Adds packaging utilities: ModelPackager pass (manifest generation) and a new olive model-package CLI command (merge multiple single-target outputs).

Reviewed changes

Copilot reviewed 20 out of 20 changed files in this pull request and generated 8 comments.

Show a summary per file
File Description
test/passes/openvino/test_openvino_encapsulation.py Adds tests for OpenVINO encapsulation multi-target behavior.
test/passes/onnx/test_multi_target_context_binary.py Adds tests for ModelPackager and generic multi-target pass dispatch.
test/passes/onnx/test_context_binary.py Adds multi-target tests for EPContextBinaryGenerator.
test/model/test_multi_target_model.py Adds unit tests for MultiTargetModelHandler JSON + attribute behavior.
test/cli/test_model_package.py Adds tests for new olive model-package command.
olive/systems/system_config.py Pydantic serialization adjustment for system config (SerializeAsAny).
olive/passes/openvino/optimum_intel.py Adds TMPDIR/tempdir workaround + improved exception chaining.
olive/passes/openvino/encapsulation.py Adds OpenVINO multi-target encapsulation and metadata population.
olive/passes/onnx/model_packager.py New pass to generate manifest.json for multi-target packages.
olive/passes/onnx/model_builder.py Improves ImportError chaining.
olive/passes/onnx/context_binary.py Adds QNN multi-target provider-options support + metadata population.
olive/passes/olive_pass.py Adds _accepts_multi_target_model and multi-target auto-dispatch in Pass.run.
olive/olive_config.json Registers new ModelPackager pass.
olive/model/handler/multi_target.py New MultiTargetModelHandler implementation.
olive/model/handler/init.py Exposes MultiTargetModelHandler from handler package.
olive/engine/engine.py Changes output path handling for artifacts directory selection.
olive/cli/optimize.py Updates excluded node name for a quantization-related config.
olive/cli/model_package.py New CLI command to merge outputs into a multi-target package manifest.
olive/cli/launcher.py Registers model-package subcommand.
olive/cache.py Adds save logic for multi-target cached models + changes output path handling.

@xiaoyu-work xiaoyu-work force-pushed the xiaoyu/context_binary branch from d357f93 to 397923c Compare April 9, 2026 21:00
@xiaoyu-work xiaoyu-work changed the title Add multi context binary package support Add model package support Apr 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants