Repository navigation
Add the testing property of pipelines to bundles #7300
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,19 @@ | ||
| bundle: | ||
| name: pipeline-testing | ||
|
|
||
| resources: | ||
| pipelines: | ||
| orders: | ||
| name: orders | ||
| libraries: | ||
| - glob: | ||
| include: ./transformations/** | ||
| testing: | ||
| sources: | ||
| - glob: | ||
| include: ./tests/** | ||
| environment: | ||
| dependencies: | ||
| - -e .[test] | ||
| - -r ./tests/requirements.txt | ||
| - ./dist/*.whl |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,2 @@ | ||
| Cloud = false | ||
| EnvMatrix.DMS = ["", "true"] |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,18 @@ | ||
|
|
||
| >>> [CLI] bundle validate -o json | ||
| { | ||
| "environment": { | ||
| "dependencies": [ | ||
| "-e .[test]", | ||
| "-r ./tests/requirements.txt", | ||
| "./dist/*.whl" | ||
| ] | ||
| }, | ||
| "sources": [ | ||
| { | ||
| "glob": { | ||
| "include": "./tests/**" | ||
| } | ||
| } | ||
| ] | ||
| } |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,6 @@ | ||
| [project] | ||
| name = "orders" | ||
| version = "0.1" | ||
|
|
||
| [project.optional-dependencies] | ||
| test = ["faker"] |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| trace $CLI bundle validate -o json | jq .resources.pipelines.orders.testing | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,4 @@ | ||
| # Runs only against the local test server. | ||
| Cloud = false | ||
| RecordRequests = true | ||
| Ignore = [".databricks"] |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| faker |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,2 @@ | ||
| def test_orders(): | ||
| pass |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| from pyspark import pipelines as dp |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -1465,6 +1465,20 @@ resources: | |
| "user_name": | ||
| "description": |- | ||
| The name of the user granted the permission level. | ||
| "testing": | ||
| "description": |- | ||
| The test configuration of the pipeline. Only test-only updates, such as those started by `databricks pipelines test`, use it. | ||
| "$fields": | ||
| "environment": | ||
| "$fields": | ||
| "dependencies": | ||
| "description": |- | ||
| A list of files that declare the packages the tests need: a project folder, a requirements file, or a wheel, in the workspace or a UC volume. A project folder can take `-e` and an extra, for example `-e ./project[test]`. A relative path, such as `-e ./project[test]` or `-r ./tests/requirements.txt`, is relative to the YAML file that defines it. Package names and URLs are rejected. | ||
| "sources": | ||
| "$fields": | ||
| "glob": | ||
| "description": |- | ||
| A `.py` test file, or a folder followed by `/**`. Other wildcards are rejected, and the sources must not overlap the pipeline's `libraries`. A relative path is relative to the YAML file that defines it. | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We don't need the override; the description comes from upstream.
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Agreed, removed. The descriptions now come from the SDK. |
||
| "trigger": | ||
| "deprecation_message": |- | ||
| Use continuous instead | ||
|
|
||
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What is this test testing now that the property comes from the SDK spec?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
On its own it covered nothing beyond the SDK type. It now lives in #7437, where it checks the translated test paths and the uploaded wheel.