Add file_extension fields to BlobType by ddl-rliu · Pull Request #3406 · flyteorg/flytekit

ddl-rliu · 2026-03-12T23:02:02Z

Tracking issue

Closes flyteorg/flyte#7024 [BUG] [copilot] File extensions are missing when copilot downloads Blob/FlyteFile inputs

Why are the changes needed?

(Keeping this PR in draft until flyteorg/flyte#7009 is merged)

After flyteorg/flyte#7009 merges, adding the new file_extension field to BlobType, this flytekit PR will enable users to configure the file extension on FlyteFile inputs. This addresses the issue where file extensions are missing when copilot downloads Blob/FlyteFile inputs.

What changes were proposed in this pull request?

Add the file_extension field to BlobType. Add annotation "FileExtension", which is used to annotate a FlyteFile when we want to download the file with a specific extension. For example,

# ContainerTask
def t1(file: Annotated[FlyteFile, FileExtension("csv")]):
    ... # copilot downloads the file to e.g. /inputs/file.csv

versus...

def t1(file: FlyteFile["csv"]):
    ... # copilot downloads the file to e.g. /inputs/file

Under the hood, this sets file_extension in BlobType.

How was this patch tested?

Setup process

Ran a workflow locally to test the changes

# Container task python code
def t1(datasetA: Annotated[FlyteFile[TypeVar('csv')], FileExtension("csv")], datasetB: FlyteFile[TypeVar('csv')]): …

# flyte-copilot logs
2026-04-09T22:15:08.506Z: {"json":{},"level":"info","msg":"Successfully copied [1703936] bytes remote data from [s3://ls-engc-flyte-data//25d4f8d8-7eb4-492b-97c4-ab3ce8bdcf74/ae] to local [/execution-vol/flows/workflow/inputs/datasetA.csv]","ts":"2026-04-09T22:15:08Z"}
2026-04-09T22:15:08.540Z: {"json":{},"level":"info","msg":"Successfully copied [196608] bytes remote data from [s3://ls-engc-flyte-data//858fd656-11ec-41ec-b235-b3b2984f230e/ex] to local [/execution-vol/flows/workflow/inputs/datasetB]","ts":"2026-04-09T22:15:08Z"}

Screenshots

Check all the applicable boxes

I updated the documentation accordingly.
All new and existing tests passed.
All commits are signed-off.

Related PRs

Docs link

ddl-rliu · 2026-04-09T22:21:37Z

    "docker>=4.0.0",
    "docstring-parser>=0.9.0",
-    "flyteidl>=1.16.1,<2.0.0a0",
+    "flyteidl @ git+https://github.com/ddl-rliu/flyte.git@1ba7c1545198a2820348323e64c23a41a19e7a7d#subdirectory=flyteidl",


Bump this after flyteorg/flyte#7009 merges

ddl-rliu · 2026-04-09T22:21:54Z

    return None


+class FileExtension:


Follows same pattern as BatchSize:

flytekit/flytekit/core/type_engine.py

Line 75 in 71194a4

class BatchSize:

Port new BlobType fields file_extension and enable_legacy_filename to flytekit. FlyteFile inputs can be annotated with the FileDownloadConfig annotation to configure the file extension to use during the copilot download phase. e.g. ```python def t1(file: Annotated[FlyteFile, FileDownloadConfig(file_extension="csv")]): ... # copilot downloads the file to e.g. /inputs/file.csv versus... def t1(file: FlyteFile["csv"]): ... # copilot downloads the file to e.g. /inputs/file ``` Signed-off-by: ddl-rliu <richard.liu@dominodatalab.com>

Signed-off-by: ddl-rliu <richard.liu@dominodatalab.com>

codecov · 2026-04-10T00:01:18Z

Codecov Report

❌ Patch coverage is 45.71429% with 19 lines in your changes missing coverage. Please review.
✅ Project coverage is 39.44%. Comparing base (c164f35) to head (3edf6e4).
⚠️ Report is 3 commits behind head on master.

Files with missing lines	Patch %	Lines
flytekit/core/type_engine.py	37.50%	10 Missing ⚠️
flytekit/types/file/file.py	25.00%	9 Missing ⚠️

❗ There is a different number of reports uploaded between BASE (c164f35) and HEAD (3edf6e4). Click for more details.

HEAD has 24 uploads less than BASE

Flag BASE (c164f35) HEAD (3edf6e4)

25 1

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #3406      +/-   ##
==========================================
- Coverage   45.78%   39.44%   -6.34%     
==========================================
  Files         317      216     -101     
  Lines       28359    22873    -5486     
  Branches     3015     3021       +6     
==========================================
- Hits        12983     9022    -3961     
+ Misses      15278    13755    -1523     
+ Partials       98       96       -2

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

machichima · 2026-04-20T08:00:45Z

+    def __init__(self, val: str = ""):
+        self._val = val
+
+        pattern = r"^[a-zA-Z0-9]+(\.[a-zA-Z0-9]+)*$"


nit: Could we add a comment saying that this matches single and multi-part file extension (e.g. tar.gz)?

machichima · 2026-04-20T08:17:59Z

+        ... # copilot downloads the file to e.g. /inputs/file
+    ```
+
+    val: (Default is "") The file extension (e.g. "csv", "parquet") to use during copilot download.


Let's move this docstring to __init__ below. And change it to:

val: The file extension (e.g. "csv", "parquet") to use during copilot download.

machichima · 2026-04-20T08:18:57Z

+    val: (Default is "") The file extension (e.g. "csv", "parquet") to use during copilot download.
+    """
+
+    def __init__(self, val: str = ""):


Suggested change

def __init__(self, val: str = ""):

def __init__(self, val: str):

val cannot be "", which will not pass following regex check

machichima · 2026-04-20T08:20:44Z

+        return _types_pb2.BlobType(
+            format=self.format,
+            dimensionality=self.dimensionality,
+            file_extension=self._file_extension,


Suggested change

file_extension=self._file_extension,

file_extension=self.file_extension,

nit

machichima · 2026-04-20T08:46:32Z

+    ".csv",
+    "my file",
+    "../../escape",
+    "csv!",


Let's also add "" here

ddl-rliu requested review from cosmicBboy, davidmirror-ops, kumare3, machichima, pingsutw, samhita-alla and wild-endeavor as code owners March 12, 2026 23:02

ddl-rliu mentioned this pull request Mar 12, 2026

Add file_extension field to BlobType flyteorg/flyte#7009

Open

3 tasks

ddl-rliu marked this pull request as draft March 12, 2026 23:06

ddl-rliu commented Mar 12, 2026

View reviewed changes

Comment thread pyproject.toml Outdated

ddl-rliu commented Mar 12, 2026

View reviewed changes

Comment thread flytekit/core/type_engine.py Outdated

ddl-rliu mentioned this pull request Mar 12, 2026

[BUG] [copilot] File extensions are missing when copilot downloads Blob/FlyteFile inputs flyteorg/flyte#7024

Open

2 tasks

ddl-rliu force-pushed the rliu.DOM-75010.file-ext2 branch from caaa6f8 to 8ec4ae4 Compare March 12, 2026 23:22

ddl-rliu changed the title ~~Add file_extension, enable_legacy_filename fields to BlobType~~ Add file_extension fields to BlobType Apr 9, 2026

ddl-rliu marked this pull request as ready for review April 9, 2026 22:20

ddl-rliu commented Apr 9, 2026

View reviewed changes

ddl-rliu marked this pull request as draft April 9, 2026 23:04

ddl-rliu force-pushed the rliu.DOM-75010.file-ext2 branch 2 times, most recently from 52b0c79 to ce58c01 Compare April 9, 2026 23:53

ddl-rliu added 5 commits April 9, 2026 16:53

Add regex match for file_extension

6fa1207

Signed-off-by: ddl-rliu <richard.liu@dominodatalab.com>

Remove enable_legacy_filename

04a502c

Signed-off-by: ddl-rliu <richard.liu@dominodatalab.com>

Fix lint

d9b5c7b

Signed-off-by: ddl-rliu <richard.liu@dominodatalab.com>

Bump commit

3edf6e4

Signed-off-by: ddl-rliu <richard.liu@dominodatalab.com>

ddl-rliu force-pushed the rliu.DOM-75010.file-ext2 branch from ce58c01 to 3edf6e4 Compare April 9, 2026 23:53

machichima reviewed Apr 20, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add file_extension fields to BlobType#3406

Add file_extension fields to BlobType#3406
ddl-rliu wants to merge 5 commits intoflyteorg:masterfrom
ddl-rliu:rliu.DOM-75010.file-ext2

ddl-rliu commented Mar 12, 2026 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

ddl-rliu Apr 9, 2026

Uh oh!

ddl-rliu Apr 9, 2026

Uh oh!

codecov Bot commented Apr 10, 2026 •

edited

Loading

Uh oh!

machichima Apr 20, 2026

Uh oh!

machichima Apr 20, 2026

Uh oh!

machichima Apr 20, 2026

Uh oh!

machichima Apr 20, 2026

Uh oh!

machichima Apr 20, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

	def __init__(self, val: str = ""):
	def __init__(self, val: str):

	file_extension=self._file_extension,
	file_extension=self.file_extension,

Conversation

ddl-rliu commented Mar 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Tracking issue

Why are the changes needed?

What changes were proposed in this pull request?

How was this patch tested?

Setup process

Screenshots

Check all the applicable boxes

Related PRs

Docs link

Uh oh!

Uh oh!

Uh oh!

ddl-rliu Apr 9, 2026

Choose a reason for hiding this comment

Uh oh!

ddl-rliu Apr 9, 2026

Choose a reason for hiding this comment

Uh oh!

codecov Bot commented Apr 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

machichima Apr 20, 2026

Choose a reason for hiding this comment

Uh oh!

machichima Apr 20, 2026

Choose a reason for hiding this comment

Uh oh!

machichima Apr 20, 2026

Choose a reason for hiding this comment

Uh oh!

machichima Apr 20, 2026

Choose a reason for hiding this comment

Uh oh!

machichima Apr 20, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

ddl-rliu commented Mar 12, 2026 •

edited

Loading

codecov Bot commented Apr 10, 2026 •

edited

Loading