-
Notifications
You must be signed in to change notification settings - Fork 1
feat: publish emulator image directly from testing library #196
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
SilanHe
wants to merge
63
commits into
main
Choose a base branch
from
feat/publish-emulator-image
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
63 commits
Select commit
Hold shift + click to select a range
3a9f1f7
feat: build emulator image
hsilan 354f0b5
chore: fix typo in workflows file ecr-release.yml
hsilan 11227c2
chore: change paths
hsilan 2c92361
chore: try changing to relative paths
hsilan a38e2a4
chore: use root directory for build
hsilan 9500f53
chore: specify build dir as emulator sub folder
hsilan 42366c1
chore: copy files to root of container
hsilan 8658345
chore: try copy all project files
hsilan 7010fae
chore: try different image
hsilan da9e837
chore: fix typo in ecr-release.yml
hsilan 890c4b9
chore: try different copy command
hsilan da5365a
chore: try allowing ssh to github
hsilan df1184d
chore: try
hsilan 0be8526
chore: use versions released to pip
hsilan d964e53
chore: remove typo from exr-release.yml
hsilan 98c65d6
chore: only use explicit tag on release
hsilan d1f3a2d
chore: enable hatch executable
hsilan a600ecc
chore: check the path of hatch executable
hsilan 7d4e0f1
chore: whereis hatch
hsilan 8cf6814
chore: use python3.13-slim
hsilan 942d191
chore: remove chmod +x
hsilan 73dafa5
chore: try directly using pip install and not hatch
hsilan 18e9356
chore: try older version of sdk and testing sdk
hsilan ce09c72
chore: try using github dependencies
hsilan 2e6f3ff
chore: try installing git into image
hsilan 30fdc78
chore: try apt-get upgrade
hsilan 1855f38
chore: remove on push trigger for the ecr-release.yml workflow
hsilan 72096f2
chore: depend on specific sdk and testing sdk version for clarity
hsilan 4476f4e
chore: update README with installation instructions
hsilan 781d053
chore: try lower version of hatch
hsilan 11718e8
chore: update hatch version
hsilan 172568f
chore: try upgrading virtual env
hsilan e64541c
chore: remove emulator pr workflow
hsilan 63ce50c
chore: revert hatch upgrade
hsilan 43ad4ca
chore: remove exclude on emulator folder
hsilan 1a2ad02
chore: downgrade virtualenv version
hsilan 91c0e16
chore: reformatted 3 files
hsilan 26e4cc9
chore: update ci.yml step name for virtual env install
hsilan c1e1e82
chore: try directly using dex-local-runner start-server command
hsilan 9898fba
chore: fetch package version after checkout and build
hsilan 2cb3a70
chore: downgrade virtualenv version
hsilan 64d521c
chore: export __version__ in __init__.py
hsilan 63de332
chore: try fetching version from init
hsilan 2796bcb
chore: fix typo in version command
hsilan 44d97f9
chore: try pip installing the project as well as fixing the path to t…
hsilan 08d30c4
chore: set docker build dir and path_to_dockerfile
hsilan 0c1f8b9
chore: temporary fix for virtualenv issue with hatch
hsilan 5e8cbc7
chore: try printing version
hsilan d31fa7f
chore: remove condition on explicit version tag manifest and images
hsilan a4446a3
chore: version is fetched from previous job
hsilan fb05a5d
chore: add v prefix to the version
hsilan 1ebdee6
chore: try with no docker HEALTHCHECK
hsilan e053b2b
chore: remove on push trigger for the ecr-release workflow
hsilan a4b3388
chore: fetch version via grep + change versionning format
hsilan dfde877
chore: remove virtualenv version pin
hsilan 4940bba
chore: remove virtualenv version pin
hsilan 5056b6a
chore: EXPOSE 9014 for image
hsilan fd7b087
chore: rename dockerfile
hsilan 19bb4a6
Merge branch 'main' into feat/publish-emulator-image
SilanHe 9e60d21
chore: removing push action trigger for feat/publish-emulator-image b…
hsilan 0963e26
chore: remove unused var
hsilan 4cb451e
chore: use readable repository alias name
hsilan 837cc37
Merge branch 'main' into feat/publish-emulator-image
hsilan File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,154 @@ | ||
| name: ecr-release.yml | ||
| on: | ||
| release: | ||
| types: [published] | ||
|
|
||
| permissions: | ||
| contents: read | ||
| id-token: write # This is required for requesting the JWT | ||
|
|
||
| env: | ||
| path_to_dockerfile: "Dockerfile" | ||
| docker_build_dir: "." | ||
| aws_region: "us-east-1" | ||
| ecr_repository_name: "durable-functions/aws-durable-execution-emulator" | ||
|
|
||
| jobs: | ||
| build-and-upload-image-to-ecr: | ||
| runs-on: ubuntu-latest | ||
| outputs: | ||
| full_image_arm64: ${{ steps.build-publish.outputs.full_image_arm64 }} | ||
| full_image_x86_64: ${{ steps.build-publish.outputs.full_image_x86_64 }} | ||
| ecr_registry_repository: ${{ steps.build-publish.outputs.ecr_registry_repository }} | ||
| version: ${{ steps.version.outputs.VERSION }} | ||
| strategy: | ||
| matrix: | ||
| include: | ||
| - arch: x86_64 | ||
| - arch: arm64 | ||
| steps: | ||
| - uses: actions/checkout@v6 | ||
| - name: Set up Python | ||
| uses: actions/setup-python@v6 | ||
| with: | ||
| python-version: "3.13" | ||
| - name: Install dependencies | ||
| run: | | ||
| python -m pip install --upgrade pip | ||
| pip install hatch | ||
| - name: Set up QEMU for multi-platform builds | ||
| if: matrix.arch == 'arm64' | ||
| uses: docker/setup-qemu-action@v3 | ||
| with: | ||
| platforms: arm64 | ||
| - name: Build distribution | ||
| run: hatch build | ||
| - name: Get version from __about__.py | ||
| id: version | ||
| run: | | ||
| VERSION=$(grep "^__version__" src/aws_durable_execution_sdk_python_testing/__about__.py | cut -d'"' -f2) | ||
| echo "VERSION=$VERSION" | ||
| echo "VERSION=${VERSION}" >> $GITHUB_OUTPUT | ||
| - name: Configure AWS Credentials | ||
| uses: aws-actions/configure-aws-credentials@v4 | ||
| with: | ||
| role-to-assume: ${{ secrets.ECR_UPLOAD_IAM_ROLE_ARN }} | ||
| aws-region: ${{ env.aws_region }} | ||
| - name: Login to Amazon ECR | ||
| id: login-ecr-public | ||
| uses: aws-actions/amazon-ecr-login@v2 | ||
| with: | ||
| registry-type: public | ||
| - name: Build, tag, and push image to Amazon ECR | ||
| id: build-publish | ||
| shell: bash | ||
| env: | ||
| ECR_REGISTRY: ${{ steps.login-ecr-public.outputs.registry }} | ||
| ECR_REPOSITORY: ${{ env.ecr_repository_name }} | ||
| PER_ARCH_IMAGE_TAG: "v${{ steps.version.outputs.VERSION }}-${{ matrix.arch }}" | ||
| run: | | ||
| if [ "${{ matrix.arch }}" = "x86_64" ]; then | ||
| docker build --platform linux/amd64 --provenance false "${{ env.docker_build_dir }}" -f "${{ env.path_to_dockerfile }}" -t "$ECR_REGISTRY/$ECR_REPOSITORY:$PER_ARCH_IMAGE_TAG" | ||
| else | ||
| docker build --platform linux/arm64 --provenance false "${{ env.docker_build_dir }}" -f "${{ env.path_to_dockerfile }}" -t "$ECR_REGISTRY/$ECR_REPOSITORY:$PER_ARCH_IMAGE_TAG" | ||
| fi | ||
| docker push "$ECR_REGISTRY/$ECR_REPOSITORY:$PER_ARCH_IMAGE_TAG" | ||
| echo "IMAGE $PER_ARCH_IMAGE_TAG is pushed to $ECR_REGISTRY/$ECR_REPOSITORY" | ||
| echo "image_tag=$PER_ARCH_IMAGE_TAG" | ||
| echo "full_image=$ECR_REGISTRY/$ECR_REPOSITORY:$PER_ARCH_IMAGE_TAG" | ||
| echo "ecr_registry_repository=$ECR_REGISTRY/$ECR_REPOSITORY" >> $GITHUB_OUTPUT | ||
| echo "full_image_${{ matrix.arch }}=$ECR_REGISTRY/$ECR_REPOSITORY:$PER_ARCH_IMAGE_TAG" >> $GITHUB_OUTPUT | ||
| create-ecr-manifest-per-arch: | ||
| runs-on: ubuntu-latest | ||
| needs: [build-and-upload-image-to-ecr] | ||
| steps: | ||
| - name: Grab image, registry/repository name, version from previous steps | ||
| id: ecr_names | ||
| env: | ||
| ECR_REGISTRY_REPOSITORY: ${{ needs.build-and-upload-image-to-ecr.outputs.ecr_registry_repository }} | ||
| FULL_IMAGE_ARM64: ${{ needs.build-and-upload-image-to-ecr.outputs.full_image_arm64 }} | ||
| FULL_IMAGE_X86_64: ${{ needs.build-and-upload-image-to-ecr.outputs.full_image_x86_64 }} | ||
| VERSION: ${{ needs.build-and-upload-image-to-ecr.outputs.version }} | ||
| run: | | ||
| echo "full_image_arm64=$FULL_IMAGE_ARM64" | ||
| echo "ecr_registry_repository=$ECR_REGISTRY_REPOSITORY" | ||
| echo "full_image_x86_64=$FULL_IMAGE_X86_64" | ||
| echo "version=$VERSION" | ||
| - name: Configure AWS Credentials | ||
| uses: aws-actions/configure-aws-credentials@v4 | ||
| with: | ||
| role-to-assume: ${{ secrets.ECR_UPLOAD_IAM_ROLE_ARN }} | ||
| aws-region: ${{ env.aws_region }} | ||
| - name: Login to Amazon ECR | ||
| id: login-ecr-public | ||
| uses: aws-actions/amazon-ecr-login@v2 | ||
| with: | ||
| registry-type: public | ||
| - name: Create ECR manifest with explicit tag | ||
| id: create-ecr-manifest-explicit | ||
| run: | | ||
| docker manifest create "${{ needs.build-and-upload-image-to-ecr.outputs.ecr_registry_repository }}:v${{ needs.build-and-upload-image-to-ecr.outputs.version }}" \ | ||
| "${{ needs.build-and-upload-image-to-ecr.outputs.full_image_x86_64 }}" \ | ||
| "${{ needs.build-and-upload-image-to-ecr.outputs.full_image_arm64 }}" | ||
| - name: Annotate ECR manifest with explicit arm64 tag | ||
| id: annotate-ecr-manifest-explicit-arm64 | ||
| run: | | ||
| docker manifest annotate "${{ needs.build-and-upload-image-to-ecr.outputs.ecr_registry_repository }}:v${{ needs.build-and-upload-image-to-ecr.outputs.version }}" \ | ||
| "${{ needs.build-and-upload-image-to-ecr.outputs.full_image_arm64 }}" \ | ||
| --arch arm64 \ | ||
| --os linux | ||
| - name: Annotate ECR manifest with explicit amd64 tag | ||
| id: annotate-ecr-manifest-explicit-amd64 | ||
| run: | | ||
| docker manifest annotate "${{ needs.build-and-upload-image-to-ecr.outputs.ecr_registry_repository }}:v${{ needs.build-and-upload-image-to-ecr.outputs.version }}" \ | ||
| "${{ needs.build-and-upload-image-to-ecr.outputs.full_image_x86_64 }}" \ | ||
| --arch amd64 \ | ||
| --os linux | ||
| - name: Push ECR manifest with explicit version | ||
| id: push-ecr-manifest-explicit | ||
| run: | | ||
| docker manifest push "${{ needs.build-and-upload-image-to-ecr.outputs.ecr_registry_repository }}:v${{ needs.build-and-upload-image-to-ecr.outputs.version }}" | ||
| - name: Create ECR manifest with latest tag | ||
| id: create-ecr-manifest-latest | ||
| run: | | ||
| docker manifest create "${{ needs.build-and-upload-image-to-ecr.outputs.ecr_registry_repository }}" \ | ||
| "${{ needs.build-and-upload-image-to-ecr.outputs.full_image_arm64 }}" \ | ||
| "${{ needs.build-and-upload-image-to-ecr.outputs.full_image_x86_64 }}" | ||
| - name: Annotate ECR manifest with latest tag arm64 | ||
| id: annotate-ecr-manifest-latest-arm64 | ||
| run: | | ||
| docker manifest annotate "${{ needs.build-and-upload-image-to-ecr.outputs.ecr_registry_repository }}" \ | ||
| "${{ needs.build-and-upload-image-to-ecr.outputs.full_image_arm64 }}" \ | ||
| --arch arm64 \ | ||
| --os linux | ||
| - name: Annotate ECR manifest with latest tag amd64 | ||
| id: annotate-ecr-manifest-latest-amd64 | ||
| run: | | ||
| docker manifest annotate "${{ needs.build-and-upload-image-to-ecr.outputs.ecr_registry_repository }}" \ | ||
| "${{ needs.build-and-upload-image-to-ecr.outputs.full_image_x86_64 }}" \ | ||
| --arch amd64 \ | ||
| --os linux | ||
| - name: Push ECR manifest with latest | ||
| id: push-ecr-manifest-latest | ||
| run: | | ||
| docker manifest push "${{ needs.build-and-upload-image-to-ecr.outputs.ecr_registry_repository }}" | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -35,3 +35,6 @@ dist/ | |
|
|
||
| examples/build/* | ||
| examples/*.zip | ||
|
|
||
| durable-executions.db* | ||
| .coverage | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,20 @@ | ||
| FROM python:3.13-slim | ||
|
|
||
| # Copy and install the wheel | ||
| COPY dist/*.whl /tmp/ | ||
| RUN pip install --no-cache-dir /tmp/*.whl && rm -rf /tmp/*.whl | ||
SilanHe marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| # AWS credentials (required for boto3) | ||
| ENV AWS_ACCESS_KEY_ID=foo \ | ||
| AWS_SECRET_ACCESS_KEY=bar \ | ||
| AWS_DEFAULT_REGION=us-east-1 | ||
SilanHe marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| EXPOSE 9014 | ||
|
|
||
| CMD ["dex-local-runner", "start-server", \ | ||
yaythomas marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| "--host", "0.0.0.0", \ | ||
| "--port", "9014", \ | ||
| "--log-level", "DEBUG", \ | ||
| "--lambda-endpoint", "http://host.docker.internal:3001", \ | ||
| "--store-type", "sqlite", \ | ||
| "--store-path", "/tmp/.durable-executions-local/durable-executions.db"] | ||
SilanHe marked this conversation as resolved.
Show resolved
Hide resolved
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.