Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
49 changes: 27 additions & 22 deletions .github/workflows/uv-lock-generation.yml
Original file line number Diff line number Diff line change
Expand Up @@ -21,17 +21,21 @@ on:

jobs:
check_uv_lock_and_update:
if: github.repository == 'NVIDIA-NeMo/Automodel'
runs-on: ubuntu-latest
permissions:
contents: write
env:
CAN_PUSH_LOCKS: ${{ github.event.pull_request.head.repo.full_name == github.repository }}
steps:
- name: Checkout
uses: actions/checkout@v6
with:
repository: ${{ github.event.pull_request.head.repo.full_name }}
ref: ${{ github.event.pull_request.head.ref }}
fetch-depth: 0
token: ${{ secrets.PAT }}
persist-credentials: ${{ env.CAN_PUSH_LOCKS }}
token: ${{ env.CAN_PUSH_LOCKS == 'true' && secrets.PAT || github.token }}

- name: Setup Python
uses: actions/setup-python@v6
Expand All @@ -42,35 +46,36 @@ jobs:
with:
version: 0.8.22

- name: Check Uv lock
- name: Check or update uv lock files
run: |
set +e
uv lock --check
EXIT_CODE=$?
if [ $EXIT_CODE -eq 0 ]; then
echo 'Uv lock file is update to date'
else
echo 'Updating uv lock file'
uv lock
fi
set -e

check_or_update_lock() {
lock_name="$1"

if uv lock --check; then
echo "$lock_name is up to date"
elif [ "$CAN_PUSH_LOCKS" = "true" ]; then
echo "Updating $lock_name"
uv lock
else
echo "::error::$lock_name is out of date. Run the lock update flow in CONTRIBUTING.md and commit the generated lock files."
exit 1
fi
}

check_or_update_lock "Uv lock file"

- name: Check Uv PyTorch lock
run: |
set +e
mv uv.lock uv_main.lock
bash docker/common/update_pyproject_pytorch.sh "$GITHUB_WORKSPACE"
uv lock --check
EXIT_CODE=$?
if [ $EXIT_CODE -eq 0 ]; then
echo 'Uv PyTorch lock file is update to date'
else
echo 'Updating uv PyTorch lock file'
uv lock
check_or_update_lock "Uv PyTorch lock file"
if [ "$CAN_PUSH_LOCKS" = "true" ]; then
mv uv.lock docker/common/uv-pytorch.lock
fi
mv uv.lock docker/common/uv-pytorch.lock
mv uv_main.lock uv.lock

- name: Commit and push uv lock updates
if: env.CAN_PUSH_LOCKS == 'true'
run: |
git config --global user.email "nemo-bot@nvidia.com"
git config --global user.name "NeMo Bot"
Expand Down
36 changes: 32 additions & 4 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
Common workflows used for setting up Automodel environment:

1. [Developing with Automodel container](#1-developing-with-automodel-container)
2. [Developing with UV sync/pip install]($2-developing-with-uv-syncpip-install)
2. [Developing with UV sync/pip install](#2-developing-with-uv-syncpip-install)
3. [Developing with custom docker build](#3-developing-with-custom-docker-build)

### 1. Developing with Automodel container
Expand Down Expand Up @@ -111,12 +111,40 @@ pip install grouped_gemm

We use [uv](https://docs.astral.sh/uv/) for managing dependencies.

New required dependencies can be added by `uv add $DEPENDENCY`.
New required dependencies can be added by `uv add $DEPENDENCY`. If
`pyproject.toml` changes, update both lock files using the same flow as
[Generate Uv lock](./.github/workflows/uv-lock-generation.yml). This updates
the default `uv.lock`, then updates the PyTorch-container lock file with the
override dependencies from `docker/common/uv-pytorch.toml`. The PyTorch helper
rewrites `pyproject.toml` for locking, so the commands below save and restore
your intended `pyproject.toml` contents:

Adding a new dependency will update UV's lock-file. Please check this into your branch:
The workflow can update lock files automatically for same-repository PRs. For
PRs opened from forks, the workflow runs in check-only mode because it cannot
push fixes back to the fork. If the fork check fails, run this lock update flow
locally and include the generated lock files in your branch.

```bash
git add uv.lock pyproject.toml
set -e

uv lock --check || uv lock

tmp_pyproject="$(mktemp)"
cp pyproject.toml "$tmp_pyproject"
mv uv.lock uv_main.lock
bash docker/common/update_pyproject_pytorch.sh "$PWD"
uv lock --check || uv lock
mv uv.lock docker/common/uv-pytorch.lock
mv uv_main.lock uv.lock
cp "$tmp_pyproject" pyproject.toml
rm "$tmp_pyproject"
```

Only commit your intended `pyproject.toml` changes plus the generated lock
files:

```bash
git add pyproject.toml uv.lock docker/common/uv-pytorch.lock
git commit -s -m "build: Adding dependencies"
git push
```
Expand Down
Loading