Skip to content

CI: Add metadata.toml validation workflow (implements #3272)#424

Open
Vanya-kapoor wants to merge 8 commits intomesa:mainfrom
Vanya-kapoor:add-metadata-validation-ci
Open

CI: Add metadata.toml validation workflow (implements #3272)#424
Vanya-kapoor wants to merge 8 commits intomesa:mainfrom
Vanya-kapoor:add-metadata-validation-ci

Conversation

@Vanya-kapoor
Copy link
Copy Markdown

Summary

Adds automated CI validation for metadata.toml files in all examples,
as discussed in #3272.

Changes

  • .github/workflows/validate_metadata.yml: GitHub Actions workflow that
    runs on every PR touching metadata files
  • scripts/validate_metadata.py: Validation script that checks:
    • All required fields are present
    • space is one of: Grid, Network, Continuous, None
    • time is one of: Discrete, Continuous
    • complexity is one of: Basic, Advanced
    • Valid TOML syntax

Why

As raised in #3272 without validation, metadata can silently drift
from the agreed standard as new examples are added. This ensures
every PR with a metadata.toml is automatically checked.

Related

@codebreaker32
Copy link
Copy Markdown
Collaborator

See my comment in #423

Copy link
Copy Markdown

@B2prakash B2prakash left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work on the CI validation! A few thoughts:

  1. The validation script looks solid — checking required fields, valid TOML syntax, and enum values for space/time/complexity covers the key cases.

  2. Question about the space values — should "None" be a valid option? Some examples like pure network models might not have a spatial component, but I'm curious if maintainers want that as an explicit value or just make the field optional.

  3. The workflow trigger — running only on PRs touching metadata files is smart, keeps CI fast. Does it also run on the scheduled checks, or is that planned separately?

  4. One concern — the PR has multiple unrelated commits mixed in (forest_fire fixes, AgentSet.to_list fix, DataCollector import fix). These should probably be in separate PRs to keep review focused. The metadata validation is its own feature.

@Ishwarpatra
Copy link
Copy Markdown

Hi @Vanya-kapoor! Great work on this. Implementing the CI validation for metadata.toml is going to be incredibly helpful for standardizing the repo, especially with all the discussions happening around #3272! The field validation logic (checking ALLOWED_SPACE, ALLOWED_COMPLEXITY, etc.) looks really clean.

However, I noticed there are some unrelated changes mixed into this PR. Starting around line 590, there are modifications to the ForestFire model's batch_run and plotting logic (plt.scatter(df["density"], df["Burned Out"])).

According to the Mesa Examples peer-review guidelines (#390), PRs should contain "no unrelated changes." >
Suggestion: I highly recommend splitting this into two separate PRs ,one dedicated strictly to the CI metadata validation, and another for your ForestFire data collection updates. That will make it much easier and faster for the maintainers to approve and merge the CI workflow!

Let me know when you update it and I'll gladly re-review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants