Repository navigation
Conversation
…ad exceeds reject thresholds
…hen load info exists
| @@ -1,3 +1,5 @@ | |||
| from __future__ import annotations | |||
There was a problem hiding this comment.
do we really need this?
It seems harmless, without it we can also just continue using "Tier" and return Tier.TIER_1 works without this import
also in 3.14 version its enabled automatically, so it might be better to keep it so we dont need to rewrite "Tier" to Tier once we upgrade(pretty sure nobody will do that)
Just making sure its intentional and not ai being enthusiastic
There was a problem hiding this comment.
yeah; I hate the "Tier" style annotations. I almost always want from __future__ import Annotations.
| if self not in ladder: | ||
| return Tier.TIER_1 |
There was a problem hiding this comment.
Can this ever happen? A query with no tier / unknown tier
There was a problem hiding this comment.
Probably not. But technically yes.
This will ensure that in those instance we route to a tier; instead of 500-ing on the query.
| @pytest.mark.parametrize( | ||
| "start, expected", | ||
| [ | ||
| (Tier.TIER_1, Tier.TIER_8), | ||
| (Tier.TIER_8, Tier.TIER_64), | ||
| (Tier.TIER_64, Tier.TIER_512), | ||
| (Tier.TIER_512, Tier.TIER_512), | ||
| (Tier.TIER_NO_TIER, Tier.TIER_1), | ||
| ], | ||
| ) | ||
| def test_lower(start: Tier, expected: Tier) -> None: | ||
| assert start.lower() == expected |
There was a problem hiding this comment.
test for the edges of .lower() behaviour
| @@ -1,3 +1,5 @@ | |||
| from __future__ import annotations | |||
There was a problem hiding this comment.
yeah; I hate the "Tier" style annotations. I almost always want from __future__ import Annotations.
…t downsampled_query
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Want fixes drafted automatically? Bugbot Autofix can create code changes for findings. A team admin can enable Autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit eb1c500. Configure here.
When cluster load exceeds any active allocation policy
thresholds.reject,NORMALandLOW_PRIORITYqueries move one step coarser on the EAP ladder (1 → 8 → 64 → 512viaTier.lower()). Already at 512 stays 512. A tier not on the ladder becomesTIER_1then can step up on a later request. Highest-accuracy modes are unchanged.can_runis not flipped.Runs after
_update_routing_decision, so it coarsens whatever outcomes already picked. Emitssampling_in_storage_allocation_policy_downsamplewhen that bump runs, so we can track how often we downsample due to load.LoadInfo.exceeds({})is false (no ceilings → nothing exceeded). An empty reject map therefore does not downsample, and a rejection is pardoned when load info is present. Missing load info still rejects.can_runbehaviour is unchanged for now. So this will only downsample on unenforced policies (to allow our enforced rate-limits to continue rate limiting).