Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
80 changes: 48 additions & 32 deletions ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -163,57 +163,73 @@ executable version of this paragraph.
Inside, in the order a tick runs:

1. **Display estimator** (`display.rs`). From tick intervals: the refresh floor (service interval, not the fastest
sample), the nominal rate (snapped to 360/240/180/165/144/120/100/90/75/72/60/50 within 3 %), stability
(coefficient of variation <= 0.12 over 32 samples), transitions in either direction (10 agreeing samples, a faster
candidate survives 3 misses). Capacity = min(raw, nominal, the host's present limit). A tick gap above
max(25 ms, 4 x floor) is a host stall.
sample), the nominal rate (snapped to 360/240/180/165/144/120/100/90/75/72/60/50 within 3 %), stability (coefficient
of variation <= 0.12 over 32 samples), transitions in either direction (10 agreeing samples, a faster candidate
survives 3 misses). Capacity = min(raw, nominal, the host's present limit). A tick gap above max(25 ms, 4 x floor) is
a host stall. The tick period itself is the mean over the last 720 ticks (missed ticks counted as the periods they
span): the floor reads a 240 Hz display on a 0.1 ms clock as 238.6 Hz, and the rate, the capacity and the exact grid
go by the period once 30 periods are in.
2. **Source estimator** (`source.rs`). From source pts: the interval (trimmed mean over 32 samples, snapped to common
video rates within 1.5 %, held through 3 % noise once nominal, a transition after 4 agreeing samples outside 12 %),
the arrival-interval EMA, the unique-frame interval (holds and cuts excluded) and the PLL schedule clock that places
presentations. While frames arrive within 1.5 display ticks of it, the clock steps by each frame's pts step scaled
by how fast frames really came against their pts over the last 2 s, and follows the arrivals at 2 %; without pts
or with a delivery that wanders further it steps by the arrival EMA and follows at 8 %; it resyncs hard beyond
80 ms and carries on over a frame the decoder skipped.
presentations. While frames arrive within 1.5 display ticks of it, the clock steps by each frame's pts step scaled by
how fast frames really came against their pts over the last 2 s, and follows the arrivals at 2 %; without pts or with
a delivery that wanders further it steps by the arrival EMA and follows at 8 %; it resyncs hard beyond 80 ms and
carries on over a frame the decoder skipped. A reset (a seek, a change of speed) keeps the decoded cadence and drops
the clock. The source counts as regular while fewer than a quarter of the last 16 intervals are more than 15 % off
the cadence.
3. **Output rate policy** (`rate.rs`). Modes: `Factor(n)`, `Auto` (as many as the display and GPU allow, capped by
motion), `DisplayHz`, `Target(fps)`; Auto with `fps_limit` is a strict-ceiling target. Even cadence: when the display
motion), `DisplayHz`, `Target(fps)`; Auto with `fps_limit` is a strict-ceiling target. Auto runs on factors where the
display is a whole multiple of a regular source, and on the exact grid where it is not (24 fps on 60 Hz, 60 fps on
144 Hz, 25 or 50 fps on anything, a source that skips frames): the display's rate or the whole fraction of it the
GPU, the motion (up to 2.5, 3.75 and 5 frames per source frame as motion rises) and the drop penalty leave, down to
1.15 times the source (the grid needs no doubling: 60 fps on 75 Hz becomes 75 even pictures; a factor and the other
exact modes keep the 2x floor); four source frames in a row must agree before it moves between the two, and a host
with a present limit below the refresh rate keeps the factors. An exact rate that is the display's or a whole
fraction of it is taken from the measured tick period and has no headroom (97 % of the rate left a tick without a new
frame every 33 ticks); the headroom remains for a strict cap and a host present limit. Even cadence: when the display
runs at a whole multiple n of the decoded source rate (within 1 %), Auto rounds the motion ceiling up to the next
factor dividing n and takes the largest dividing factor the GPU allows (never x1 for it; a GPU that cannot reach an
even rate keeps an uneven one below half the display's rate), and `DisplayHz` outputs n x source exactly; the
display's own rate needs no headroom, every tick showing one frame (not with `low_latency`). A factor change the
factor in use does not force waits until Auto has asked for it 250 ms and the last change is 1 s old. The 2x floor,
the display headroom (0.97), compute capacity from the pair / mid / present costs within `compute_budget` of the
source interval, the queue capacity, the clamp reason (display / gpu / runtime / minimum) and a warning text. Exact
modes hold their plan until a different one has been proposed 16 frames in a row. Until the costs are measured Auto
sizes by the host's seeds (`mid_cost_seed_ms`, `pair_cost_seed_ms`). When the GPU looks too slow for 2x, one real
pair with tiny assumed costs every 2.5 s measures it (`probe`).
modes hold their plan until a different one has been proposed 16 frames in a row (the first plan after measuring
starts at once). Until the costs are measured Auto sizes by the host's seeds (`mid_cost_seed_ms`,
`pair_cost_seed_ms`). When the GPU looks too slow for 2x, one real pair with tiny assumed costs every 2.5 s measures
it (`probe`).
4. **Presentation planner** (`plan.rs`). Exact modes: target ticks on the exact output grid in media phase, carried
across intervals; each tick becomes one presentation, an anchor (`previous` / `current`) or an `interpolate` at phase
t; cuts and holds disable interpolation but keep the grid; a tick that lands on a source frame (within 5 % of a step)
shows that frame, every other tick interpolates at its own phase. Factor modes: mids at k / n of the interval and the
source frame at its arrival plus the delay. The delay target (pair lookahead in exact modes + pair cost + 2 x mid
cost + 25 ms + burst pad, floor 60 ms or 42 ms when a frame is cheap and the steady factor is 3 or less, max 180 ms)
sets how far ahead frames are due; it drifts 0.25 ms per frame and rises 2 ms per frame only while frames run late
(exact modes: a whole interval). Factor-mode mids are spread over the clock's own step. With `low_latency` (games)
the target is what the first in-between needs and no more: it is due 1/N of an interval after its source frame and
needs the next one, so (1 - 1/N) x interval + pair + frame + present cost + the arrival jitter (a decaying envelope
of how late frames come against the schedule clock) + burst pad, with no floor; it rises 2 ms per frame and falls
0.25, so the jitter it follows does not squeeze the timeline.
across intervals, each pair as long as its timestamps say (a browser at 1.5x delivers steps of one and two frames);
each tick becomes one presentation, an anchor (`previous` / `current`) or an `interpolate` at phase t; cuts and holds
disable interpolation but keep the grid; a tick that lands on a source frame (within 5 % of a step) shows that frame,
every other tick interpolates at its own phase. Factor modes: mids at k / n of the interval and the source frame at
its arrival plus the delay. The delay target (pair lookahead in exact modes + pair cost + 2 x mid cost + 25 ms +
burst pad, floor 60 ms or 42 ms when a frame is cheap and the steady factor is 3 or less, max 180 ms) sets how far
ahead frames are due; it drifts 0.25 ms per frame and rises 2 ms per frame only while frames run late (exact modes: a
whole interval). Factor-mode mids are spread over the clock's own step. With `low_latency` (games) the target is what
the first in-between needs and no more: it is due 1/N of an interval after its source frame and needs the next one,
so (1 - 1/N) x interval + pair + frame + present cost + the arrival jitter (a decaying envelope of how late frames
come against the schedule clock) + burst pad, with no floor; it rises 2 ms per frame and falls 0.25, so the jitter it
follows does not squeeze the timeline.
5. **Presentation queue** (`queue.rs`). Bounded (24), ordered by due time; one presentation per tick; the oldest entry
inside the lateness contract (3 output intervals) is presented when the display can service the target, else the
newest due. Entries before it are dropped and counted; entries due before the last presented one are stale. A
requested mid that is not there in time shows its anchor instead (`fallback`). The tick lock: a frame due a whole
number n of ticks (2 to 8) after the one before it is not presented sooner than n ticks after that one was, so
due-time jitter crossing a tick boundary no longer leaves one picture up three ticks and the next one; past a tick
plus 4 to 8 ms late it goes out at once, the slip two drifting clocks force.
6. **Auto controller** (`control.rs`, AIMD). Drop pressure is a leaky integrator (tau 300 ms); several frames due in
one tick count as one event; drops within 20 ms after a host stall are not charged while stalls are rarer than 4
per 2 s, and a missed vsync (a tick gap over 1.5 floors) or a single drop at the display's own rate counts as
such a stall; pressure > 1.2 lowers the factor by one (by two above 3), not more often than every 500 ms, along
the even factors when the even cadence holds (x4, x2, x1 at 60 fps on 240 Hz); after 6 s with
pressure < 0.15 the factor recovers one step. Tick strain (ticks 1.7 x slower than the floor) adds a provisional
one-step penalty that lifts 500 ms after the strain ends. A settle window (1200 ms, the extension's measured
fullscreen reflow) after a mode change exempts drops. The motion-adaptive ceiling (> 45 motion units: 2, > 28: 3,
> 16: 4, else 6) is an option. Auto that cannot run by the budget probes one 2x pair at 2.5 / 5 / 10 / 20 s.
6. **Auto controller** (`control.rs`, AIMD). Drop pressure is a leaky integrator (tau 300 ms); several frames due in one
tick count as one event; drops within 20 ms after a host stall are not charged while stalls are rarer than 4 per 2 s,
and a missed vsync (a tick gap over 1.5 floors) or a single drop at the display's own rate counts as such a stall;
pressure > 1.2 lowers the factor by one (by two above 3), not more often than every 500 ms, along the even factors
when the even cadence holds (x4, x2, x1 at 60 fps on 240 Hz); after 6 s with pressure < 0.15 the factor recovers one
step. A penalty that does not help is lifted: two seconds after a step down with the stall load no lower than 0.7 of
what it was (and at least half the stall limit), the stalls are the host's own, the penalty goes and the drops that
follow a stall are exempt for 30 s. Tick strain (ticks 1.7 x slower than the floor) adds a provisional one-step
penalty that lifts 500 ms after the strain ends. A settle window (1200 ms, the extension's measured fullscreen
reflow) after a mode change exempts drops. The motion-adaptive ceiling (> 45 motion units: 2, > 28: 3, > 16: 4, else
6) is an option. Auto that cannot run by the budget probes one 2x pair at 2.5 / 5 / 10 / 20 s.
7. **Just-in-time requests.** A mid is requested `pair + 2 x (mid + present) + 8` ms before its slot, so present
blits interleave with computes instead of the first mid waiting for the whole batch.
8. **Host present limit.** No two presents closer than `min_present_interval_ms` (Firefox: 7 ms); the display
Expand Down
51 changes: 51 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,57 @@ follow semantic versioning as described in [CONTRIBUTING.md](CONTRIBUTING.md).

## [Unreleased]

## [0.3.6] - 2026-11-10

### Changed

- Auto on a display that is not a whole multiple of the source rate runs on the exact grid: every tick of the display
(or every nth) shows a picture at its own media time. Factors put in-betweens at k / n between two frames, which is
even on screen only at whole multiples; 24 fps on a 60 Hz display as x2 was 48 pictures on 60 ticks. Pictures off
pace in Chrome (a 240 Hz panel presenting on every 4th or 2nd tick for 60 and 120 Hz): 24 fps on 60 Hz 25.5 % to
2.5 %, 25 fps on 60 Hz 20 % to 1.9 %, 50 fps on 120 Hz 20 % to 2.5 %, 25 and 50 fps on 240 Hz 17 to 40 % to 2 to
4 % (what remains is scene cuts). The same holds for 60 fps on 144 or 165 Hz. A display at a whole multiple keeps
the factors and the source frames with them; a host with a present limit below the refresh rate keeps the factors
too. The grid needs no doubling, so Auto also serves displays under twice the source rate (down to 1.15 times):
60 fps on 75 Hz, where every fourth frame was shown twice, is 75 pictures at their own time (measured as 60 fps on
80 Hz: 33 % off pace to 1.6 %; 50 fps on 60 Hz: 20 % to 0.9 %).
- Display Hz mode and a target at the display's rate use the whole rate, on the measured tick period: the 97 %
headroom left a tick without a new frame every 33 ticks, a held picture twice a second on a 60 Hz display (a 24 fps
pan on 240 Hz: 1.5 to 5 % of the pictures off pace to 0.5 %). `display_headroom` remains for Auto's factors, a
capped Auto and a host present limit.

### Fixed

- A source that skips frames is paced by its timestamps. A browser playing 60 fps at 1.5x delivers two of every three
frames; Auto read the rate as more than the display has and generated nothing, and the exact modes gave every pair
the average length (50 to 75 % of the pictures off pace). Auto takes the exact grid for such a source and each
pair is as long as its timestamps say: 0.5 % off pace at 120 pictures a second.
- The display's rate was measured half a percent low on a clock with 0.1 ms steps (240 Hz as 238.6), and the
presentation target of a factor went by the arrival average (x4 of 60 fps as 241 to 247 fps): at the display's own
rate the queue dropped frames it could have shown a tick later, the drops drew a penalty, and fast motion lost its
in-betweens for six seconds in about one run in three. The rate goes by the mean tick period and the target by the
decoded cadence.
- An exact mode started generating a second after every start and seek (the first plan waited 16 frames like a
change of plan): at once now, 0.5 s to 0.1 s at 24 fps.
- A seek or a change of speed threw the decoded cadence away, eight frames without in-betweens while it was
relearned (1.2 s after a change of speed). It is kept.
- Video mode (the lookahead, cadence re-timing): the in-betweens of a pair were spread over the clock's step of the
frame after it, 7.2 to 9.5 ms apart at x5 and off their frames; they take their own pair's span (a 24 fps pan,
x5 on 240 Hz: 18.7 % of the pictures off pace to 2.2 %).
- The browser `CostSampler` timed every 30th generate, and in video mode, where an interval's requests start with the
re-timing of a real frame, a stride over 2 or 5 requests only ever met those, which have no cost: the frame cost
was never measured and Auto stayed on the seed. A sample that did not land (nothing written, or refused in a settle
window) is taken again on the next call.
- Stalls that a lower factor does not cure no longer cost frames. Firefox stalls its display loop 50 to 100 ms a few
times a second while a page element lies over the canvas, whatever is generated; the drops after the stalls drew
penalties down to x1 (21 pictures a second, under the video's own rate, where x5 showed 44 to 90 between the same
stalls). Two seconds after a step down with the stall load no lower, the penalty is lifted and the drops that
follow a stall count as the host's for half a minute (Firefox with the pointer over the video: 26 to 61-77
pictures a second). Drops without a stall are charged as before.
- `settle_active` in the report was false before the first tick, so a host measuring its GPU costs once measured
inside the settle window, had the samples refused and stayed on the 10 ms seed (the reference player: x4 instead
of x5 in about one run in four). The report is right from the first event, and the player measures until taken.

## [0.3.5] - 2026-11-09

### Fixed
Expand Down
12 changes: 6 additions & 6 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ members = ["crates/flyhigh-options", "crates/flyhigh-engine", "crates/flyhigh-ca
default-members = ["crates/flyhigh-options", "crates/flyhigh-engine", "crates/flyhigh-capi", "crates/flyhigh-cli", "examples/player"]

[workspace.package]
version = "0.3.5"
version = "0.3.6"
edition = "2021"
license = "Apache-2.0"
repository = "https://github.com/getflyhigh/flyhigh-engine"
Expand Down
Loading
Loading