Skip to content

Upgrade testthat to 3rd edition. - #476

Open
mcol wants to merge 9 commits into
stan-dev:masterfrom
mcol:issue_458
Open

mcol wants to merge 9 commits into
stan-dev:masterfrom
mcol:issue_458

Conversation

@mcol

@mcol mcol commented Nov 6, 2026 •

Copy link
Copy Markdown
Member

Summary

This replaces uses of deprecated expect_is() and expected_equivalent(). Messages were automatically suppressed in 2e, but need to be manually suppressed (or captured with expect_message()) under 3.

Changes like the following:

-  expect_equal(as_draws_df(x), draws_df(x = 1:10))
+  expect_equal(as.data.frame(as_draws_df(x)), as.data.frame(draws_df(x = 1:10)))

are used to avoid the "Dropping 'draws_df' class as required metadata was removed" warnings.

Also runs tests in parallel.

Fixes #458.

Copyright and Licensing

By submitting this pull request, the copyright holder is agreeing to
license the submitted work under the following licenses:

@VisruthSK

VisruthSK commented Nov 7, 2026 •

Copy link
Copy Markdown
Member

Didn't put this in the issue, but should probably also see if test times can be dropped. I'm looking to see if parallelization would be worth it, initially it seems like one file dominates timings and should be split up.

EDIT: cut times by half locally using 4 workers, should speed up CI a little too.

I haven't looked through the test suite, but snapshots may be useful too.

@github-actions

github-actions Bot commented Nov 7, 2026 •

Copy link
Copy Markdown

This is how benchmark results would change (along with a 95% confidence interval in relative change) if 4cf4511 is merged into master:

  • 🚀as_draws_array: 92.3ms -> 88.7ms [-5.75%, -2.05%]
  • ✔️as_draws_df: 29.4ms -> 29.2ms [-1.98%, +0.38%]
  • ✔️as_draws_list: 144ms -> 144ms [-1.29%, +1.63%]
  • ✔️as_draws_matrix: 23ms -> 23.1ms [-0.24%, +1.34%]
  • ✔️as_draws_rvars: 69.6ms -> 69.7ms [-0.64%, +1.01%]
  • ✔️summarise_draws_100_variables: 582ms -> 581ms [-0.59%, +0.1%]
  • ✔️summarise_draws_10_variables: 64.7ms -> 64.6ms [-0.44%, +0.27%]
    Further explanation regarding interpretation and methodology can be found in the documentation.

@jgabry

jgabry commented Nov 10, 2026

Copy link
Copy Markdown
Member

Thanks @mcol! @VisruthSK do you have time to review this?

@VisruthSK

Copy link
Copy Markdown
Member

Yup, will take a proper look soon then.

@jgabry

jgabry commented Nov 10, 2026

Copy link
Copy Markdown
Member

Thanks!

@VisruthSK VisruthSK left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great! I noted a few patterns where I'm being nitpicky and would prefer a change in how we're checking some things, the ignore_attr comment is the only serious one in that it weakens the assertion significantly enough that I think we should try to avoid that.

There are two places where I think snapshots make sense (though that could go in another PR): in the Pareto tests, and most of the rvar print tests. Both would be good to simplify the code and make updating messages easier.

Comment thread tests/testthat/test-as_draws.R Outdated
draws_matrix <- draws_matrix(a = 1:10, b = 11:20, c = 1)
draws_matrix2 <- as_draws_matrix(cbind(1:10, 11:20, 1))
expect_equivalent(draws_matrix, draws_matrix2)
expect_equal(draws_matrix, draws_matrix2, ignore_attr = TRUE)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you check if we can weaken the ignore_attr bypass? I think ignore_attr = "cache" might suffice across the board.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I now ignore only the specific attributes that are different. Ignoring "cache" is fine in a few cases, In most cases it's "dimnames", but there are also others ("dim", "names", "nchains") that crop up.

w <- runif(ndraws(x), 0, 10)

x <- as_draws_matrix(x)
x_rs <- resample_draws(x, weights = w, method = "stratified")

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think suppressing the messages is good since it cleans up test run logs, but maybe we could try something like

 # Store and restore option to ensure test isolation
old_option <- getOption("posterior.warn_on_merge_chains")
on.exit(options(posterior.warn_on_merge_chains = old_option))
  
options(posterior.warn_on_merge_chains = TRUE)

which is already used elsewhere? We could write a tiny helper or even add withr since its a transitive dep anyway.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I used the existing pattern.

Comment thread tests/testthat/test-pareto_smooth.R Outdated
expect_equal(pareto_khat(tau_chains, ndraws_tail = 20),
pareto_khat(tau_nochains, ndraws_tail = 20))

suppressMessages({

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wouldn't verbose = FALSE work here instead of explicit suppression? I think that's cleaner.

@VisruthSK

Copy link
Copy Markdown
Member

Made a few tiny fixes, if its all green on the rerun, should be okay to merge.

@VisruthSK VisruthSK mentioned this pull request Nov 15, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Use testthat 3e

4 participants