Skip to content

Fix S4 method dispatch on imported S3 generics (e.g. setMethod("as.data.table", "Basket", ...))#8

Draft
Copilot wants to merge 4 commits intomainfrom
copilot/research-s4-method-imported-function
Draft

Fix S4 method dispatch on imported S3 generics (e.g. setMethod("as.data.table", "Basket", ...))#8
Copilot wants to merge 4 commits intomainfrom
copilot/research-s4-method-imported-function

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Apr 10, 2026

setMethod() on an imported S3 generic (like data.table::as.data.table) creates an implicit S4 generic in ns_env. The pkg_env population step was overwriting that implicit generic with the original S3 function from impenv, silently breaking search-path dispatch while internal namespace dispatch worked fine. Additionally, exportMethods()/exportClasses() directives in NAMESPACE were ignored, so exported generics never appeared in getNamespaceExports().

Fixes in R/loadfast.R

  • pkg_env copy order reversed (both full and incremental paths): impenv is now the base layer, ns_env applied second so any shadows created during sourcing (implicit generics, deliberate overrides) win over raw imports
  • exportMethods/exportClasses now processed: namespace export block extended to include nsInfo$exportMethods and nsInfo$exportClasses alongside nsInfo$exports
# Before — impenv overwrote the implicit generic back to the original S3 fn:
list2env(as.list(ns_env, all.names = FALSE), envir = pkg_env)
list2env(as.list(impenv, all.names = TRUE),  envir = pkg_env)  # ← clobbered

# After — ns_env wins on conflicts:
list2env(as.list(impenv, all.names = TRUE),  envir = pkg_env)
list2env(as.list(ns_env, all.names = FALSE), envir = pkg_env)

Test coverage

  • Added Basket S4 class + setMethod("as.data.table", "Basket", ...) to devpackage fixture; as.data.table is imported but intentionally not exported (exportMethods omitted from NAMESPACE)
  • Stage 1 checks (full load): method registration via existsMethod(), implicit generic is a genericFunction in both ns_env and pkg_env (not overwritten by the raw imported S3 fn), dispatch via get(), direct calls via get(..., pos = "package:devpackage") and evalq() in pkg_env scope, single-element and empty Basket edge cases, non-Basket data.frame dispatch regression, confirms as.data.table is not in getNamespaceExports()
  • Stage 4b checks (incremental reload): same battery of direct-call and edge-case checks after an incremental reload triggered by mutating base.R, plus explicit checks that as.data.table remains a genericFunction in both ns_env and pkg_env after the reload (not reverted to the plain S3 function), confirming the method remains registered and dispatches correctly through all call paths

- Reverse pkg_env copy order in full load (impenv first, ns_env second) so
  implicit generics created by setMethod() are not overwritten by the
  original imported function from impenv
- Same fix in incremental load path
- Process nsInfo$exportMethods and nsInfo$exportClasses when calling
  namespaceExport(), so exportMethods() directives in NAMESPACE are honoured
- Add Basket class + setMethod("as.data.table","Basket") to devpackage fixture
- Add exportMethods(as.data.table) to devpackage/NAMESPACE
- Add Stage 1 tests covering implicit generic in ns_env and pkg_env,
  search-path dispatch, and namespace exports
- Add Stage 4b incremental reload test for Basket method via pkg_env

Agent-Logs-Url: https://github.com/finccam/loadfast/sessions/1e692b27-bdbc-40af-987c-8cde7d9c14cd

Co-authored-by: felix-andreas-finccam <266759557+felix-andreas-finccam@users.noreply.github.com>
Comment thread test_loadfast.R
"as.data.table" %in% getNamespaceExports("devpackage")
))


Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot can we also test if normal call works? also for incmrental test. are these new tests exhaustive? can you think of more

basket <- new("Basket", ...)
as.data.table(new()

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added in e1d67be. Both Stage 1 (full load) and Stage 4b (incremental) now cover:

  • existsMethod("as.data.table", "Basket") — method is registered
  • Direct call via getExportedValue("devpackage", "as.data.table")(basket) — no get() wrapper
  • Direct call via evalq(as.data.table(basket), envir = as.environment("package:devpackage")) — natural call syntax in pkg_env scope
  • Single-element Basket
  • Empty Basket → 0-row DT with item column
  • Non-Basket data.frame dispatch still works (regression guard)

Stage 1 additions:
- existsMethod("as.data.table", "Basket") registration check
- Direct call via getExportedValue("devpackage", "as.data.table")(b1)
- Direct call via evalq(as.data.table(b1), envir = pkg_env)
- Single-element Basket edge case
- Empty Basket → 0-row DT edge case
- Non-Basket data.frame dispatch regression test

Stage 4b additions (incremental path):
- existsMethod() still true after reload
- Direct call via getExportedValue() after reload
- Direct call via evalq in pkg_env scope after reload
- Empty Basket edge case after reload

Agent-Logs-Url: https://github.com/finccam/loadfast/sessions/b04d79a4-ae26-4d2c-8d17-24aaa4c3ee10

Co-authored-by: felix-andreas-finccam <266759557+felix-andreas-finccam@users.noreply.github.com>
Comment thread devpackage/NAMESPACE Outdated
importFrom(data.table,":=")
importFrom(data.table,as.data.table)
importFrom(data.table,data.table)
exportMethods(as.data.table)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot remove this. we shouldn't export this method. we only import it. we shouldn't rely on an export

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in dcc3a12. Removed exportMethods(as.data.table) from devpackage/NAMESPACE. The export test now asserts that as.data.table is not in getNamespaceExports("devpackage"), and all direct-call tests were updated from getExportedValue() (which requires export) to get("as.data.table", pos = "package:devpackage") (search-path lookup).

@felix-andreas-finccam
Copy link
Copy Markdown
Contributor

@copilot can we also add a test for implicit generics? (normal an incremental?)

…pdate tests accordingly

Agent-Logs-Url: https://github.com/finccam/loadfast/sessions/867ce387-0440-413c-a48f-d463aca793c4

Co-authored-by: felix-andreas-finccam <266759557+felix-andreas-finccam@users.noreply.github.com>
Agent-Logs-Url: https://github.com/finccam/loadfast/sessions/1626ac4c-08cf-457f-bddc-8ad5aa513549

Co-authored-by: felix-andreas-finccam <266759557+felix-andreas-finccam@users.noreply.github.com>
Copy link
Copy Markdown
Contributor Author

Copilot AI commented Apr 10, 2026

@copilot can we also add a test for implicit generics? (normal an incremental?)

Added in 387678b. Stage 1 already had the two implicit generic checks (ns_env + pkg_env). Stage 4b now has the same two checks after the incremental reload — verifying as.data.table remains a genericFunction in both ns4b (namespace env) and package:devpackage (search-path pkg env), not reverted to the raw imported S3 function.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants