Skip to content

Add support for SQLAlchemy 2.1 - #831

Closed
ollz272 wants to merge 1 commit into
kvesteri:masterfrom
ollz272:sqlalchemy-2.1-support
Closed

ollz272 wants to merge 1 commit into
kvesteri:masterfrom
ollz272:sqlalchemy-2.1-support

Conversation

@ollz272

@ollz272 ollz272 commented Sep 25, 2026

Copy link
Copy Markdown

Follow-up to #830, reworked based on the feedback there: this no longer touches any SQLAlchemy internals.

Summary

import sqlalchemy_utils fails on SQLAlchemy 2.1:

AttributeError: module 'sqlalchemy.orm.attributes' has no attribute 'ScalarAttributeImpl'

The only user of those internals was generic_relationship. Rather than importing the now-private _ScalarAttributeImpl / _register_attribute, this PR reimplements it as a hybrid_property with a custom comparator, using only public APIs:

  • Lazy load via object_session() + Session.get() (replaces the private _state_session).
  • Target class lookup via inspect(cls).registry.mappers (replaces registry._class_registry).
  • Column → attribute resolution via Mapper.get_property_by_column().
  • is_type() via Mapper.self_and_descendants (replaces _inheriting_mappers).
  • The GenericAttributeImpl constructor version-sniffing hack is gone.

The public interface is unchanged: generic_relationship(discriminator, id, doc=None), column/string/hybrid discriminators, composite ids, declared_attr on abstract bases, Event(object=obj), filter_by(object=obj), Event.object == obj, !=, and is_type().

Behaviour changes

  • Assignment now sets the discriminator and id attributes through normal instrumentation instead of writing directly to the instance dict. As a result, re-assigning a generic relationship on a persistent object is now actually flushed (previously the change was not tracked).
  • Event.object is a hybrid rather than a MapperProperty, so it appears in mapper.all_orm_descriptors but not mapper.attrs.
  • A misconfigured discriminator/id column raises ImproperlyConfigured on first use rather than at mapper configuration.

Tests

Also pins the test PostgreSQL DSN to postgresql+psycopg2://. SQLAlchemy 2.1 changed the default postgresql:// driver to psycopg (v3), which CompositeType doesn't support.

Testing

  • All tests/generic_relationship tests pass on SQLAlchemy 2.0.54 and 2.1.0.
  • Checked manually on both versions: reading back before any session, re-assigning on a persistent object and flushing, select().where(Event.object == obj), is_type(), and doc=.
  • The rest of the suite was run against SQLite only for this revision (no PostgreSQL/MySQL/MSSQL available locally). Results were identical before and after the change on both SQLAlchemy versions.

Notes

  • CompositeType still only supports psycopg2; on 2.1, a plain postgresql:// URL gets psycopg3. psycopg3 support could be a follow-up.
  • tox installs the latest SQLAlchemy, so CI will test 2.1 only. It may be worth adding a 2.0 job to the matrix.

SQLAlchemy 2.1 made ScalarAttributeImpl and register_attribute private,
breaking import of sqlalchemy_utils. Rather than depending on those
internals, reimplement generic_relationship as a hybrid_property with a
custom comparator, using only public APIs (object_session, Session.get,
Mapper.get_property_by_column, registry.mappers, self_and_descendants).

Assignment now goes through regular attribute instrumentation, so
re-assigning a generic relationship on a persistent object is flushed.

Also pin the test PostgreSQL DSN to psycopg2 since 2.1 changed the
default postgresql:// driver to psycopg (v3).
@danilogco

Copy link
Copy Markdown

#829

tonyskapunk added a commit to distributedci/dci-control-server that referenced this pull request Sep 28, 2026
SQLAlchemy-Utils does not have support for SQLAlchemy v2.1 yet
causing issues while importing
kvesteri/sqlalchemy-utils#829
kvesteri/sqlalchemy-utils#800
Should be solved by: kvesteri/sqlalchemy-utils#831

SQLAlchemy on 2.1 is causing this issue during db init:
Traceback (most recent call last):
  File "/opt/dci-control-server/bin/dci-dbinit", line 27, in <module>
    from sqlalchemy_utils import functions as sa_funcs
  File "/usr/local/lib/python3.12/site-packages/sqlalchemy_utils/__init__.py", line 46, in <module>
    from .generic import generic_relationship  # noqa
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/sqlalchemy_utils/generic.py", line 15, in <module>
    class GenericAttributeImpl(attributes.ScalarAttributeImpl):
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: module 'sqlalchemy.orm.attributes' has no attribute 'ScalarAttributeImpl'. Did you mean: '_ScalarAttributeImpl'?
Unable to init the DB. Exiting.
tiborsimko added a commit to tiborsimko/reana-db that referenced this pull request Sep 30, 2026
SQLAlchemy-Utils 0.41.2 imports ScalarAttributeImpl, which was made
private in SQLAlchemy 2.1. This prevents REANA database consumers
from starting despite passing pip check.

Keep SQLAlchemy on the compatible 2.0 series until the upstream fix is
released and validated.
See kvesteri/sqlalchemy-utils#831
@ollz272

ollz272 commented Oct 5, 2026

Copy link
Copy Markdown
Author

Hi @kurtmckee

Just wondering if you could take a look at this one? The previous was rejected quite swiftly, really hoping we can get a release out that supports 2.1.

tobias-urdin added a commit to gnocchixyz/gnocchi that referenced this pull request Oct 5, 2026
SQLAlchemy 2.1 has breaking changes that are not
yet supported in sqlalchemy-utils [1].

[1] kvesteri/sqlalchemy-utils#831

Change-Id: I67afc2f8199b12823a85a07b05170f7d48da8bb6
Signed-off-by: Tobias Urdin <tobias.urdin@binero.com>
tobias-urdin added a commit to gnocchixyz/gnocchi that referenced this pull request Oct 5, 2026
SQLAlchemy 2.1 has breaking changes that are not
yet supported in sqlalchemy-utils [1].

[1] kvesteri/sqlalchemy-utils#831

Change-Id: I3b1854519e7897a54b95e376cd2d7d83b99d409c
Signed-off-by: Tobias Urdin <tobias.urdin@binero.com>
tobias-urdin added a commit to gnocchixyz/gnocchi that referenced this pull request Oct 5, 2026
SQLAlchemy 2.1 has breaking changes that are not
yet supported in sqlalchemy-utils [1].

[1] kvesteri/sqlalchemy-utils#831

Change-Id: Ia0bea063118d0026e7cdbd2a9c01560aeda46373
Signed-off-by: Tobias Urdin <tobias.urdin@binero.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants