Skip to content

fix(duckdb): do not cascade table/view drops in DuckLake catalogs - #6034

Open
kingjaiteh wants to merge 2 commits into
SQLMesh:mainfrom
kingjaiteh:fix/6032-ducklake-drop-cascade
Open

kingjaiteh wants to merge 2 commits into
SQLMesh:mainfrom
kingjaiteh:fix/6032-ducklake-drop-cascade

Conversation

@kingjaiteh

@kingjaiteh kingjaiteh commented Sep 8, 2026 •

Copy link
Copy Markdown

Fixes #6032

DuckLake does not implement DROP TABLE ... CASCADE or DROP VIEW ... CASCADE and raises "Cascade Drop not supported in DuckLake". Since #5133 the janitor always deletes snapshot tables with cascade=True, so on a DuckLake catalog it can never reclaim space and silently reports "Cleanup complete." until a snapshot expires.

Cascade support is declared per engine adapter (SUPPORTED_DROP_CASCADE_OBJECT_KINDS), but on DuckDB it varies per attached catalog: a native catalog accepts DROP TABLE ... CASCADE, a DuckLake catalog attached in the same connection does not.

Changes:

  • DuckDBEngineAdapter._drop_object resolves the target catalog (explicit catalog on the name, else the current one) and looks up its type in duckdb_databases(). For ducklake it drops TABLE/VIEW without CASCADE. SCHEMA drops are untouched, DuckLake supports cascade there.
  • The lookup is factored into _get_catalog_type, which _create_table already did inline for the DuckLake partitioning check, so both share it.
  • Native DuckDB catalogs behave exactly as before, so users running with enable_view_dependencies keep cascade.

The list-only alternative (removing TABLE/VIEW from the DuckDB cascade list) would also fix DuckLake but would regress enable_view_dependencies on native catalogs, which is what #4767 added it for.

Test Plan

  • test_drop_table_ducklake_no_cascade attaches a real DuckLake catalog (same setup as test_ducklake_partitioning), creates a table with a dependent view in it and a native table/view in memory, and drops all four with cascade=True. Fails on main with the NotImplementedException above.
  • test_drop_object_cascade_by_catalog_type asserts the emitted SQL with a mocked adapter: no CASCADE for DuckLake table/view, CASCADE kept for DuckLake schema and for a native catalog table.
  • tests/core/engine_adapter/test_duckdb.py and tests/core/test_snapshot_evaluator.py pass on DuckDB 1.5.5, Python 3.12, Windows.
  • The issue reporter has offered to run the branch against their DuckLake project.

Checklist

  • I have run make style and fixed any issues
  • I have added tests for my changes (if applicable)
  • All existing tests pass (make fast-test)
  • My commits are signed off (git commit -s) per the DCO

DuckLake does not implement `DROP TABLE ... CASCADE` or `DROP VIEW ... CASCADE`
and raises "Cascade Drop not supported in DuckLake". Since SQLMesh#5133 the janitor
always deletes snapshot tables with cascade=True, so on a DuckLake catalog it
can never reclaim space and silently reports "Cleanup complete." until a
snapshot expires.

Cascade support is declared per engine adapter, but on DuckDB it varies per
attached catalog. Resolve the target catalog's type from duckdb_databases()
and omit CASCADE for TABLE/VIEW drops in DuckLake catalogs. SCHEMA cascade is
still supported by DuckLake and is left untouched, as is behaviour for native
DuckDB catalogs.

Fixes SQLMesh#6032

Signed-off-by: kingjaiteh <omarjaiteh453@gmail.com>
@StuffbyYuki
StuffbyYuki force-pushed the fix/6032-ducklake-drop-cascade branch from 8f4e72b to 9d3a6fb Compare October 2, 2026 07:14
@StuffbyYuki

Copy link
Copy Markdown
Collaborator

@kingjaiteh Thanks for the PR! No blocker but a few things:

  • Case-insensitive catalog match: _drop_object takes the catalog from the table name, which SQLMesh lowercases. Aliases in catalogs: are attached with their case kept, though (e.g. MyLake), so database_name = 'mylake' misses and CASCADE comes back. Comparing LOWER(database_name) = LOWER(catalog) in _get_catalog_type would fix it.
  • Please add "Fixes # 6032" to the description so the issue closes on merge.

@kingjaiteh

Copy link
Copy Markdown
Author

Thanks @StuffbyYuki! Both done: _get_catalog_type now compares LOWER(database_name) = LOWER(catalog), with a test that attaches a DuckLake catalog as "MyLake" and drops through the lowercased name. Also added Fixes #6032 to the description.

Catalog aliases keep their case in duckdb_databases() (e.g. MyLake), but
SQLMesh lowercases table names, so the DuckLake lookup missed and CASCADE
was emitted again.

Signed-off-by: kingjaiteh <omarjaiteh453@gmail.com>
@kingjaiteh
kingjaiteh force-pushed the fix/6032-ducklake-drop-cascade branch from 96d44e2 to 938ef89 Compare October 3, 2026 15:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Janitor cannot delete snapshots on DuckLake: DROP ... CASCADE is unsupported, and fails silently until a snapshot expires

2 participants