Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions .github/workflows/pr.yml
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,13 @@ jobs:
cache-dependency-glob: ${{ env.UV_CACHE_GLOB }}
- run: make install-py
- run: make ci-python-lint
# The checked-in typed-key files must match what the installed modules'
# catalogs emit. Catches a catalog edit committed without regenerating, and
# (with the namespace guard in gen_i18n.py) a regeneration that dropped keys.
- name: i18n generated files are up to date
run: |
uv run --project host python scripts/gen_i18n.py
git diff --exit-code packages/i18n/src/

python-typecheck:
name: Python typecheck
Expand Down
2 changes: 1 addition & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -84,7 +84,7 @@ hence `SM022`/`SM023`. See `docs/module-authoring.md` § Styling.

Standard mixins in `simple_module_db.mixins`: `AuditMixin`, `SoftDeleteMixin` (bypass the read filter with `stmt.execution_options(include_deleted=True)`; purge by deleting an already-trashed row, or with `hard_delete(session, obj)`), `MultiTenantMixin`, `VersionedMixin`. The soft-delete/tenant filters cover every statement shape, not only selects that name the entity — joins, ORM subqueries, `select(func.count()).select_from(Model)`, and Core statements over `Model.__table__` (GH #332). **Tenancy fails closed**: with `multi_tenant` on, a query or insert on a `MultiTenantMixin` model with no `current_tenant_id` raises `TenantIsolationError` instead of reading every tenant; cross-tenant code says so with `all_tenants()` / `execution_options(all_tenants=True)`, and jobs/CLI act for one tenant with `tenant_context(id)`. With `multi_tenant` off and no tenant bound, inserts are stamped `DEFAULT_TENANT_ID` (`"default"`, from `simple_module_db`) and reads stay unfiltered; adoption migrations backfill with the same constant. Tenant roles reach the principal as `tenant:<role>` — map them with `tenant_role(TenantRole.MEMBER)` from `simple_module_core.tenancy`, never by importing `tenants`; tests use the `tenant_client(role)` fixture. Unique keys on such tables must include `tenant_id` (`SM024`). The `tenants` module owns organisations, memberships and `app.state.tenant_resolver`; tenant-level routes act on the *active* tenant, never a tenant id from the URL. See [docs/framework/multi-tenancy.md](docs/framework/multi-tenancy.md). The per-request session (`get_db`) auto-commits **only if** there are pending writes (via `after_flush` listener); otherwise rollback. Service code should **not** call `session.commit()` — flush if you need DB-assigned values. DML executed through the session (`session.execute(update(Model)...)`) counts as a write; a raw `text("UPDATE ...")` does not, and needs `mark_written(session)`. The commit fires in `CommitBeforeResponseMiddleware`, at the ASGI `http.response.start` message, so a client that creates a row and immediately reads it back in a second request sees it — FastAPI runs a `yield` dependency's exit code *after* the response is delivered, which used to make that a deterministic 404 (GH #257). `get_db` keeps the same commit in its own exit code as a fallback for when the middleware isn't in the stack; whichever runs first wins.

**Migrations** live in `host/migrations/versions/` — not in module packages. `host/alembic/env.py` calls `build_module_metadata()` + `make_include_object()` so autogenerate covers every installed module and ignores host-owned tables. First migration of each module should set `branch_labels = ("<module_name>",)` to enable per-module `downgrade <module>@base`.
**Migrations** live in `host/migrations/versions/` — not in module packages. `host/alembic/env.py` calls `build_module_metadata()` + `make_include_object()` so autogenerate covers every installed module and ignores host-owned tables. First migration of each module should set `branch_labels = ("<module_name>",)`. The label is only a **named target** (`alembic upgrade/downgrade <module>@<rev>`, and the `module` column of the Doctor migration list): because autogenerate chains each first revision off the current head, the history is one linear chain, and `downgrade <module>@base` walks the **entire chain beneath that revision, other modules included** — it is not a per-module rollback. To remove one module's schema, downgrade to the `down_revision` of its first revision, which is safe only while no other module's revisions sit above it; otherwise write a dedicated migration that drops that module's tables. See [docs/database/migrations.md](docs/database/migrations.md) § Removing one module's schema.

**Admin section**. Administrative screens live under `/admin/*`, register into `MenuSection.ADMIN_SIDEBAR`, and render in `AdminLayout` — all three together, not one of the three. `SidebarLayout` renders whichever menu its `menuKey` names, so a page left on `AuthenticatedLayout` after its menu item moved shows a sidebar that no longer contains it. `group=` sub-clusters *within* the admin sidebar (`Access`, `Appearance`, `System`); it is no longer used to carve an admin area out of the main sidebar. `/admin` itself is a host route (`host/routes.py`) that renders from the `adminSidebar` shared prop, so an installed module contributes a card without touching it. Old URLs 301 from `host/routes_legacy.py`. Only view URLs moved — `/api/*` is a separate contract and stays put.

Expand Down
4 changes: 2 additions & 2 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -29,9 +29,9 @@ gen-pages:

# Regenerate packages/i18n/src/{generated-resources,keys.generated}.ts from every
# installed module's locales/, plus host/locales and packages/ui/locales — without
# booting the app.
# booting the app. Pass flags with ARGS, e.g. `make gen-i18n ARGS=--allow-removals`.
gen-i18n:
uv run --project host python scripts/gen_i18n.py
uv run --project host python scripts/gen_i18n.py $(ARGS)

# Install JS deps declared by installed modules into host/client_app/node_modules.
# Wheel-installed modules need this; in-repo workspace modules do not.
Expand Down
12 changes: 8 additions & 4 deletions docs/database/migrations.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ All migrations live in `host/migrations/versions/` — **not** in module package
- **Autogenerate sees everything.** `host/migrations/env.py` calls `build_module_metadata()` to union every installed module's `MetaData`. Autogenerate diffs the DB against that union and writes one migration covering all changes.
- **Operators run one command.** `make migrate` is the only target. No "did you also run `orders/migrate`?" footgun.

Each module's *first* migration sets `branch_labels = ("<module_name>",)` so you can still downgrade one module at a time with `alembic downgrade <module>@base`.
Each module's *first* migration sets `branch_labels = ("<module_name>",)`. The label is a named target for that revision; it does **not** isolate the module's history, because every first revision chains off the previous head. `alembic downgrade <module>@base` therefore walks the whole chain beneath it, other modules included. See [Removing one module's schema](#removing-one-modules-schema).

## Day-to-day workflow

Expand Down Expand Up @@ -39,10 +39,14 @@ Runs `alembic -c host/alembic.ini upgrade heads`. Idempotent.
```bash
make downgrade # back one revision
uv run --project host alembic -c host/alembic.ini downgrade <revision_id> # to a specific revision
uv run --project host alembic -c host/alembic.ini downgrade orders@base # back to the state before the orders module existed
```

`orders@base` uses the `branch_labels` marker from the module's first migration. Module-level downgrade is the mechanism for uninstalling a module cleanly.
Do **not** use `downgrade orders@base` to roll back just the orders module. The `branch_labels` marker names a revision, but `orders@base` resolves to the base of the linear chain that revision sits on, so it rolls back every revision beneath it, including other modules'.

### Removing one module's schema

- If the module's revisions are the latest in the chain (nothing from another module sits above them), downgrade to the `down_revision` of the module's *first* revision: `alembic -c host/alembic.ini downgrade <that down_revision id>`.
- Otherwise write a dedicated migration that drops the module's tables (and anything depending on them). History stays linear and other modules are untouched.

## First migration of a new module

Expand All @@ -57,7 +61,7 @@ branch_labels = ("orders",) # ← add this
depends_on = None
```

Once the marker is in place, all future `orders` migrations inherit the branch.
The marker names the revision so you can target it (for example `alembic upgrade orders@head`); later `orders` revisions do not need it. It does not turn the module into an independent branch.

## Alembic environment setup

Expand Down
2 changes: 1 addition & 1 deletion docs/database/models.md
Original file line number Diff line number Diff line change
Expand Up @@ -170,4 +170,4 @@ Do not re-enable these rules in module-local configs. Real bugs caused by wrong
- [Per-module Base](/database/per-module-base) — how `create_module_base` and `build_module_metadata` work.
- [Mixins](/database/mixins) — `AuditMixin`, `SoftDeleteMixin`, `MultiTenantMixin`, `VersionedMixin`.
- [Session lifecycle](/database/sessions) — the `get_db` dependency and why you don't call `commit()`.
- [Migrations](/database/migrations) — Alembic autogenerate and per-module branch labels.
- [Migrations](/database/migrations) — Alembic autogenerate and branch labels (named revision targets).
2 changes: 1 addition & 1 deletion docs/guide/first-module.md
Original file line number Diff line number Diff line change
Expand Up @@ -80,7 +80,7 @@ make migration msg="add orders tables"
Open `host/migrations/versions/XXXX_add_orders_tables.py` and eyeball it (`make migration` runs Alembic from the host dir):

- It should create the `orders_order` table.
- Add `branch_labels = ("orders",)` to the revision so you can later `alembic downgrade orders@base` to roll the module back to empty without touching other modules.
- Add `branch_labels = ("orders",)` to the revision to give it a named target. The label does not isolate the module: `alembic downgrade orders@base` rolls back the whole chain beneath it, other modules included. See [Removing one module's schema](/database/migrations#removing-one-modules-schema).

Apply:

Expand Down
2 changes: 1 addition & 1 deletion docs/guide/project-structure.md
Original file line number Diff line number Diff line change
Expand Up @@ -101,7 +101,7 @@ Never hand-edit the `.generated.*` files — they are overwritten on the next `m

## Where things intentionally *don't* live

- **No per-module `migrations/` folder.** All migrations live in `host/migrations/versions/`. Autogenerate discovers every installed module's metadata via `build_module_metadata()` in `host/migrations/env.py`. Each module's first migration sets a `branch_labels` marker to enable `alembic downgrade <module>@base`.
- **No per-module `migrations/` folder.** All migrations live in `host/migrations/versions/`. Autogenerate discovers every installed module's metadata via `build_module_metadata()` in `host/migrations/env.py`. Each module's first migration sets a `branch_labels` marker, a named target for that revision (it does not make `alembic downgrade <module>@base` a per-module rollback; see [Migrations](/database/migrations)).
- **No host-level `api/` folder.** REST endpoints are attached by each module via `register_routes(api_router, view_router)`. `/api/*` is the union of every module's API router.
- **No `schemas/` top-level folder.** DTOs live inside the owning module's `contracts/` so other modules import them by name — the reverse of a monolith's "shared schemas" directory.

Expand Down
2 changes: 1 addition & 1 deletion docs/guide/quickstart.md
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,7 @@ make migration msg="add orders tables"
make migrate
```

`make migration` runs Alembic from the host directory (where `alembic.ini` lives). Autogenerate picks up the new `orders_*` tables and writes `host/migrations/versions/XXXX_add_orders_tables.py`. Add `branch_labels = ("orders",)` to that revision so you can later `alembic downgrade orders@base` to roll the module back in isolation.
`make migration` runs Alembic from the host directory (where `alembic.ini` lives). Autogenerate picks up the new `orders_*` tables and writes `host/migrations/versions/XXXX_add_orders_tables.py`. Add `branch_labels = ("orders",)` to that revision to give it a named target. The label does not isolate the module: `alembic downgrade orders@base` rolls back the whole chain beneath it. See [Removing one module's schema](/database/migrations#removing-one-modules-schema).

## 7. Hit the module

Expand Down
19 changes: 17 additions & 2 deletions docs/module-authoring.md
Original file line number Diff line number Diff line change
Expand Up @@ -312,8 +312,23 @@ the module name:
branch_labels = ("my_module",)
```

This lets operators roll back a single module's schema with
`alembic downgrade my_module@base` without touching other modules.
The label gives the revision a stable, readable name you can target
(`alembic downgrade my_module@<rev>`, `alembic upgrade my_module@head`), and
it is what the Doctor's migration list shows as the owning module. It does
**not** isolate the module's history. `alembic revision --autogenerate` sets
`down_revision` to the current head, so every module's first revision chains
linearly off the previous one, and a label on a revision that has a
`down_revision` does not make it a separate branch. `<label>@base` resolves to
the base of the *chain*, so `alembic downgrade my_module@base` rolls back every
revision beneath it, other modules' included.

To remove one module's schema safely:

- If the module's revisions are the latest in the chain (no other module has
added a revision above them), downgrade to the `down_revision` of the
module's *first* revision: `alembic downgrade <that down_revision id>`.
- Otherwise write a dedicated migration that drops that module's tables (and
their dependents), so history stays linear and other modules are untouched.

## Frontend assets

Expand Down
2 changes: 1 addition & 1 deletion docs/reference/deployment.md
Original file line number Diff line number Diff line change
Expand Up @@ -96,7 +96,7 @@ The shipped `CMD` runs a single Uvicorn worker. Tune worker count with `--worker

### Migrations on container start

The shipped `CMD` runs `alembic upgrade heads && uvicorn …`. Note **`heads` (plural)**: each module's first migration sets its own `branch_labels`, so once a second module ships one, `upgrade head` (singular) errors out.
The shipped `CMD` runs `alembic upgrade heads && uvicorn …`. Note **`heads` (plural)**: a history that carries per-module `branch_labels` can end up with several heads, and `upgrade head` (singular) errors out when it does.

That default is right for a single container. Once you run more than one replica, move migrations to a one-shot job instead — see [Running migrations on deploy](#running-migrations-on-deploy).

Expand Down
2 changes: 1 addition & 1 deletion docs/reference/make-commands.md
Original file line number Diff line number Diff line change
Expand Up @@ -91,7 +91,7 @@ The scaffolded Makefile intentionally doesn't wrap one-off things. Run them dire
|---|---|
| New Alembic migration | `uv run alembic revision --autogenerate -m "..."` |
| Downgrade one revision | `uv run alembic downgrade -1` |
| Roll back a single module | `uv run alembic downgrade <module>@base` |
| Remove one module's schema | See [Removing one module's schema](/database/migrations#removing-one-modules-schema); `<module>@base` rolls back the whole chain beneath it |
| Single Python test | `uv run pytest path/to/test_file.py::test_name` |
| Single JS test | `cd client_app && npx vitest run <path>` |
| Format + lint Python | `uv run ruff format . && uv run ruff check .` |
Expand Down
64 changes: 61 additions & 3 deletions framework/hosting/simple_module_hosting/i18n_manifest.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,8 @@
from __future__ import annotations

import logging
import re
from collections import Counter
from pathlib import Path
from typing import Any

Expand Down Expand Up @@ -68,6 +70,7 @@ def emit_frontend_types_for_modules(
project_root: Path,
*,
strict: bool = False,
allow_removals: bool = False,
) -> None:
"""Emit the TS key union covering every *installed* module.

Expand All @@ -79,11 +82,15 @@ def emit_frontend_types_for_modules(
inactive module's strings are typed, never served.
"""
registry, _ = build_i18n_registry(settings, installed_modules, project_root)
emit_frontend_types(registry, project_root, strict=strict)
emit_frontend_types(registry, project_root, strict=strict, allow_removals=allow_removals)


def emit_frontend_types(
registry: I18nRegistry, project_root: Path, *, strict: bool = False
registry: I18nRegistry,
project_root: Path,
*,
strict: bool = False,
allow_removals: bool = False,
) -> None:
"""Write the TS augmentation files into @simple-module-py/i18n if present.

Expand All @@ -95,12 +102,31 @@ def emit_frontend_types(
error to a ``tsc`` run several steps removed from the cause. Under strict a
missing package directory is an error too, where on the boot path it is the
normal shape of a wheel-installed app that ships no i18n workspace.

The checked-in files are shared by everyone, so a regeneration from a venv
missing a module (``uv sync --all-packages`` not run) would silently delete
that module's whole namespace (GH #329). When the new emission would drop a
namespace the existing file has, ``strict`` raises
:class:`NamespaceRemovalError` and the boot path logs a warning and writes
nothing, so a dev server never crashes over it. ``allow_removals=True``
skips the check, for a namespace that was deleted on purpose or a
deliberate ``SM_MODULES_ENABLED`` subset.
"""
pkg_src = project_root / "packages" / "i18n" / "src"
if not pkg_src.is_dir():
if strict:
raise FileNotFoundError(f"{pkg_src} does not exist — nothing to regenerate")
return
if not allow_removals:
dropped = find_dropped_namespaces(
pkg_src / "generated-resources.ts", registry.messages(registry.default_locale)
)
if dropped:
error = NamespaceRemovalError(dropped)
if strict:
raise error
logger.warning("%s Not writing the generated i18n files.", error)
return
try:
write_generated_resources(registry, pkg_src)
except Exception:
Expand All @@ -109,9 +135,41 @@ def emit_frontend_types(
logger.exception("Failed to write generated-resources.ts — frontend types will be stale")


_RESOURCE_KEY_RE = re.compile(r"^\s+'([^']+)': '',$", re.MULTILINE)


class NamespaceRemovalError(RuntimeError):
"""Regenerating would delete whole key namespaces from the checked-in files."""

def __init__(self, dropped: dict[str, int]) -> None:
self.dropped = dropped
detail = ", ".join(f"'{ns}.*' ({n} keys)" for ns, n in sorted(dropped.items()))
super().__init__(
f"regenerating would drop {detail} — the module(s) are not installed in "
"this environment. Run `uv sync --all-packages` first, or pass "
"--allow-removals (`make gen-i18n ARGS=--allow-removals`) if the module was "
"genuinely deleted."
)


def find_dropped_namespaces(resources_path: Path, new_messages: Any) -> dict[str, int]:
"""Top-level namespaces in the existing resources file absent from ``new_messages``.

Returns ``{namespace: existing key count}``. Empty when the file is absent
or unreadable (nothing to lose).
"""
try:
existing = resources_path.read_text(encoding="utf-8")
except OSError:
return {}
old = Counter(key.split(".", 1)[0] for key in _RESOURCE_KEY_RE.findall(existing))
new = {key.split(".", 1)[0] for key in new_messages}
return {ns: n for ns, n in old.items() if ns not in new}


_RESOURCES_HEADER = (
"// AUTO-GENERATED by simple_module_hosting.i18n_manifest — do not edit by hand.\n"
"// Regenerate by booting the host in development mode."
"// Regenerate by booting the host in development mode or `make gen-i18n`."
)

_KEYS_HEADER = _RESOURCES_HEADER
Expand Down
2 changes: 1 addition & 1 deletion packages/i18n/src/generated-resources.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
// AUTO-GENERATED by simple_module_hosting.i18n_manifest — do not edit by hand.
// Regenerate by booting the host in development mode.
// Regenerate by booting the host in development mode or `make gen-i18n`.
export default {
translation: {
'audit_log.actions.created': '',
Expand Down
2 changes: 1 addition & 1 deletion packages/i18n/src/keys.generated.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
// AUTO-GENERATED by simple_module_hosting.i18n_manifest — do not edit by hand.
// Regenerate by booting the host in development mode.
// Regenerate by booting the host in development mode or `make gen-i18n`.

export const keys = {
audit_log: {
Expand Down
Loading
Loading