Skip to content

Migrations

msm schema creation and schema evolution are admin workflows. Runtime code attaches to already-registered MetaTable and TimeIndexMetaTable resources through direct backend lookups keyed by SQLAlchemy table name; it does not create tables, apply DDL, or repair schema drift.

The package exposes one SDK Alembic provider:

migrations:migration

That single provider covers core msm, msm_portfolios, and msm_pricing MetaTables. Do not create separate migration configurations for those packages.

Admin Commands

Use the SDK CLI directly:

mainsequence migrations current --provider migrations:migration --json
mainsequence migrations revision --provider migrations:migration -m "describe change" --autogenerate
mainsequence migrations upgrade --provider migrations:migration head
mainsequence migrations downgrade --provider migrations:migration <revision>

There is no msm migrations ... command group. The msm package integration is the provider object.

revision creates normal Alembic revision files at the SDK-derived version location for the provider namespace. Revision files and namespace-specific revision directories are generated authoring output; documentation must not treat them as pre-existing checkout state. Revisions are generated by Alembic; they are not hand-authored SDK operation manifests.

The package migration environment stays on the standard SDK scaffold path:

  • src/migrations/env.py;
  • src/migrations/script.py.mako;
  • SDK-managed namespace version locations.

env.py, provider construction, provider model registry, version-table class construction, metadata extraction, namespace version-location calculation, and the revision template should stay on the SDK helper path instead of local boilerplate.

Default Schema Rule

PostgreSQL public is the default schema. In this provider's SQLAlchemy metadata, default-schema tables must be authored with schema=None, not schema="public".

This matters for Alembic autogenerate. PostgreSQL reflection reports default-schema foreign keys as schema=None. If model metadata says schema="public", Alembic treats identical foreign keys as changed and emits false drop_constraint(...) / create_foreign_key(...) pairs. Reject that generated revision; the model or migration environment is wrong.

Use explicit schema metadata only for real non-default schemas.

upgrade runs the SDK provider flow, applies the Alembic migration through the backend-scoped migration connection, and registers or updates provider MetaTable resources.

downgrade uses the same provider and Alembic revision graph to move the schema back to an earlier revision.

Lifecycle

  1. Add or change SQLAlchemy model declarations.
  2. Ensure the model is returned by the package model graph: markets_sqlalchemy_models(), portfolio_sqlalchemy_models(), or pricing_sqlalchemy_models().
  3. Let mainsequence migrations revision --provider migrations:migration generate a normal Alembic revision.
  4. Review the generated Alembic operations. A no-op model state must not produce FK drop/create churn, index churn, or public versus default-schema churn.
  5. Run the SDK apply and registration command: 
    mainsequence migrations upgrade --provider migrations:migration head
  6. Start runtime code with msm.start_engine(...).

msm.start_engine(...) is direct and read-only. It resolves selected backend tables by model.__table__.name and fails if required platform MetaTable or TimeIndexMetaTable resources are missing.

Registry

src/migrations/registry.py defines the package-owned table universe used by the SDK provider. It is the msm equivalent of an installed-app registry, not migration history.

The registry is derived from:

  • msm.models.markets_sqlalchemy_models();
  • msm_portfolios.models.portfolio_sqlalchemy_models();
  • msm_pricing.meta_tables.pricing_sqlalchemy_models().

Managed models must inherit normal SDK authoring bases. Plain MetaTables use PlatformManagedMetaTable through MarketsMetaTableMixin; time-indexed DataNode storage uses PlatformTimeIndexMetaTable through MarketsTimeIndexMetaTableMixin.

The registry is built through the SDK build_metatable_model_registry(...) helper so filtering, duplicate identifier detection, and provider order follow the same rules as other SDK migration providers.

Time-index storage identifiers use the same CamelCase style as domain MetaTables plus a TS suffix, for example OrdersTS and AssetSnapshotsTS.

See the platform-focused migration reference for provider details: MetaTable Migrations.