Migrations¶

Migrations version the declarations in your project: the schema of a dataset, the hyperparameters of a model, the configuration of an agent. They are ordinary Python files you read and edit, generated by diffing what your code says now against what the migration history says it used to.

python manage.py makemigrations
python manage.py migrate

Why an ML project needs them¶

A dataset version recorded six months ago carries a schema fingerprint. Without history, that fingerprint is an opaque hash. With it, you can say exactly which fields existed, what their bounds were, and which classes a LabelField accepted — which is what makes an old run interpretable rather than merely recorded.

What a migration looks like¶

reviews/migrations/0001_initial.py

"""Generated by mlango 0.2.0 on 2026-07-31 12:00."""

from mlango import migrations
from mlango.core import fields


class Migration(migrations.Migration):
    initial = True

    dependencies = []

    operations = [
        migrations.CreateObject(
            name="Reviews",
            kind="dataset",
            fields=[
                ("id", fields.IntegerField()),
                ("text", fields.TextField()),
                ("label", fields.LabelField(["negative", "positive"])),
            ],
            options={
                "description": "Customer product reviews.",
                "primary_key": "id",
            },
        ),
    ]

Operations¶

Operation	Meaning
`CreateObject(name, kind, fields, options)`	A new declaration
`DeleteObject(name)`	A declaration removed
`RenameObject(old_name, new_name)`	A rename — written by hand
`AddField(object_name, name, field)`	A field added
`RemoveField(object_name, name)`	A field removed
`AlterField(object_name, name, field)`	A field's definition changed
`AlterOptions(object_name, options)`	`Meta` options changed
`RunPython(code, reverse_code=None)`	Arbitrary code — backfills, re-encodings

Renames are never guessed¶

Rename a field and the autodetector writes a RemoveField and an AddField, not a RenameObject.

That is deliberate. Guessing wrong about a rename silently changes what stored data means — a body column quietly treated as the old text column would make every historical dataset version subtly wrong, with nothing to notice. If you meant a rename, say so:

operations = [
    migrations.RenameObject("Reviews", "Feedback"),
]

Data migrations¶

RunPython receives the project state and a context carrying a metastore session, so a backfill can read and write versions without opening its own connection:

from mlango import migrations


def backfill_language(state, context):
    """Tag every materialised version with a detected language."""
    from mlango.metastore import DatasetVersion

    session = context["session"]
    for version in session.query(DatasetVersion).filter_by(label="reviews.Reviews"):
        version.notes = f"{version.notes} language=en".strip()


class Migration(migrations.Migration):
    dependencies = [("reviews", "0003_reviews_language")]

    operations = [
        migrations.RunPython(backfill_language, description="Tag versions with a language"),
    ]

Start one with:

python manage.py makemigrations reviews --empty -n backfill_language

RunPython executes on migrate

Review migrations from outside contributors as you would any other code.

Commands¶

python manage.py makemigrations                    # every app
python manage.py makemigrations reviews            # one app
python manage.py makemigrations -n add_language    # name it yourself
python manage.py makemigrations --dry-run          # show, do not write
python manage.py makemigrations reviews --empty    # a stub to fill in

python manage.py migrate                           # apply everything pending
python manage.py migrate reviews                   # one app
python manage.py migrate --plan                    # what would run
python manage.py migrate --fake                    # mark applied without running

python manage.py showmigrations                    # what exists, and what ran
python manage.py showmigrations -v 2               # with each operation listed

The metastore's own tables¶

The ten metastore tables are framework-owned and never change shape at your request, so they are created on demand — you do not need migrate before your first materialize(). migrate creates them too, and then applies your declarative migrations.

That split is deliberate: bookkeeping tables should never be something a newcomer has to think about, while their schema history should be explicit.

Upgrading mlango is handled in the same place. When a release adds a column to a metastore table, the next connection adds it to your database — with the declared default, so rows written by the old version keep working. Only additive changes qualify; a rename or a type change would be a real migration, and one that silently rewrote your run history at import time would be a bad idea. If a new column cannot be filled in safely, the log says which one and leaves the database alone rather than guessing.

Ordering and dependencies¶

Within an app, the numeric prefix is the order. Across apps, declare it:

class Migration(migrations.Migration):
    dependencies = [
        ("reviews", "0002_reviews_language"),
        ("shared", "0001_initial"),
    ]

A cycle is a hard error rather than something to sort around.

Not every `Meta` option is recorded¶

An option that cannot survive a round trip through a file is not part of migration state — a source pointing at a live Python generator, for instance. Pretending otherwise would produce migrations that crash on import. Simple values (strings, numbers, booleans, and flat lists and dicts of them) are recorded; callables and live objects are not.

Squashing¶

Not implemented yet. For now, a project that wants a clean slate can delete its migration files, regenerate 0001_initial, and migrate --fake. That loses history, so do it deliberately.