research-tracker

A small, local-first research metadata store for connecting training runs to evaluations, experimental conditions, sample-level metrics, and generated artifacts. Records live in a single SQLite database inside a store directory, and config, checkpoint, and artifact paths are stored relative to that directory so a copied store can be merged back into the main store.

The store is one directory:

artifacts/
├── schema.json     # {"schema_version": 2}, the version marker
├── store.sqlite    # runs, evaluations, conditions, metrics, artifacts
└── .sync.lock      # file lock used for writes and sync

30-second example

from pathlib import Path

from research_tracker import ExperimentStore
from research_tracker.trackers import Status

root = Path("artifacts")
store = ExperimentStore(root)

run = store.create_run(
    model="inverse-operator",
    model_class="my_project.models:InverseOperator",
    dataset="dynamic-pet",
    dataset_version="irr-v1",
    config=root / "config.yaml",
    status=Status.RUNNING,
)

with store.evaluation(run.run_id, "test") as evaluation:
    evaluation.add_condition("split", "out-of-distribution")
    evaluation.log_metric(
        sample_id="mouse-01",
        target="parametric-map",
        metric="mse",
        value=0.012,
    )
    evaluation.log_metric(metric="mean_mse", value=0.012)
    evaluation.log_artifact(
        kind="prediction",
        path=root / "predictions" / "mouse-01.npz",
        sample_id="mouse-01",
    )

store.set_status(Status.COMPLETED, run.run_id)

The evaluation context flushes buffered metrics and conditions when the block succeeds. If the block or its flush raises, that evaluation and its metric, condition, and artifact records are removed; artifact files themselves are left in place because they may be shared.

Install

pip install research-tracker

Migration from a 0.1.x Parquet store additionally needs the migrate extra:

pip install "research-tracker[migrate]"

Where to go next

• Getting started — install, first store, the evaluation context manager, and the Lightning callback.
• Runs and evaluations — the record model, IDs, buffering, rollback, and relative paths.
• Sync — merging a copied store back into the main one.
• CLI — every subcommand with a runnable example.
• Migration — upgrading a Parquet v1 store in place.
• API reference — the ExperimentStore API and helpers.
• Architecture — why the storage looks the way it does.
• Limitations — the honest edges of the design.