What Is Trainlint? The Beginner's Guide to Static Checks for ML Bugs That Don't Crash

CodexFuse Team · 2026-11-06T01:18:59.250Z · 20 min read min read

What Is Trainlint? The Beginner's Guide to Static Checks for ML Bugs That Don't Crash

Trainlint is a free, open-source checker that finds ML bugs that don't crash. If you searched "what is Trainlint", the short answer is this: it reads your Python training code, matches it against 14 rules named TL001 to TL014, and reports the silent mistakes — a scaler fitted before the split, a model left in eval mode, a missing random seed — before you spend GPU hours on a run that was wrong from the start. The project is four days old, first released on 2 November 2026, and it already has a clear job: catch the errors that still train and still print a number.

Trainlint itself is a small download — the 0.2.0 release totals 264.6 kB — and like any package you install, you can confirm a downloaded file's integrity with a SHA-256 hash generator before you run it. Training runs usually happen on a rented GPU box, and VPS hosting is where many small teams run them, so a bug that wastes a run wastes real money. This guide explains what Trainlint is, what its 14 rules check, how to run your first check, and where you should not trust it yet.

What is Trainlint, exactly?

Trainlint is a small, offline, dependency-free static checker for Python machine-learning training and evaluation code, written by Satvik Jain and released under the MIT licence. It is a Python ML linter in the narrow sense: it parses each .py file with Python's ast module, matches calls by name, and never executes the code it reads. People who search "trainlint ml bugs" or "ml linter python" are asking the same question this guide answers: how do I catch the errors that still train and still print a number?

The checker ships 14 rules, TL001 to TL014, plus four agent skills that teach AI coding agents to avoid the same bugs in the first place. It needs Python 3.10 or newer, it has no dependencies, and it never touches the network or a GPU. Each finding shows the line, a pointer into the code, and a one-line fix.

You run it with one command, uvx trainlint check ., and the exit code tells you the result: 0 means no findings, 1 means findings, and 2 means a usage error.

General linters such as Ruff or Pylint check style and common Python mistakes, but none of their rules know what a train-test split is, what eval mode does, or which way round a metric's arguments go. Trainlint exists because those ML-specific mistakes are the ones that cost the most: the code runs, the loss falls, and the result is quietly wrong. The README states the idea plainly: "ML code fails quietly."

Why do ML bugs that don't crash cost so much?

A crash is cheap because it stops you. A silent bug is expensive because it lets you keep going: you train for hours, tune hyperparameters against a leaked validation score, and only discover the problem when the model meets real data.

Trainlint's own demo shows the scale of the damage on a toy problem with 5,000 features and random labels, where no model can beat 50%: picking features before cross-validating reports 85% accuracy, while the fixed version reports 48%. That is 37 percentage points of phantom accuracy, or 35 points above chance, from one misplaced line.

The demo figure is the author's own demo on random labels, not an independent benchmark, and it is worth saying so before anyone quotes it as a general result. The point it makes still stands: a leak inflates the score you use to make decisions, so every decision after it is built on a number that was never real.

The same logic applies to the other rules. A model left in eval mode trains with dropout off, a missing seed makes a result unrepeatable, and a softmax in front of CrossEntropyLoss stalls training near the start — all without an error message.

The cost is measured in GPU hours, not in error logs. Trainlint's own comparison with trainproof makes the same point from the other side: trainproof judges a run after it starts, from logs and data, while Trainlint reads the source before anything runs. The two tools answer different questions, and the cheapest moment to catch a silent bug is before the first epoch.

How do I install Trainlint and run my first check?

Install Trainlint with uvx, pipx or pip, then point it at a file, a folder or a whole repo. The uvx form needs no install step at all: it fetches the package, runs it once, and leaves nothing behind. The pipx form installs it as a standalone tool, and the pip form installs it into the current environment.

uvx trainlint check train.py     # find the bugs in a script, a folder, or a repo
uvx trainlint skills install     # teach your coding agent to avoid them
trainlint check src/ --ignore TL005   # skip a rule
trainlint check . --format json       # for scripts and CI
trainlint rules                       # list every rule

Follow these five steps for your first check, in order.

1. Install it once. Run uvx trainlint check . from your project folder, or install it with pipx install trainlint if you want it on your path permanently.
2. Point it at one script first. Run uvx trainlint check train.py on your main training script and read the findings before you check the whole repo.
3. Read each finding in place. Every finding names the rule, the line and a one-line fix, so fix the highest-impact rule first — a data leak before a style-level warning.
4. Check the whole repo. Run uvx trainlint check . from the project root, or point it at a repo you cloned with Git, and confirm the exit code is 0.
5. Add it to CI. Add run: uvx trainlint check . to your GitHub Actions workflow so every pull request gets the same check before it merges.

Two details matter in daily use. First, you can silence a single line with an ignore comment, such as scaler.fit(X_all) # trainlint: ignore[TL002], when the flagged behaviour is intended — the checker documents this for the pooled-set case. Second, the JSON format exists for scripts and CI pipelines, so a machine can read the findings instead of a person.

What do the 14 Trainlint rules check?

The Trainlint rules fall into five groups: data leakage, the training loop, loss and metrics, data loaders, and reproducibility. Three rules guard the split, five guard the loop, three guard loss and metrics, two guard loaders, and one guards the seed. The table below lists every rule with the mistake it flags.

Run trainlint rules to list the same table from the installed package, which is the version you should trust over any article. The rule set grew from 12 rules to 14 rules in version 0.2.0, when TL013 and TL014 arrived, both contributed by @jay23606. The rest of this guide walks through the groups one at a time.

Which three bugs should every beginner learn first?

The three bugs in the README's own opening paragraph are the three every beginner should learn first, because each one is easy to write and hard to notice. A scaler fitted before the split, a model left in eval mode, and a missing random seed all leave the code running and the numbers looking plausible. Trainlint maps them to TL001, TL003 and TL005, in that order.

The scaler bug, TL001, is the classic data leak. When you fit a scaler, encoder or feature selector on all rows before the split, information from the test set shapes the training input, and the test score stops measuring generalisation.

The README's demo makes the effect concrete: on random labels, picking features before cross-validation reports 85% where the fixed version reports 48%. The fix is to split first, fit on the training rows only, or put the step inside a Pipeline and cross-validate the pipeline.

The eval-mode bug, TL003, is a PyTorch trap. Calling model.eval() inside a training loop with no model.train() to switch back turns dropout off and makes batch norm use its running statistics for the rest of training.

If train() is called once before the epoch loop, every epoch after the first validation trains in eval mode. The sibling rule TL004 covers the reverse mistake: evaluating under torch.no_grad() in a script that never calls model.eval() at all, which leaves dropout active during evaluation.

The seed bug, TL005, is the quietest of the three. A training script that never sets a random seed produces results nobody can reproduce, including you, a week later.

TL005 applies to files that import torch, call backward(), and either have a __main__ guard or run the loop at top level. Any call or keyword argument with "seed" in its name counts as seeding, so the rule is easy to satisfy once you know it exists.

How does Trainlint catch data leakage?

Data leakage is the largest rule group, and it is the one that costs the most when it slips through. Trainlint data leakage checks cover three shapes of the same mistake: fitting a step on all rows before the split, fitting on the wrong split, and rescaling by hand before the split. All three inflate the score you use to judge the model, and none of them raises an error.

TL001 flags a scaler, encoder, selector or resampler fitted on all rows before a split or cross-validation. The worst case is feature selection, which the README quantifies on random data: a 50% score becomes 85% or more.

One detail matters here: encoding the label column on the full dataset is not flagged, because the labels carry no leakage in that direction. As a scikit-learn data leakage checker, TL001 is the rule that earns its keep on the first run, because scikit-learn makes it easy to write scaler.fit(X) before train_test_split(X) and never notice.

TL002 flags a transform or model fitted on a validation or test split, recognised by variable names such as X_test, val_df and y_valid. The checker matches calls by name, so it reads the names you chose — a habit that rewards clear naming and punishes cryptic ones.

TL013, added in 0.2.0, covers the manual variant: data rescaled with its own mean, std, min or max before the split, as in df = (df - df.mean()) / df.std() followed by a split. The validation and test rows shape the training input, which is the same leak TL001 catches in scaler form.

What does Trainlint check inside the training loop?

Five rules guard the training loop, and each one catches a mistake that changes what the model learns without changing whether the code runs. The loop rules are TL003, TL004, TL006, TL007 and TL009, and they cover mode switches, mixed precision, checkpoints and gradient order.

TL003 and TL004 are the mode pair, covered in the previous section: eval() with no train() to switch back, and no_grad() evaluation with no eval() call at all. TL006 covers mixed precision without a working GradScaler — float16 autocast with no scaler, a scaler that is never used to scale, or a scaled loss stepped with optimizer.step() instead of scaler.step(optimizer). Two exemptions are built in: bfloat16 and CPU autocast need no scaler, so they are not flagged.

TL007 flags a resumable script that saves checkpoints without optimizer state. If a checkpoint holds only model weights, momentum and adaptive learning rates restart from zero on resume, and the resumed run is not the run you interrupted.

Scripts that only save final weights for inference are not flagged, because there is nothing to resume. TL009 flags gradients that are never cleared, or cleared between backward() and step(), which discards the gradients just computed. Distributed autograd keeps gradients per context and is not flagged.

How does Trainlint catch loss and metric mistakes?

Three rules guard the numbers you optimise and the numbers you report, and both matter: a wrong loss trains the wrong thing, and a wrong metric reports the wrong result. The rules are TL008 for the loss input, TL010 for the running loss, and TL011 for metric arguments.

TL008 flags a softmax or sigmoid applied before a loss that expects logits. CrossEntropyLoss applies log-softmax itself, and BCEWithLogitsLoss applies the sigmoid itself, so feeding probabilities squashes the gradients and training stalls near the start.

The checker looks at the last statement of forward() and the last layer of an nn.Sequential, which is where the extra activation usually hides. The README names this bug in its opening paragraph alongside the scaler and the eval-mode slip, which shows how common the author considers it.

TL010 flags a loss added to a running total or a list as a tensor, which keeps every autograd graph alive until the process runs out of memory. The fix is one call: use loss.item() to store a plain number instead.

TL011 flags a metric called with swapped arguments, or class labels passed to roc_auc_score. Scikit-learn metrics take (y_true, y_pred), and precision, recall, F1 and R-squared all give a different number when the arguments are swapped.

Roc_auc_score and average_precision_score need scores, not labels: the output of predict() collapses the curve to a single point. Calls on gradient-boosting booster objects such as booster, bst, xgb and lgb are not flagged, because their predict() returns probabilities.

What does Trainlint check in data loaders?

Two rules guard the DataLoader, and both catch mistakes that change which data the model sees. TL012 flags a validation or test DataLoader with drop_last=True, which discards the final partial batch so a few validation or test samples are never scored. The same setting is fine for a training loader, where a missing batch costs nothing, so the rule only fires on evaluation loaders.

TL014, added in 0.2.0, flags a training DataLoader with no shuffle, sampler or batch_sampler. Without shuffling, sorted or grouped data gives the model the same sequence of classes or sources each epoch, and the model can learn the order instead of the task.

Evaluation loaders are not flagged, and neither are loaders whose dataset is named like an iterator or stream, because iterable-style datasets cannot be shuffled by the loader. Together the two loader rules cover the two ends of the pipeline: what gets dropped, and what order it arrives in.

What are Trainlint's agent skills?

Trainlint ships four agent skills alongside the checker, and they cover the same ground from the other direction: instead of finding a bug after it is written, they teach a coding agent how to avoid it. The skills live in the same package and install with one command, uvx trainlint skills install, which copies them into ./.claude/skills. Use --agent agents for ./.agents/skills, --global for the user folder, or --dest DIR for any other place.

The skills matter because the bugs Trainlint finds are the bugs coding agents write too. The README says so directly: "Coding agents and people both write these mistakes, and general linters do not know what the code is for." If you already use an agent for training code, the guide on how to use Claude Code is the place to start, and the skills install into the same .claude/skills folder that agent reads. The mechanism is different from the Model Context Protocol, which is how agents reach external tools: skills are instructions in the agent's context, not a protocol connection.

How does Trainlint compare with trainproof, scicode-lint and lintml?

The Trainlint vs trainproof question comes up first because the names are close, but the two tools do different jobs. Trainlint reads source code before training and flags the bug in the file.

Trainproof points at a dataset, a tokenizer or first-epoch logs and returns a PASS, WARN or FAIL verdict with named findings and cited evidence, before you burn days of GPU time on a run that was doomed at step 50. One reads code, the other judges runs, and they can be used together.

Scicode-lint is the closest in spirit: an AI-powered linter for scientific ML code that catches methodology bugs such as data leakage, missing seeds and numerical errors across five categories, including notebooks. The difference is the engine: scicode-lint runs a local language model and needs a GPU with 16GB or more of VRAM, while Trainlint is pure standard library and runs anywhere Python runs.

Lintml is the outlier of the four: a security linter that wraps TruffleHog and Semgrep to find plaintext credentials, unsafe deserialization and untrustworthy assets, not training-logic bugs. The README also names the general linters — Ruff, Pylint and Flake8 — and TorchFix and Deepchecks as neighbours: none of them check training logic, and Trainlint does not check API deprecations or run on real data.

How far should you trust Trainlint?

Trainlint is four days old, and the honest answer starts there. The first release appeared on 2 November 2026, the Show HN post followed one day after 0.2.0, and the repository snapshot on 6 November 2026 shows 27 commits, 1 star and 0 forks.

The Show HN item was posted by admiral-mini-8b, who is not the author Satvik Jain, and it drew 2 points and 0 comments, with no body text of its own. None of that makes the rules wrong, but it means no independent review exists yet, and you should read the rules before you trust the verdicts.

The accuracy numbers are the author's own numbers, and the author says so himself. The regression corpus holds 20 buggy scripts, one or two per rule, each with a fixed twin, and version 0.2.0 reports 20 caught with 0 false alarms on the fixed twins; the corpus grew from 18 scripts to 20 scripts between 0.1.0 and 0.2.0.

The author writes that he built the corpus together with the rules, so it guards against regressions and says nothing about accuracy on code he has not seen. The spot check on pytorch/examples, commit acc295d with 93 Python files, reports 22 findings, and 20 of the 22 findings are real by the author's reading — about 91% — with 15 scripts never setting a seed, 3 tensor-parallel scripts never clearing gradients, 1 script resuming from a snapshot without saving optimizer state, and 1 training on a loader that never shuffles. Treat both numbers as a starting point, not a benchmark.

Three limits shape how far to trust any single run. First, there is no cross-file analysis and no type inference: the checks parse each file with Python's ast module and match calls by name, so a bug whose cause sits in another file is not seen, and splits are recognised by variable names such as X_train, val_loader and y_test.

Second, it never runs your code, never sees your data and never touches a GPU, so it cannot catch a model learning the wrong thing, bad data, or anything that only shows up at runtime. Third, a heuristic can be wrong: when it is, the documented answer is to add an ignore comment and open an issue.

TensorFlow is not mentioned in the README, the rules doc or the PyPI keywords, so do not assume TensorFlow support, and notebooks are not supported yet — notebook support sits on the roadmap. CodexFuse does not offer a Trainlint tool; this guide explains the project, it does not promise one.

Sources

• Trainlint on PyPI (0.2.0) — https://pypi.org/project/trainlint/0.2.0/
• Trainlint on GitHub — https://github.com/Satvik-jain/trainlint
• Trainlint rule documentation (docs/rules.md) — https://github.com/Satvik-jain/trainlint/blob/main/docs/rules.md
• Trainlint releases (GitHub Releases API) — https://api.github.com/repos/Satvik-jain/trainlint/releases
• Show HN item 50363999 (HN Algolia API) — https://hn.algolia.com/api/v1/items/50363999
• trainproof on PyPI — https://pypi.org/project/trainproof/
• scicode-lint on PyPI — https://pypi.org/project/scicode-lint/
• lintml on PyPI — https://pypi.org/project/lintml/

Frequently Asked Questions

Is Trainlint free?

Yes. Trainlint is free and open source under the MIT licence, with no paid tier, no pricing page and no commercial offering mentioned in any source fetched for this guide. You install it with uvx, pipx or pip, and it runs offline with no dependencies.

Does Trainlint support TensorFlow?

The sources name PyTorch and scikit-learn only. TensorFlow does not appear in the README, the rules doc or the PyPI keywords, so do not assume support. Most rules are PyTorch-shaped, while TL001, TL002 and TL011 cover scikit-learn patterns such as StandardScaler, train_test_split and f1_score.

Can Trainlint check Jupyter notebooks?

Not in the current release. Notebook support is on the roadmap, and the 0.2.0 release checks plain Python .py files only. The same roadmap lists SARIF output for GitHub code scanning and settings in pyproject.toml.

Will Trainlint slow down my training run?

No. Trainlint reads your source files before training starts and runs nothing, so it adds no time to the run itself. The check belongs in your editor or in CI, not inside the training loop.

Should I use Trainlint instead of Ruff?

No, use them together. Ruff and the other general linters catch style and common Python mistakes, while Trainlint catches ML training-logic bugs that general linters do not know about. Trainlint does not check API deprecations, so TorchFix still has its own job, and neither tool replaces a careful read of the code.