How many images to parameters?

🧨 Diffusers
1

I’m training a latent diffusion model and wondering what the scaling laws were, I couldnt find much info online. My current is around 430M parameters training on around 2 Million images. I’m thinking of possibly scaling the parameters up a bit to increase the quality. Is this a good idea or should I scale down my images or parameters? Based on SD, they seemed to use billions for their models which were at 860M, half of mine

2

Hmm… for now, I don’t think the number of images per parameter alone gives us enough information to decide:

I would not reduce the number of useful images just to make an images-per-parameter ratio look more similar to Stable Diffusion. I also would not assume that increasing 430M to 860M will automatically improve quality.

The main reason is that “430M parameters / 2M images” combines several different constraints:

  • what the 430M parameters belong to;
  • whether this is training from scratch or continued pretraining;
  • how many times the images are actually presented;
  • the semantic, caption, resolution, and aspect-ratio distribution of those images;
  • the training compute available;
  • and what kind of “quality” is currently missing.

There is an unusually close reference point: Bigger is not Always Better: Scaling Properties of Latent Diffusion Models includes an exact 430M-parameter denoising UNet in a controlled family ranging from 39M to 5B parameters. That at least shows that 430M is a perfectly plausible model size rather than an obviously undersized configuration.

However, that 430M model was trained from scratch under a very different regime—about 600M image-text pairs, 500K updates, and global batch 2048—so I would not transfer its image/model ratio directly to a 2M-image dataset. The useful lesson from that paper is instead that model size, training compute, and inference budget need to be compared separately.

My default route would be:

  1. Keep the 2M useful images.
  2. Check whether the current 430M model is still improving on held-out data.
  3. Separate the current failure mode—underfitting, prompt alignment, rare concepts, composition, high-resolution detail, or overfitting.
  4. Compare the current model with a moderately larger model in a small controlled run, rather than immediately committing to a full larger-model training run.
  5. Compare at both:
    • matched sample presentations; and
    • matched training compute or wall-clock budget.

If the larger model wins under both comparisons, scaling it up becomes much easier to justify. If all model sizes fail in the same way, the bottleneck is more likely to be the data, captions, VAE, resolution pipeline, or training objective.

The most useful distinctions

Information Why it changes the answer
Training from scratch vs. continued pretraining Scratch training must learn the broad visual/text prior; CPT can reuse it but may forget old capabilities
430M denoiser vs. 430M total pipeline Stable Diffusion parameter figures often count only the denoising network
Updates and global batch size These determine total sample presentations; unique image count alone does not
Source and training resolutions Two million 512px crops and two million native 1536px samples are not equivalent training regimes
Caption format and density Short tags, long natural-language captions, and noisy web captions teach different conditional mappings
Specific quality failure More capacity may help fine detail but not necessarily bad crops, missing concepts, or caption noise
Training and inference budget A larger model costs more per update and per denoising step
What the ratio hides—and the closest controlled evidence

1. First make sure the parameter counts are comparable

If by “860M” you mean the Stable Diffusion 1.x denoising UNet, then 430M is approximately half of that—not the entire Stable Diffusion pipeline.

The controlled 430M study explicitly notes that its parameter, training-cost, and inference-cost figures count only the latent denoising UNet, excluding its text encoder and latent encoder/decoder. Its models were produced by changing the base channel width of an SD1.5-style UNet while keeping the rest of that architecture family broadly fixed.

The tested denoisers were:

Denoiser size 39M 83M 145M 223M 318M 430M 558M 704M 866M 2B 5B

Source: Bigger is not Always Better.

This makes it a close reference for model-size behavior, but not necessarily for your architecture. A 430M DiT, an SD1-style UNet, and an SDXL-style UNet can allocate their parameters very differently.

A second controlled study, On the Scalability of Diffusion-based Text-to-Image Generation, trained UNet and Transformer variants from roughly 0.4B to 4B parameters under controlled conditions. It found that:

  • the location and amount of cross-attention mattered;
  • increasing Transformer depth was more parameter-efficient for text-image alignment than simply increasing channel width;
  • an efficiently designed UNet could be smaller and faster than the SDXL UNet while performing better in the study’s controlled setting;
  • data quality and diversity mattered more than raw dataset size alone;
  • caption density and diversity changed learning efficiency.

So even after the denominator is corrected, the total number of parameters does not fully specify the model’s effective capacity.

2. Replace one ratio with three budgets

I would track at least three separate quantities.

A. Unique-data support

This is the part closest to “number of images,” but it should include:

  • unique and near-unique image-caption pairs;
  • domain breadth;
  • frequency of each concept;
  • frequency of compositions and relationships;
  • caption correctness and coverage;
  • distribution of image quality;
  • distribution of source resolutions and aspect ratios.

Two million images can be enormous for a narrow domain and still be sparse for broad general-purpose text-to-image generation.

For example, a dataset may contain many portraits but very few:

  • multiple-character interactions;
  • unusual camera angles;
  • hands manipulating objects;
  • rare species or objects;
  • spatial relationships;
  • text in images;
  • uncommon attribute combinations.

In that case, increasing the total number of portraits does not directly repair those missing regions of the distribution.

B. Exposure and spatial budget

A useful first accounting value is:

sample presentations = optimizer updates × global batch size

This is still approximate because sampling weights, dropped examples, repeats, and distributed-loader behavior can change the real exposure distribution, but it is much more informative than the number of files alone.

The exact-430M study used:

  • 500,000 updates;
  • global batch size 2048;
  • about 600M source pairs.

That is roughly 1.024B total sample presentations. This does not mean a 2M-image dataset should also receive 1.024B presentations; doing so would imply hundreds of average repeats and would be a completely different generalization/memorization regime. It only illustrates why “600M images” and “2M images” are not enough to compare the two training runs.

Resolution adds another dimension. With the same VAE downsampling factor, a 1536×1536 sample has about 2.25 times as many latent spatial positions as a 1024×1024 sample. It also costs substantially more to process, although the exact compute ratio depends on the architecture and attention implementation.

I would therefore record:

  • source resolution;
  • post-resize resolution;
  • crop size;
  • aspect-ratio bucket;
  • latent height and width;
  • presentation count per bucket.

This is not a proposal for a universal “latent tokens per parameter” law. It is just a more informative accounting method than treating every image file as one equivalent unit.

C. Compute budget

The larger model is more expensive per update. Therefore:

  • at equal update count, the larger model receives more training compute;
  • at equal compute, the larger model sees fewer updates or fewer samples;
  • at equal sampling steps, the larger model costs more at inference;
  • at equal inference cost, the smaller model may be allowed more denoising steps.

The exact-430M scaling paper found that under moderate training budgets, performance correlated strongly with cumulative training compute. With relaxed compute, larger models reached better final quality and detail. Under a fixed inference budget, however, smaller models frequently performed better because they could use more denoising steps for the same cost.

That means there are at least two valid questions:

  1. Which model has the higher quality ceiling?
  2. Which model produces the best quality within my actual training and inference budget?

They need not have the same answer.

3. Scaling laws exist, but they are conditional

I would avoid saying that diffusion models have no scaling laws. There are controlled scaling studies, including scaling laws for Diffusion Transformers, which fit relationships among model size, data, compute, and pretraining loss.

The limitation is transferability.

Those curves are estimated under particular choices of:

  • model family;
  • optimizer and learning-rate schedule;
  • caption distribution;
  • data distribution;
  • image and latent resolution;
  • objective;
  • compute range;
  • repetition regime.

A fitted coefficient from a data-rich DiT experiment is not automatically valid for a finite 2M-image latent-diffusion corpus. It is better used as evidence that controlled pilot runs can produce a useful local scaling curve for your own setup.

4. Image count is not information count

Recaptioning work such as RECAP shows that the same images can provide more useful conditional supervision when their captions are made more faithful and informative.

However, “make every caption maximally long” is not a universal answer either. A later controlled study on synthetic-caption design found that dense captions improved text alignment but could trade off against aesthetics or diversity, while varying caption length produced a more balanced result.

I would therefore treat these as separate dataset variables:

  • caption correctness;
  • caption density;
  • caption length distribution;
  • tags vs. natural language;
  • terminology consistency;
  • train-time caption style vs. inference-time prompt style;
  • whether relationships and attributes are actually described.
How the answer changes by training type and failure mode

1. If this is training from scratch

For scratch training, the 2M-image corpus is responsible for teaching much more than the target visual style. It must provide enough support for:

  • object and scene concepts;
  • visual composition;
  • relationships between objects;
  • text-image correspondence;
  • rendering statistics;
  • resolution and aspect-ratio behavior;
  • the intended prompt language.

In this branch, I would initially consider four possibilities before increasing parameter count:

Undertraining

If held-out loss and fixed validation generations are still improving, the current 430M model may simply need more compute.

A larger model can make this worse under a fixed budget because each update becomes more expensive.

Insufficient semantic coverage

If the model handles frequent concepts but fails on rare concepts, relations, or unusual compositions, the issue may be the frequency distribution rather than total capacity.

A useful audit is to group validation prompts by:

  • frequent vs. rare concepts;
  • one object vs. multiple objects;
  • common vs. unusual aspect ratios;
  • simple vs. relational captions;
  • common vs. rare attribute combinations.

Conditioning bottleneck

If visual quality is acceptable but prompt following is weak, increasing only the image denoiser may not be the most efficient use of parameters.

The text encoder, tokenizer, caption quality, cross-attention placement, and number of Transformer blocks may matter more. This is consistent with the controlled architecture results in On the Scalability of Diffusion-based Text-to-Image Generation.

Optimization bottleneck

Diffusion training is sensitive to how timesteps are sampled and weighted. Min-SNR weighting, for example, was motivated by conflicting optimization directions between timesteps and substantially accelerated convergence in its experiments.

This does not mean Min-SNR is necessarily correct for every architecture, but it is a reminder that a training curve that looks capacity-limited may instead reflect:

  • loss weighting;
  • timestep sampling;
  • prediction target;
  • noise schedule;
  • learning rate;
  • warmup;
  • effective batch size;
  • optimizer stability.

The current Diffusers SDXL training example exposes several of these variables, including SNR weighting, timestep bias, prediction type, EMA, LR scaling, and gradient accumulation.

2. If this is continued pretraining

If the 430M model begins from a pretrained checkpoint, the interpretation changes substantially.

The new 2M images do not need to relearn the complete visual world from zero. They can modify an existing representation, but then the important questions become:

  • How much does the target domain improve?
  • Which base capabilities are retained?
  • Does prompt compositionality regress?
  • Do unrelated domains deteriorate?
  • Does the model become more dependent on the new caption style?
  • Does the new resolution regime damage old resolution behavior?

T2I-ConBench proposes four useful evaluation dimensions for continual text-to-image post-training:

  1. retention of generality;
  2. target-task performance;
  3. catastrophic forgetting;
  4. cross-task generalization.

You would not necessarily need to run that full benchmark, but its categories are useful for constructing a small retention suite.

For example:

  • a fixed prompt set from the original model domain;
  • a fixed prompt set from the new target domain;
  • prompts combining old and new concepts;
  • multiple resolutions and aspect ratios;
  • the same seeds before and after post-training.

3. If the main problem is high-resolution quality

I would not immediately interpret high-resolution failure as insufficient parameter count.

Possible bottlenecks include:

  • low-resolution source images;
  • small images enlarged to the nominal training size;
  • destructive square crops;
  • poorly distributed aspect-ratio buckets;
  • insufficient exposure to large latent grids;
  • a VAE that already loses the relevant details;
  • a noise schedule or objective that behaves poorly at the new resolution;
  • insufficient compute per high-resolution sample.

SDXL itself introduced original-size, crop-coordinate, and target-size conditioning and was trained across multiple aspect ratios; see the SDXL technical report.

The current Diffusers SDXL example resizes images to a configured resolution and then uses center or random square cropping. The still-open aspect-ratio bucketing request specifically notes that fixed 1024×1024 crops can remove important image content and reduce model quality.

That does not mean bucketing is unavailable in the wider ecosystem—other trainers implement it—but it does mean the actual preprocessing code is part of the model definition in practice.

A useful boundary-breaking example: Illustrious XL

Illustrious XL’s technical report explicitly states that it is built on the SDXL architecture without changes. Its official model card describes v1.0 as an SDXL model with a native 1536×1536 resolution.

In other words, it remains an SDXL-compatible checkpoint rather than requiring a new denoising architecture, yet its practical high-resolution behavior differs from original SDXL.

That is a strong example of why “supported resolution” should be split into:

  • resolutions the architecture and code can technically process;
  • resolutions the learned weights handle reliably.

It also shows why parameter count alone cannot define the practical operating range.

However, it is not a clean resolution-only ablation. The Illustrious work also discusses changes to training resolution, dataset construction, batch/dropout behavior, and multi-level captions. It supports the claim that the learned operating regime can move without changing the architecture, but it does not isolate one single cause.

4. If fine detail is the problem, check the VAE first

Before blaming denoiser capacity, I would compare held-out source images with a plain:

image → VAE encode → VAE decode → reconstructed image

round trip.

If the missing details are already gone after reconstruction, a larger denoiser cannot recover supervision that is absent from its latent target.

Useful categories to inspect include:

  • faces at small scale;
  • fingers;
  • thin lines;
  • small text;
  • repeating patterns;
  • distant objects;
  • subtle textures;
  • strong highlights and dark regions.

The original Latent Diffusion Models paper frames latent compression as a trade-off between computational efficiency and perceptual detail. The right VAE and latent resolution can therefore matter as much as denoiser size for some failure modes.

5. If train quality is good but validation quality is poor

This makes data repetition, duplicates, and narrow coverage more plausible.

I would check:

  • exact duplicates;
  • near-duplicates;
  • repeated captions;
  • overrepresented artists, subjects, or compositions;
  • train/validation near-neighbor leakage;
  • whether a small subset receives very high sampling weight.

Research using an inversion-based memorization measure found that duplicating training examples rapidly increased memorization in its diffusion experiments.

The practical implication is not that every repeated image is harmful. It is that “2M files” may represent far fewer than 2M independent training examples, and the repeat distribution matters.

A controlled comparison that could answer the question

A small scaling experiment would probably be more useful than trying to infer a universal ratio from Stable Diffusion.

Recommended default comparison

Run A: current 430M model, trained longer

Purpose:

  • establish whether the current model is undertrained;
  • produce a learning curve under the existing recipe;
  • estimate the return from additional compute without changing capacity.

Run B: a moderately larger model

This does not need to jump immediately to 860M. A smaller step may be more informative and cheaper.

Keep as much as possible fixed:

  • training data and sampling weights;
  • preprocessing;
  • captions;
  • VAE;
  • text encoder;
  • objective;
  • optimizer family;
  • validation prompts;
  • evaluation seeds.

A learning rate that works for 430M may not be optimal for the larger model, so I would allow a small stability/LR check before deciding that the larger model is worse.

Optional Run C: similar parameter count, different allocation

If architecture changes are feasible, this can distinguish total capacity from parameter placement.

Possible comparisons include:

  • more depth rather than more width;
  • additional Transformer blocks;
  • different cross-attention placement;
  • different compute allocation across spatial resolutions.

The controlled AWS study found that these choices affected efficiency enough that parameter count alone did not predict the result.

Compare at more than one budget

1. Matched sample presentations

Both models see approximately the same number of examples.

This is useful for observing the effect of capacity when exposure is held roughly constant, but the larger model consumes more compute.

2. Matched training compute or wall-clock

Both models receive approximately the same practical training budget.

The larger model will usually get fewer updates or see fewer samples. This comparison answers the operational question: which model is better for the budget actually available?

3. Matched inference cost

Comparing both models at the same number of denoising steps is useful as a control, but it is not the same as equal deployment cost.

A smaller model may be able to use more steps within the same latency budget. The exact-430M study found that this can reverse the apparent ranking.

I would therefore record:

  • quality at the same step count;
  • quality after a small per-model step/CFG sweep;
  • quality at approximately matched latency or FLOPs.

What to log

Category Suggested record
Model denoiser parameters, total/trainable parameters, architecture, GFLOPs if available
Data unique examples, duplicate estimate, domain and concept distribution
Exposure optimizer updates, global batch, total presentations, average repeats
Spatial source-resolution histogram, bucket histogram, crops, latent dimensions
Conditioning caption type, length distribution, text encoder, prompt dropout
Training LR, warmup, optimizer, objective, noise schedule, timestep weighting, EMA
Cost GPU-hours, peak memory, samples/sec, estimated cumulative compute
Inference sampler, steps, CFG, resolution, latency, peak memory
Evaluation held-out loss, quality categories, prompt adherence, diversity, retention

A compact validation suite

A custom held-out suite will often be more useful than a single public benchmark.

I would include:

  • common concepts from the training domain;
  • rare concepts;
  • single-object prompts;
  • multi-object prompts;
  • spatial relationships;
  • attribute binding;
  • short prompts;
  • detailed prompts;
  • common aspect ratios;
  • extreme aspect ratios;
  • low, native, and high-resolution outputs.

For monitoring during training:

  • use fixed prompts and fixed seeds so changes are visually attributable to the checkpoint;
  • also evaluate the final candidates with multiple seeds, because one fixed seed can hide or exaggerate a difference.

If this is CPT, add:

  • base-domain prompts;
  • target-domain prompts;
  • mixed old/new prompts;
  • old and new caption styles.
How I would interpret the results, plus references

Interpreting the experiment

Observation More plausible interpretation Reasonable next route
Larger model wins on train and held-out metrics at matched presentations and matched compute Capacity or architectural scaling is helping Scale up more confidently
Larger model wins at matched presentations but loses at matched compute Higher ceiling, lower budget efficiency Choose based on available training budget
Current 430M catches up when trained longer Current model was undertrained Continue the current model
All sizes fail on the same prompt categories Shared data, caption, VAE, or objective bottleneck Improve the shared pipeline
Only high-resolution outputs fail Spatial supervision, buckets/crops, VAE, or resolution-specific optimization Audit the resolution pipeline
Prompt alignment improves with added attention/depth but not channel width Conditioning/architecture allocation bottleneck Change allocation rather than only total size
Larger model fits training data much better but validation degrades Overfit, repeated data, or insufficient coverage Deduplicate, rebalance, regularize, or add coverage
CPT improves the target domain but hurts base prompts Forgetting or mixture imbalance Add retention evaluation and adjust data mixing
Results change mostly with sampler, CFG, or step count Inference configuration is confounding the comparison Compare tuned and cost-matched inference
VAE round-trip already destroys the target details Latent bottleneck Change VAE/latent setup before enlarging the denoiser

About “quality”

A single score can conceal why one model is preferable.

At minimum, I would separate:

  • visual fidelity/aesthetics;
  • prompt alignment;
  • object presence;
  • object count;
  • colors and attributes;
  • spatial relations;
  • sample diversity;
  • high-resolution detail;
  • inference cost.

GenEval is one example of decomposing text-image alignment into object co-occurrence, position, count, and color rather than relying only on a holistic FID or CLIP score.

It may not match a specialized dataset, so I would use its categories as a template rather than treating its aggregate score as the final authority.

Practical implementation caveats

If using or adapting the Diffusers example scripts, check the exact script version rather than relying on a remembered default.

In the current SDXL text-to-image example:

  • --resolution resizes all training images toward one configured resolution;
  • the image is then center-cropped or randomly cropped;
  • --max_train_steps overrides epoch-derived training length;
  • effective batch depends on per-device batch, gradient accumulation, and process count;
  • --scale_lr changes the LR with batch/process configuration;
  • SNR weighting, timestep bias, EMA, and prediction type are optional controls.

Changing model size often changes memory use, which then changes batch size, accumulation, throughput, and possibly LR. If those move together, it becomes difficult to attribute the result to parameter count.

References most directly relevant here

So my tentative answer would be:

  • Do not shrink a useful 2M-image dataset merely to match an apparent Stable Diffusion ratio.
  • 430M is not obviously too small; there is even a controlled 430M LDM reference point.
  • A larger model may raise the quality ceiling, especially for fine detail, but it may be worse within a fixed training or inference budget.
  • The next useful step is a controlled learning-curve comparison, not a universal ratio calculation.

The three details that would narrow the recommendation most are whether this is scratch training or CPT, what exactly the 430M count includes, and which specific quality failures are currently visible.

3

Thank you, this is extremely useful. I have decided to do option B: scale up parameters a bit

A big thing my past models struggled on was following more complex prompts. For example, in trying a prompt like “A girl with one blue eye and one red eye”, it would only get that correct MAYBE 1/30 images~. With enhancing prompts to make them longer and more detailed on simple prompts it also significantly struggles, results become ‘deep-fried’, and more glitchy.

I decided to go up to about 600 million parameters because of this, and because more difficult things such as hands, it does poorly; even sometimes trying to hide them all together (although I know that parameters alone will not solve that)

If scaling up parameters helps, I may shift it up slightly again after. At the same 70,000-step, effective-batch-64 budget, the new model is estimated to require approximately 12.69 EFLOP, compared with 8.31 EFLOP for the previous model, a 52.7% increase. The final total will depend on the eventual training length.

From this I’ve also taken much more time into the quality of the dataset, removing over 100,000 images from the dataset.

The U-Net is trained from scratch here.

4

yw. Hmm. Based on that information, there may be a few things worth adding, just in case:

Since the U-Net is being trained from scratch, the step and batch figures give another useful way to interpret the run:

70,000 updates × effective batch 64 = 4.48 million sample presentations

If the cleaned dataset is now approximately 1.9 million images and sampling is roughly uniform, that is only about 2.3–2.4 presentations per image on average. This is not evidence by itself that the model is undertrained—there is no universal target number of epochs—but it does mean this is a relatively low-repeat, data-rich regime. The 70K checkpoint may therefore be measuring both model capacity and how quickly each model learns under limited exposure.

Your EFLOP estimates also provide a convenient extra checkpoint for the 600M run. Assuming cumulative compute remains approximately linear with steps and the rest of the setup stays comparable:

70,000 × 8.31 / 12.69 ≈ 45,800 steps

So saving a checkpoint near 45.8K steps would give three useful comparisons:

Comparison Approximately controlled variable
430M @ 70K vs. 600M @ 45.8K Similar cumulative training compute
430M @ 70K vs. 600M @ 70K Similar sample presentations
430M @ 70K vs. a later 430M checkpoint, if available Additional-training / undertraining control

They will not be perfectly controlled experiments, but preserving those checkpoints would make the new run much easier to interpret. This distinction between model size, training compute, and inference cost is also one of the main findings of the latent-diffusion scaling study containing the exact 430M configuration.

The different-eye-color example is also a useful diagnostic, although I would classify it more specifically as fine-grained attribute binding plus local spatial binding, rather than a pure test of parameter count. The model has to preserve two instances of the same small feature, distinguish their locations, and attach a different color to each one.

Attribute binding, color attribution, and spatial relationships are known difficult categories in T2I evaluation; GenEval and T2I-CompBench provide nearby evaluation taxonomies, although neither is an exact test of two differently colored eyes on one face.

A small fixed prompt panel could make that result easier to generalize beyond one prompt:

  • one blue eye and one red eye;
  • a red left glove and a blue right glove;
  • shoes of two different colors;
  • two characters with separately specified hair or clothing colors;
  • one object to the left of another;
  • common versus rare attribute combinations.

Using the same prompts and seeds at each checkpoint is useful for tracking development. For the final comparison, multiple seeds per prompt are still important—the observed 1/30 success rate already shows why a single generation would be noisy.

For the longer prompts becoming “deep-fried,” two inexpensive controls may be worth recording before attributing the effect entirely to model capacity:

  1. Inspect the actual tokenized prompt and any truncation or chunking.
    If the conditioning encoder is CLIP-based, the standard Hugging Face CLIP configuration defaults to 77 positional tokens. Some pipelines truncate, while some interfaces implement prompt chunking, so the effective conditioning sequence may differ from the visible prompt.

  2. Run a small CFG sweep on the same long prompts.
    High classifier-free guidance can produce oversaturation and unrealistic artifacts; this behavior is discussed directly in work such as Adaptive Projected Guidance. If the “deep-fried” effect changes substantially with guidance while the prompt stays fixed, inference configuration is contributing to the symptom. If it remains across reasonable guidance settings, the training distribution or conditioning capacity becomes more plausible.

For the dataset cleanup, one useful record to keep is the removal reason and the before/after counts for difficult categories. Removing duplicates, corrupted images, bad captions, and genuinely out-of-domain material can improve the usable signal. At the same time, valid but difficult examples—hands, multiple people, unusual poses, rare compositions, and precise attribute combinations—may be exactly the cases needed for the failures you want to improve.

A small held-out set selected before further filtering, grouped by those failure categories, would make the 430M→600M comparison especially informative.

So the additional information I would preserve from this run is:

  • the approximately 45.8K matched-compute checkpoint;
  • the 70K matched-presentation checkpoint;
  • fixed short-prompt and long-prompt validation panels;
  • a small attribute-binding panel;
  • tokenized prompt lengths and truncation behavior;
  • a small CFG sweep;
  • dataset counts by removal reason and difficult-example category.

None of these require changing the 600M plan. They mainly help ensure that, whatever the result is, the run tells you whether the improvement came from greater capacity, greater compute, better data, or a change in conditioning behavior.