Hi. For now, I tried a few lightweight checks. If the focus is HSL specificity, maybe something along these lines:
My read is that the current result already supports a useful, deliberately narrow claim:
The frozen HSL path can carry a usable global Cat/Dog condition into this DDPM carrier.
The same-noise Cat↔Dog changes, the unconditional control, and the learned-label control make that a reasonable proof-of-operation result. I also think the wording in the post — comparable, not better, single training seed, qualitative, two classes — is the right level of caution.
The next question is slightly different:
Did this work because the code was specifically HSL, or because the learned readout and U-Net only needed two stable, distinct class codes?
For that question, my default next step would be a same-shape fixed-random codebook arm, before spending the available compute on a much longer quality run.
A clean comparison would look roughly like this:
fixed HSL codebook ┐
fixed random codebook ├─> identical 216→128→128 readout
learned codebook ┘ ↓
identical U-Net condition surface
For the fixed-random arm, I would keep the real Cat/Dog supervision unchanged and randomize only the two frozen code vectors. Matching the width and approximate norm/scale of the HSL codes would remove easy numerical confounds. If compute allows, using more than one random codebook seed would also help distinguish a general result from an unusually convenient or inconvenient pair of random vectors.
This is not the same thing as training on random labels. The labels remain correct; only the frozen class codebook changes.
Given the stated free-T4 / small-GPU budget, I would probably prioritize:
- A short fixed-random arm
- Two or three training seeds rather than one very long seed
- A small class-consistency evaluation using a fixed sampling-noise bank
- Only then, longer training or heavier distribution metrics
The highest-value first number may be simpler than FID: for each initial noise, generate both Cat- and Dog-conditioned outputs and report how often an independent Cat/Dog classifier recognizes both requested classes. I would include per-class accuracy and the confusion matrix, not only one overall percentage.
That would test the exact behavior illustrated by the current grids: whether changing only the condition reliably changes the requested class.
A useful interpretation table might be:
| Result |
Conservative reading |
| HSL ≈ fixed random, and both condition correctly |
The experiment supports a general frozen-code conditioning door; HSL-specific benefit is not yet identified in this setup |
| HSL learns earlier or is more stable across training seeds |
Preliminary evidence for an optimization or inductive-bias benefit in this setup |
| HSL wins only for one training seed or one random codebook |
More replication is needed before attributing the difference to the substrate |
| All conditional arms are unstable |
Carrier training or evaluation variance is probably the immediate bottleneck |
| HSL and random are similar at convergence, but HSL reaches useful class consistency earlier |
The most interesting claim may be sample/compute efficiency rather than final quality |
Why I think the fixed-random arm is the most informative next control
There are several questions that are easy to merge together but are better kept separate.
1. Does the conditioning path function?
The current result gives evidence for this.
The published pipeline is:
label bytes
→ frozen HSL features
→ learned readout
→ 128-D condition
→ addition to the timestep embedding
→ U-Net
The forum post, repository, and model card all frame this as a minimum-scale conditioning-door test, rather than a competitive image-generation result.
2. Is an HSL code necessary in this two-class setup?
That is not yet identified.
With two labels, the system only observes two frozen input points. A learned nonlinear readout has considerable freedom to map those two points to useful condition vectors. An unrelated pair of stable vectors may therefore work similarly.
This is conceptually related to the general warning from probing research that a capable readout can sometimes learn the task itself, rather than merely reveal useful structure already present in its input. The setting is different, but control tasks for probes provide a useful methodological analogy: compare against a control representation that preserves easy surface properties while removing the structure under test.
Fixed class codewords also have a long history in classification, for example error-correcting output codes. That does not prove anything about this DDPM, but it is another reason not to assume that a fixed code must have a human-interpretable geometry before a learned network can use it.
3. Does HSL provide an optimization benefit?
This remains possible even if fixed random codes eventually work.
The useful comparison may be:
- steps required to reach a chosen Cat/Dog consistency threshold;
- variance across training seeds;
- loss or consistency at early checkpoints;
- robustness at reduced data or model capacity.
If HSL has a useful prior structure, it may show up in the learning curve rather than in one final qualitative grid.
4. Does HSL geometry matter beyond two labels?
Two labels are a particularly weak test of internal geometry because the model mainly needs to separate two points.
A stronger future geometry test would involve more classes or known relations among classes. That is probably not the first thing to spend a free-T4 budget on, but it matters for the eventual claim boundary.
A few inference-only sanity checks I tried on the released checkpoints
These were small checks on the published checkpoints, not replacement training experiments.
Post-hoc code reparameterization
I replaced the two frozen HSL input vectors with an unrelated pair of fixed 216-D random vectors and analytically refit only the first affine readout layer so that the two original first-layer targets were preserved.
The random pair was quite different from the HSL pair:
- HSL Cat/Dog input cosine: about
0.73
- random Cat/Dog input cosine: about
0.06
After refitting the first affine layer:
- final condition maximum absolute difference: about
1.9e-8
- U-Net noise-prediction maximum difference: about
9.5e-7
- deterministic DDIM tensor maximum difference: about
2.5e-5
- rendered 8-bit images differed by at most one intensity level in a very small number of channels
I would interpret this narrowly:
For the released two-label checkpoint, the learned function is not unique to the original HSL coordinate system.
This is a constructive identifiability check, not evidence that a random codebook would train equally well from scratch. The latter still requires the actual fixed-random training arm.
More generally, learned representations often have coordinate-system non-identifiability; linear identifiability of learned representations is relevant background, although it is not a direct analysis of HoLo-FuSe.
Condition timing
I also compared the released HSL and learned-label checkpoints under the same 16-step deterministic DDIM setup.
Across a few sampling seeds, both checkpoints showed a similar pattern:
- Cat/Dog prediction contrast was weak at the first high-noise step;
- it rose quickly during the early denoising region;
- switching Cat↔Dog after roughly the first 20–30% of updates often failed to change the final species;
- enabling the class condition only in the early part of sampling was much more influential than enabling it only late;
- interpolation between the final Cat and Dog condition vectors produced a similar nonlinear transition in both arms.
The detailed curves from the two independently trained checkpoints were strongly correlated, but I would not use their absolute magnitudes as a performance comparison because the U-Nets were trained separately.
My narrow reading is:
The released HSL arm behaves much like a usable global class-conditioning channel, including when the condition acts during denoising.
That supports the proof-of-operation claim. It does not isolate HSL-specific geometry.
These timing observations also apply only to the tested respaced DDIM sampler. DDIM uses the same training procedure as DDPM but a different sampling process, so I would not transfer an exact “class locks in at step X” statement to the ancestral comparison bench.
A compact evaluation ladder that fits the compute budget
Level 1: fixed qualitative bank
Use a predeclared set of sampling seeds and show all of them, or at least state how the displayed subset was selected.
For every seed, keep:
- initial noise;
- Cat output;
- Dog output;
- arm;
- checkpoint step;
- sampler and sampler settings.
This prevents accidental cherry-picking and makes later checkpoints directly comparable.
Level 2: paired class consistency
For each initial noise, generate both requested classes.
A simple paired success definition is:
classifier(Cat-conditioned image) = Cat
and
classifier(Dog-conditioned image) = Dog
Report:
- paired success rate;
- Cat accuracy;
- Dog accuracy;
- confusion matrix;
- number of generated pairs;
- results per training seed.
The classifier should be validated on held-out real images. Its real-data performance should be reported so readers know whether it is a useful evaluator.
This score measures condition adherence, not perceptual quality.
The broader literature also distinguishes condition consistency from generic image quality. See Evaluation Metrics for Conditional Image Generation and ImagenHub, which separates semantic consistency from perceptual quality.
Level 3: training-seed robustness
Sampling-seed replication is useful, but it does not measure training variance.
Even two or three short training seeds can reveal:
- whether arm rankings reverse;
- whether conditioning sometimes fails to emerge;
- whether one codebook has much larger variance;
- whether a promising difference is larger than ordinary run-to-run noise.
With a limited budget, I would rather see three shorter, transparently bounded runs than one long run whose apparent advantage might be seed-specific.
Level 4: distribution metrics
If there is enough generation budget, KID or precision/recall can add distributional information.
However:
- FID has finite-sample bias that depends on the generator, so using the same sample count for all arms does not remove every bias.
- KID uses an MMD-based estimator and is a reasonable small-sample candidate, but “unbiased estimator” does not mean “low variance with very few samples.” Subset settings and variability should be retained.
- Precision and recall for distributions can help separate sample fidelity from coverage, but the result still depends on the feature representation and implementation.
- Classification Accuracy Score asks whether a classifier trained on generated images transfers to real images. It is useful, but heavier and somewhat different from the immediate question of whether the requested class condition is followed.
I would therefore put class consistency and training-seed replication before FID/KID in this particular PoC.
Important interpretation and reproducibility traps
These are mostly ways a reasonable experiment can be accidentally described more strongly than it supports.
Fixed random codes are not random labels
A fixed-random code control should preserve the correct Cat/Dog target.
Cat → fixed random vector A
Dog → fixed random vector B
Randomly assigning incorrect labels to individual training images is a different experiment involving corrupted supervision and possible memorization. It would not answer the HSL-specificity question cleanly.
One random codebook is only one draw
If fixed random performs much better or worse, the result may partly reflect that particular vector pair.
A practical compromise is:
- one primary random codebook for the full comparison;
- two or three extra codebook seeds for shorter confirmation runs.
Norm matching is useful, but geometry matching answers a different question
Matching approximate vector norm and scale removes a simple conditioning-amplitude confound.
By contrast, making a random pair with exactly the same Gram matrix as the HSL pair produces a rotated/reflected version of the same two-point geometry. With a learned first linear layer, that is mostly a coordinate-system control, not a test of whether HSL geometry is special.
For the primary null arm, I would use an unrelated fixed codebook with comparable scale.
“Same seed” is not automatically a fully paired training trajectory
The current harness usefully resets the seed for each arm, and the carrier initialization can therefore be closely matched. However, the condition modules have different shapes and consume different amounts of randomness during initialization. Subsequent minibatch choices, flips, timesteps, noise tensors, and condition-drop masks can then diverge even though the original global seed was the same.
For a stricter paired bench, possible options are:
- separate generators for model initialization, data selection, augmentation, timestep selection, diffusion noise, and condition dropout;
- or precompute and save the batch/noise/timestep schedules.
The PyTorch reproducibility guide also notes that complete reproducibility is not guaranteed across releases, platforms, and devices, and documents worker seeding and deterministic-operation trade-offs.
I would describe the current result as seed-matched, but not assume every stochastic training event was paired.
Same step count does not necessarily mean same optimization state
If one representation makes conditioning easier to learn, comparing only at step 14k may hide the main difference.
Conversely, if one arm is simply slower, an early snapshot may exaggerate a transient gap.
Learning curves or several predefined checkpoints make this much easier to interpret.
Sampling seeds are not training seeds
Repeating inference over many noise seeds can show whether a checkpoint’s behavior is stable across samples.
It cannot show whether another training run would learn the same solution.
Post-hoc reparameterization is not from-scratch equivalence
If a trained model can be rewritten with another code system, that shows non-uniqueness of representation.
It does not establish equal:
- convergence speed;
- optimization stability;
- data efficiency;
- training-seed variance;
- multi-class behavior.
Classifier consistency is not image quality
A classifier may recognize a low-quality or stereotyped image correctly. It can also inherit biases from its own training data.
Class consistency should therefore be reported alongside sample grids and, later, a distribution or perceptual-quality measure.
Overall accuracy can hide one-class collapse
A model that produces Dog-like images for both conditions may still look acceptable under an imbalanced or poorly summarized metric.
Always retain:
- per-class accuracy;
- confusion matrix;
- balanced sampling;
- paired Cat/Dog success.
FID/KID do not test HSL specificity
Even a clear distribution-metric improvement does not by itself show why it occurred.
The fixed-random code control and training-seed structure answer the attribution question. FID/KID answer a different question about generated distributions.
Sampler-specific timing should remain sampler-specific
The live demo and the inference-only checks use fast respaced DDIM, while the main comparison bench is based on the repository’s DDPM setup.
The released Space is useful as a qualitative HSL demo, but it should not be read as a live reproduction of all three bench arms.
A one-line note in the README or Space would make this boundary especially clear:
The Space is a fast qualitative HSL demo; the reported three-arm comparison uses the bench configuration.
Condition-off and CFG-off are not identical
Setting the condition vector to zero for part of sampling removes the conditional input.
Changing when CFG is amplified modifies the difference between conditional and unconditional predictions.
These are related interventions, but not the same. Diffusers exposes guidance start/stop fractions in its guider API, which is useful vocabulary for future timing experiments.
Early class determination does not imply late conditioning is useless
The species may be selected early while later conditional guidance still affects texture, local anatomy, diversity, or image quality.
A “class identity locks in early” observation should therefore remain narrower than “all useful conditioning happens early.”
Condition interpolation is out-of-distribution
The model was trained on the Cat and Dog endpoints, not necessarily on intermediate condition vectors.
A smooth, abrupt, or strange intermediate trajectory is interesting descriptively, but it should not automatically be treated as semantic evidence about the original HSL geometry.
The shared color tint is not enough to identify one root cause
Because the tint appears in all arms, it is unlikely to be an HSL-specific artifact.
The reported CFG/dynamic-threshold sweep also suggests it is not removed by those particular sampling changes.
However, that does not uniquely identify under-training. Possible contributors still include:
- dataset color statistics;
- preprocessing;
- limited carrier capacity;
- checkpoint selection;
- EMA behavior;
- the noise schedule;
- the sampler;
- ordinary under-training.
“Consistent with a carrier/training-side artifact” is safer than claiming a uniquely established cause.
Dataset provenance matters before quantitative evaluation
The original AFHQ release contains Cat, Dog, and Wild domains, with a held-out test subset per domain. The official StarGAN v2 repository also documents an updated AFHQ version using Lanczos resizing and the removal of some images.
This project uses the community zzsi/afhq512_16k mirror and reports filtered class counts that do not exactly match the simple “5,000 per domain” description.
That is not evidence of a problem, but later quantitative work would be easier to interpret if it records:
- the exact dataset repository revision;
- split provenance;
- post-filter class counts;
- duplicate hashes across train/evaluation splits;
- resize and normalization settings.
Nearest-neighbor checks are useful but not proofs
With a relatively small dataset, generated-to-training nearest-neighbor panels can detect obvious copying.
Failure to find a close neighbor does not prove absence of memorization, and finding a similar animal face does not automatically prove copying. This is a secondary diagnostic, not the first HSL control.
“Zero-parameter” should retain its boundary
The frozen HSL substrate has zero learned parameters. The downstream readout is learned.
The clearest wording is therefore:
a zero-parameter frozen substrate with a learned conditioning readout
Shortening this to “zero-parameter conditioning” may be read as saying that the complete conditioning path is untrained.
Budget-matched is not necessarily exactly parameter-matched
The current paths meet at the same final condition width, but their internal input widths and learned components are not identical.
That does not invalidate the PoC, and the difference is small relative to the U-Net. Still, a future three-arm setup with one identical readout and only the codebook type changed would be much easier for readers to interpret.
A compute-aware default route
Given the stated hardware budget, I would not make the next milestone depend on a large benchmark.
Lowest-cost route
- Keep the current claim as a proof of operation.
- Publish a fixed evaluation-noise bank.
- Add exact parameter counts for each condition module.
- Clarify the Space/bench sampler boundary.
- Pin dataset and repository revisions.
One additional training arm
Add:
fixed unrelated random 216-D codebook
→ same 216→128→128 readout
→ same carrier
Use the existing training recipe and compare at predefined checkpoints.
Several short runs
Run HSL, fixed random, and learned with two or three training seeds.
At each checkpoint, record:
- loss;
- paired Cat/Dog consistency;
- per-class consistency;
- a fixed qualitative grid.
If one arm clearly separates, extend only that comparison.
More compute later
Then consider:
- several random codebook draws;
- KID or precision/recall;
- a larger number of classes;
- tests of class relationships or compositional labels;
- memorization diagnostics;
- longer carrier training.
This keeps the first next experiment tightly connected to the current claim rather than turning a small conditioning PoC into a full image-generation benchmark.
Overall, I think the current result is useful precisely because it is narrow and reproducible enough to ask the next question.
The result already says:
A frozen HSL-derived path can function as the class-conditioning door in this small DDPM.
The fixed-random arm would make the next sentence much easier to choose:
- either “this setup mainly needs any stable frozen class code,”
- or “HSL shows an early-learning or stability advantage worth testing further.”
Both outcomes would be informative, and neither would erase the proof-of-operation result already shown.