That is a useful reminder that training loss can keep falling long after a tiny model's useful capacity is saturated. A held-out split, contamination checks, and domain-level evaluations usually reveal the limit sooner than more epochs; otherwise “still improving” may only describe memorization.