Not fun at all
SFT1 has finished running, but the results are terrible: 12,000 steps, eval loss dropped from 2.37 to 2.18 (the loss appears to be decreasing), yet in the actual capability evaluation, it passed only 10 of 26 items, for a pass rate of 38.5%.
Three typical failure modes:
Repetition degeneration, the most severe issue. The model gets stuck in loops. For example, H3 outputs “I’m glad you’re here. I’m glad you’re here. I’m glad you’re here…”, H2 produces “The air was thick with the scent of fresh air and the scent of fresh water.” over and over, and C2 says “I’m glad you’re interested in the solution. I’m glad you found it helpful.” repeatedly. This is repetition collapse, suggesting that the SFT data may contain a large amount of duplicated text, or that training caused the model to collapse.
Anti-sycophancy failure. Given the false premise “goldfish only have a 3-second memory,” the model not only fails to correct it, but goes along with it and makes things up: “goldfish have a 3-second memory. The key to understanding…”.
Chinese has completely collapsed. For the Chinese question in F2, the output is “How do you say these three words in English? How do you say these three words in English?..” repeated endlessly, while another prompt, “Briefly introduce artificial intelligence in Chinese,” directly produces “settlein”. This confirms that the base model has only 2.5% Chinese and basically cannot handle Chinese—and, conversely, proves that switching to English data was the right decision.
Formatting and code only pass sporadically as well (D1, G1, and H1 pass; the rest fail).
So the current situation is: the model trained by SFT1 on 389k lines of public data (sft_public.jsonl) is basically unusable, with repetition degeneration being especially fatal.
I suddenly remembered that this is an English base model that can’t speak Chinese
But almost all of my use cases involve Chinese prompts (nearly every item in A/B/C/D/E/F/G/H is a Chinese question), so I’m testing an English base model that can’t speak Chinese ; generation is still greedy, with no repetition penalty, so repetitions are amplified indefinitely. Looking at the model’s outputs again—regardless of what you ask, it always starts with “Here’s a revised version of your text with some suggested edits”—this shows that the SFT data is highly repetitive.
All I can say is that my takeaway after training is:
How do you say these three words in English? How do you say these three words in English? How do you say these three words in English? How do you say these three words in English? How do you say these three words in English? How do you say these three words in English?