🩹 Help BOBAI (fix the model)
A model is broken or must grow new abilities — repair and extend instead of rebuilding.
📜The task, in plain English
IOAI 2024's NLP thread: work with a language model on ciphered text of an unknown language — fine-tune it for classification, then extend the classifier to more classes than it was trained for. You can't rely on pretrained knowledge of the language (it's ciphered!) — the model must learn structure from the data itself.
🔧The baseline you're given
A standard fine-tuning setup for the original label set. Limitation by design: the head has the wrong number of outputs the moment new classes appear, and naive re-training risks losing what was learned.
🚀Baseline vs. solution
- Fixed-size head for the original classes
- Naive full re-training when classes change
- Head surgery: expand the classification layer, preserve learned weights for old classes, initialize new rows fresh
- Keep old-class data in the training mix to avoid forgetting
- Sound familiar? This is 2026 Night Watch's exact skeleton, two years earlier, in text instead of audio
🧒Explain it like I'm brand new
The lesson that repeats every year: competition models are rarely built from scratch — they're adapted. Load pretrained thing → change the last layer → train carefully without destroying what works. 2024 did it with text, 2025 with chicken counting, 2026 with audio. Learn the move once.
💬The Gemma 4 playthrough (2000-token limit)
Here is the printed module tree of my model: <paste print(model)>. I need the final classification layer expanded from N to M outputs keeping existing weights. Give ONLY the surgery code for THIS structure. Max 15 lines.
🎯Takeaways & what Day 1 might do with this
- Pattern family: model repair / head surgery — a yearly ritual since the first edition.