2025 At-Home Round

🦎 Chameleon

Guess the secret word from a sequence of icons — embeddings and semantic similarity.

NLPEmbeddingssentence-transformers★★☆☆ great first embeddings task

📜The task, in plain English

A word-guessing game: you see an ordered sequence of icons (each icon comes with a text description), chosen by a "clue-giver" to hint at a secret word. Your model must predict the secret word. Example: icons for water + fall + mist → "waterfall". The order and combination of clues carry meaning.

Input	Sequence of icons with text descriptions
Output	The secret word (from a vocabulary)
Really tests	Text embeddings + semantic similarity: mapping words and descriptions into vector space and measuring closeness

🔧The baseline you're given

The baseline embeds icon descriptions and candidate words with a pretrained sentence encoder (sentence-transformers) and picks the candidate whose embedding is most similar (cosine) to a simple average of the clue embeddings. Limitations: naive averaging treats clues as an unordered bag, ignores that some clues matter more, and never adapts the embedding space to the game.

🚀Baseline vs. solution

⚠️ The baseline (what you are given)

Mean of clue embeddings → nearest word
Ignores clue order and importance
Off-the-shelf encoder, no task adaptation

✅ The winning approach

Better pooling: weighted combination of clues (position weighting, max-sim per clue)
Score candidates by sum of similarities to each clue rather than similarity to the average — combinations like water+fall need both parts to match
Try stronger sentence encoders from the provided model list; ensemble two encoders' rankings
Validate on the provided games; inspect misses — most fixes come from reading errors, not more math

🧒Explain it like I'm brand new

What's an embedding? A function that turns text into a list of ~400 numbers such that similar meanings get nearby numbers. "dog" and "puppy" land close together; "dog" and "carburetor" don't. Once everything is a point in space, "which word do these clues point at?" becomes geometry: measure distances (cosine similarity) and pick the closest. That one idea — embed, then compare — solves Chameleon, Restroom, Concepts, and appears at IOAI every year.

💬The Gemma 4 playthrough (2000-token limit)

Chat 1 · Recipe

YOU

I have clue descriptions (list of strings) and candidate words (list of strings). Using sentence-transformers (model already cached locally), write ONLY code to embed both, and rank candidates by the SUM of cosine similarities to each clue. Code only, max 25 lines.

GEMMA 4

from sentence_transformers import SentenceTransformer
import torch

model = SentenceTransformer('/path/to/local/model')
E_clues = model.encode(clues, convert_to_tensor=True, normalize_embeddings=True)
E_cand  = model.encode(candidates, convert_to_tensor=True, normalize_embeddings=True)

sims = E_cand @ E_clues.T          # (n_cand, n_clues)
scores = sims.sum(dim=1)
best = [candidates[i] for i in scores.argsort(descending=True)[:5]]
print(best)

Verify on 3 games where you know the answer before trusting the pipeline.

🎯Takeaways & what Day 1 might do with this

Pattern family: embeddings + cosine similarity — the most reliably recurring IOAI pattern.
On the real Day 1 of 2025, this at-home task evolved into Concepts — same semantic-similarity core with an LLM judge added. Master the simple version and the extension is incremental.