🌌 Lost in Hyperspace
Navigate embedding space itself — vectors, distances, and what 'nearby' means.
📜The task, in plain English
IOAI 2024's embedding-space thread: tasks built around working directly with high-dimensional vector representations — measuring similarity, navigating neighborhoods, and engineering features for a fixed model (one 2024 ML task literally fixed the model and judged only your features).
🔧The baseline you're given
Raw features / raw embeddings fed straight to the fixed model. Limitation: the representation, not the model, is the score bottleneck — which is the point.
🚀Baseline vs. solution
- Raw vectors as-is
- No normalization or feature thinking
- Normalize, combine, and transform features; measure what actually helps via validation
- Distances in high dimensions behave unintuitively — cosine similarity over Euclidean for embeddings
- Feature engineering is experimentation with a scoreboard: try → measure → keep
🧒Explain it like I'm brand new
Why embeddings keep appearing: modern AI's universal trick is turning anything (words, icons, sounds) into vectors where geometry = meaning. Every year at least one task boils down to "embed, then do geometry." 2024 made the geometry itself the task.
💬The Gemma 4 playthrough (2000-token limit)
X is (n, 512) embeddings. Give ONLY code for L2-normalizing rows and computing the full cosine similarity matrix with numpy. Max 6 lines.
import numpy as np Xn = X / np.linalg.norm(X, axis=1, keepdims=True) sim = Xn @ Xn.T
🎯Takeaways & what Day 1 might do with this
- Pattern family: embeddings + similarity — the through-line of the entire olympiad.