🌧️ Weather
Predict rain from satellite images plus context features — fusing two data types in one model.
📜The task, in plain English
Predict rainfall from GOES-16 satellite imagery combined with context features (sun angle, time of day, location). Neither source is enough alone: clouds look different at different sun angles and locations.
| Input | Satellite image bands + a small vector of numeric context features |
|---|---|
| Output | Rain prediction |
| Really tests | Fusing image features with tabular features in one network — a pattern 2026's Robot Delivery reuses (grid + 13-dim vector) |
🔧The baseline you're given
Typically an image-only CNN or a features-only model. Limitation: ignores half the signal — the fusion is the point.
🚀Baseline vs. solution
- Uses image OR features, not both
- Basic CNN, default preprocessing of satellite bands
- Two-branch network: CNN encodes the image → feature vector; concatenate the context features; a small MLP head makes the prediction
- Normalize satellite bands per channel; normalize tabular features too (models hate mixed scales)
- Sanity baseline first: gradient boosting on context features alone — know what the "cheap" signal is worth before fusing
🧒Explain it like I'm brand new
Fusion in one sentence: run the image through a CNN until it becomes a vector of learned features, then just staple the extra numbers onto that vector (concatenate) and let a final small network combine them. That "encode → concat → head" template is the standard way to mix images/audio/text with plain numbers, and it appears in 2026's Robot task verbatim.
💬The Gemma 4 playthrough (2000-token limit)
Write ONLY a PyTorch nn.Module: branch A = a provided encoder `enc` mapping (B,C,H,W) to (B,512); input B = (B,7) float features. Concat both, then Linear(519→64) → ReLU → Linear(64→1). Code only, max 20 lines.
🎯Takeaways & what Day 1 might do with this
- Pattern family: multimodal fusion (encode → concat → head).
- Also a lesson in cheap baselines: always know what tabular-only or majority-class scores before investing in the fancy model.