Why Does My AI Think Math Is a Fishing Trip?
I asked my model to solve a simple integral. It responded with a detailed description of trout migration patterns. This is not the answer I was looking for, though I admit the trout explanation was surprisingly well-structured.
Training a small language model is like teaching a very enthusiastic puppy. It wants to please you. It really does. But sometimes it hears "calculate" and thinks you said "salmon" and suddenly you are knee-deep in aquatic biology when you just wanted to find the area under a curve.
The Great Derivative Debacle
There is a special kind of pain in watching your model confidently explain the chain rule by describing how fishing nets are woven. The word "chain" apparently triggered some deep association with tackle boxes and I lost an afternoon debugging why my math tutor suddenly thinks it's a marine supply catalog.
I have tried everything. More data. Better tokenization. Careful prompt engineering. The model still looks at "polynomial" and whispers "poly-wobbly fish" under its digital breath. At this point I am not even sure if the problem is the architecture or if I accidentally fed it too many nature documentaries during pre-training.
Attention mechanisms are supposed to focus on what matters. Mine focuses on anything with scales, fins, or the word "scale" which in fairness is a legitimate ambiguity but also deeply unhelpful when doing calculus.
Self-Reflection Time
Maybe the issue is me. Maybe my prompts are weird. Maybe I have a subconscious bias toward aquatic themes that leaks into my training data. I did once spend a summer working at an aquarium. The model remembers. The model never forgets. The model will not let me forget either.
I have started keeping a log of these incidents. "Asked: what is the derivative of x squared. Received: a poem about swordfish." It is both hilarious and deeply concerning. My thesis advisor asked for progress updates. I sent the swordfish poem. They did not respond. I think they are reconsidering their life choices.