👀 tempting to reproduce this with [model to come]
an interesting result that doesn’t surprised me at all: dropping memorization affects significantly math results. from what I’ve seen, similarly to humans, models need to memorize standard operations with small numbers.
