Memorization Makes Progress