There is a moment where your benchmark says a new approach is clearly better, and you have to decide how to ship it. The tempting path is to graft the new idea onto the old code so you keep everything that already works.
I have learned to resist that. The old code carries assumptions from the old approach, and stacking a new idea on top of them gives you something nobody can fully reason about. Once I threw away a broken piece that had a pile of failing tests and rebuilt it clean from the new approach. It took a fraction of the time that untangling it would have.
This only works because the parts around it are cheap to recreate: the eval that proves the rebuild is better, the small tools, the glue. Versions are cheap. Iteration is cheap.
What is not cheap is complexity you cannot hold in your head. When the approach changes, let the code change with it, all the way down.