That sounds a lot like the strategy behind Quake 3's magic inverse sqrt function.
And I agree 100%. If you really want to "optimize the !@$!" out of something, you need to start looking at it from a numeric analysis standpoint. What are the input and output ranges, and what is the acceptable error deviation from correct?
Books can be had on that topic for really cheap used.
I bought this one years ago:
There are a hundred ways to skin a cat. And many combinations of techniques... in different ORDERS (let alone combinations) will lead to a variety of results.
But the point is, instead of a simple hardcoded lookup, a modern implementation will involve a lookup, some approximation function, and maybe some more approximation function. (See the Quake 3 function link.) That way, you're benefiting from the CPU time and not just accessing a value from RAM and waiting for the next one if it's not in cache. You really want to pack things into units of single cachelines whenever possible (64-bytes on almost all systems these days).
Another option is to take advantage of cachelines in a different way. Let's say you really just need X multiplied by a value from your lookup. Well, instead of multiplying ONE X at a time, why not multiply a whole ROW of X's? That's the principle behind Data-oriented design and practically REQUIRED to do any kind of PS3 programming. You pack like-minded data together, and then operate on many at the same time using the same function. So if you were using a blind lookup table, that lookup table would remain hot in cache the entire time you're using it--instead of doing ONE lookup and then running the next algorithm then the next, and then repeating for the NEXT OBJECT. This is where traditional OOP education becomes "slow as dirt". Treating each object as an independent unit by-definition means the compiler can't do anything to help you.
//to clarify, you go from this:
obj1.get_stuff(); //caches get_stuff() data/code
obj1.think_about_stuff(); //get_stuff no longer needed.
obj1.apply_stuff(); //think_about_stuff no longer needed.
obj2.get_stuff(); //apply_stuff no longer needed, start over.
get_stuff(obj1...objN); //all get_stuff routines are hot in cache after the FIRST object is called
Or for a full lecture on how DoD rules by a industry great, game programmer, Mike Acton:
p.s. I'm so happy that Bob posted here.