I have had some feedback along the lines of why write a micro-benchmark such as Tokyo Cabinet vs Berkeley DB. Everyone knows that:
and
That kinda misses the point. Micro-benchmarking is nothing to do with optimization. Its purpose is to avoid premature pessimization due to choosing the wrong technology.
I have a project to replace an existing system where the backing store is by far the biggest bottleneck. The new system needs to be signicantly faster than the existing system and I have a lot of flexibility in choosing the backing store.
The two alternatives that immediately come to mind are.
- create a nice abstraction layer that can easily switch between Perl DBI, Redis, Tokyo Cabinet and a bunch of other alternatives… or
- micro-benchmark them with something similar to my read/write data profile.
Time pressure unfortunately compels me to choose #2. Maybe it’s time to take a second look at KiokuDB, for inspiration at least, even if I can’t use it directly.
For DBs a microbenchmark might not discover the issues that will plague you later. You might want to write a lot of crap data in there in between your test data.
Hi Anonymous,
One of my benchmarks is a volume benchmark – populate the test database with a couple of years worth of fake data.
Admittedly, there could still be problems that show up later, but to ensure there isn’t would require an abstraction layer which allows us to plugin each of the different candidates and we’ve decided the effort of doing this is too great.