Improve Linux performance
By Cameron Laird
2004-04-08
Reader Rating: 
500 times as quick
Independent consultant Alex Martelli experienced an even more dramatic breakthrough just last year. As he related in the Python Business Forum mailing list, he had implemented an XML processor that required eight hours to complete its task. That was unacceptable; end-users simply couldn't wait that long. He combined several fixes to bring the time down to one minute -- a factor of 500 over where he started!
First, he switched from the built-in Python XML libraries to the specialized pyRXP parser. This is the same pyRXP that made an appearance in a previous developerWorks article on the performance of XML processing; see Resources for a link to the article. Along with its thriftier use of processing cycles, pyRXP also dramatically slashes memory footprint, when compared to other Python-bound XML engines. This makes an enormous difference, and is typical of many performance bottlenecks: applications are starved for memory. Even formally favorable algorithms can slow catastrophically if they use so much memory as to require swapping. That was Martelli's biggest problem.
So, when evaluating alternative products or algorithms, benchmark not just their apparent throughput but also their memory impact. The latter often dominates the effective scalability of production applications. Moreover, if you decide to try solving a performance problem by throwing hardware at it, as is often wise, start by considering more memory. An increase in main memory is an inexpensive experiment that often yields dramatic results.
That, then is the second easy principle: get more memory, and make better use of what you have. Lots of people, including managers, seem to appreciate that memory matters.
First published by IBM developerWorks
If you found this article interesting, you may want to read these as well:
|
