// Internet Duct Tape

Programming Best Practices: Profiling

Posted in Firefox and Greasemonkey, Programming and Software Development, Technology by engtech on November 14, 2007

Programming Tips

My first task coming back from my work stress blogging hiatus is to finally fix problems with Akismet Auntie Spam that Lorelle reported over a month ago — if your Akismet spambox has over 10,000 spam comments then Auntie Spam is going to crash hard. Viewing that many comments at once will make Firefox use eight times more memory than normal web browsing, even without using Auntie Spam [1].
This means it’s time to do some code profiling [2]. In programming, profiling means to measure your code and find out which parts are using the most time and the most memory. Profiling gives you performance analysis measurements so that you can optimize your program for speed and/or memory.

“Don’t prematurely optimize” is a programming Best Practice, and it can be summed up in the words of my grandfather: “measure twice and cut once”. You can guess at what parts need fixing, but it is much more effective to measure how your program performs so that you can focus on the worst parts. They have the most room for improvement. Without profiling you could easily spend several hours optimizing a loop that executes in negligible time and ignore the three lines that copy huge chunks of memory for No Apparent Reason. Get it working, and then use your profiler to get it working fast.

Profiling is a Skill

I’ve been creating Greasemonkey scripts using javascript for a year now, and this is my first time firing up any kind of javascript profiler. It really struck me that I waited too long to do this. Don’t prematurely optimize, but also don’t waste any time learning how to run a profiler on your code and interpret the results. If you’ve never gone through the process of optimizing code in a language you regularly use, then you’ve been relying on all kinds of bad habits [3]. Learn how to integrate a profiler with your program as soon as possible so that performance analysis doesn’t become one of those “I’ll get around to it” tasks that never happens.

Another good rule is to always test with large data sets. Ideally you want a fast case for rapid prototyping of new features, and a worst case for stressful testing of that new feature. To often we use small sets of data for development and testing. We never realize how badly our code performs in real world conditions. Speed and responsiveness play a greater factor in whether or not someone becomes a regular user of your program than you might realize.

Footnotes

[1] One thing WordPress does wrong is it includes all of your comment spam in their WordPress export files. One friend saw his export file decrease from 83 MB to 8 MB once he deleted the comment spam.

[2] The best way to profile Javascript is with FireBug, but it doesn’t recognize Greasemonkey scripts unless you embed them in the page so FireBug can find them. Wikipedia has a list of profilers for popular languages.

[3] Some of the bad habits that were lurking in Auntie Spam:

  • I was using a custom getElementsByClassName instead of an XPATH call. XPATH can be so much faster that walking the DOM.
  • I had too many innerHTML assignments instead of leaving HTML as a string and then giving it to the web page to process as a final step
  • Inefficient regular expressions
  • Too many copies of the comments in memory

4 Responses

Subscribe to comments with RSS.

  1. adam said, on November 14, 2007 at 3:27 pm

    [1] One thing WordPress does wrong is it includes all of your comment spam in their WordPress export files.

    agreed. i’ve been meaning to post this as a bug in trac for a while now.

  2. Lorelle VanFossen said, on November 15, 2007 at 2:19 am

    Wow! I can’t believe that working with Auntie Akismet on my blog will lead to a bug fix in the next version of WordPress! That’s pretty impressive for a non-Plugin. I’m impressed. Be proud, my friend.

    I’m looking forward to the new improved version of Auntie Akismet, especially if it won’t crash my browser. I’ve found that anything over 1,000 comment spams slows everything down, and I get a lot more than that every day.

    I would so much love to see much of what you have introduced into this powerful Greasemonkey script put into Akismet. It makes it so much easier to manage comment spam, which totally overwhelms so many bloggers.

  3. [...] Skip navigation Subscription OptionsMost Popular PostsIDT Labs – Free Software ToolsWordPress.com Resources – Tips, Tricks and ToolsWordpress.com Theme ReviewsWordpress.com Theme Review HelpGreasemonkey script: WordPress Category ResizerWordpress.com 7 Day Referrer ParserPerl Script – WordPress.com 7 Day Referrer ParserGreasemonkey Script: Akismet Auntie Spam for WordpressGreasemonkey Script: Find images that are too wideTag Cloud Generator for Wordpress.comTag Cloud Generator AdvancedTag Cloud Generator – Release NotesWordPress Themes by InternetDuctTape.comBlack and Blue and Read All Over Theme for WordPress SandboxMoon Under Uranus Theme for WordPress SandboxMiscellanious WordPress Scripts and ToolsGreaseMonkey Script: WordPress Comment NinjaTechnorati Favorite Your FansTechnorati Favorite Your Fans – Release NotesComic BloggerGreasemonkey script: Flickr always search for Creative Commons licensed photosGreasemonkey Script: Yahoo Pipe CleanerTag CloudAll Posts by Category and TitleSeriesGift Guide for Geekseaster eggsReader Appreciation for RSS subscribersWelcome to Internet Duct Tapegoogle1ec000b3808eedbf.htmlAbout MeDisclosureImages Credits « Programming Best Practices: Profiling [...]

  4. [...] week I’ve been talking about code profiling and how if you want to analyze the performance of your application you need to work w…. Application efficiency isn’t free, it requires measurement, analysis and change. Unsurprisingly, [...]


Comments are closed.

Follow

Get every new post delivered to your Inbox.

Join 281 other followers

%d bloggers like this: