Stefan does a great job going through the whole rails performance problems. From choosing session containers to caching to his own customized piggy backing technique. There is another piece missing, and I’d like to bring it up, again…

Rails, err… ActiveRecord cannot efficiently handle large sets of data. Thanks to eager associations in Rails and the piggy backing technique that Stefan wrote the finder methods are doing pretty good. The area that is not getting attention has to do with inserts, updates and merges.

If normal ActiveRecord can get you 100,000 records created in 505 seconds, I can get them created for you in under 15 seconds. And this is running on a good old 7200rpm drive. I’m sure when I test this on a 10k and 15k SCSI drive at the office we’ll be seeing some blazing numbers.

There are a lot of Rails sites out there and in development. If you’re hosting your Typo blog then using standard ActiveRecord is fine, but if you’re trying to deal with the enterprise, or tables that will house thousands upon thousands, or millions upon millions of records you better start paying attention.

Here’s the point of this post, I want to help ActiveRecord help you. I’ve been working with millions of records with rails for quite some time now. I’ve got a few tricks up my sleeves and a few goodies in my bag that will be coming out shortly. And most of the code has been refactored and planned out over the past few months as to make it easy to integrate to ActiveRecord itself, and once it’s out I’ll be all over the Rails-core ML to get it patched onto ActiveRecord itself.

In the past I wrote an ActiveRecord::TemporaryTable plugin, and I brought it up on rails-core and the rails-ml and despite some good response I never released it. Well after much ado about something that will be integrated with additional optimizations packaged in ActiveRecord::Optimizations .

I’ll be going more indepth on the optimizations for and limitations of, ActiveRecord at the 2006 RubyConf*MI conference held at Calvin College in Grand Rapids, MI on August 26th.

Any questions, wants or feedback in regards to ActiveRecord please feel free to comment here or email me.

I am NOT affiliated with the rails-core develpment team. I am affiliated with millions of records in databases serving thousands of users using ActiveRecord. It's time my inside-scoop got leaked to the public.