Query data in chunks with ActiveRecord

If you have to iterate over massive data sets, you don't want to have to use the standard #find method, as it will be extremely memory intensive. Instead, you should work on data in chunks. This code uses limit and offset to chunk up your data. You can then work on the pieces within the block.

n = 500
offsets = (0..size).step(n)

  offsets.each do |offset|
    company.customers.find(:all, :order=>:id, :limit=>n, :offset=>offset).each do |c|
     #
     #  Do the work on this chunk of size 500
     #
  end
end