<?xml version="1.0" encoding="utf-8"?><feed xmlns="http://www.w3.org/2005/Atom" ><generator uri="https://bridgetownrb.com/" version="2.2.2">Bridgetown</generator><link href="https://peterboling.com/feed/each_in_batches.xml" rel="self" type="application/atom+xml" /><link href="https://peterboling.com/" rel="alternate" type="text/html" /><updated>2026-11-15T23:03:36+00:00</updated><id>https://peterboling.com/feed/each_in_batches.xml</id><title type="html">Peter Boling | Each_in_batches</title><subtitle>Yet Another Paradigm by Peter Boling, featuring technical blog posts about Ruby, JavaScript, TypeScript, Go, Python, DevOps, and various open source projects. Ramblings, mumblings, cv/resume, technical blog, and documentation for many popular, and unpopular, RubyGems and other FLOSS projects, including oauth2, omniauth-identity, sanitize_email, debug_logging, flag_shih_tzu, gem_bench, etc. Semi-lifely state-of-the-petris updates.</subtitle><author><name>Peter Boling</name><email>floss@galtzo.com</email></author><entry><title type="html">each_in_batches v0.1.3 released!</title><link href="https://peterboling.com/each_in_batches/each_in_batches-v0.1.3-released!/" rel="alternate" type="text/html" title="each_in_batches v0.1.3 released!" /><published>2025-06-01T00:40:59+00:00</published><updated>2025-06-01T00:40:59+00:00</updated><id>https://peterboling.com/each_in_batches/each_in_batches-v0.1.3-released!/</id><content type="html" xml:base="https://peterboling.com/each_in_batches/each_in_batches-v0.1.3-released!/">&lt;h2 id=&quot;whats-changed&quot;&gt;What’s Changed&lt;/h2&gt;
&lt;ul&gt;
  &lt;li&gt;Respect the value of `@verbose’ by @RobertAudi in https://github.com/pboling/each_in_batches/pull/3&lt;/li&gt;
  &lt;li&gt;Update the ‘activerecord’ dependency by @RobertAudi in https://github.com/pboling/each_in_batches/pull/2&lt;/li&gt;
&lt;/ul&gt;

&lt;h2 id=&quot;new-contributors&quot;&gt;New Contributors&lt;/h2&gt;
&lt;ul&gt;
  &lt;li&gt;@RobertAudi made their first contribution in https://github.com/pboling/each_in_batches/pull/3&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;strong&gt;Full Changelog&lt;/strong&gt;: https://github.com/pboling/each_in_batches/compare/v0.1.2…v0.1.3&lt;/p&gt;</content><category term="release" /><category term="each_in_batches" /><category term="v0.1.3" /></entry><entry><title type="html">Introducing each_in_batches</title><link href="https://peterboling.com/each_in_batches/introducing-each_in_batches/" rel="alternate" type="text/html" title="Introducing each_in_batches" /><published>2015-06-18T19:30:16+00:00</published><updated>2015-06-18T19:30:16+00:00</updated><id>https://peterboling.com/each_in_batches/introducing-each_in_batches/</id><content type="html" xml:base="https://peterboling.com/each_in_batches/introducing-each_in_batches/">&lt;h2 id=&quot;performance-measured-and-tuned-batch-processing-of-records-with-blocks-in-rails&quot;&gt;Performance Measured and Tuned Batch Processing of Records with Blocks in Rails.&lt;/h2&gt;

&lt;p&gt;&lt;a href=&quot;https://github.com/pboling/each_in_batches&quot;&gt;&lt;img src=&quot;https://img.shields.io/gem/rt/each_in_batches.svg&quot; alt=&quot;Total Downloads&quot; /&gt;&lt;/a&gt;
&lt;a href=&quot;https://github.com/pboling/each_in_batches&quot;&gt;&lt;img src=&quot;https://img.shields.io/gem/rd/each_in_batches.svg&quot; alt=&quot;Downloads Today&quot; /&gt;&lt;/a&gt;
&lt;a href=&quot;https://codeclimate.com/github/pboling/each_in_batches&quot;&gt;&lt;img src=&quot;https://img.shields.io/codeclimate/github/pboling/each_in_batches.svg&quot; alt=&quot;Code Quality&quot; /&gt;&lt;/a&gt;
&lt;a href=&quot;https://github.com/pboling/each_in_batches/network&quot;&gt;&lt;img src=&quot;https://img.shields.io/github/forks/pboling/each_in_batches.svg?style=social&quot; alt=&quot;Network&quot; /&gt;&lt;/a&gt;
&lt;a href=&quot;https://github.com/pboling/each_in_batches/stargazers&quot;&gt;&lt;img src=&quot;https://img.shields.io/github/stars/pboling/each_in_batches.svg?style=social&quot; alt=&quot;Stars&quot; /&gt;&lt;/a&gt;
&lt;a href=&quot;https://rubygems.org/gems/each_in_batches&quot;&gt;&lt;img src=&quot;https://img.shields.io/gem/v/each_in_batches.svg&quot; alt=&quot;Version&quot; /&gt;&lt;/a&gt;
&lt;a href=&quot;https://travis-ci.org/pboling/each_in_batches&quot;&gt;&lt;img src=&quot;https://img.shields.io/travis/pboling/each_in_batches.svg&quot; alt=&quot;Build&quot; /&gt;&lt;/a&gt;
&lt;a href=&quot;http://inch-ci.org/github/pboling/each_in_batches&quot;&gt;&lt;img src=&quot;http://inch-ci.org/github/pboling/each_in_batches.svg&quot; alt=&quot;Documentation&quot; /&gt;&lt;/a&gt;
&lt;a href=&quot;https://depfu.com/github/pboling/each_in_batches?project_id=2682&quot;&gt;&lt;img src=&quot;https://badges.depfu.com/badges/0d506c43cf01b41e1937d7cb151a8c0d/count.svg&quot; alt=&quot;Depfu&quot; /&gt;&lt;/a&gt;
&lt;a href=&quot;https://gitter.im/pboling/each_in_batches&quot;&gt;&lt;img src=&quot;https://img.shields.io/gitter/room/pboling/each_in_batches.svg&quot; alt=&quot;Chat&quot; /&gt;&lt;/a&gt;
&lt;a href=&quot;https://opensource.org/licenses/MIT&quot;&gt;&lt;img src=&quot;https://img.shields.io/badge/License-MIT-green.svg&quot; alt=&quot;License: MIT&quot; /&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h1 id=&quot;eachinbatches&quot;&gt;EachInBatches&lt;/h1&gt;

&lt;p&gt;(Originally BolingForBatches)&lt;/p&gt;

&lt;h3 id=&quot;note&quot;&gt;NOTE:&lt;/h3&gt;
&lt;p&gt;I am resurrecting this code because I still have this recurring need, and Rail’s native batching doesn’t cut mustard.&lt;br /&gt;
It is some of my most ancient code, and it isn’t pretty, but I hope to improve it over time.&lt;/p&gt;

&lt;p&gt;I often need to execute really large computations on really large data sets.
I usually end up writing a rake task to do it, which calls methods in my models.
But something about the process bugged me.  Each time I had to re-implement my
‘batching code’ that allowed me to not chew up GB after GB of memory due to
&lt;code class=&quot;highlighter-rouge&quot;&gt;klass.find(:all, :include =&amp;gt; [:everything_under_the_sun])&lt;/code&gt;. Re-implementation of
the same logic over and over across many projects is not very DRY, so I got out
my blow torch and lit it up.  The difficulty was that the part that was different
each time I batched was at the center of the code, right in the middle of the
batch loop.  But I didn’t let that stop me!&lt;/p&gt;

&lt;h2 id=&quot;why-this-plugin-is-way-better-than-standard-rails-batching&quot;&gt;Why this plugin is way better than standard Rails batching&lt;/h2&gt;
&lt;ol&gt;
  &lt;li&gt;I’ve been doing batching in Rails a lot longer than Rails has.&lt;/li&gt;
  &lt;li&gt;Metrics.  I measure stuff.&lt;/li&gt;
  &lt;li&gt;I can batch from the top down (a.k.a backwards), making it possible to DELETE things in batches.
     A. If you’ve never tried using the built-in rails batching for deleting millions of records… don’t start now.  Use this gem instead.&lt;/li&gt;
  &lt;li&gt;Exception Handling.  Exceptions occurring within the batching can be rescued, in a customizable fashion, which means that the process doesn’t need to die on batch 309,675 of 402,540.&lt;/li&gt;
  &lt;li&gt;Merged in the EachInBatches fork (from Brian Kidd):
 I needed to iterate over the results and perform more actions than a single
 method would provide.  I didn’t want to write a method in my app that performed
 the needed functionality as I felt the plugin should support this directly.
 I modified the original plugin so that it takes a block instead of a method.
 It will pass the object instance to the block.  It works pretty much the same
 as &lt;code class=&quot;highlighter-rouge&quot;&gt;Class.find(:all).each {|x| do something}&lt;/code&gt;, except in batches n that you
 specify with :batch_size.&lt;/li&gt;
&lt;/ol&gt;

&lt;h2 id=&quot;installation&quot;&gt;Installation&lt;/h2&gt;

&lt;p&gt;Add this line to your application’s Gemfile:&lt;/p&gt;

&lt;div class=&quot;language-ruby highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&lt;span class=&quot;n&quot;&gt;gem&lt;/span&gt; &lt;span class=&quot;s1&quot;&gt;&apos;each_in_batches&apos;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;And then execute:&lt;/p&gt;

&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;$ bundle
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Or install it yourself as:&lt;/p&gt;

&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;$ gem install each_in_batches
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;h2 id=&quot;usage&quot;&gt;Usage&lt;/h2&gt;

&lt;p&gt;To create a new Batch, call &lt;code class=&quot;highlighter-rouge&quot;&gt;Batch#new&lt;/code&gt; pass it the class and any additional arguments (all as a hash).&lt;/p&gt;

&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;batch = EachInBatches::Batch.new(:arel =&amp;gt; Payment.canceled.order(&quot;transaction_id ASC&quot;), :batch_size =&amp;gt; 50)
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;To process the batched data, pass a block to &lt;code class=&quot;highlighter-rouge&quot;&gt;Batch#run&lt;/code&gt; the same way you would to an object in a block like &lt;code class=&quot;highlighter-rouge&quot;&gt;Klass.all.each {|x| x.do_something }&lt;/code&gt;.
&lt;code class=&quot;highlighter-rouge&quot;&gt;Batch#run&lt;/code&gt; will pass the data to your block, one at a time, in batches set by the :batch_size argument.&lt;/p&gt;

&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;batch.run {|x| puts x.id; puts x.transaction_id}
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Print the results!&lt;/p&gt;

&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;batch.print_results
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Or…&lt;/p&gt;

&lt;p&gt;Consolidate your code if you prefer&lt;/p&gt;

&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;EachInBatches::Batch.new(:arel =&amp;gt; Payment.canceled.order(&quot;transaction_id ASC&quot;), batch_size =&amp;gt; 50, :show_results =&amp;gt; true).run{|x| puts x.id; puts x.transaction_id}
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;h2 id=&quot;configuration&quot;&gt;Configuration&lt;/h2&gt;

&lt;p&gt;Arguements for the initializer (Batch.new) method are:&lt;/p&gt;

&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;Required:

  :arel          - Usage: :arel =&amp;gt; Payment.canceled.order(&quot;transaction_id ASC&quot;)
                    Required, as this is the class that will be batched

Optional:

  :verbose       - Usage: :verbose =&amp;gt; true or false
                    Sets verbosity of output
                    Default: false (if not provided)

  :batch_size    - Usage: :batch_size =&amp;gt; x
                    Where x is some number.
                    How many AR Objects should be processed at once?
                    Default: 50 (if not provided)

  :last_batch    - Usage: :last_batch =&amp;gt; x
                    Where x is some number.
                    Only process up to and including batch #x.
                      Batch numbers start at 0 for the first batch.
                    Default: won&apos;t be used (no limit if not provided)

  :first_batch   - Usage: first_batch =&amp;gt; x
                    Where x is some number.
                    Begin processing batches beginning at batch #x.
                      Batch numbers start at 0 for the first batch.
                    Default: won&apos;t be used (no offset if not provided)

  :show_results  - Usage: :show_results =&amp;gt; true or false
                    Prints statistics about the results of Batch#run.
                    Default: true if verbose is set to true and :show_results is not provided, otherwise false
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;h2 id=&quot;output&quot;&gt;Output&lt;/h2&gt;

&lt;p&gt;Interpreting the output:&lt;/p&gt;

&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&apos;[O]&apos; means the batch was skipped due to an offset.
&apos;[L]&apos; means the batch was skipped due to a limit.
&apos;[P]&apos; means the batch is processing.
&apos;[C]&apos; means the batch is complete.
and yes... it was a coincidence.  This class is not affiliated with &apos;one laptop per child&apos;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;h2 id=&quot;license&quot;&gt;License&lt;/h2&gt;

&lt;p&gt;Copyright ©2008-2015 Peter H. Boling, Brian Kidd, released under the MIT license&lt;/p&gt;

&lt;h2 id=&quot;development&quot;&gt;Development&lt;/h2&gt;

&lt;p&gt;After checking out the repo, run &lt;code class=&quot;highlighter-rouge&quot;&gt;bin/setup&lt;/code&gt; to install dependencies. Then, run &lt;code class=&quot;highlighter-rouge&quot;&gt;bin/console&lt;/code&gt; for an interactive prompt that will allow you to experiment.&lt;/p&gt;

&lt;p&gt;To install this gem onto your local machine, run &lt;code class=&quot;highlighter-rouge&quot;&gt;bundle exec rake install&lt;/code&gt;. To release a new version, update the version number in &lt;code class=&quot;highlighter-rouge&quot;&gt;version.rb&lt;/code&gt;, and then run &lt;code class=&quot;highlighter-rouge&quot;&gt;bundle exec rake release&lt;/code&gt; to create a git tag for the version, push git commits and tags, and push the &lt;code class=&quot;highlighter-rouge&quot;&gt;.gem&lt;/code&gt; file to &lt;a href=&quot;https://rubygems.org&quot;&gt;rubygems.org&lt;/a&gt;.&lt;/p&gt;

&lt;h2 id=&quot;maintenance&quot;&gt;Maintenance&lt;/h2&gt;

&lt;p&gt;To release a new version, update the version number in &lt;code class=&quot;highlighter-rouge&quot;&gt;version.rb&lt;/code&gt;, and then run &lt;code class=&quot;highlighter-rouge&quot;&gt;bundle exec rake release&lt;/code&gt; to create a git tag for the version, push git commits and tags, and push the &lt;code class=&quot;highlighter-rouge&quot;&gt;.gem&lt;/code&gt; file to &lt;a href=&quot;https://rubygems.org&quot;&gt;rubygems.org&lt;/a&gt;.&lt;/p&gt;

&lt;h2 id=&quot;versioning&quot;&gt;Versioning&lt;/h2&gt;

&lt;p&gt;This library aims to adhere to &lt;a href=&quot;http://semver.org/&quot;&gt;Semantic Versioning 2.0.0&lt;/a&gt;.
Violations of this scheme should be reported as bugs. Specifically,
if a minor or patch version is released that breaks backward
compatibility, a new version should be immediately released that
restores compatibility. Breaking changes to the public API will
only be introduced with new major versions.&lt;/p&gt;

&lt;p&gt;As a result of this policy, you can (and should) specify a
dependency on this gem using the &lt;a href=&quot;http://docs.rubygems.org/read/chapter/16#page74&quot;&gt;Pessimistic Version Constraint&lt;/a&gt; with two digits of precision.&lt;/p&gt;

&lt;p&gt;For example:&lt;/p&gt;

&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;spec.add_dependency &apos;each_in_batches&apos;, &apos;~&amp;gt; 0.0&apos;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;h2 id=&quot;contributing&quot;&gt;Contributing&lt;/h2&gt;

&lt;ol&gt;
  &lt;li&gt;Fork it ( https://github.com/[my-github-username]/each_in_batches/fork )&lt;/li&gt;
  &lt;li&gt;Create your feature branch (&lt;code class=&quot;highlighter-rouge&quot;&gt;git checkout -b my-new-feature&lt;/code&gt;)&lt;/li&gt;
  &lt;li&gt;Commit your changes (&lt;code class=&quot;highlighter-rouge&quot;&gt;git commit -am &apos;Add some feature&apos;&lt;/code&gt;)&lt;/li&gt;
  &lt;li&gt;Push to the branch (&lt;code class=&quot;highlighter-rouge&quot;&gt;git push origin my-new-feature&lt;/code&gt;)&lt;/li&gt;
  &lt;li&gt;Make sure to add tests!&lt;/li&gt;
  &lt;li&gt;Create a new Pull Request&lt;/li&gt;
&lt;/ol&gt;

&lt;h2 id=&quot;contributors&quot;&gt;Contributors&lt;/h2&gt;

&lt;p&gt;See the &lt;a href=&quot;https://github.com/pboling/each_in_batches/network&quot;&gt;Network View&lt;/a&gt;&lt;/p&gt;</content><category term="development" /><category term="ruby" /><category term="rails" /><category term="rubygems" /><category term="each_in_batches" /><category term="database" /><category term="active_record" /></entry><entry><title type="html">Introducing each_in_batches</title><link href="https://peterboling.com/each_in_batches/introducing-each_in_batches/" rel="alternate" type="text/html" title="Introducing each_in_batches" /><published>2008-11-15T00:00:00+00:00</published><updated>2008-11-15T00:00:00+00:00</updated><id>https://peterboling.com/each_in_batches/introducing-each_in_batches/</id><content type="html" xml:base="https://peterboling.com/each_in_batches/introducing-each_in_batches/">&lt;p&gt;&lt;a href=&quot;https://discord.gg/3qme4XHNKN&quot;&gt;&lt;img src=&quot;https://logos.galtzo.com/assets/images/galtzo-floss/avatar-192px.svg&quot; alt=&quot;Galtzo FLOSS Logo by Aboling0, CC BY-SA 4.0&quot; /&gt;&lt;/a&gt; &lt;a href=&quot;https://www.ruby-lang.org/&quot;&gt;&lt;img src=&quot;https://logos.galtzo.com/assets/images/ruby-lang/avatar-192px.svg&quot; alt=&quot;ruby-lang Logo, Yukihiro Matsumoto, Ruby Visual Identity Team, CC BY-SA 2.5&quot; /&gt;&lt;/a&gt; &lt;a href=&quot;https://github.com/galtzo-floss/each_in_batches&quot;&gt;&lt;img src=&quot;https://logos.galtzo.com/assets/images/galtzo-floss/each_in_batches/avatar-192px.svg&quot; alt=&quot;each_in_batches Logo by Aboling0, CC BY-SA 4.0&quot; /&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h3 id=&quot;getting-started&quot;&gt;Getting Started&lt;/h3&gt;

&lt;p&gt;For more information and detailed examples, please visit the &lt;a href=&quot;https://github.com/galtzo-floss/each_in_batches&quot;&gt;documentation&lt;/a&gt;.&lt;/p&gt;

&lt;h3 id=&quot;supporting-this-project&quot;&gt;Supporting this Project&lt;/h3&gt;
&lt;ul&gt;
  &lt;li&gt;&lt;a href=&quot;https://opencollective.com/galtzo-floss&quot;&gt;OpenCollective&lt;/a&gt;&lt;/li&gt;
  &lt;li&gt;&lt;a href=&quot;https://git.sr.ht/~galtzo/each_in_batches&quot;&gt;SourceHut&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;</content><category term="introduction" /><category term="each_in_batches" /></entry></feed>