Coding recipe: extracting loop functionality (via callbacks and generators)

[2018-04-29] dev, javascript, coding
(Ad, please don’t block)

In this blog post, we look at two ways of extracting the functionality of a loop: internal iteration and external iteration.

The loop  

As an example, take the following function logFiles():

const fs = require('fs');
const path = require('path');

function logFiles(dir) {
  for (const fileName of fs.readdirSync(dir)) { // (A)
    const filePath = path.resolve(dir, fileName);
    console.log(filePath);
    const stats = fs.statSync(filePath);
    if (stats.isDirectory()) {
      logFiles(filePath); // (B)
    }
  }
}
logFiles(process.argv[2]);

The loop starting in line A logs file paths. It is a combination of a for-of loop and recursion (the recursive call is in line B).

What if you find the functionality of the loop (iterating over files) useful, but don’t want to log?

Internal iteration  

The first option for extracting the loop functionality is internal iteration:

const fs = require('fs');
const path = require('path');

function logFiles(dir, callback) {
  for (const fileName of fs.readdirSync(dir)) {
    const filePath = path.resolve(dir, fileName);
    callback(filePath); // (A)
    const stats = fs.statSync(filePath);
    if (stats.isDirectory()) {
      logFiles(filePath, callback);
    }
  }
}
logFiles(process.argv[2], p => console.log(p));

This way of iterating is similar to the Array method .forEach(): logFiles() implements the loop and invokes its callback for every iteration value (line A).

External iteration  

An alternative to internal iteration is external iteration: We implement an iterable and a generator helps us with it:

const fs = require('fs');
const path = require('path');

function* logFiles(dir) {
  for (const fileName of fs.readdirSync(dir)) {
    const filePath = path.resolve(dir, fileName);
    yield filePath;
    const stats = fs.statSync(filePath);
    if (stats.isDirectory()) {
      yield* logFiles(filePath); // (A)
    }
  }
}
for (const p of logFiles(process.argv[2])) {
  console.log(p);
}

With internal iteration, logFiles() called us (“push”). This time, we call it (“pull”).

Note that in generators, you must make recursive calls via yield* (line A): If you just call logFiles() then it returns an iterable. But what we want is to yield every item in that iterable. That’s what yield* does.

One nice trait of generators is that processing is just as interlocked as with internal iteration: Whenever logFiles() has created another filePath, we get to look at it immediately, then logFiles() continues. It’s a form of simple cooperative multitasking, with yield pausing the current task and switching to a different one.

Further reading