Testing impure modules
In a node application, a single JavaScript file forms a self-contained unit known as a 'module'. When executing your program, each one of these modules is executed and loaded into memory each time you call require
(for commonjs) or import
(for ES6 modules).
Fans of functional programming will be familiar with the idea of "pure functions", that is "functions that don't have any side effects". We use this terminology in a similar way when referring to JavaScript modules and it turns out that this is an important concept that you can use to make your code simpler and easier to test.
Note that most of my sample code is written in TypeScript as that's the language I mostly work in. Where I refer to 'execution' of a .ts
file, it should be understood that I'm really referring to execution of the transpiled JavaScript. The syntax for TypeScript is closer to ES6, but everything in this article also applies to both commonjs
and es6
module systems. For testing, I use jest - the problems I'm discussing here apply equally to other testing frameworks, but you might need to use different mechanisms to get around them.
What is a pure module?
A pure module is a JavaScript module that only exports functions and variables but has no actual logic of its own. For example:
// pure-uppercase.ts
const uppercaseFirstLetter = (name: string) => `${name.substring(0, 1).toUpperCase()}${name.slice(1)}`
export uppercaseFirstLetter
this module when imported will define a single function, then export the reference to that function. It doesn't actually do anything until the function itself is called.
Here is an example of an impure module that I wrote recently.
// impure-set-tag.ts
import tracer from 'third-party-tracing-library';
tracer.init();
export const setTag = (tagName: string, tagValue: unknown) => {
const span = tracer.scope().active();
if (!span) {
throw new Error('Active span not available');
}
span.setTag(tagName, tagValue);
};
Note the use of tracer.init()
at the top level of the module. This module is impure because the simple act of importing it will cause tracer.init()
to be executed which very clearly has the side effect of initialising the tracer.
In this case there was a good reason why the module was written in this way as I'm taking advantage of the very mechanism that often trips people up. I'd like to show you, however, that testing this kind of module is tricky, and consequently avoiding them will probably help you have a happier life.
Import caching
Modules in node are singletons; that is to say that because modules are cached by the runtime (using the file path as the cache key), the modules themselves are executed once and only once at the point of import. The runtime then stores the results of that execution in the cache and uses the result any time a further import is made for that path. Lets look at this in action
First the pure module. Consider the following program:
// program.ts
import "./file-one"
import "./file-two"
// file-one.ts
import { uppercaseFirstLetter } from "./pure-uppercase" // execution
console.log(uppercaseFirstLetter('foo'))
// file-two.ts
import { uppercaseFirstLetter } from "./pure-uppercase" // no-execution
console.log(uppercaseFirstLetter('bar'))
When you execute program.ts
, the following happens
./file-one
is imported. There are no cached executions, so./file-one
is loaded and executed. This imports a function from./pure-uppercase
.- Since that module isn't in our cache yet it is executed, and so defines and exports the function which is then used to log to the console.
./file-two
is imported. There are no cached executions, so./file-two
is loaded and executed. Again, this imports a function from./pure-uppercase
.- This time the result is in our cache, so we reuse the same function to log a message to the console without executing
./pure-uppercase
again.
While interesting, this is fairly trivial since the caching mechanism doesn't impact the behaviour of the application. Lets now try again with our impure module. Consider the following program:
// program.ts
import "./file-one"
import "./file-two"
// file-one.ts
import { setTag } from "./impure-set-tag"
setTag('foo', 'bar')
// file-two.ts
import { setTag } from "./impure-set-tag"
setTag('bar', 'baz')
When you execute program.ts
from the command line, the following happens
./file-one
is imported. There are no cached executions, so./file-one
is loaded and executed. This imports a function from./impure-set-tag
.- Again, this is not in the cache so it is executed. As part of this execution, a
tracer
is imported fromthird-party-tracing-library
, andtracer.init()
is called. - Finally, we define and export a function that captures a reference to
tracer
within its scope. ./file-two
is imported. There are no cached executions, so./file-two
is loaded and executed.- Again, this imports a function from
./impure-set-tag
. The result is already in our cache so instead of executing the module a second time the already definedsetTag
function is returned.
Because this module is impure, there are two interesting consequences - regardless of how many times you import/call setTag
:
- The
setTag
function is always acting on the same instance oftracer
tracer.init()
will only ever be called once
In the case of our distributed tracing library, this is useful, as we only want the tracer to be initialised once. Writing modules in this way however, has consequences for testing which we will explore in the next section.
Testing Impure Modules
Since the above single initialisation behavior sounds useful, lets test it. Consider the following test
import tracer, { Scope } from "third-party-tracing-library"
import { mock } from 'jest-mock-extended';
import { mocked } from 'ts-jest/utils';
import { setTag } from "./impure-set-tag"
jest.mock("third-party-tracing-library")
beforeEach(() => jest.resetAllMocks())
test("the impure setTag function only calls tracer.init method once", () => {
const mockScope = mock<Scope>();
mockScope.active.mockReturnValue(mock());
mocked(tracer.scope).mockReturnValue(mockScope);
setTag('foo', 'bar');
setTag('baz', 'bash');
expect(mocked(tracer.init)).toHaveBeenCalledTimes(1);
})
This seems like it should work... except it doesn't.
Expected number of calls: 1
Received number of calls: 0
Let's walk through what is happening here.
- Before
jest
executes the test, it transpiles the code with either babel-jest or ts-jest depending on your configuration. - In either case, this will result in the call to
jest.mock()
being moved ('hoisted') to the top of the file - Consequently when the code is executed, the first thing that happens is
jest.mock('third-party-tracing-library')
automagically replaces any import ofthird-party-tracing-library
with ajest.fn()
instances (mock function), with all known properties replaced with furtherjest.fn()
instances. - When execution reaches
import { setTag } from "./impure-set-tag
, this immediately executes the code at the top level of the module. - This calls
import tracer from "third-party-tracing-library"
, which returns the mock created above. tracer.init()
is executed; since this is a mock function, it increments its call counts by one.- A few lines later, we call
beforeEach(() => jest.resetAllMocks())
which resets the state of all jest mock functions - The test then executes and fails because it expects the call count to be 1, despite the fact that we reset the mock state in the previous step
The problem here is that we are trying to test a behaviour that occurs
at the point of import, which is outside the test. We could
of course change our beforeEach
to an afterEach
so that we don't reset mock state before the test runs, but we shouldn't have to do that; resetting mock state before each test is a very uncontroversial best practise, and this solution is easily broken - if I add a second test before the one I've written, the state will get reset and the same issue will appear.
Move the import into the test
Lets try to move the behaviour under test into the test. We can do this by removing the original import, making the test callback asynchronous and importing the module dynamically inside the test, like this:
// ... stuff
test("the impure setTag function only calls tracer.init method once", () => {
const { setTag } = await import("./impure-set-tag")
// ...stuff
})
Congratulations, you now have a passing test!
While this appears to work, there is a big problem with it. We can demonstrate it by being diligent engineers and adding some more test coverage. Lets start by adding the following test above our existing test (order is important here).
test('the impure setTag function if you pass in an empty string for the name of a tag', async () => {
const { setTag } = await import('./impure-set-tag');
expect(() => setTag('', 'bar')).toThrow();
});
Go ahead and change the implementation to make that test pass. Once you do, you'll find your original test is failing again in exactly the same way.
If you think back to the section on
module caching, you'll understand why. When the first test executes it imports
./impure-set-tag
which executes the module and calls tracer.init()
. Since
this is a mock function, its call count is incremented by one. Before
our next test executes however, we reset all mock state so the count goes back to zero. In the next test,
./impure-set-tag
does not get executed again which means tracer.init()
does not get called again, and thus the expectation fails.
Clear the module cache
To resolve this problem, you need to ensure there is a fresh module cache for
every test. jest
provides us with two APIs to do this
jest.resetModules()
- a single function call similar tojest.clearMocks()
jest.isolateModules()
- takes a callback function. When this function is executed, everything inside its scope gets a fresh module cache
If every test in the module needs a fresh cache, jest.resetModules()
should
go into a beforeEach
hook. If you only need this kind of isolation for a
single test, I'd encourage use of jest.isolateModules()
, since it is more
explicit and calling jest.resetModules()
half way through a test file may
cause surprising behaviour for subsequent tests.
Note that
jest.isolateModules()
cannot accept an asynchronous callback which means
if your test is asynchronous (as ours is), you need to use jest.resetModules()
. In our current
example, you could also overcome this by replacing await import()
with require()
(which is synchronous)
Lets try resetting the cache. Go ahead and add beforeEach(() => jest.resetModules())
at the start of your test module and run jest. Settle down and watch as your tests... fail... again.
Move automocks
Lets look again at the test file and try to understand what is happening
import tracer, { Scope } from 'third-party-tracing-library';
// ...imports
jest.mock('third-party-tracing-library');
beforeEach(() => {
jest.resetAllMocks();
jest.resetModules();
});
test('the impure setTag function throws an error if you pass in an empty string for the name of a tag', async () => {
const { setTag } = await import('./impure-set-tag');
// ...stuff
});
test('the impure setTag function only calls tracer.init method once', async () => {
const { setTag } = await import('./impure-set-tag');
// ...call function
expect(mocked(tracer.init)).toHaveBeenCalledTimes(1);
});
Walking through (hopefully for the last time) the relevant parts, we discover the following:
- First, because of hoisting, 'third-party-tracing-library' is automocked
- Next, the mocked
tracer
is imported fromthird-party-tracing-library
- We then call
jest.resetAllModules()
, which resets the module cache before each test - In each test we import
./impure-set-tag
, which performs a fresh execution each time due to the reset - Also because of the reset,
third-party-tracing-library
is no longer automocked and so returns the real tracer. This means that the mockedtracer
that was returned earlier doesn't get any calls
To solve this problem, we need to perform automocking inside the test itself.
Deep breath... because we are doing this we cannot use jest.mock()
as this is designed to be
used at the top level of a module. Instead we use jest.doMock()
which
doesn't get hoisted, and so we need to do it before the import, not after. Since you
are performing automocking inside the test, you also need to import the mock
tracer
inside the test in order to set an expectation on it.
This is what the final setup looks like:
import { Scope } from 'third-party-tracing-library';
// ...imports
beforeEach(() => {
jest.resetAllMocks();
jest.resetModules();
});
// .. other test(s)
test('the impure setTag function only calls tracer.init method once', async () => {
jest.doMock("third-party-tracing-library");
const { default: tracer } = await import("third-party-tracing-library");
const { setTag } = await import('./impure-set-tag');
const mockScope = mock<Scope>();
mockScope.active.mockReturnValue(mock());
mocked(tracer.scope).mockReturnValue(mockScope);
setTag('foo', 'bar');
setTag('baz', 'bash');
expect(mocked(tracer.init)).toHaveBeenCalledTimes(1);
});
Final Thoughts
To summarise, if you want to test module level 'impure' logic, you need to
- Move the import, all automocking, and imports of any mocks inside the test and
- Ensure the module cache is fresh for each test
There are good reasons to write impure modules, as I have done. But I would recommend not doing so unless you specifically want to make use of the module caching mechanism, or the 'impure' behaviour is trivial enough that you are happy not testing it.
As you have already seen, the resulting tests are verbose and more complicated than they need to be considering the simplicity of the behaviour under test. Mistakes are very easy to make, and they result in subtle and hard to understand failures that eat up whole evenings.
Since you have read this post, you know how to handle them properly. But many engineers don't and you can make their lives much easier in the first place by favouring pure modules by default.