Node.js Internals

Module System: CommonJS vs ESM

JavaScript was the only mainstream language WITHOUT a built-in module system until 2015. PHP had `include`, Python - `import`, Java - packages. But JavaScript lived in browsers, where everything was loaded through `<script>` tags into the global scope. Node.js changed the game by introducing CommonJS - the first practical module system for JS. Today we live in a transitional period: the old standard (CJS) versus the official one (ESM). Understanding both systems is understanding the history and future of JavaScript.

**Migration from require() to import:** Half of npm packages have already migrated to pure ESM (node-fetch, chalk, execa). If you don't understand interop, your project will be stuck on old versions of dependencies.
**Monorepositories (Nx, Turborepo):** Mix CJS and ESM packages. Module resolution should work through workspace aliases and symlinks. Without understanding the resolution algorithm - continuous MODULE_NOT_FOUND.
**Tree-shaking and bundle size:** ESM allows webpack/rollup to remove unused code. Lodash weighs 70KB, but `import { map }` will add only 2KB to the bundle. In CJS, it doesn't work that way - you have to use `lodash.map` (a separate package for each function).
**Serverless (Lambda, CloudFlare Workers):** Cold start time is critical. ESM loads asynchronously and in parallel, CJS - synchronously and sequentially. The difference in startup time can be 2-3x.

Intro

Imagine a library where all the books are piled into one huge heap without any division into sections. Finding the right chapter is a nightmare. Before the advent of modules, all JavaScript was like this: one global scope, name conflicts, and the inability to reuse code. Node.js solved this problem by introducing a **module system** - a way to break an application into independent files with clear boundaries and dependencies.

But the story got complicated: Node.js was born with **CommonJS** (2009), and then JavaScript received an official standard - **ES Modules** (2015). Now we have two module systems that work differently and require understanding their internals. Why is `require()` synchronous, while `import` is asynchronous? Can CJS and ESM be mixed? How does module caching work? Why can't `import` be used conditionally?

**Key difference:** CommonJS loads modules **synchronously** and **at runtime**. ES Modules are parsed **statically** and loaded **asynchronously**. This is not just a syntactic difference - these are two design philosophies with different trade-offs.

The real problem of migration

You are working on an Express application using CommonJS (thousands of lines of code). You decide to migrate to ESM to use modern packages. But: 1. Change `package.json`: `"type": "module"` 2. Replace `require()` with `import` 3. Run it - the application crashes with the error `ERR_REQUIRE_ESM` **Why?** You have dependencies that export only ESM (e.g., `node-fetch v3`), but other packages still use CommonJS. You need to understand interoperability, use `.mjs` / `.cjs` extensions, change the build pipeline. This is why understanding the internals is critical - migration requires understanding how Node.js distinguishes formats and resolves dependencies.

Why can't computed paths (variables) be used in import in ES Modules, unlike CommonJS require()?

CommonJS Internals

CommonJS is not just `require()` and `module.exports`. Under the hood, Node.js performs magic: it wraps each file in a function, creates a `module` object with metadata, caches results, resolves relative and absolute paths through a complex algorithm. Let's break down the mechanics.

When you write `const foo = require('./bar')`, the following happens: **1. Module Wrapping** - Node.js reads the `bar.js` file and wraps it in a function: ``` (function(exports, require, module, __filename, __dirname) { // your code from bar.js }); ``` **2. Creation of the module object:** ``` module = { id: '/path/to/bar.js', exports: {}, // Empty object by default parent: [current module], filename: '/path/to/bar.js', loaded: false, children: [], paths: [list of paths for searching node_modules] }; ``` **3. Function Execution** - Node.js calls the wrapped function with these parameters. Inside your code, `module.exports` refers to `module.exports` from the parameters. **4. Caching** - the result is saved in `require.cache['/path/to/bar.js']`. **5. Return** - `module.exports` is returned.

**module.exports vs exports:** `exports` is just a reference to `module.exports`. You can do `exports.foo = 'bar'`, but you SHOULD NOT reassign `exports = { ... }`, because this will create a new local variable. Always use `module.exports` to export entire objects.

Circular Dependencies - How CommonJS Resolves Them

**Problem:** ```js // a.js const b = require('./b'); module.exports = { name: 'A', b }; // b.js const a = require('./a'); // ← Circular dependency! module.exports = { name: 'B', a }; ``` **What happens:** 1. `a.js` starts loading, `module.exports = {}` 2. `a.js` calls `require('./b')` 3. `b.js` starts loading, calls `require('./a')` 4. Node.js sees that `a.js` is already in the process of loading (flag `loaded: false`) 5. Returns a **partially loaded** `module.exports` from `a.js` (empty object at this point) 6. `b.js` receives `a = {}` 7. `b.js` completes, returns control to `a.js` 8. `a.js` completes with full `module.exports` **Result:** `b.a` will be an empty object, while `a.b` will be complete. Circular dependencies work but lead to bugs. Use dependency injection or refactoring.

What happens if you write `exports = { foo: 'bar' }` inside a module instead of `module.exports = { foo: 'bar' }`?

ES Modules Internals

ES Modules (ESM) is the official JavaScript standard adopted in ECMAScript 2015. Unlike CommonJS, which was invented for Node.js, ESM works the same in browsers and on the server. But the devil is in the details: ESM in Node.js has nuances related to backward compatibility and performance.

**Key Differences from CommonJS:** **1. Static Analysis** - the dependency graph is built BEFORE code execution. This allows for: - Tree-shaking (removal of unused exports in bundlers) - Static import error checking at the parsing stage - Circular dependencies without issues (all exports are known before execution) **2. Asynchronous Loading** - `import` returns a Promise in a dynamic manner. Modules can be loaded in parallel. **3. Live Bindings** - exports in ESM refer to the original values rather than copying them. Changes in the exporting module are visible in the importing one. **4. Strict Mode** - all ESM files are automatically in strict mode. You cannot use deprecated features like `arguments.caller` or `with`.

**Top-level await:** In ESM, you can use `await` at the top level of a module without an async function. This blocks the loading of dependent modules until the Promise is resolved. Use with caution - you can block the entire dependency graph.

Tree-shaking in action

**Library (lodash-es):** ```js // lodash.mjs export function map(arr, fn) { ... } export function filter(arr, fn) { ... } export function reduce(arr, fn, init) { ... } // ... 300+ functions ``` **Your code:** ```js import { map } from 'lodash-es'; const result = map([1, 2, 3], x => x * 2); ``` **Webpack/Rollup analysis:** - Parses `import { map }` → only `map` is used - The remaining 299 functions are marked as "dead code" - Final bundle: ~2KB instead of 70KB **Why is this not possible with CommonJS?** ```js const lodash = require('lodash'); // Node.js MUST execute the entire lodash/index.js const map = lodash.map; // The bundler doesn't know that only map is used ``` This is why modern libraries (lodash-es, date-fns) offer ESM versions.

Your module exports a counter. In CommonJS, count remains 0 after increment(), in ESM it becomes 1. Why?

Module Resolution Algorithm

When you write `import express from 'express'`, how does Node.js find this package? Behind the simple syntax lies a complex path resolution algorithm that checks dozens of possible files and directories. Understanding this algorithm is critical for debugging `MODULE_NOT_FOUND` errors and configuring monorepositories.

**Detailed algorithm for `import 'foo'` from `/project/src/app.mjs`:** **1. Check built-in modules** - if 'foo' matches a built-in module (fs, path, http, etc.), it is used. Node.js 16+ requires the `node:` prefix for clarity: `import fs from 'node:fs'`. **2. Search in the node_modules hierarchy:** ``` /project/src/node_modules/foo /project/node_modules/foo /node_modules/foo ``` **3. For each directory, check:** - `package.json` → `"exports"` field (new standard) - `package.json` → `"main"` field (old standard) - `index.js` / `index.mjs` / `index.json` **4. Package.json "exports" (conditional exports):** A package can export different files for different environments: ```json { "exports": { "." : { "import": "./dist/esm/index.mjs", "require": "./dist/cjs/index.cjs", "types": "./dist/index.d.ts" }, "./submodule": "./dist/submodule.mjs" } } ``` ESM import uses `"import"`, CommonJS require uses `"require"`.

**Important:** In ESM, file extensions are mandatory for relative paths: `import './foo.js'`, not `import './foo'`. This is done for compatibility with browsers, where import must specify the full URL.

Real problem: dual packages

You are publishing a library that should work in both CommonJS and ESM projects. How can this be done? **Bad solution (double instantiation):** ```json { "main": "./dist/cjs/index.js", "module": "./dist/esm/index.mjs" } ``` **Problem:** If an ESM project imports your library and one of the dependencies uses require(), the module will load TWICE - separate copies with separate state. The Singleton pattern will break. **Correct solution ("exports" with conditions):** ```json { "exports": { "import": "./dist/esm/index.mjs", "require": "./dist/cjs/index.js" } } ``` Node.js ensures that the module loads once, even if both import and require are used (through an internal wrapper).

Project with `"type": "module"` in package.json. You are doing `import './config'`, the file is named config.js. What will happen?

Module Caching

Modules are loaded once and cached. This is critical for performance - without caching, each `require('./db')` would read the file from disk and parse the code again. But caching creates interesting effects and problems that need to be understood.

**How caching works:** **CommonJS:** `require.cache` - a regular object `{ [absolute path]: module }`. **ESM:** An internal Map in V8, not directly accessible (no public API for clearing). **Cache key:** Absolute path after resolution. This means: - `require('./foo')` and `require('./FOO')` on a case-insensitive FS (macOS/Windows) - one module - `require('lodash')` from different node_modules - different modules - Symlinks are resolved to real paths (one module even through different symlinks)

**Singleton pattern for free:** Module caching automatically makes exports singletons. `const db = require('./db')` in 100 different files will return the same connection object. This is convenient, but dangerous in tests - you need to clear the state between tests.

Circular dependencies through cache

Cache also allows cyclic dependencies: ```js // a.js console.log('a starting'); const b = require('./b'); // b starts loading console.log('in a, b.done =', b.done); module.exports = { done: true }; console.log('a done'); // b.js console.log('b starting'); const a = require('./a'); // a is ALREADY in cache (loaded: false) console.log('in b, a.done =', a.done); // undefined! Module not completed module.exports = { done: true }; console.log('b done'); ``` **Output:** ``` a starting b starting in b, a.done = undefined ← Partially loaded module b done in a, b.done = true a done ``` **Why it works:** Node.js immediately adds the module to the cache with `loaded: false`. When `b` does `require('./a')`, it gets a partially filled `module.exports` (empty at that moment). After `a` completes, its `module.exports` is updated, but `b` already has a reference to the old object. **Conclusion:** Avoid circular dependencies or use lazy require inside functions.

You use `require('./config')` in 50 different files. How many times will the code in `config.js` be executed?

CJS <-> ESM Interoperability

The main question of migration: can CommonJS and ES Modules be mixed? The short answer is: **yes, but with limitations**. Node.js allows ESM to import CJS, but not vice versa (only through dynamic `import()`). This creates an asymmetry that needs to be understood.

**Interaction Rules:** **1. ESM → CJS (works):** ```js // lib.cjs (CommonJS) module.exports = { foo: 'bar' }; // app.mjs (ESM) import lib from './lib.cjs'; // ✅ Works console.log(lib.foo); // 'bar' // Named imports DO NOT work: import { foo } from './lib.cjs'; // ❌ ERROR in most cases // Works only if Node.js can statically analyze exports ``` **2. CJS → ESM (DOES NOT work synchronously):** ```js // lib.mjs (ESM) export const foo = 'bar'; // app.cjs (CommonJS) const lib = require('./lib.mjs'); // ❌ ERROR: ERR_REQUIRE_ESM // require() is synchronous, ESM is asynchronous. Incompatible! // Solution: dynamic import() (async () => { const lib = await import('./lib.mjs'); // ✅ Works console.log(lib.foo); })(); ```

**Named exports from CommonJS:** Node.js tries to statically analyze CommonJS modules and create named exports for ESM. But this only works for simple cases (`exports.foo = ...`). If exports are created dynamically, only the default import is available.

Migration path: hybrid package

**Task:** You have a library in CommonJS, and you need to support ESM without breaking changes. **Solution (recommended by Node.js):** **1. Build in both formats:** ```bash /dist /cjs index.js # CommonJS /esm index.mjs # ES Module ``` **2. package.json with "exports":** ```json { "type": "commonjs", "main": "./dist/cjs/index.js", "exports": { ".": { "import": "./dist/esm/index.mjs", "require": "./dist/cjs/index.js" } } } ``` **3. Build script (TypeScript example):** ```bash # ESM tsc --module esnext --outDir dist/esm renameext dist/esm/**/*.js .mjs # CJS tsc --module commonjs --outDir dist/cjs ``` **Result:** - ESM projects: `import lib from 'your-lib'` → loads .mjs - CJS projects: `const lib = require('your-lib')` → loads .js - One module, different entry points, state is correctly separated

ESM is just a new syntax for modules. You can convert `require()` to `import` and everything will work.

ESM and CommonJS are different systems with different loading models (static vs dynamic, asynchronous vs synchronous). Migration requires understanding the resolution algorithm, caching, interop limitations, and changes to the build pipeline.

Many developers think that migration is simply replacing syntax: `const foo = require('foo')` → `import foo from 'foo'`. But in practice, problems arise: 1. **Top-level await** blocks dependent modules 2. **Dynamic imports** (`require(variable)`) need to be rewritten to `import()` 3. **Pure ESM packages** (node-fetch v3+, chalk v5+) break CJS projects 4. **Dual package hazard** duplicates state 5. **File extensions** become mandatory Migration is an architectural change, not a syntax refactor. You need to change package.json, build configuration, understand module resolution, and test interop scenarios.

Key Ideas

**CommonJS (require/module.exports):** Synchronous, dynamic system. Modules are executed at runtime, cached in `require.cache`, wrapped in a function. You can use computed paths, conditional imports. Ideal for Node.js, does not work in browsers without a bundler.
**ES Modules (import/export):** Asynchronous, static system. The dependency graph is built before code execution (tree-shaking, static errors). Live bindings instead of copies. Extensions are mandatory. Works in both browsers and Node.js. The future of JavaScript.
**Module Resolution:** A complex algorithm with checking package.json "exports", searching in the node_modules hierarchy, and trying extensions. The "type" field in package.json determines the format of .js files. .mjs is always ESM, .cjs is always CommonJS.
**Caching:** Modules are loaded once, the result is cached by the absolute path. Automatic singletons. Circular dependencies are resolved through partial loading. In ESM, the cache is not directly accessible.
**Interop:** ESM can import CJS (default import always, named sometimes). CJS CANNOT require() ESM (only async import()). Dual packages require "exports" to avoid state duplication. Migrating to ESM is an architectural change, not just a syntax replacement.

Вопросы для размышления

Your project uses CommonJS. One of the dependencies has been updated to pure ESM. What migration strategies are available? In what order would you apply them?
Why is static analysis of ESM useful for optimizations (tree-shaking, bundling), but limits flexibility (no conditional imports at the top-level)?
You are publishing a library that should work in both CommonJS and ESM projects. How to configure package.json "exports" to avoid dual package hazard?

Связанные уроки

comp-01-intro