Under the hood webpack: core library behind the event-driven architecture

What: this is interactive story build with Codecrumbs.

Quick tip: click code mentions in the text (e.g. file-name.js) or visual elements (e.g. , 4) to highlight them on scheme above.

TL;DR

Webpack architecture is heavily based on events. Each webpack plugin is basically a set of listeners hooked on different events during compilation phases. Under the hood, webpack uses a library called tapable to encapsulate "publish-subscribe" implementation.

Tapable provides different "hooks" classes (SyncBailHook, AsyncParallelHook, etc.) to "hook" on events with some extra rich functionality (e.g. interceptions or cross-listeners integration).

For example, DefinePugin (used to define environment variables, e.g. NODE_ENV) and SizeLimitsPlugin (reports oversized chunks, e.g. size > 250kb) tap into compiler instance hooks: the first one listens to compilation event in order to insert extra variables and the latter to afterEmit event - to proceed with assets evaluation once they were emitted.

DefinePlugin
Compiler
SizeLimitsPlugin
hooks.compilation.tap
hooks.compilation.call
hooks.afterEmit.tap
hooks.afterEmit.callAsync
DefinePlugin
Compiler
SizeLimitsPlugin

 
Let's have a quick look under the hood of webpack at SizeLimitsPlugin integration.

..lib..performancelib/WebpackOptionsApply.jsWebpackOptionsApply.jslib/performance/SizeLimitsPlugin.jsSizeLimitsPlugin.js
/lib/WebpackOptionsApply.js
483 }
484 }
485
486 if (options.performance) {
487 const SizeLimitsPlugin = require("./performance/SizeLimitsPlugin");
488 new SizeLimitsPlugin(options.performance).apply(compiler);
489 }
490
491 new TemplatedPathPlugin().apply(compiler);
492
● ● ●
/lib/performance/SizeLimitsPlugin.js
10/** @typedef {import("../Compiler")} Compiler */
11/** @typedef {import("../Entrypoint")} Entrypoint */
12
13module.exports = class SizeLimitsPlugin {
+
25 apply(compiler) {
+
32 compiler.hooks.afterEmit.tap("SizeLimitsPlugin", compilation => {
33 const warnings = [];
34
35 /**
● ● ●
0
1
0SizeLimitsPlugin is instantiated and assigned to compiler in WebpackOptionsApply if performance option is enabled from webpack config (highlight )
1Then SizeLimitsPlugin taps on afterEmit event and will sit still until most of the compilation flow is done and that particular event is called.

 

..lib..performancelib/Compiler.jsCompiler.jslib/performance/SizeLimitsPlugin.jsSizeLimitsPlugin.js
/lib/performance/SizeLimitsPlugin.js
29 const assetFilter =
30 this.assetFilter || ((name, source, info) => !info.development);
31
32 compiler.hooks.afterEmit.tap("SizeLimitsPlugin", compilation => {
+
74 const entrypointsOverLimit = [];
75 for (const [name, entry] of compilation.entrypoints) {
76 const size = getEntrypointSize(entry);
77
78 if (size > entrypointSizeLimit) {
79 entrypointsOverLimit.push({
80 name: name,
81 size: size,
82 files: entry.getFiles().filter(fileFilter)
83 });
84 /** @type {any} */ (entry).isOverSizeLimit = true;
85 }
● ● ●
/lib/Compiler.js
350 }
351 }
352
353 emitAssets(compilation, callback) {
+
358 asyncLib.forEachLimit(
359 compilation.getAssets(),
360 15,
361 ({ name: file, source }, callback) => {
+
482 this.hooks.afterEmit.callAsync(compilation, err => {
483 if (err) return callback(err);
484
485 return callback();
● ● ●
Using a simple "pub-sub" mechanism, performance evaluation plugin `SizeLimitsPlugin` can be loosely hooked into compilation flow.
2
3
1
0
`Compiler` calls (0)
 asynchronously `afterEmit` event (providing "compilation" instance).
`SizeLimitsPlugin` subscribes (1) to that to evaluate oversized chunks.

Once the event is called the plugin will do its job (collecting oversized chunks in our case 2, 3).

Indeed, all plugins are assigned similarly, so then you have a pipeline of plugins performing different operations but staying very loosely coupled.

Slightly longer read

Let's learn more about tapable library.

Have a look at this classic events-based architecture all of us have used many times.

..liblib/webpack.jswebpack.js
/lib/webpack.js
1// pseudo code
2
3pubSub.subscribe('myEvent', () => console.log('Handled by 1st subscriber'))
4
5pubSub.subscribe('myEvent', () => console.log('Handled by 2nd subscriber'))
6
7pubSub.subscribe('myEvent', () => console.log('Handled by 3rd subscriber'))
8
9
10pubSub.publish('myEvent')
11
● ● ●
All registered subscribers will be notified once particular event is published
1
0

Usually, Pub-Sub (Observer or Listener) behavioral pattern is used to establish a one-to-many relationship: publisher dispatches events 0 and multiple subscribers can listen to them accordingly 1.

Tapable is the Pub-Sub implementation on steroids. One key difference - instead of just string event names, uses the "Hook" class instance per event. It implements tap/tapAsync with call/callAsync methods to listen and publish events, or even intercept them.

..liblib/Compiler.jsCompiler.js
/lib/Compiler.js
39 * @property {Set<string>} compilationDependencies
40 */
41
42class Compiler extends Tapable {
43 constructor(context) {
44 super();
45 this.hooks = {
+
72 /** @type {AsyncSeriesHook<CompilationParams>} */
73 beforeCompile: new AsyncSeriesHook(["params"]),
74 /** @type {SyncHook<CompilationParams>} */
75 compile: new SyncHook(["params"]),
76 /** @type {AsyncParallelHook<Compilation>} */
77 make: new AsyncParallelHook(["compilation"]),
78 /** @type {AsyncSeriesHook<Compilation>} */
79 afterCompile: new AsyncSeriesHook(["compilation"]),
80
● ● ●
2
0
1

On top of classic "Sync" (0) events handling, it also adds "AsyncSeries" (1), "AsyncParallel" (2) (handlers called in a row or parallel, respectively).

On the scheme below you can see another example where plugins use a few more advanced hooks sub-classes from tapable ("Waterfall" and "Bail").

..lib..optimizelib/Compilation.jsCompilation.jslib/FlagDependencyUsagePlugin.jsFlagDependencyUsagePlugin.jslib/optimize/SideEffectsFlagPlugin.jsSideEffectsFlagPlugin.js
/lib/FlagDependencyUsagePlugin.js
25class FlagDependencyUsagePlugin {
26 apply(compiler) {
27 compiler.hooks.compilation.tap("FlagDependencyUsagePlugin", compilation => {
28 compilation.hooks.optimizeDependencies.tap(
29 "FlagDependencyUsagePlugin",
30 modules => {
31 const processModule = (module, usedExports) => {
32 module.used = true;
33 if (module.usedExports === true) return;
34 if (usedExports === true) {
● ● ●
/lib/optimize/SideEffectsFlagPlugin.js
49 });
50 });
51 compiler.hooks.compilation.tap("SideEffectsFlagPlugin", compilation => {
52 compilation.hooks.optimizeDependencies.tap(
53 "SideEffectsFlagPlugin",
54 modules => {
55 /** @type {Map<Module, Map<string, ExportInModule>>} */
56 const reexportMaps = new Map();
57
58 // Capture reexports of sideEffectFree modules
● ● ●
/lib/Compilation.js
1283 seal(callback) {
1284 this.hooks.seal.call();
1285
1286 while (
1287 this.hooks.optimizeDependenciesBasic.call(this.modules) ||
1288 this.hooks.optimizeDependencies.call(this.modules) ||
1289 this.hooks.optimizeDependenciesAdvanced.call(this.modules)
1290 ) {
1291 /* empty */
1292 }
1293 this.hooks.afterOptimizeDependencies.call(this.modules);
● ● ●
`FlagDependencyUsagePlugin` optimize chunks by identifying unused dependencis.
1
`SideEffectsFlagPlugin` handling specified side-effects from webpack options.
2
`optimizeDependencies` is an instance of `SyncBailHook`.
0

Using them with callback allows us to pass data between different subscribers (Waterfall) or exit early (Bail).

Event-driven architecture has some pros&cons. Like any other architectures there is no silver bullet. Within webpack, however, it brings endless scalability and works very well with a pipeline of loosely-coupled plugins.

To learn more about tapable library checkout tapable repository.

javascriptwebpackwebpack@4.43.0tapable@1.1.3

Follow me on Twitter

Bohdan Liashenko

@bliashenko
Code geek. I tweet about interesting code tips and findings I am discovering under the hood of popular open source projects.

Join the Newsletter

Join the newsletter to receive interesting code tips and findings I am discovering under the hood of popular open source projects.