Under the hood webpack: core library behind the event-driven architecture

What: this is interactive story build with Codecrumbs.

Quick tip: click code mentions in the text (e.g. file-name.js) or visual elements (e.g. , 4) to highlight them on scheme above.

TL;DR

Webpack architecture is heavily based on events. Each webpack plugin is basically a set of listeners hooked on different events during compilation phases. Under the hood, webpack uses a library called tapable to encapsulate "publish-subscribe" implementation.

Tapable provides different "hooks" classes (SyncBailHook, AsyncParallelHook, etc.) to "hook" on events with some extra rich functionality (e.g. interceptions or cross-listeners integration).

For example, DefinePugin (used to define environment variables, e.g. NODE_ENV) and SizeLimitsPlugin (reports oversized chunks, e.g. size > 250kb) tap into compiler instance hooks: the first one listens to compilation event in order to insert extra variables and the latter to afterEmit event - to proceed with assets evaluation once they were emitted.

 
Let's have a quick look under the hood of webpack at SizeLimitsPlugin integration.

..lib..performancelib/WebpackOptionsApply.jsWebpackOptionsApply.jslib/performance/SizeLimitsPlugin.jsSizeLimitsPlugin.js)
483 }
484 }
485
486 if (options.performance) {
487 const SizeLimitsPlugin = require("./performance/SizeLimitsPlugin");
488 new SizeLimitsPlugin(options.performance).apply(compiler);
489 }
490
491 new TemplatedPathPlugin().apply(compiler);
492
10/** @typedef {import("../Compiler")} Compiler */
11/** @typedef {import("../Entrypoint")} Entrypoint */
12
13module.exports = class SizeLimitsPlugin {
+
25 apply(compiler) {
+
32 compiler.hooks.afterEmit.tap("SizeLimitsPlugin", compilation => {
33 const warnings = [];
34
35 /**
`SizeLimitsPlugin` is instantiated and assigned to compiler in `WebpackOptionsApply`
0
`SizeLimitsPlugin` taps on afterEmit event
1
Involved projects:

webpack@4.43.0

Scheme created
by Bohdan Liashenko
with codecrumbs.io
SizeLimitsPlugin initialisation
0SizeLimitsPlugin is instantiated and assigned to compiler in WebpackOptionsApply if performance option is enabled from webpack config (highlight )
1Then SizeLimitsPlugin taps on afterEmit event and will sit still until most of the compilation flow is done and that particular event is called.

 

..lib..performancelib/Compiler.jsCompiler.jslib/performance/SizeLimitsPlugin.jsSizeLimitsPlugin.js)
29 const assetFilter =
30 this.assetFilter || ((name, source, info) => !info.development);
31
32 compiler.hooks.afterEmit.tap("SizeLimitsPlugin", compilation => {
+
74 const entrypointsOverLimit = [];
75 for (const [name, entry] of compilation.entrypoints) {
76 const size = getEntrypointSize(entry);
77
78 if (size > entrypointSizeLimit) {
79 entrypointsOverLimit.push({
80 name: name,
81 size: size,
82 files: entry.getFiles().filter(fileFilter)
83 });
84 /** @type {any} */ (entry).isOverSizeLimit = true;
85 }
351 }
352
353 emitAssets(compilation, callback) {
+
358 asyncLib.forEachLimit(
359 compilation.getAssets(),
360 15,
361 ({ name: file, source }, callback) => {
+
482 this.hooks.afterEmit.callAsync(compilation, err => {
483 if (err) return callback(err);
484
Using a simple "pub-sub" mechanism, performance evaluation plugin `SizeLimitsPlugin` can be loosely hooked into compilation flow.
0
1
2
3
`Compiler` calls (0)
 asynchronously `afterEmit` event (providing "compilation" instance).
`SizeLimitsPlugin` subscribes (1) to that to evaluate oversized chunks.
Involved projects:

webpack@4.43.0

Scheme created
by Bohdan Liashenko
with codecrumbs.io
SizeLimitsPlugin and Compiler

Once the event is called the plugin will do its job (collecting oversized chunks in our case 2, 3).

Indeed, all plugins are assigned similarly, so then you have a pipeline of plugins performing different operations but staying very loosely coupled.

Slightly longer read

Let's learn more about tapable library.

Have a look at this classic events-based architecture all of us have used many times.

1// pseudo code
2
3pubSub.subscribe('myEvent', () => console.log('Handled by 1st subscriber'))
4
5pubSub.subscribe('myEvent', () => console.log('Handled by 2nd subscriber'))
6
7pubSub.subscribe('myEvent', () => console.log('Handled by 3rd subscriber'))
8
9
10pubSub.publish('myEvent')
11
All registered subscribers will be notified once particular event is published
1
Publisher dispatches events
0
Scheme created
by Bohdan Liashenko
with codecrumbs.io
PubSub example

Usually, Pub-Sub (Observer or Listener) behavioral pattern is used to establish a one-to-many relationship: publisher dispatches events 0 and multiple subscribers can listen to them accordingly 1.

Tapable is the Pub-Sub implementation on steroids. One key difference - instead of just string event names, uses the "Hook" class instance per event. It implements tap/tapAsync with call/callAsync methods to listen and publish events, or even intercept them.

39 * @property {Set<string>} compilationDependencies
40 */
41
42class Compiler extends Tapable {
43 constructor(context) {
44 super();
45 this.hooks = {
+
72 /** @type {AsyncSeriesHook<CompilationParams>} */
73 beforeCompile: new AsyncSeriesHook(["params"]),
74 /** @type {SyncHook<CompilationParams>} */
75 compile: new SyncHook(["params"]),
76 /** @type {AsyncParallelHook<Compilation>} */
77 make: new AsyncParallelHook(["compilation"]),
78 /** @type {AsyncSeriesHook<Compilation>} */
79 afterCompile: new AsyncSeriesHook(["compilation"]),
80
0
1
2
Involved projects:

webpack@4.43.0

Scheme created
by Bohdan Liashenko
with codecrumbs.io
Hooks example

On top of classic "Sync" (0) events handling, it also adds "AsyncSeries" (1), "AsyncParallel" (2) (handlers called in a row or parallel, respectively).

On the scheme below you can see another example where plugins use a few more advanced hooks sub-classes from tapable ("Waterfall" and "Bail").

))
49 });
50 });
51 compiler.hooks.compilation.tap("SideEffectsFlagPlugin", compilation => {
52 compilation.hooks.optimizeDependencies.tap(
53 "SideEffectsFlagPlugin",
54 modules => {
55 /** @type {Map<Module, Map<string, ExportInModule>>} */
56 const reexportMaps = new Map();
57
58 // Capture reexports of sideEffectFree modules
25class FlagDependencyUsagePlugin {
26 apply(compiler) {
27 compiler.hooks.compilation.tap("FlagDependencyUsagePlugin", compilation => {
28 compilation.hooks.optimizeDependencies.tap(
29 "FlagDependencyUsagePlugin",
30 modules => {
31 const processModule = (module, usedExports) => {
32 module.used = true;
33 if (module.usedExports === true) return;
34 if (usedExports === true) {
1284 this.hooks.seal.call();
1285
1286 while (
1287 this.hooks.optimizeDependenciesBasic.call(this.modules) ||
1288 this.hooks.optimizeDependencies.call(this.modules) ||
1289 this.hooks.optimizeDependenciesAdvanced.call(this.modules)
1290 ) {
1291 /* empty */
1292 }
1293 this.hooks.afterOptimizeDependencies.call(this.modules);
`optimizeDependencies` is an instance of `SyncBailHook`.
0
`SideEffectsFlagPlugin` handling specified side-effects from webpack options.
2
`FlagDependencyUsagePlugin` optimize chunks by identifying unused dependencis.
1
Involved projects:

webpack@4.43.0

Scheme created
by Bohdan Liashenko
with codecrumbs.io
Advanced Hooks flow

Using them with callback allows us to pass data between different subscribers (Waterfall) or exit early (Bail).

Event-driven architecture has some pros&cons. Like any other architectures there is no silver bullet. Within webpack, however, it brings endless scalability and works very well with a pipeline of loosely-coupled plugins.

To learn more about tapable library checkout tapable repository.

javascriptwebpackwebpack@4.43.0tapable@1.1.3

Follow me on Twitter

Bohdan Liashenko

@bliashenko
Code geek. I tweet about interesting code tips and findings I am discovering under the hood of popular open source projects.

Join the Newsletter

Join the newsletter to receive interesting code tips and findings I am discovering under the hood of popular open source projects.