Most popular mistake to ruin Webpack bundle optimization

Quick tip: click code mentions (file.js) or visual elements (, 4) to highlight them on scheme. Use player () at the bottom to guide through scheme steps.

Intro

Working on big projects brings many difficult challenges, keeping applications bundle size in check is one of them. When project grows, inevitably you will start separating big sections of features into separate modules or sub-applications, delegating development to other teams or, sometimes, even other companies. Not after long you have huge application, tens teams building hundreds of modules, all to be packed, bundled and shipped towards user.

Control of bundle size becomes critical at this point, one module, one bad apple, can just ruin everything. Fortunately webpack does a lot of optimisation under the hood, to make sure you ship as minimum code as required. However, and I witnessed this over and over again, there is still one simple mistake you can do that will prevent webpack from working its magic. Let's talk about that.

TL;DR

We all know at this point, webpack does "tree shaking" to optimise bundle size. Just in case, "tree shaking" is a term commonly used in the JavaScript context for dead-code elimination, or in simple words - export-ed code that wasn't import-ed and executed will be detected as unused, so it can be safely removed to decrease bundle size.

What you might not know, it's not the webpack that cleans up dead code per se. Of course, it does bulk of "preparation" work, but it is terser package that actually will cut off unused code. Terser is JavaScript parser, mangler and compressor toolkit for ES6+.

Let's lay this out - webpack will take your modules, concatenate them into chunks and feed to terser for minification (all of this, obviously, will happen only if optimization is enabled).

 
Time to highlight few key points:

  • By default, webpack always will try to concatenate your code from different modules (files) into one scope and create a chunk from it later. E.g. moduleA.js imports few methods from moduleB.js will end up being chunk-[hash].js containing code from both files mentioned before within one scope, like it was written inside one file in the first place (essentially removing "module" concept). When it can't be concatenated though, webpack will register those files as modules, so they can be accessed globally via internal helper __webpack_require__ later.

  • By default terser doesn't cut off global references in your code (topLevel flag is false). E.g. you build some library with global scope API, you don't want it to be removed during minification. In essence, only somewhat "obviously" dead (unreachable) code or unused in near scopes code will be removed.

You probably saw this coming - terser can remove your unused export-s only if webpack scoped them in a way that unused declarations can be easily detected.

For optimization webpack heavily relies on the static structure of ES2015 module syntax, i.e. import and export key-words, and, as for now, doesn't work for other module types. We can see this ourself from the source.

59 return reason;
60 };
61
62 compilation.hooks.optimizeChunkModules.tap(
63 "ModuleConcatenationPlugin",
64 (allChunks, modules) => {
+
67 for (const module of modules) {
68 // Only harmony modules are valid for optimization
69 if (
+
72 !module.dependencies.some(
73 d => d instanceof HarmonyCompatibilityDependency
74 )
75 ) {
76 setBailoutReason(module, "Module is not an ECMAScript module");
77 continue;
78 }
+
94 // Exports must be known (and not dynamic)
95 if (!Array.isArray(module.buildMeta.providedExports)) {
96 setBailoutReason(module, "Module exports are unknown");
97 continue;
98 }
99
100 // Using dependency variables is not possible as this wraps the code in a function
101 if (module.variables.length > 0) {
`ModuleConcatenationPlugin` puts code from JavaScript files into "chunks" (basically concatenating code from different files into one scope). It bails though if it can not identify ECMAScript module exports.
Involved projects:

webpack

Scheme created
by Bohdan Liashenko
with codecrumbs.io
ModuleConcatenationPlugin


Like you can see, messing up module interfaces prevents ModuleConcatenationPlugin (plugin for optimization) to do its job.

We all love and use babel to transpile modern ES syntax in our modules, but in this situation the babel-preset-env becomes a bad friend of ours - by default modules are transpiled to "commonjs" standard and that's precisely what we don't want when pulling together multiple packages into one application! We have to make sure to set modules: false in preset config. Webpack can do majority of its optimizations only for Harmony modules!

Do not transpile modules with your babel setup! If you do so, webpack will treat files as a blobs, loosing ability to prepare chunks properly for terser to cut off unused code.

 
Well, technically it's not that straightforward, of course. Webpack does ton of processing on its side in order to build the concatenated code, it does track of provided and used export-s on its side as well, before even calling terser, so "combined" code with all modules is still valid for terser. But once again - it won't work for anything else but static ES module syntax.

Under the hood

There is quite a complex process going under the hood, starting from you passing webpack.config.js to compiler and before bundle is generated. We'll touch slightly the parts that are interesting for our discussion.

 
 
Compilation phase is where all fun happens, below you can see its main steps.

 
Ultimately, during compilation webpack builds dependency graph for the entry point specified in your webpack.config.js (or several of them, if configuration specifies multiple entry points).

In essence, the idea is to start from the entry point, go through dependent modules, build them and connect together - modules are nodes and dependencies are connections of the graph.

789 * @param {ModuleCallback} callback callback to be triggered
790 * @returns {void}
791 */
792 processModuleDependencies(module, callback) {
793 const dependencies = new Map();
+
843 this.addModuleDependencies(
844 module,
845 sortedDependencies,
846 this.bail,
847 null,
848 true,
849 callback
850 );
851 }
+
1033 _addModuleChain(context, dependency, onModule, callback) {
+
1093 const afterBuild = () => {
1094 if (addModuleResult.dependencies) {
1095 this.processModuleDependencies(module, err => {
1096 if (err) return callback(err);
1097 callback(null, module);
1098 });
+
1102 };
+
1111 this.buildModule(module, false, null, null, err => {
+
1123 afterBuild();
1124 });
1125 } else {
1126 this.semaphore.release();
1127 this.waitForBuildingFinished(module, afterBuild);
0
1
2
3
Involved projects:

webpack

Scheme created
by Bohdan Liashenko
with codecrumbs.io
Compilation
0Start for entry module (Compilation.js#1033)
1Build module (Compilation.js#1111)
2After build process module dependencies (Compilation.js#1095)
3Add dependencies to module (Compilation.js#843)

To build module means to generate AST while extracting all needed information (export-s, import-s etc.). Webpack relies on acorn.Parser (from acorn) to build and process AST.
 
 
Next comes optimization phase.

..webpack..libwebpack/lib/FlagDependencyUsagePlugin.jsFlagDependencyUsagePlugin.jswebpack/lib/WebpackOptionsApply.jsWebpackOptionsApply.js)
345 const SideEffectsFlagPlugin = require("./optimize/SideEffectsFlagPlugin");
346 new SideEffectsFlagPlugin().apply(compiler);
347 }
348 if (options.optimization.providedExports) {
349 const FlagDependencyExportsPlugin = require("./FlagDependencyExportsPlugin");
350 new FlagDependencyExportsPlugin().apply(compiler);
351 }
352 if (options.optimization.usedExports) {
353 const FlagDependencyUsagePlugin = require("./FlagDependencyUsagePlugin");
354 new FlagDependencyUsagePlugin().apply(compiler);
355 }
356 if (options.optimization.concatenateModules) {
357 const ModuleConcatenationPlugin = require("./optimize/ModuleConcatenationPlugin");
358 new ModuleConcatenationPlugin().apply(compiler);
359 }
360 if (options.optimization.splitChunks) {
361 const SplitChunksPlugin = require("./optimize/SplitChunksPlugin");
23};
24
25class FlagDependencyUsagePlugin {
26 apply(compiler) {
27 compiler.hooks.compilation.tap("FlagDependencyUsagePlugin", compilation => {
28 compilation.hooks.optimizeDependencies.tap(
29 "FlagDependencyUsagePlugin",
30 modules => {
+
38 module.usedExports = addToSet(
39 module.usedExports || [],
40 usedExports
41 );
42 if (module.usedExports.length === old) {
43 return;
`FlagDependencyUsagePlugin` identifies `usedExports` and assign to the `module`.
Involved projects:

webpack

Scheme created
by Bohdan Liashenko
with codecrumbs.io
FlagDependencyUsagePlugin

FlagDependencyUsagePlugin hooks into the compilation phase and identifies usedExports. Basically, the idea is to find what "moduleA" imports from "moduleB", to set its usedExports. This process requires a lot of recursive traversing and "counting references".

As you know, webpack has pipe of plugins working on events, if you want to learn more, check out my other post Tapable library as a core of webpack architecture.

FlagDependencyUsagePlugin.js follows what HarmonyImportDependencyParserPlugin.js found about dependencies usage.

)
37 return true;
38 }
39 );
40 parser.hooks.importSpecifier.tap(
41 "HarmonyImportDependencyParserPlugin",
42 (statement, source, id, name) => {
43 parser.scope.definitions.delete(name);
44 parser.scope.renames.set(name, "imported var");
+
55 );
+
128 parser.hooks.call
129 .for("imported var")
130 .tap("HarmonyImportDependencyParserPlugin", expr => {
+
136 const settings = parser.state.harmonySpecifier.get(name);
137 const dep = new HarmonyImportSpecifierDependency(
+
143 name,
144 expr.range,
145 this.strictExportPresence
146 );
147 dep.directImport = true;
+
151 parser.state.module.addDependency(dep);
152 if (args) parser.walkExpressions(args);
153 return true;
134 info,
135 exportName,
+
141) => {
142 switch (info.type) {
143 case "concatenated": {
+
153 } else if (!info.module.isUsed(exportName)) {
154 return "/* unused export */ undefined";
155 }
156 if (info.globalExports.has(directExport)) {
157 return directExport;
158 }
159 const name = info.internalNames.get(directExport);
160 if (!name) {
0
once `importSpecifier` is detected, variable will be marked as "imported var" for further tracking
1
Listen to calls (AST element `method` call), i.e. webpack is smart, imported method doesn't necessary means it's used, it needs to make sure it's called as well
2
Called imported method detected and saved as dependency (later going to be inside `usedExports` for imported module)
3
Mark unused export
Involved projects:

webpack

Scheme created
by Bohdan Liashenko
with codecrumbs.io
HarmonyImportDependencyParserPlugin

Webpack is smart, imported method doesn't necessary means it's used, we need to make sure it's called as well.

Once again, in order for this to work, import-s/export-s should remain in the package (not transpiled).

Interesting finds

There are way too many interesting things I've noticed in the source code of webpack that should be to mentioned. It probably needs a separate post.

I'll highlight just a few of them.

Remember that error when you run webpack for the first time, but forgot to install webpack-cli package? They are not peerDependencies, so webpack provides quite useful guidance for users on how to solve it.

))
34 * @param {string} packageName name of the package
35 * @returns {boolean} is the package installed?
36 */
37const isInstalled = packageName => {
38 try {
39 require.resolve(packageName);
40
41 return true;
42 } catch (err) {
43 return false;
44 }
45};
+
60const CLIs = [
61 {
62 name: "webpack-cli",
63 package: "webpack-cli",
64 binName: "webpack-cli",
65 alias: "cli",
66 installed: isInstalled("webpack-cli"),
+
70 },
+
81];
82
83const installedClis = CLIs.filter(cli => cli.installed);
84
85if (installedClis.length === 0) {
+
90 let notify =
91 "One CLI for webpack must be installed. These are recommended choices, delivered as separate packages:";
92
93 for (const item of CLIs) {
94 if (item.recommended) {
Simply try to resolve the package, thrown error means it's not installed!
Involved projects:

webpack

Scheme created
by Bohdan Liashenko
with codecrumbs.io
bin/webpack


Another rather big surprise, how many independent packages-dependencies webpack has. Literally for everything:

1) tapable package for event-driven architecture
2) terser for minification
3) acorn for AST processing
4) watchpack to watch file changes

That's obviously very nice, hence all of them can be reused for different purposes in other tools!

Interactive code schemes in this post were created with Codecrumbs App. You can try it out for your own codebase here, check out Getting started guide for more details.

webpackwebpack@4.43.0javascript

Follow me on Twitter

Bohdan Liashenko

@bliashenko
Code geek. I tweet about interesting code tips and findings I am discovering under the hood of popular open source projects.

Join the Newsletter

Join the newsletter to receive interesting code tips and findings I am discovering under the hood of popular open source projects.