Most popular mistake to ruin Webpack bundle optimization

Quick tip: click code mentions (file.js) or visual elements (, 4) to highlight them on scheme. Use player at the bottom to guide through scheme steps.

Intro

Working on big projects brings many difficult challenges, keeping applications bundle size in check is one of them. When project grows, inevitably you will start separating big sections of features into separate modules or sub-applications, delegating development to other teams or, sometimes, even other companies. Not after long you have huge application, tens teams building hundreds of modules, all to be packed, bundled and shipped towards user.

Control of bundle size becomes critical at this point, one module, one bad apple, can just ruin everything. Fortunately webpack does a lot of optimisation under the hood, to make sure you ship as minimum code as required. However, and I witnessed this over and over again, there is still one simple mistake you can do that will prevent webpack from working its magic. Let's talk about that.

TL;DR

We all know at this point, webpack does "tree shaking" to optimise bundle size. Just in case, "tree shaking" is a term commonly used in the JavaScript context for dead-code elimination, or in simple words - export-ed code that wasn't import-ed and executed will be detected as unused, so it can be safely removed to decrease bundle size.

What you might not know, it's not the webpack that cleans up dead code per se. Of course, it does bulk of "preparation" work, but it is terser package that actually will cut off unused code. Terser is JavaScript parser, mangler and compressor toolkit for ES6+.

Let's lay this out - webpack will take your modules, concatenate them into chunks and feed to terser for minification (all of this, obviously, will happen only if optimization is enabled).

Time to highlight few key points:

By default, webpack always will try to concatenate your code from different modules (files) into one scope and create a chunk from it later. E.g. moduleA.js imports few methods from moduleB.js will end up being chunk-[hash].js containing code from both files mentioned before within one scope, like it was written inside one file in the first place (essentially removing "module" concept). When it can't be concatenated though, webpack will register those files as modules, so they can be accessed globally via internal helper __webpack_require__ later.
By default terser doesn't cut off global references in your code (topLevel flag is false). E.g. you build some library with global scope API, you don't want it to be removed during minification. In essence, only somewhat "obviously" dead (unreachable) code or unused in near scopes code will be removed.

You probably saw this coming - terser can remove your unused export-s only if webpack scoped them in a way that unused declarations can be easily detected.

For optimization webpack heavily relies on the static structure of ES2015 module syntax, i.e. import and export key-words, and, as for now, doesn't work for other module types. We can see this ourself from the source.

59					return reason;
60				};
61
62				compilation.hooks.optimizeChunkModules.tap(
63					"ModuleConcatenationPlugin",
64					(allChunks, modules) => {
+ 
67						for (const module of modules) {
68							// Only harmony modules are valid for optimization
69							if (
+ 
72								!module.dependencies.some(
73									d => d instanceof HarmonyCompatibilityDependency
74								)
75							) {
76								setBailoutReason(module, "Module is not an ECMAScript module");
77								continue;
78							}
+ 
94							// Exports must be known (and not dynamic)
95							if (!Array.isArray(module.buildMeta.providedExports)) {
96								setBailoutReason(module, "Module exports are unknown");
97								continue;
98							}
99
100							// Using dependency variables is not possible as this wraps the code in a function
101							if (module.variables.length > 0) {

`ModuleConcatenationPlugin` puts code from JavaScript files into "chunks" (basically concatenating code from different files into one scope). It bails though if it can not identify ECMAScript module exports.

Involved projects:

webpack

Scheme created
by Bohdan Liashenko
with codecrumbs.io

ModuleConcatenationPlugin

Like you can see, messing up module interfaces prevents ModuleConcatenationPlugin (plugin for optimization) to do its job.

We all love and use babel to transpile modern ES syntax in our modules, but in this situation the babel-preset-env becomes a bad friend of ours - by default modules are transpiled to "commonjs" standard and that's precisely what we don't want when pulling together multiple packages into one application! We have to make sure to set modules: false in preset config. Webpack can do majority of its optimizations only for Harmony modules!

Do not transpile modules with your babel setup! If you do so, webpack will treat files as a blobs, loosing ability to prepare chunks properly for terser to cut off unused code.

Well, technically it's not that straightforward, of course. Webpack does ton of processing on its side in order to build the concatenated code, it does track of provided and used export-s on its side as well, before even calling terser, so "combined" code with all modules is still valid for terser. But once again - it won't work for anything else but static ES module syntax.

Under the hood

There is quite a complex process going under the hood, starting from you passing webpack.config.js to compiler and before bundle is generated. We'll touch slightly the parts that are interesting for our discussion.

Compilation phase is where all fun happens, below you can see its main steps.

Ultimately, during compilation webpack builds dependency graph for the entry point specified in your webpack.config.js (or several of them, if configuration specifies multiple entry points).

In essence, the idea is to start from the entry point, go through dependent modules, build them and connect together - modules are nodes and dependencies are connections of the graph.

789	 * @param {ModuleCallback} callback callback to be triggered
790	 * @returns {void}
791	 */
792	processModuleDependencies(module, callback) {
793		const dependencies = new Map();
+ 
843		this.addModuleDependencies(
844			module,
845			sortedDependencies,
846			this.bail,
847			null,
848			true,
849			callback
850		);
851	}
+ 
1033	_addModuleChain(context, dependency, onModule, callback) {
+ 
1093					const afterBuild = () => {
1094						if (addModuleResult.dependencies) {
1095							this.processModuleDependencies(module, err => {
1096								if (err) return callback(err);
1097								callback(null, module);
1098							});
+ 
1102					};
+ 
1111						this.buildModule(module, false, null, null, err => {
+ 
1123							afterBuild();
1124						});
1125					} else {
1126						this.semaphore.release();
1127						this.waitForBuildingFinished(module, afterBuild);

Involved projects:

webpack

Scheme created
by Bohdan Liashenko
with codecrumbs.io

Compilation


0	Start for entry module (`Compilation.js#1033`)
1	Build module (`Compilation.js#1111`)
2	After build process module dependencies (`Compilation.js#1095`)
3	Add dependencies to module (`Compilation.js#843`)

To build module means to generate AST while extracting all needed information (export-s, import-s etc.). Webpack relies on acorn.Parser (from acorn) to build and process AST.

Next comes optimization phase.

345			const SideEffectsFlagPlugin = require("./optimize/SideEffectsFlagPlugin");
346			new SideEffectsFlagPlugin().apply(compiler);
347		}
348		if (options.optimization.providedExports) {
349			const FlagDependencyExportsPlugin = require("./FlagDependencyExportsPlugin");
350			new FlagDependencyExportsPlugin().apply(compiler);
351		}
352		if (options.optimization.usedExports) {
353			const FlagDependencyUsagePlugin = require("./FlagDependencyUsagePlugin");
354			new FlagDependencyUsagePlugin().apply(compiler);
355		}
356		if (options.optimization.concatenateModules) {
357			const ModuleConcatenationPlugin = require("./optimize/ModuleConcatenationPlugin");
358			new ModuleConcatenationPlugin().apply(compiler);
359		}
360		if (options.optimization.splitChunks) {
361			const SplitChunksPlugin = require("./optimize/SplitChunksPlugin");

23};
24
25class FlagDependencyUsagePlugin {
26	apply(compiler) {
27		compiler.hooks.compilation.tap("FlagDependencyUsagePlugin", compilation => {
28			compilation.hooks.optimizeDependencies.tap(
29				"FlagDependencyUsagePlugin",
30				modules => {
+ 
38							module.usedExports = addToSet(
39								module.usedExports || [],
40								usedExports
41							);
42							if (module.usedExports.length === old) {
43								return;

`FlagDependencyUsagePlugin` identifies `usedExports` and assign to the `module`.

Involved projects:

webpack

Scheme created
by Bohdan Liashenko
with codecrumbs.io

FlagDependencyUsagePlugin

FlagDependencyUsagePlugin hooks into the compilation phase and identifies usedExports. Basically, the idea is to find what "moduleA" imports from "moduleB", to set its usedExports. This process requires a lot of recursive traversing and "counting references".

As you know, webpack has pipe of plugins working on events, if you want to learn more, check out my other post Tapable library as a core of webpack architecture.

FlagDependencyUsagePlugin.js follows what HarmonyImportDependencyParserPlugin.js found about dependencies usage.

37				return true;
38			}
39		);
40		parser.hooks.importSpecifier.tap(
41			"HarmonyImportDependencyParserPlugin",
42			(statement, source, id, name) => {
43				parser.scope.definitions.delete(name);
44				parser.scope.renames.set(name, "imported var");
+ 
55		);
+ 
128		parser.hooks.call
129			.for("imported var")
130			.tap("HarmonyImportDependencyParserPlugin", expr => {
+ 
136				const settings = parser.state.harmonySpecifier.get(name);
137				const dep = new HarmonyImportSpecifierDependency(
+ 
143					name,
144					expr.range,
145					this.strictExportPresence
146				);
147				dep.directImport = true;
+ 
151				parser.state.module.addDependency(dep);
152				if (args) parser.walkExpressions(args);
153				return true;

134	info,
135	exportName,
+ 
141) => {
142	switch (info.type) {
143		case "concatenated": {
+ 
153				} else if (!info.module.isUsed(exportName)) {
154					return "/* unused export */ undefined";
155				}
156				if (info.globalExports.has(directExport)) {
157					return directExport;
158				}
159				const name = info.internalNames.get(directExport);
160				if (!name) {

once `importSpecifier` is detected, variable will be marked as "imported var" for further tracking

Listen to calls (AST element `method` call), i.e. webpack is smart, imported method doesn't necessary means it's used, it needs to make sure it's called as well

Called imported method detected and saved as dependency (later going to be inside `usedExports` for imported module)

Mark unused export

Involved projects:

webpack

Scheme created
by Bohdan Liashenko
with codecrumbs.io

HarmonyImportDependencyParserPlugin

Webpack is smart, imported method doesn't necessary means it's used, we need to make sure it's called as well.

Once again, in order for this to work, import-s/export-s should remain in the package (not transpiled).

Interesting finds

There are way too many interesting things I've noticed in the source code of webpack that should be to mentioned. It probably needs a separate post.

I'll highlight just a few of them.

Remember that error when you run webpack for the first time, but forgot to install webpack-cli package? They are not peerDependencies, so webpack provides quite useful guidance for users on how to solve it.

34 * @param {string} packageName name of the package
35 * @returns {boolean} is the package installed?
36 */
37const isInstalled = packageName => {
38	try {
39		require.resolve(packageName);
40
41		return true;
42	} catch (err) {
43		return false;
44	}
45};
+ 
60const CLIs = [
61	{
62		name: "webpack-cli",
63		package: "webpack-cli",
64		binName: "webpack-cli",
65		alias: "cli",
66		installed: isInstalled("webpack-cli"),
+ 
70	},
+ 
81];
82
83const installedClis = CLIs.filter(cli => cli.installed);
84
85if (installedClis.length === 0) {
+ 
90	let notify =
91		"One CLI for webpack must be installed. These are recommended choices, delivered as separate packages:";
92
93	for (const item of CLIs) {
94		if (item.recommended) {

Simply try to resolve the package, thrown error means it's not installed!

Involved projects:

webpack

Scheme created
by Bohdan Liashenko
with codecrumbs.io

bin/webpack

Another rather big surprise, how many independent packages-dependencies webpack has. Literally for everything:

1) tapable package for event-driven architecture
2) terser for minification
3) acorn for AST processing
4) watchpack to watch file changes

That's obviously very nice, hence all of them can be reused for different purposes in other tools!

Interactive code schemes in this post were created with Codecrumbs App. You can try it out for your own codebase here, check out Getting started guide for more details.

Stay tuned for more content like this

Follow me on Twitter

Bohdan Liashenko

@bliashenko

Code geek. I tweet about interesting code tips and findings I am discovering under the hood of popular open source projects.

Most popular mistake to ruin Webpack bundle optimization

Intro

TL;DR

Under the hood

Interesting finds

Stay tuned for more content like this

Follow me on Twitter

Bohdan Liashenko

Join the Newsletter