The Numbers on Webpack 1 vs 2
Things are not always what they seem
So last week I wrote about my experiences with upgrading to Webpack 2. In that story, I mentioned that it seemed like Webpack v2 was much faster than v1. So why not try and benchmark each version and provide some numbers?
Disclaimer: I’m not a performance engineer or a statistician. Take this story as a data-point and do your own comparisons. I have no vested interests in any of the technologies mentioned, other than being a contributor to Confit.
Test Setup
Test cases
See the appendix for the exact setup for each case. The only test-case that differed from the others was the React test, which compared Webpack v2.1 beta 21 vs beta 25. (The reason for this exception was that React-support in Confit was only introduced recently, when Webpack 2.1-beta.21 was the current version. So when you see “v1 React …”, it’s really “v2 beta 21”).
- Sample App — Angular 1 ES6
- Sample App — Angular 2 Typescript
- Sample App — React
- “Real world example” Angular Form Lib
Factors to compare
- Included files (modules)
- Production build-time (run 5 times and take an average)
- Development build-time (run 5 times and take an average)
- Code size
Included files (modules)
One of Webpack 2’s new features is called Tree Shaking. Tree shaking basically removes unused ES6 files (modules) from the bundled code. Code has to be written a certain way for this to work.
The chart reveals a couple of things:
- Only Angular 2 shows a notable reduction in the number of modules included in the JS bundle. This may be due to tree shaking, but it may also be due to differences in the Angular 2 API (Webpack v1 uses Angular 2 beta 17, v2 uses Angular 2.1). It is more likely a combination of both of these factors.
- Angular 2 is much larger than React, and Angular 1 is tiny by comparison!
It is worth noting that the sample applications (results 1, 2 & 4) have identical behaviour and are interpreted by the browser (not using ahead-of-time compilation, which would reduce the code size (and number of modules) for Angular 2). So if you are building an application, framework-size should be one of your considerations.
Production build-time
Let’s look at the time it takes to create a minified, optimised JS bundle. Each build used the “source-map” option, which is what I normally use for production builds.
- In every case except “Angular 1”, production builds took longer!
Development build-time
Production builds take a little longer (up to 20%), but what about development builds (using the “cheap-module-source-map” option)?
- Again, most development builds took longer with Webpack 2!
- The “Angular Form Lib” test is interesting in that the build time was almost the same. This could be due to this test-case containing a few more non-JS assets (images, fonts, stylesheets) in the build than the other test-cases.
Code size
The last factor that I measured was the actual code-size (in bytes) of the application code and the framework (vendor) code.
- Angular 2’s code-size increased substantially (102kB) even though the number of modules included (see first graph) decreased by 130! Again, this will be partially due to changes in the Angular 2 framework from beta 17 to v2.1. But, it means that using Angular v2.1 adds 100kB for the same functionality??! That’s a 12% size increase for no discernible benefit, despite less modules being bundled.
- React is interesting as it shows a reduction in code-size (of 4kB) between Webpack v2 beta 21 and beta 25! I have no idea why this would be the case. Tree shaking?
- Except for “Angular Form Lib” (which does not include a copy of the framework), the application bundle is tiny in comparison to the vendor code. This is expected because the sample application is a tiny application.
Summary
Webpack 2.1.0 beta 25 is approximately 10% slower when creating development & production builds, for the test-cases tested. Code size remains about the same, though there is some evidence of tree-shaking working to reduce the number of modules in the final bundle.
I’m personally surprised by these results as when using Webpack 2 it felt faster. Upon reflection, this may be due to the fact that the console UI provides better feedback on what the tool is doing (compared to v1). So even though it is actually a little slower, I will still use it going forward due to the potential for smaller bundles on modern ES6 code-bases.
Appendix
All tests were performed 5 times to ensure some degree of consistency (only affects build time, which is an average of the 5 runs).
All tests were run with the following configuration:
- MacOS 10.12 (Sierra), 2.4GHz Intel Core i7, 8GB RAM
- NodeJS 6.2.2
- NPM v4.0.2
- Webpack source-maps: cheap-module-source-map (dev), source-map (prod)
Angular 1 (v1) Test Details
- Confit 6.0.1
- Webpack 1.12.11
- Webpack Dev Server 1.14.1
- Angular 1.5.5
Angular 1 (v2) Test Details
- Confit 8.1.1
- Webpack 2.1.0-beta.25
- Webpack Dev Server 2.1.0-beta.9
- Angular 1.5.8
- Sample App code
Angular 2 (v1) Test Details
- Confit 6.0.1
- Webpack 1.12.11
- Webpack Dev Server 1.14.1
- Angular 2.0.0-beta.17
- TypeScript 1.8.9
- Dynamic compilation (in browser)
Angular 2 (v2) Test Details
- Confit 8.1.1
- Webpack 2.1.0-beta.25
- Webpack Dev Server 2.1.0-beta.9
- Angular 2.1.1
- TypeScript 2.0.6
- Dynamic compilation (in browser)
- Sample App code
React (v2b21) Test Details
- Confit 7.4.0
- Webpack 2.1.0-beta.21
- React 15.3.2
React (v2b25) Test Details
- Confit 8.1.1
- Webpack 2.1.0-beta.25
- Webpack Dev Server 2.1.0-beta.9
- React 15.3.2
- Sample App code
Angular Form Library (v1) Test Details
- Confit 6.0.1
- Webpack 1.12.11
- Webpack Dev Server 1.14.1
- Angular v1.2.29 (required for IE8 compatibility)
- Note: For the purposes of comparison, the form library produced a production build (
webpack -p). Normally (for the sake of distribution) the library does not use the-pflag.
Angular Form Library (v2) Test Details
- Confit 8.1.1
- Webpack 2.1.0-beta.25
- Webpack Dev Server 2.1.0-beta.9
- Angular v1.2.29
- Sample App code
