Duplicate Dependency Problem#

Rsdoctor will report cases where multiple duplicate dependencies exist in the same builder's artifact.

Hazards of Duplicate Packages#

• Security
  • Many packages adopt singleton mode and are only loaded once by default. Examples include core-js, react, and some component libraries. Having multiple versions coexist can cause runtime errors.
• Runtime Perßformance
  • Increases artifact size, slowing down network transmission
  • Same code for the same functionality runs multiple times

How to Solve Duplicate Package Problems?#

To solve the issue of multiple versions of dependencies, you can address it from both the dependency and build aspects.

Dependency Aspect#

1. Use the npm dedupe command#

Generally, package managers try to install the same version of a package based on the semver range. However, long-term projects may have some duplicate dependencies due to the existence of lock files.

Package managers provide the dedupe command, such as npm/yarn/pnpm dedupe, to optimize duplicate dependencies within the correct semver range.

2. Use resolutions#

Under the constraints of semver, the effectiveness of the dedupe command may not be ideal. For example, if the artifact contains dependencies debug@4.3.4 and debug@3.0.0, where they are respectively depended on by "debug": "^4" and another package's "debug": "^3".

In this case, you can try using the resolutions feature of the package manager, such as pnpm's pnpm.overrides, .pnpmfile.cjs, or yarn's resolutions.

The advantage of these features is that they can break free from the constraints of semver and change the version declared in package.json during installation to precisely control the installed version.

However, before using them, it is important to consider the compatibility between package versions and evaluate whether optimization is necessary. For example, whether the logic changes between different versions of the same package will affect the functionality of the project.

Build Aspect#

Use resolve.alias#

Almost all builders support modifying the paths for resolving npm packages. Therefore, we can eliminate duplicate dependencies by manually specifying the resolve paths for packages during compilation. For example, using Rspack or Webpack, if lodash is duplicated in the build, we can configure it as follows to specify the resolve paths for all lodash packages to the node_modules directory in the current directory.

// webpack.config.js/rspack.config.js
const path = require('path');

module.exports = {
  //...
  resolve: {
    alias: {
      "lodash": path.resolve(__dirname, 'node_modules/lodash')
    }
  }
};

This method also requires attention to the compatibility between package versions.

Cases of Handling Duplicate Packages#

Duplicate Packages in pnpm-workspace#

In this project, the web app depends on react@18.2.0 and imports component using "component": "workspace:*". The component package, in turn, depends on react@18.1.0. The project structure is as follows:

├── node_modules
├── apps
│   └── web
│       ├── node_modules
│       │   ├── component  -> ../../packages/component
│       │   └── react      -> ../../../node_modules/.pnpm/react@18.2.0
│       ├── src   
│       │   └── index.js
│       ├── webpack.config.js
│       └── package.json
└── packages
    └── component
        ├── node_modules
        │   └── react      -> ../../../node_modules/.pnpm/react@18.1.0
        ├── src   
        │   └── index.js
        └── package.json

When executing webpack build under apps/web, the code in the web directory will be resolved to react@18.2.0, and then the code in the component directory will be resolved to react@18.1.0. This results in the output of the web project containing two versions of React.

Solution#

This issue can be resolved using the resolve.alias configuration in the builder, such as Rspack or Webpack. By specifying the resolve path for React to only resolve to apps/web/node_modules/react, you can ensure that only one version of React is included in the output. Here is an example code:

// apps/web/webpack.config.js
const path = require('path');

module.exports = {
  //...
  resolve: {
    alias: {
      'react': path.dirname(require.resolve('react/package.json'))
    },
  },
};

Duplicate Packages Caused by Peer Dependency#

This handling method also applies to projects with duplicate packages caused by multiple instances of peerDependencies in pnpm workspace. The project directory structure is as follows:

├── node_modules
├── apps
│   └── web
│       ├── node_modules
│       │   ├── component  ->  ../../packages/component
│       │   ├── axios      ->  ../../../node_modules/.pnpm/axios@0.27.2_debug@4.3.4
│       │   └── debug      ->  ../../../node_modules/.pnpm/debug@4.3.4
│       ├── src   
│       │   └── index.js
│       ├── webpack.config.js
│       └── package.json
└── packages
    └── component
        ├── node_modules
        │   └── axios      ->  ../../../node_modules/.pnpm/axios@0.27.2
        ├── src   
        │   └── index.js
        └── package.json

In this project, when executing webpack build under apps/web, the code in the web directory will be resolved to axios@0.27.2_debug@4.3.4, and then the code in the packages/component directory will be resolved to axios@0.27.2.

Although they are the same version, they have different paths, resulting in two copies of axios in the output.

The solution is to configure the web project to only resolve the axios package under the web directory's node_modules.

// apps/web/webpack.config.js
alias: {
  'axios': path.resolve(__dirname, 'node_modules/axios')
}