What I Learned This Week - Merge Conflicts in Your Lockfile 😱

Posted on: 29th Apr, 2020

This is gonna be a short but sweet post.

Hopefully sweet, anyway.

First, a little backstory about lockfiles. If you just want to get to the main content, Skip ahead

What's a lock file?

A lock file is an auto generated file that contains a list of versions of third party libraries that are used in a project.

In the NodeJS world, we usually use either NPM or Yarn to manage our dependencies.

We are probably used to seeing a package.json file created, which lists out the dependencies that are used in our project under the dependencies and devDependencies keys. For example, the package.json file for this website looks something like this (shortened):

{
  "name": "harrietrydercouk",
  "description": "Harriet Ryder personal blog and portfolio",
  "license": "MIT",
  "scripts": {
    "develop": "gatsby develop",
    "build": "gatsby build --prefix-paths"
  },
  "engines": {
    "node": "10.14"
  },
  "dependencies": {
    "axios": "^0.19.1",
    "gatsby": "^2.18.21"
  },
  "devDependencies": {
    "eslint": "^6.8.0",
    "gh-pages": "^2.0.1"
  }
}

But you'll notice that the dependency versions listed in this file are quite vague. "^6.8.0" means "Install anything from 6.8.0 up to but not including 7.0.0" (the next major version). The syntax for specifying versions and ranges is known as Semver (Semantic Versioning). You can read more about it here.

This means that different people on a team might clone the repository, run the install command, and end up with different minor versions or patched versions to one another. Often this is fine - there should not be any breaking changes in minor/patch versions, but sometimes there can be problems. So for that reason, NPM creates a package-lock.json file and Yarn creates a yarn.lock file which lists the exact versions of all the dependencies that should be installed. We commit these lockfiles to Git so we can share them round our team.

If you're wondering why not just put the exact versions in the package.json file in the first place, this quora answer provides some interesting insight!

Anyway, that's the point of a lockfile. To keep a record of exact dependency and subdependency versions installed so that anyone (or anything) that builds the project does so with the exact same third party code - and you don't get any nasty surprises!

Lockfiles are autogenerated

One more thing to mention - lockfiles are autogenerated. When you run npm install or yarn install the lockfile gets updated automatically. You should never find yourself editing it yourself.

Merge Conflicts

So what happens if you end up with a merge conflict in your lockfile?

This is something that is bound to happen at some point, when team members explicitly upgrade dependencies to different versions. In my team, we experience it a lot because we publish a lot of our own projects as private NPM packages and use them in other projects.

Basically, whenever someone makes a change to one of these packages (which happens daily!) we have an opportunity to update the package version. Because versions change so much, on different branches we'll likely have upgraded to different versions and this causes merge conflicts in our package.json file and our package-lock.json file.

Fixing conflicts in a package.json file isn't so bad. But in an autogenerated lockfile, it's a nightmare.

Until a few months ago, I was diligently sifting through these merge conflicts one by one every time the happened, picking which change to keep. It was hard to see which version was required, especially with subdependencies, and basically a real pain in the arse.

Auto Merge

One day, I was in the middle of resolving a merge conflict when I accidentially went back to the terminal and ran npm install.

I expected the install to fail since my lockfile was full of merge conflict markers like:

<<<<<<< HEAD
 "@ant-design/icons": {
  "version": "4.0.5",
  "resolved": "https://registry.npmjs.org/@ant-design/icons/-/icons-4.0.5.tgz",
  "integrity": "sha512-qYme6WcHF5J+/dMLBpYTvWx4fSAk3rtWu3wxt8bHRjiB5BlKWME
},
=======
 "@ant-design/icons": {
  "version": "4.0.3",
  "resolved": "https://registry.npmjs.org/@ant-design/icons/-/icons-4.0.3.tgz",
  "integrity": "sha512-jkldjfkldscnjknc/fjdsklfjdsi83/2489skjfnn=="
},
>>>>>>> branch-a

But amazingly, NPM informed me that it detected a merge conflict in the lockfile and was attempting to automerge. Which it did, with no problem, saving me 20 minutes of work and a headache.

I immediately shared this glorious news with my team, who also had no idea this was a feature of NPM! Yarn works exactly the same - it identifies merge conflict markers and fixes the file for you.

So now when I have a merge conflict, all I need to do is sort the package.json file out and then run npm install.

If you want NPM to fix your conflicts for you whilst automatically when you merge (i.e. without having to run the install yourself) check out npm-merge-driver.