Skip to main content

Javascript Broke, and no one noticed

So, on Tuesday, at around 11:30, the Javascript world went into cardiac arrest.  The details are pretty interesting only if you're as deep in the code as I am, so here's a summary for the tl;dr; crowd.

What happened?

Code builds on itself.  No one (well, almost no one) codes in binary anymore because we came out with code that wraps groups of binary into smaller, more readable pieces.  That then got wrapped the same way, and so on.  That's how the software world works.  Some articles about this incident even use a Jenga tower as a reference, and that's not very far off (sadly).  This isn't just from language to language either.  Particularly in a language that's been around for a while (like Javascript), there are libraries of code within the language to do the same thing (wrap complex bits in smaller, more readable pieces).  One of those libraries has been around for a loooooooooong time and pretty much everyone relied on it, somewhere deep down the Jenga tower (we call it the software stack).  For the devs in my audience, some of the software that directly relied on left-pad (the library that was the most important here) included Node.js and Babel.  For the non-devs: it was pretty deep down the stack--so deep that most people weren't even aware they were relying on it.

Azer Koçulu is the man of the hour.  He built the tiny piece of code (literally 11 lines) that everyone was using.  He was responsible for hundreds of emergency meetings on Tuesday (I don't know the exact number, but you can imagine a lot of execs calling their IT department screaming).  He's a big deal in the OSS (open source software) community.  He's published over 270 packages to npm, the place most people (including Node.js and Babel) get their Javascript code.  On Tuesday he un-published all 270+ of his packages, including one innocuous 11 line piece of code called left-pad.  After that moment, everyone whose code relied on left-pad at some level was fine (really, it was).  But new code starting out, and deployment builds all failed, because they couldn't retrieve one little dependency.

Why did he do it?

I'm not going to steal the man's thunder.  He published a blog post in his own words saying why he did it.  Here's the tl;dr version.

Koçulu had a package named kik.  A messenger company named kik asked Koçulu to give them the package name so they could use it instead.  Koçulu refused.  The company pleaded with npm.  npm sided with kik.  Koçulu was offended and decided not to be involved with npm in the future.

It's a bit more complicated than that: Koçulu was good friends with most of the npm team, including the guy who took away his package.  kik sent Koçulu threatening emails, and Koçulu sent kik rude emails.  The whole thing took just under 3 weeks to come to this.

For some perspective, here are some of the highlights from the email chain, which was published by the kik team in their own blog post on the matter.

our trademark lawyers are going to be banging on your door and taking down your accounts and stuff like that


fuck you. don’t e-mail me back.


Is there something we could do for you in compensation to get you to change the name?


you can buy it for $30.000 for the hassle of giving up with my pet project for bunch of corporate dicks

What I expected

While Koçulu was kind of a dick to kik, and kik was kind of a dick to Koçulu, all of that was more or less expected as far as I'm concerned.  Open source developers, especially those as active as Azer are notoriously anti-establishment and are likely to respond to even gentle requests from them harshly.  And kik has trademark rights to consider, so you expect them not to be nice when someone tells them no.  They actually mentioned it in the email chain:

we’d have no choice but to do all that because you have to enforce trademarks or you lose them.

All of that is more or less, the sort of banter you'd expect if you're as deep in the software world as I am.  But that's not where it ended.  kik went and got npm involved.

npm's take

kik sent numerous emails over the 3 weeks following the initial email thread with Koçulu about how rude he was to them asking "can you guys help?" repeatedly.  Eventually, npm sent one email to both parties involved with this message.


Hi, Azer.
I hear your frustration. The desire to continue to use the kik and kik-starter package names, is clear.
Our goal is to make publishing and installing packages as frictionless as possible. In this case, we believe that most users who would come across a kik package, would reasonably expect it to be related to In this context, transferring ownership of these two package names achieves that goal. I understand that you’ve committed time and energy to the packages already, and we don’t take that lightly. I’m hopeful that you’ll be able to republish this project with a new name.
Can you provide an npm account to transfer the name to?
Thank you both for your patience and understanding.

So npm's decision was to transfer the package from one account to another.  As anyone who's lost a battle would be, Koçulu was offended and decided to remove all of his code from this particular package manager.  He even sent a reply with an explanation.


Isaac; I’m very disappointed with your decision here. I know you for years and would never imagine you siding with corporate patent lawyers threatening open source contributors.
There are hundreds of modules like Kik, for example, Square;
So you’ll let these corporate lawyers register whatever name they want ? Noone is looking for a Kik package because they don’t have one.
I want all my modules to be deleted including my account, along with this package. I don’t wanna be a part of NPM anymore. If you don’t do it, let me know how do it quickly. I think I have the right of deleting all my stuff from NPM.

He deleted code?

Well, no actually.  He didn't delete anything.  He just "un-published" them.  You can still get access to all of his open source code on his github account.  He simply doesn't want his code in the package management system that screwed him over.  Sadly, the package management system he pulled his code from is the one that everyone uses so it broke everyone's code (remember that Jenga tower again?).  It was relatively easy to fix things, but most people weren't aware of what was broken, so it resulted in a very big freak out (which is why I called it cardiac arrest and not death).

But I didn't notice

OK.  To be fair, most people didn't notice.  Most build processes are smarter than to publish code to production when something like that breaks.  But the developers who were trying to publish code to production on Tuesday noticed, and that's what the big freakout was.  In the software world, many companies publish code to production multiple-times per day, so you can see how quickly this becomes a corporate meeting.

In fact, npm "fixed" the brokenness by re-publishing the left-pad package.   npm's Laurie Voss made the call here.  And later, npm published a followup about the decision.

What's the contraversy here?

So most of the discussion is about the argument between Azer and kik, and about this "unprecedented" move by npm to re-publish an un-published package.  I'm ok with all of that.  I expected the argument, and I am even ok with npm choosing to publish someone's package without their consent.  It is open source after all.  I tend to side with Voss on this one:

In the meantime, several thousand open source projects have been repaired, and I'm sleeping fine tonight.

I have the same problem that Koçulu had to begin with.  I think that npm should not be handing over projects to other users.  That's a dispute between the parties involved.  For npm to act as arbitrator in the situation is inappropriate.  Moreover, to hand over a project from someone like Koçulu who has published over 270 open source packages to npm and knows the team on a first name basis to someone like the kik team, who sends nasty threatening emails to that same developer is wrong.

We're talking about a community here, not just one guy.  They may have made the kik team happier, but they damaged the community as a whole.  They took the non-contributor and put him above one of their top contributors.  That's very wrong to me.

For completeness, and those who want it, here's the code that broke the "internet" for 2.5 hours.

module.exports = leftpad;

function leftpad (str, len, ch) {
  str = String(str);

  var i = -1;

  if (!ch && ch !== 0) ch = ' ';

  len = len - str.length;

  while (++i < len) {
    str = ch + str;

  return str;


Popular posts from this blog

Teams and Complexity

Let's pretend you're a car mechanic (I don't know, maybe you are).  But you don't work at some shop in town, you work at a bigtime auto-maker onsite at the factory.  You help build cars from scratch.  You work with a lot of other car mechanics, and various engineers of different levels.  If we were looking at the 1910s, you might be working for Henry Ford on one of the first ever assembly lines.  Either way, you have a specific part of the car you're responsible for.  Your skills probably extend beyond that, and you might work on a different part of the car each day as your manager moves you around to keep the factory efficient.

One of your coworkers, Bob, is a superb engineer.  He is very good at building cars, far better than you, and he does it faster than everyone else.  Your boss really likes him.  You often get stuck after him in the assembly line, so you know exactly what sorts of things he does.  He's not sloppy, but he likes to do things his way.  He w…

Managing Programmers

Working with other programmers is tricky.  That said, it's nothing compared to the job of managing programmers.  One of my favorite quotes about Perl is that (paraphrased) "a Perl developer is like a rockstar.  Now imaging having a bunch of rockstars in one room together and you will understand why you don't want an entire team of Perl developers."  It's not about Perl here though. What's important to understand is that any developer worth his salt is going to be like a rockstar.  And yes, there are a lot of professional developers out there who aren't worth their salt, but that's for another post another day.  Rockstar may not be the right term here, but think of it this way.  These guys are smart.  They may not be geniuses, but there's going to be things that they know that you don't and probably never will.

I've seen it more than once and it's not going to make some Product Managers happy, but I'm going to state a fact, an eleph…

How to identify a skilled programmer during an interview

How does one identify a skilled programmer?  No company that has interviewed me could tell the difference between myself and other programmers they'd interview.  The interview process is truly a game of luck in this industry--on both sides.  Both the programmer and the company are basing their actions entirely on luck.

Companies have come up with numerous methods to attempt to discern a good programmer from a bad one.  The best tricks they have include a series of math problems, algorithms, problem solving technique tests, and even obscure programming questions, some without definitive answers.  As an example: Is there an authoritative source of information on the core principles that define object oriented programming?  I've heard everywhere from 3 to 7.  In a field of research about a synthetic concept, an authoritative answer is almost impossible to obtain.

Programmers were then forced to study to the interview.  Careercup is one of my favorite sites for this.  This almost …