Migrating Legacy JS to TypeScript

At Qualtrics nothing can be said to be certain except death, taxes, and absurd quarterly growth. Enter TypeScript, a powerful tool for growing JavaScript codebases. I am going to talk about why we chose to convert our automation framework to TypeScript and our methodology in carrying out an agile rewrite.

This article will assume you know enough TypeScript to understand a few code snippets. We will not talk about any super complex type system sorcery, favoring a simple subset of TypeScript. Finally, we won’t be talking about TypeScript versus Flow or other similar languages.

Why TypeScript?

The Qualtrics team grows fast. Sticking to languages that are similar to the ones we already use makes it quicker for new engineers to ramp up on our codebase. TypeScript looks like a child of C#, Java, and JavaScript.  A guiding design principle for the TypeScript project is that it attempts to stay as true to the current JavaScript syntax as it can. Any valid JavaScript should be valid TypeScript with a few compiler options checked off. At Qualtrics my job is to write code that makes it easier for product developers to test code, so if Java and JavaScript are what my developers know, my framework should reflect that.

Many at Qualtrics use Visual Studio Code for our JavaScript. It’s a great editor, but it isn’t a superhero. A lot of the time it is impossible to get any IntelliSense on our code. It’s not the poor IDE’s fault—dynamic languages like JavaScript just don’t give it enough to go on. There are other solutions. We used JSDoc to try to give typing information to all of our functions, but comments can lie to you. All it takes is one commit where someone forgets to update the comment to reflect their changes and one code review where someone misses it. By contrast, the information that types give us is never out of date and seldom lies.

Designing interoperability layers

Our codebase is sandwiched between both upstream and downstream vanilla JavaScript code. They both use function modules instead of prototypes or ES6 classes, so we can’t even inherit from them. It’s unfortunate, but TypeScript does little to give us safety in this situation.

We can make this code easier to interact with in the rest of our TypeScript code and still get a lot of the benefits to our development environment that types give us. There are two main tools at our disposal:

1. We can wrap the code in a class responsible for communicating with unmanaged code. A concrete example of this can be found in the section below on External Auto-Magic.
   • Gives us a lot of flexibility in the communication protocol between our code and outside code.
   • Slower to write and a lot more fragile.
2. We can create interfaces that are strongly typed and cast references to the upstream class to those interfaces. A concrete example of this can be found in the section below on Typing External Dependencies with Interfaces.
   • Dirt cheap to write.
   • There are less tools at your disposal for doing things like error handling or run-time type assertions in this layer.
   • Doesn’t work in all situations, like auto-magic.

Both of these require a lot of maintenance. Any upstream changes make you update this layer. This is why it is critical to only type or wrap what you need to keep maintenance cost low.

It is still hands-down worth investing in these layers. I see two main alternatives. You can leak interop code all over the codebase with even worse maintenance cost, or just bite the bullet and import the JS dependency without typing information. The latter makes it significantly harder to lean on your compiler when you’re refactoring this JS back into TypeScript. Both of these seem worse than storing references in one location and accepting some churn in that layer.

Preservation of Signatures

The most important skill when refactoring is preservation of signatures. Try as hard as you can to not alter functionality or form. I recommend Working Effectively with Legacy Code for more information on this idea as it applies to code. Preserving signatures is important even for build pipelines, especially since JS to TS transitions may introduce a build step to a project for the first time.

For instance, if this is my original directory structure:

This should be my new structure:

This may make the more sensitive readers gag, but it preserves a contract with downstream clients. The code in the src directory is still runnable JS. Later this can be cleaned up, but this keeps changes in behavior atomic.

Split Implementation and The Importance of Pinch Points

The notion of Preservation of Signatures can be applied when moving a JS class to TS with a pattern I call Split Implementation. This pattern keeps around the old JS class that downstream dependencies know about. Say a class is exported in foo-bar.js, when I converted that class to TypeScript I put it in typed-foo-bar.ts. The implementation in foo-bar.js then became just instantiating my implementation in typed-foo-bar.ts and returning references to that. This lets me preserve the import structure and convert a class one function at a time.

If your code is decently architected, you’ll notice there are “pinch points” in your architecture. A common example is a base class that many things inherit from. These provide a natural place to put up an interop wall and start your conversion. You can use Split Implementation to provide a JS instance of your new base class to the rest of your code base. Starting here isolates the amount your code has to interact with outside codebases you don’t control, and it gives you the greatest bang for your buck for the rest of the refactor. With the Split Implementation conversion style, you can run your unit test suite to rapidly verify your refactor.

Say we has some JS base class in “base.js”:

module.exports = function() {
  var self = this;
  self.foo = "bar";
  return self;
};

We can start here by making a “typedBase.ts”:

export class Base {
   protected foo: string;
   constructor() {
       this.foo = "bar";
   }
}

Then we can change “base.js” to:

var TypedBase = require("./typedBase").Base;
module.exports = function() {
  return new TypedBase();
};

Now we can support both TS and JS children using this pattern! Here's an example of a TypeScript child:

import { Base } from "./typedBase";
export class ChildB extends Base {
  public say() {
      console.log(this.foo.toLowerCase());
  }
}

Here's an example of a JavaScript child:

var Base = require("./base");
module.exports = function() {
  var self = Base();
  self.say = function() {
      console.log(self.foo.toUpperCase());
  };
  return self;
}

From here on, the pace that we convert the rest of the code doesn’t matter too much. We can easily and cleanly maintain a partition between converted children of this class and JS children of this class. Furthermore, this gives a solid base if we need to stop refactoring and add a new class to the system.

Challenges we faced

External Auto-Magic

Upstream we have some auto-magic code that executes when we register data items with a handler, like this:

module.exports = function () {
  var self = this;
  self.registerPets = function (cats) {
      var ids = Object.keys(cats);
      ids.forEach(function (id, _) {
          self["meowAt" + id] = function () {
              console.log("Meow, meow. " + cats[id] + "!");
          } ;
      });
  };
  return self;
};

I don’t know how much you know about statically typed compilers, reader, but let me tell you: they don’t like this much. Since we don’t own this code, we had to design around it. This is an adaptation of a classic factory pattern where the objects created reference the original JS code.

const JSCatGreeter = require("./cat-greeter");

class Cat {
  private jsCatGreeterInstance: any;
  private interopId: string;
  public readonly name: string;

  constructor(jsCatGreeterInstance: any, interopId: string, name: string) {
      this.jsCatGreeterInstance = jsCatGreeterInstance;
      this.interopId = interopId;
      this.name = name;
  }

  public meowAt() {
      this.jsCatGreeterInstance["meowAt" + this.interopId]();
  }
}



export class Cattery {
  private jsGreeter: any;
  private static currentUuid: number = 0;

  constructor() {
      this.jsGreeter = JSCatGreeter();
  }

  public getNewCat(name: string) {
      const newId = this.getUniqueIdentifier();
      const cat = new Cat(this.jsGreeter, newId, name);
      let interopObj: any = {};
      interopObj[newId] = name;
      this.jsGreeter.registerPets(interopObj);
      return cat;
  }

  private getUniqueIdentifier() {
      const id = `__probablyNotTaken${Cattery.currentUuid}`;
      Cattery.currentUuid += 1;
      return id;
  }
}

Using our new typed Cattery, we see everything works!

This seems like overkill for our simple little Cattery, but we used this same pattern during our refactor. This enabled us to keep a nice layer of types over our auto-magic and still share implementations with the company-wide library.

Typing External Libraries with Interfaces

For most packages, the DefinitelyTyped project has your back. If a project isn’t there you can make a lib.d.ts file yourself, or you can create an interface and cast objects to that interface. For example, if I have some JS library:

module.exports = function() {
  var self = this;
  self.countMatches = function(s) {
      return s.filter(function (val) {
          return val == 1;
      }).length;
  };
  return self;
};

We can give it the following interface:

export interface SillyAPI {
  countMatches: (xs: number[]) => number;
}

And cast an instance to this interface for typing information:

import { SillyAPI } from "./interfaces";
const sillyAPI = require("./sillyAPI")();

const typedSillyAPI = sillyAPI as SillyAPI;
typedSillyAPI.countMatches([1,2,3]);

We had a particularly tricky instance in our codebase. There are types packages for chai and chai-as-promised, but to get an instance of the typed chai-as-promised dependency you must invoke a method on chai. To get around this, we ended up exporting our own version of chai:

import * as chai from "chai";
import * as chaiAsPromised from "chai-as-promised";

chai.should();
chai.use(chaiAsPromised);
export const expect = chai.expect;

Then the rest of the code uses it like this:

import { expect } from "shared-package";
expect(Bluebird.resolve(1)).to.eventually.equal(1);

A little dirty, but this has the added benefit of further separating our code from our dependencies.

Don’t Convert the World

Huge refactors are usually a disaster. When your unit of change is large, so is your chance of introducing bugs. A lot of articles recommend you change all of your .js files to .ts files, and mark everything with the any type. That seems like a lot of work that doesn’t really get me much. Implicit any checking can also be disabled in the tsconfig, but that handicaps the compiler for the rest of the code I write. There are other tools at your disposal like the allowJs compiler flag. For more fine-grained control, you can also use something like grunt-contrib-copy to copy all *.js files into a build artifact directory. This actually paid off for us during our refactor. There was a client facing that arose and required our immediate attention. Instead of having to manage a tricky patchwork of branches, or drop our changes and try again later, we were able to work in the codebase reasonably while it was mid-transition.

Scrappiness is a valuable refactoring tool

Responsible refactoring is some of the most creative programming engineers get the chance to do. There will never be one true practice to follow. A few guiding principles: keep changes granular, limit scope to just refactoring, and be diligent about the contracts between code you are changing and the code you are not. What matters is what works for you and your team. Be scrappy and creative.

Further Reading

I would highly recommend Basarat Ali Syed’s book on TypeScript. It provides an overview of JavaScript, ES6 and ESNext syntax, TypeScript, tips and best practices, common stumbling blocks, and it even has a walk through of the internals of the TypeScript compiler. All for the low price of free.
https://basarat.gitbooks.io/typescript/docs/types/migrating.html

Also recommended is the You Don’t Know JS series. It goes into more detail on the nooks and crannies of JavaScript. Life is easier when you know your language.

https://github.com/getify/You-Dont-Know-JS