Type definitions to retrieve objects from localStorage

If you have ever written an object into localStorage, you most likely have had the nasty surprise that reading it back didn't instantly return a full object with its associated methods. As with all bugs, you swore that it wouldn't happen again, but how many times do we trip over the same stone? I recently decided that the last bug occurrence was one too many and that I would solve this once and for all. Let me take you on a trip down type manipulation lane...

The original problem

I have a Db class, a simple controller that talks to a Storage interface and stores data with JSON format:

export class Db {
    private db: Storage;

    constructor(db: Storage) {
        this.db = db;
    }

    public get<T>(key: string, mapFn: DbMapFn<T>|null = null): T {
        const valueStr = this.db.getItem(key);

        if (valueStr == null) {
            throw new Error(`Could not get value from key "${key}"!`);
        }

        return JSON.parse(valueStr);
    }

    public set<T>(key: string, value: T): void {
        this.db.setItem(key, JSON.stringify(value));
    }

    // ...and other goodies!
}

As simple as it looks, this has a fatal flaw that's not obvious from the beginning. Let's reproduce it with a short code example:

class Widget {
    constructor(public readonly value: number) { }

    public isEven(): boolean {
        return this.value % 2 == 0;
    }
}

(() => {
    const db = new Db(localStorage);
    const widget = new Widget(10);
    console.log(widget.isEven()); // true, as expected

    db.set("widget", widget);
    const widget2 = db.get<Widget>("widget");
    console.log(widget2.isEven()); // ERROR: widget2.isEven is not a function
})()

This looks like a compiler bug, but it's not! The issue here is that JSON.parse returns any, so the type checker will happily cast it to any other type and trust that we're adults who know what they're doing. Dumb compiler, I don't even know what I'm doing for dinner!

Now, instead of having a Widget object, we have an object that contains the properties of Widget but none of its methods. We can still manually test the evenness with widget2.value % 2, but you and I know that it won't pass a code review. The issue now is that we need to map the value we got from the storage into a Widget object, but we also don't want this to turn into a ceremony, since tons of apostates regularly visit our codebase. That's to say, this isn't going to cut it:

const widget2Data = db.get<{ value: number }>("widget");
const widget2 = new Widget(widget2Data.value);
console.log(widget2.isEven()); // true, but at what cost?

Duplicating the declaration of every property and its type gets old really fast. Let's look at a first approach to the solution.

Mapping the storage return value

We need a bit of ceremony, that's for sure, but we also want the database API to make it easy for us. Instead of trusting that our user will convert all the objects, we can expand the get function to receive a mapping strategy:

type DbMapFn<T> = (obj: T) => T;

class Db {
    // ...
    public get<T>(key: string, mapFn: DbMapFn<T>|null = null): T {
        const valueStr = this.db.getItem(key);

        if (valueStr == null) {
            throw new Error(`Could not get value from key "${key}"!`);
        }

        const value: T = JSON.parse(valueStr);
        return (mapFn != null) ? mapFn(value) : value;
    }
}

This solves our problem! Almost, it's not quite right. It works at runtime, but the types are flaky at best. Still, the idea is there: When we receive a mapFn argument, we can use it to construct the target object with user-provided instructions. This is how it looks with our Widget:

const widget2 = db.get<Widget>("widget", obj => new Widget(obj.value));
console.log(widget2.isEven()); // true, but I'm not satisfied yet

The issue here is that the compiler treats obj as a Widget when it's still an object without functions, so an unsuspecting user would be baffled at us seemingly re-creating Widget objects and could try to call some of the functions and crash the program. We could cast the parsed type to any and trust that the user would get the property names right, but we don't want to trust the user! It could perfectly be our future selves and I'm not trusting that guy to remember what he wrote a week ago!

How can we solve this conundrum then? As promised, let's have fun with types!

Having fun with types

So here's what we want: A type that represents our Widget (or any other object) in its just-deserialised state. That is, with its methods stripped and with access to its public data properties. In fact, at the time of writing, there's no way to access private properties when mapping types and it's probably a good idea to keep it this way. I'm going to show you the full type transformation and walk you through it:

type NonMethodPropertiesOf<T> = {
    [K in keyof T]: T[K] extends Function ? never : K
}[keyof T];

This is a generic type map, it maps generic types. This means that it takes a type T and applies a transformation (maps) to each of its properties and constructs a new type (NonMethodPropertiesOf<T>) with them. The most obvious part of the type's syntax is the generic <T>, meaning that we can apply this map to any type.

Next, there keyof T, which you'll see that appears twice. This is a TypeScript operator that creates a union type from the keys (the property names of) our generic T. As an example, take this simple class:

class XYZTuple {
    constructor(
        public x: number,
        public y: string,
        public z: boolean) { }
}

Here, keyof XYZTuple is "x"|"y"|"z", as we can index the object with those three strings. We then pass keyof T to the body of the map, telling which keys to iterate over.

Knowing this, we can read the body of the type map as for each property K in T only keep it if its type is not function-like. The first [K in keyof T] makes the fact that we're examining some keys generically named K we expect to be part of keyof T explicit. Thus, T[K] is the type of the member K in T. In the case of XYZTuple, T[K] will be number when the map looks at x, string when it looks at y and boolean when it looks at z.

To transform the type, we ask if it extends Function which is asking if it's a function-like type. If it is, we want to strip it, so we convert it to never, a type that cannot be instantiated. Otherwise, we keep it as it is.

This looks like it should be enough, but this map just lists the properties of T that are not functions. It doesn't create a new type. For that, we need to apply the transformation:

type WithoutMethods<T> = Pick<T, NonMethodPropertiesOf<T>>;

The Pick<T,U> type creates a new type from a subset of T that contains only the properties listed in the union type U. As it turns out, an union type is exactly what NonMethodPropertiesOf<T> returns, after it takes care of listing all non-function-like properties of T, we use Pick to apply the transformation by creating a new type that only contains those properties.

However, there's a slight problem here. What if T is an array? We're removing the method properties from the array, not from the type it contains. That's certainly not what we want to do, so we need to update our transformer:

type WithoutMethods<T> = T extends any[]
    ? Pick<T[0], NonMethodPropertiesOf<T[0]>>[]
    : Pick<T, NonMethodPropertiesOf<T>>;

There it is! Now, if T is an array, i.e. it extends any[], we will map it to an array of types without methods. Here, T[0] retrieves what would be the type of the first element of the array. Remember that we're not accessing the element, but asking for the type of the expression T[0], which is the underlying type of T.

And so, we can finally update our database driver class to work with this new type!

type DbMapFn<T> = (obj: WithoutMethods<T>) => T;

class Db {
    // ...
    public get<T>(key: string, mapFn: DbMapFn<T>|null = null): T {
        const valueStr = this.db.getItem(key);

        if (valueStr == null) {
            throw new Error(`Could not get value from key "${key}"!`);
        }

        const value: WithoutMethods<T> = JSON.parse(valueStr);
        return (mapFn != null) ? mapFn(value) : value as T;
    }
}

Now, our users will clearly see what's going on, as they won't be able to access T's methods any more. They'll also see the type WithoutMethods<T> in their mapFn function signature, further hinting that there are limitations to what they can do.

Note that, if we don't receive a mapFn, we are assuming that the object we're returning doesn't need a mapping to construct it with its methods. This is the case for primitive types such as number or string[], whose methods are part of JavaScript's core engine.

Limitations

There are two limitations to this technique that you must be aware of. First, as we said earlier, it doesn't work with private properties, as keyof T won't list them. The other one is a bit more subtle. TypeScript treats getters as data properties. This means that this will crash:

class WithGet {
    constructor (public x_: number) { }
    get x(): number { return this.x_; }
}

// ERROR: obj.x is not a function!
db.get<WithGet>("with-get", obj => new WithGet(obj.x));

This pattern works best when you use it with immutable data objects that you store in your database, as they hardly ever carry private types, making getter functions irrelevant for their use case.

And... We're done!

I like to use type systems to make the compiler impose the restrictions I would only be able to express with comments and tribal knowledge. This is a fine example of how a minimal amount of plumbing can make our code much more reliable and easier to extend. I hope you've enjoyed this small rambling about mapping types and are now ready to apply it in your projects!