Skip to main content

Command Palette

Search for a command to run...

Node.js ORMs overview and comparison

Updated
25 min read

I believe that ORM is the most important tool for backend development, the tool we spend the most of our time working on a feature, the tool we scold the most, always wanting something better. Here I'm doing a review of 6 popular ORMs to compare and find the best one.

ORM is a library to access the database, a high-level library where you can define models, and relations. Another option is to write pure SQL, and it's a valid option when performance is the biggest concern, but this way is not well suited for dynamic queries.

ORM is needed to make you more productive, to have automatically returned TS types instead of defining them each time by hand, and to make it more safely so you can't accidentally pass a dangerous string into SQL. Ideal ORM allows defining queries and then combining and reusing them to construct more complex queries.

I'll share my experience of working with 6 popular ORMs including knex which is a query builder, I won't retell the docs and it's not a usage guide, just sharing the experience of how it was easy or difficult to use the ORM on a close to reality project. I'll do benchmarks in the end.

Not touching the topic of migrations here, with any ORM it's possible to use its own migration tools or to use standalone libraries for this. In my opinion, this is a must to have up and down migration for every table and every schema change, so if I used Prisma which doesn't have it, I would use a separate library for migrations together with it. Basically, that's what I did when developing API for this article.

To get a real-world feel of the ORM, I have used realworld API spec, and implemented the same API of the blog site again and again with each one.

The most complex endpoint is GET /articles, the response should include:

  • columns of article

  • tag list of articles, via "has and belongs to many relation" between Article and Tag

  • count of favorites, each user can mark an article as a favorite, also via "has and belongs to many"

  • favorited - boolean if current user marked as favorite or not, false if not logged in

  • profile of the author

  • profile of the author must include following field - boolean to know if the current user follows the author, false if not logged in

Also quite tricky was the endpoint for creating an article, user can provide a tag list for the article, and then in the code, I have to check if the such tag exists, if not need to create the tag, and connect the tag to the new article, and do all of this in a single transaction.

All of the code is published on github.

Not included in the overview:

  • bookshelf, waterline, OpenRecord: seems to be obsolete, no TS support

  • deepkit ORM: it hacks typescript compiler, I tried it and wasn't able to manage setup

  • kysely: actually it looks interesting and worth a try, but it lacks good documentation, so it's complex to write complex queries with it

  • Raw SQL: too hard to implement this specific API because queries are constructed on the fly, and with Raw SQL it would be too hard and messy to implement.

Sequelize

tldr; Sequelize is very difficult to work with, coming across issues all the time, TS support is poor, and requires lots of boilerplates. Kinda works. It is actively developing lately, perhaps things will get better in the future.

Code examples:

Setup

Sequelize has a default behavior that may be unwanted, so when configuring the main Sequelize instance need to turn this behavior off:

export const db = new Sequelize(config.dbUrl, {
  logging: false,
  define: {
    freezeTableName: true, // with `false` default Sequelize will transform table names
    timestamps: false, // by default every model will implicitly have `updatedAt` and `createdAt` columns
  },
});

Modeling

I started writing the code for this article in 2020, but now writing models in Sequelize is not so bad. Here I'm leaving the code as it was. In recent versions of Sequelize they try to make modeling similar to how it's done in TypeORM/MikroORM.

Sequelize offers the most bloated way to define a model, first define a TS type:

type UserType = {
  id: number
  // ...
}

Then define the type of create parameters:

type UserCreate = Optional<User, 'id'>;

Then define the model:

export class User
  extends Model<UserType, UserCreate>
  implements UserType {
  id!: number;
  // ...
}

Then define columns with JSON, table name and Sequelize instance:

User.init(
  {
    id: {
      type: DataTypes.INTEGER,
      autoIncrement: true,
      primaryKey: true,
    },
  },
  {
    tableName: 'user',
    sequelize: db,
  }
}

And then define relations:

User.hasMany(UserFollow, {
  as: 'followedBy',
  foreignKey: 'followingId',
});

Each column is defined three times: in the type, in the model and in the column definition, it doesn't have to be this way.

Querying

During developing the API with Sequelize I jumped across open issues on Github quite often, here for an example: Can't include "has-many": issue Draw attention to how old is this open issue. So with Sequelize you will have N+1 problem, in my case, it means that for each article it will perform an additional query to load tags.

To query articles by the tag I tried to use belongsToMany relation:

// In Article model:
Article.belongsToMany(Tag, {
  as: 'tags',
  through: ArticleTag as any, // any because ArticleTag is not accepted without a reason
  foreignKey: 'articleId',
  otherKey: 'tagId',
});

// in ArticleTag model:
ArticleTag.belongsTo(Tag, {
  as: 'tag',
  foreignKey: 'tagId',
});

// querying:
Article.findAndCountAll({
  include: [
    required: true,
    model: Tag,
    as: 'tags',
  ]
})

Which gives the error: column ArticleTag.id does not exist.

Second attempt to join article to tags explicitly in calling site:

// Article model:
Article.hasMany(ArticleTag, {
  as: 'articleTags',
  foreignKey: 'articleId',
});

// ArticleTag model:
ArticleTag.belongsTo(Tag, {
  as: 'tag',
  foreignKey: 'tagId',
});

// In service:
Article.findAndCountAll({
  include: [{
    required: true,
    model: ArticleTag,
    as: 'articleTags',
    include: [
      {
        required: true,
        model: Tag,
        as: 'tag',
      },
    ],
  }]
});

This gives the error: 'missing FROM-clause entry for table "articleTags"' Probably it is related to retrieving count.

So the final working version is to use raw SQL in where:

import { Sequelize, Op } from 'sequelize'

Article.findAll({
  where: {
    [Op.and]: Sequelize.literal(`
      EXISTS (
        SELECT 1
        FROM "articleTag"
        JOIN "tag"
          ON "articleId" = "SequelizeArticle"."id"
         AND "tagId" = "tag"."id"
         AND "tag"."tag" = '${params.tag}')
    `);
    ...otherWhereClauses,
  },
  include: includes
})

Sequelize was not designed to use with TypeScript, so it is not guilty that writing models is very verbose, or that sometimes I had to write as any to make it work.

Inserting

Inserting and updating is alright, it even has .findOrCreate for simplicity.

There is one gotcha to be very careful with when you're working with a transaction:

await ArticleTag.create(
  { articleId: article.id, tagId: tag.id },
  { transaction },
);
  • it's very easy to forget the second argument with transaction, I was developing a large project with Sequelize and was encountering this mistake many times. So it works fine, seems like not a problem at all, but in a specific situation, part of the records will be rolled back with the transaction, and some of them will be saved. Writing tests is a must on serious projects, and testing a real database, only by writing tests I could find missing { transaction } options.

TypeORM

tldr; Problematic, very verbose, most of the time is spent on fighting with the ORM. Defining models is easier than with Sequelize, TypeORM is shipped with its own query builder so it's more flexible than Seqeulize, with fewer hacks.

Code examples:

Setup

Setup is straightforward, define a DataSource with a db config and use it everywhere to perform queries.

Modeling

Works just fine, no gotchas.

Querying

TypeORM offers two ways to fetch data: to use it as ORM similar to Sequelize, or to use its own query builder.

For blog API I've been using its query builder almost all the time because the main interface or TypeORM covers only basic queries.

One of the purposes of the ORM is to define relationships, i.e in my case to define that Article is connected to Tag via middle table ArticleTag by matching articleId and tagId columns. Idea is to define this just once and then use it without repeating it. Unfortunately, no way to use it in TypeORM with sub-queries, I had to explicitly define the relationship right in the query:

// select tagList
const tagListSubquery = query
  .subQuery()
  .select('"tag"."tag"')
  .from(Tag, 'tag')
  .innerJoin(
    ArticleTag,
    'articleTag',
    // in a good ORM I should not have to write this:
    '"articleTag"."tagId" = "tag"."id" AND "articleTag"."articleId" = "article"."id"',
  );

I found a problem with the query builder: it is selecting fields not the way I want, here is a sample output:

{
  article_id: 19,
  article_authorId: 1,
  article_slug: 'slug-19',
  ...other article fields
  author_id: 1,
  author_email: 'email-1@mail.com',
  ...other author fields
  ...etc
},

If you try to join one-to-many relations, such as tags, it will result in repeating article records with different tags:

[
  {
    article_id: 1,
    ...article fields
    tags_id: 1
    tags_name: 'tag 1'
  },
  {
    article_id: 1, // the same id
    ...article fields
    tags_id: 2
    tags_name: 'tag 2'
  }
]

Converting this response to the desired one would be too tedious, so I decided not to use a query builder.

To resolve the problem from above I tried to use the ManyToMany relation instead of OneToMany and it kinda works:

@Entity()
export class Article implements ArticleType {
  // ...columns

  @ManyToMany(() => User)
  @JoinTable({
    name: 'userArticleFavorite',
    joinColumn: {
      name: 'articleId',
      referencedColumnName: 'id',
    },
    inverseJoinColumn: {
      name: 'userId',
      referencedColumnName: 'id',
    },
  })
  favoritedUsers?: User[];
}

But there is one downside of it: each article now has to load ALL favorited users, and I check if the current user liked the article on the javascript side.

Let me say it again: instead of loading a simple boolean field from the database, we need to load a bunch of records from two tables to get that boolean.

But later, when I was trying to filter by tag, - it is impossible to do this without query builder.

So I returned to the query builder approach again.

The final approach to resolving real-world app: I used query builder with selecting related records using json_agg and row_to_json from Postgres. I don't even know how it could possible to do that with MySQL.

Inserting

The model doesn't have an automatic constructor, so I had to assign parameters in a bit awkward and unsafe way:

const repo = dataSource.getRepository(User);
const user = Object.assign(new User(), params);
await repo.save(user);

Unsafe because params here possibly can contain unknown key values and TS won't complain about this.

MIkroorm

tldr; similar to TypeORM, takes longer to set up, again it's very problematic to work with, has its own surprises, and has Knex for query builder which is a win.

Code examples:

Setup

Complex to configure. Docs are suggesting to set both entities and entitiesTs, but where should entities point if we don't have compiled output during development? Does it mean we have to compile the project after every change?

const orm = await MikroORM.init<PostgreSqlDriver>({
  entities: ['./dist/entities/**/*.js'], // path to our JS entities (dist)
  entitiesTs: ['./src/entities/**/*.ts'], // path to our TS entities (source)
  // ...
});

Later I found that it's possible to explicitly provide entities, and it works:

const orm = await MikroORM.init<PostgreSqlDriver>({
  entities: [Author, Book, BookTag],
  // ...
});

Documentation is not very good: embed search doesn't work, searching only via google.

It was a surprise to find that Mikroorm is translating table and column name to underscore case, as I have all tables and columns named in camelCase it was a problem.

So when selecting some camelCased field, I received the error no such field, I wanted to select authorId but the error said "no such column author_id"

To fix that I had to define my own NamingStrategy class. That mostly fixed situation, except for table names, though I defined each method of NamedStrategy, camel-cased table name still was translated to underscore.

The way to solve this is to set static tableName on the entity, which doesn't seem to be documented anywhere at all:

@Entity()
export class UserFollow {
  static tableName = 'userFollow';
  // ...
}

Unline other ORMs, MikroORM requires you to fork EntityManager in the middleware for all requests to use it later. Some people like dependency injection, but I prefer how you can simply import db instance in any other ORM to work with it, less code and no problems.

Modeling

We need to define a constructor in the model, explicitly define arguments and assign them to the entity. I guess it's just the same as in TypeORM, in the case of MikroORM I've added a constructor because docs suggested to do so, and in TypeORM I've used Object.assign(new Model(), params) because didn't see how to do it better in the docs.

@Entity()
export class Article {
  @PrimaryKey() id!: number;
  @Property() authorId!: number;
  @Property() slug!: string;
  // ...

  constructor(
    params: Pick<
      Article,
      // here is a list of fields required for new record:
      'slug' | 'title' | 'description' | 'body' | 'authorId'
    >,
  ) {
    // assign all fields to the entity:
    Object.assign(this, params);
  }
}

Apart from this and from undocumented static tableName modeling is fine.

Querying

Just like TypeORM, for more than basics you have to use the query builder of MikroORM.

MikroORM query builder is a wrapper around Knex, which is a huge plus for me. Here is an example of how to build a sub-query to select article tags, you can see that joining articleTags to tag is done implicitly, which is good, but to join this query to the article it's explicit where condition:

const tagsQuery = em
  .qb(Tag, 'tag')
  .select(`coalesce(json_agg(tag.tag), '[]') AS "tagList"`)
  // must provide alias as a second parameter
  .join('articleTags', 'articleTags')
  .where({ articleTags: { articleId: knex.ref('article.id') } });

Inserting

We need to instantiate the Article object and .persistAndFlush it:

const article = new Article(params);
await em.persistAndFlush(article);

Because of Unit of Work pattern in Mikroorm, saving to db takes two steps: first .persist which is only marking record to be saved, and then .flush to really write changes. .persistAndFlush the method does both for us.

But here I have a problem: The article has a relation to the User:

@Entity()
export class Article {
  // ...
  @ManyToOne({ entity: () => User, inversedBy: (user) => user.articles })
  author!: User;
}

And when trying to save an article with just .authorId it fails with the error:

Value for Article.author is required, 'undefined' found

Ok, so instead of passing .authorId we can load the whole user record and provide it to the constructor, but in such case, it gives an error that authorId is required.

Ok, then we can provide both .authorId and .author user record. Error again, this time it says that the article has no column author, and this time it is an SQL error.

I give up at this point, just use .nativeInsert method to avoid Unit of Work patter and related issues.

Finally, creating new article code looks like this:

const article = new Article({ ...params, authorId: currentUser.id });
article.id = await em.nativeInsert(article);

When using .nativeInsert we also need to assign returned id to article.

Bulk insert is broken, this code:

const articleTagsToInsert = tagIds.map((tagId) => {
  return new ArticleTag({ tagId, articleId: article.id });
});
await em.nativeInsert(articleTagsToInsert);

Throws an error the query is empty, it works when I change this to saving records one by one:

await Promise.all(
  tagIds.map(async (tagId) => {
    const articleTag = new ArticleTag({ tagId, articleId: article.id });
    await em.nativeInsert(articleTag);
  }),
);

Prisma

tldr; Prisma is most pleasant to work with, best query interface, and TS support. The grain of salt is it has a limited query interface, for specific advanced queries you'll have to fall back to writing complete queries in raw SQL. One year ago this "Next generation ORM" could not even handle transactions, so I am very suspicious of Prisma. But now the transactions feature is in preview and overall my experience with Prisma was highly positive. Try it and most likely you'll enjoy it, especially on smaller projects, though I would not be confident to use it in a large long-running projects yet.

Code examples:

An interesting fact about Prisma, it launches a separate server under the hood written in Rust. The main node.js program is sending queries encoded with GraphQL to the Rust server, Rust server calls db and sends results back to the node.js server.

Setup, modeling

The setup is easy. Defining schema in custom DSL language, and having it in a single file is very pleasant. So you can open a single file and easily see all table definitions, and all relations.

Some features of Prisma are currently available "in preview", which means need to specify a feature name in client section of the schema:

generator client {
  provider = "prisma-client-js"
  previewFeatures = ["interactiveTransactions"]
}

After changing the schema run prisma generate in console.

I've found one gotcha with time stamps when I define fields in this way:

model article {
  ...skip
  createdAt           DateTime              @default(now()) @db.Timestamp(6)
  updatedAt           DateTime              @default(now()) @db.Timestamp(6)
}

Prisma will serialize dates in the current time zone, and this is unexpected because all other ORMs are saving date in UTC time zone.

To fix this behavior need to set the time zone to UTC in the code:

process.env.TZ = 'UTC';

Querying

I was really excited about how good this part is, really a great job on the query interface! Despite I am skeptical about Prisma, I was able to code all my quite complex queries without a need to fallback to passing raw sql.

So my feeling about this is dual.

On the one hand, Prisma offers the simplest way to perform queries, with the best TS support, and I must admit it's quite powerful, selecting related records is well worked out and quite easy to do.

On another hand, there is a limit somewhere. I didn't reach it in this blog API, Prisma offered what I needed. But the limit exists, in my past projects I used specific database features like Postgres full-text search and trigram search, Postgis, and window functions. I don't believe it's possible to cover all database features in a single interface of the ORM, at some point you will want to add just one small piece of raw SQL to a where condition, but Prisma doesn't support this.

Imagine you have a big complex query with lots of conditions, selecting related records, and it's all fine with the Prisma interface. But then someday you need to add just one small condition where custom SQL is needed, Postgis condition, for example, then you have to rewrite all the queries to a raw SQL.

Inserting, updating, deleting

The interface for data modification is very good, it allows the creation or deletes related record right in the single update, you can see examples in the nested writes section of Prisma docs.

In 2021 Prisma received a long waited "interactive transaction" feature (currently in preview), I tested it on various endpoints and it works just fine.

Objection

tldr; It's an ORM built upon Knex to allow defining models and relations between them. Not as good with TypeScript as Prisma, so I had to typecast results to the needed type. But otherwise, I enjoyed using it, easy to use and allows me to build any query. Definitely better than Sequelize, TypeORM, and MikroORM. But unfortunately, it become deprecated and won't be actively maintained in the future, author has switched to developing Kysely.

Code examples:

Setup

Setup of Objection is easy, simply configure Knex and provide a Knex instance to the objection Model.

Modeling

Defining model class is easy. Defining relation looks like this:

  static relationMappings = {
    author: {
      relation: Model.BelongsToOneRelation,
      modelClass: User,
      join: {
        from: 'article.authorId',
        to: 'user.id',
      },
    },
  }

Instead of decorators in TypeORM or MikroORM, not a big deal but for me, syntax without decorators feel cleaner.

The objection has a special feature modifiers, here in the Article model I have:

  static modifiers = {
    selectTagList(query: QueryBuilder<Article>) {
      query.select(
        Tag.query()
          .select(db.raw("coalesce(json_agg(tag), '[]')"))
          .join('articleTag', 'articleTag.tagId', 'tag.id')
          .where('articleTag.articleId', db.raw('article.id'))
          .as('tagList'),
      );
    },
}

And later I can use it in such ways:

const articlesWithTags = await Article.query()
  .select('article.*')
  .modify('selectTagList')

So in such a way, we can keep repeated parts right in the model and use them later when building queries. Unfortunately, modify doesn't have TS check and auto-completion, Objection was designed for plain JS and not so good at TS.

Querying

Querying with Objection feels very similar to querying with Knex, and I like it a lot, easy and straightforward. The big bonus of Objection is the ability to define relations in the model and then use it when querying with different methods, for example, query.joinRelated('tag') to join tags.

The downside of Objection is its bad support of TypeScript at the part of the query result type, for example:

const count = await Article.query().count()

Here variable count has a type Article[], because Objection simply uses model type as the result. And we need type cast it to what we want explicitly:

const count = (await Article.query().count()) as unknown as { count: string }
// yes, count is actually an object with string field for count

Inserting, updating, deleting

Just as with querying, I found mutating with Objection very simple to use and powerful.

Knex

tldr; if you know SQL (it's a must if you dealing with the backend) - you know Knex, which means you don't have to read through the docs and learn how it works, just start using it! Simple, maximum support for database features. Downsides: usually result type is any so you have to cast it, which means need to be careful so the query matches the type. This is a query builder, it doesn't contain models or relations. If you'd like to use Knex on a bigger project, need to organize queries in some reusable way so you won't have to write the same relation joins again and again - that would be error-prone.

Code examples:

Setup, modeling

The setup is straightforward. We don't have models at all in Knex so no need to define them.

To define types of schema, you can write special declare module block to tell Knex what tables and columns you have:

declare module 'knex/types/tables' {
  interface Tables {
    user: {
      id: number
      username: string
      // ...
    };
    // ...
  }
}

And then use it:

const user = await db('user').select('username').first();

Here username will be autocompleted because TS knows fields of the user, but still, you can provide any string to the select so it won't catch typos of fields.

The result in this example will have a proper type Pick<User, 'username'> | undefined, and if you provide some custom SQL to the select the result will be of type any.

Anyway, friendship with TS is not a good part of Knex. You can provide a type in a generic:

const record1 = await db<{ foo: string }>('table')
const record2 = await db<{ foo: string }>('table').select('foo')
const record3 = await db<{ foo: string }>('table').select('custom sql')

Record1 and 2 have types that we provided, but record3 has type any[], perhaps it won't be a problem in many situations, but I relied on custom SQL in select a lot, for instance, to select tagList of article.

So when working with Knex we should expect that usually return type is any[] or any and cast it explicitly to the needed type.

Querying, inserting, updating, deleting

The only downside of Knex is it doesn't have a way to define relations as ORMs do, otherwise, it's just wonderful, interface is very close to SQL, and gazillions of methods.

Knex is translating SQL into JavaScript DSL, and as a bonus, it's providing handy methods to save us some time.

As the result, when working with Knex you think "how would I do this in SQL" and intuitively translate this to Knex, while in the case of other ORMs you have to figure out how this specific ORM is doing this thing, read conversations in github issues, and quite often you may find that ORM lacks the support of the specific feature.

OrchidORM

tl;dr this is the best ORM available for node.js with perfect TypeScript support. It comprises all the best parts of other ORMs and query builders, plus adds additional features. I highly suggest checking on it for yourself, because as I'm the author of it I may be a little biased :)

Code examples:

Setup

To set it up, we need two files: in one file we configure and export db the instance, and in the second file we export a BaseTable - a class to extend other tables from. Having this BaseTable the class allows us to customize columns: we can configure it to return all timestamps as strings, as numbers, or as Date objects, and do similar overrides for all the columns if needed.

Modeling

Columns are defined similarly to Zod schemas (or like Yup, Joi, etc). This gives a short syntax that allows for various customizations, setting validations right in place, and automatically inferred TS types. And no need to run code generation as in Prisma.

Relations are defined nearby.

export class MyTable extends BaseTable {
  table = 'tableName'
  columns = this.setColumns((t) => ({
    id: t.serial().primaryKey(),
    name: t.text(3, 100), // min 3, max 100
    ...t.timestamps(), // adds createdAt and updatedAt
  }))

  relations = {
    items: this.hasMany(() => ItemTable, {
      primaryKey: 'id',
      foreignKey: 'itemId,
    })
  }
}

Querying

Orchid ORM offers an interesting mix of a query builder that enables writing complex custom queries, and a higher-level interface partly inspired by Prisma, that makes working with relations very easy. And everything is type-safe, without manual annotations.

Example: here we load articles, and we can see a list of requested article columns followed by a special nested select. In the nested select, we are free to name properties as needed (not like in Prisma where you have to map the result afterward). It has various methods for how to return the data: pluck in the example will return a flat array of values. It has a feature called repositories, so the query can offload complexity to another more specific query, in the example we have userRepo with custom method defaultSelect. It has exists query to load only a boolean from db, not like in Sequelize we have to load a full record to see if it exists. It's possible to have conditions, here if the currentUser variable is set (it is an authorized request) this will construct an inner query to know if the article is favorited by the user, and if not set it will load a plain false value by using raw. In Orchid ORM it's possible to have just a little piece of the query be a raw SQL, unlike Prisma where you need to rewrite a full query for this.

Orchid ORM converts even such a complex query into just a single SQL command, which makes it the most performant of node.js ORMs.

const articles = await db.article.select(
  'id',
  'slug',
  'title',
  'description',
  'body',
  'createdAt',
  'updatedAt',
  'favoritesCount',
  {
    tagList: (q) => q.tags.order('tag').pluck('tag'),
    author: (q) => userRepo(q.author).defaultSelect(currentUser),
      favorited: currentUser
        ? (q) =>
          q.userArticleFavorites
            .where({ userId: currentUser.id })
            .exists()
        : db.article.raw((t) => t.boolean(), 'false'),
  },
);

Inserting
By default, create returns a full record, but it can be customized to return only an id or a specific set of columns. Orchid ORM has various nested methods to insert related records together in one place. This side of Orchid ORM is inspired by Prisma, but it's more flexible: allows to have only some of the inserting properties to be a raw SQL.

const id = await db.article.get('id').create({
  ...params,
  authorId: currentUser.id,
  articleTags: {
    create: tagList.map((tag) => ({
      tag: {
        connectOrCreate: {
          where: { tag },
          create: { tag },
        },
      },
    })),
  },
});

Benchmarks

I measured how long it takes for each ORM to serve the articles endpoint (as in API spec of "real world app"), and each request returns 100 articles with tags, author, and other nested fields. Performed 300 such requests per ORM to compare total time:

(Sequelize is not included in this chart because it's too slow for comparison)

Slow to fast:

Sequelize: 8.62s

TypeORM: 0.85s

Knex: 1.03s

Prisma: 1.92s

Objection: 0.95s

Mikroorm: 0.96s

OrchidORM: 0.86

Sequelize is the slowest for two reasons: it's indeed the slowest and you can find the same conclusion in other benchmarks over the internet, and also due to its limited interface I could not figure out how to avoid the "N+1" problem, so it performs a separate query for each article to get tags. For other ORMs, there is no N+1.

Prisma is the second slowest. Under the hood, it performs a separate request for each relation, so when we load articles with tags and author it loads articles, then tags, then author - 3 queries. In other ORMs, I was able to load the whole result with just one query. The second reason is overhead, under the hood Prisma is communicating with the database through its own server written in Rust, this server is a middleman between node.js and the database, and other ORMs don't have a such middleman.

For other ORMs, I was surprised to see there is no big difference, almost the same time.


The next thing to know is how fast they can insert. Measured creation of 1000 articles with 5 tags each:

Sequelize: 6.65s

TypeORM: 5.2s

Knex: 4.43s

Prisma: 6.77s

Objection: 5.58s

MikroORM: 7.13s

OrchidORM: 4.48s

Sequelize, Prisma, and MikroORM are a bit slower, but in my opinion, the difference here is negligible.

Conclusion

Which ORM to take for your next project? I've spent a lot of time here finding out the answer. But there is no answer. Each ORM in node.js is another way of pain.

Ironically, the most popular Sequelize was the biggest struggle to use and the slowest. If you want to choose it for the next project - choose anything else. Anything.

TypeORM and MikroORM - with these two I constantly had issues, and experienced unexpected behavior. Can't recommend them.

Bad TypeScript support of Knex and Objection can be compensated with writing tests, DX is suffering of course, but they are bearable. I would prefer Objection since it has models and relations.

Prisma has really nice TS support and makes it easy to query relations, has sweet nested creates, and updates. My experience with this API was highly positive. I was skeptical about it before, it gained the support of long-running transactions only in 2021 which is ridiculous. But now it became a mainstream choice and is slowly but steadily gaining the support of requested features. If the performance of queries is not necessary, and the ability to customize queries is not required - Prisma is a good tool for the job. As a compromise, some people use Prisma as the main ORM and a standalone query builder (or raw SQL) for specific queries.

One popular response I hear from the community: it's normal that ORM covers basic CRUD, and when you go beyond that, just use raw SQL or a query builder. And I really want the community to change minds about this, this doesn't have to be so. I was a ruby developer and developed complex apps with complex queries, and there wasn't a single time when I was restricted by the ORM. Because proper ORM is designed so it's easy to do basic things, it has thousands of methods for different cases to make your life easier, it's built upon query builder which allows writing any possible query, and it allows you to put a piece of SQL when needed, and it's designed to be easily extended. If in ruby they had such ORM for decades, I don't understand why it is not possible to do in node.js with proper TS support. And I believe it's only a question of time, if not now, maybe in a year, or two.

Orchid ORM update: now I'm developing such ORM, and it shows that type safety, flexibility and simplicity of usage are achievable together!

A
Ahmed Pat2y ago

You should add Kysely to the list. Without the benchmarks part would be fine.

R
Roman2y ago

Kysely is great and I was happy to use it in my last work project.

In theory, implementing "real world app" with Kysely might have some quirks (or maybe not), but implementing this app one more time is time-consuming so not gonna do that.

Though it would be interesting to see if it's much easier to use than Knex, as they are similar, but take different approaches for query builder design.

K

I usually don't comment on stuff, but well done on this really good article. Thanks for the sweet summary!

1
S

Roman very nice writeup!

Understood if you're tired of learning yet-another-ORM at this point :-), but if you want to check out Joist (joist-orm.io), it's an ORM we've written and would appreciate your input/thoughts on it.

To your chagrin, we'd also don't have a robust query builder yet :-) (we defer complex query use cases to Knex for now), but we do consider ActiveRecord to be a big source of inspiration (a lot of our domain objects, validation rules, and lifecycle hooks are inspired by ActiveRecord).

I think in a few cases we've surpassed ActiveRecord in ergonomics (specifically since we build on dataloader, we get N+1 avoidance basically for free; and our reactive rules & derived fields is pretty unique I think), but overall it's still a high bar to meet!

R
Roman3y ago

Thanks for reading!

These are my thoughts on Joist after briefly reading through the docs, sorry that not examining it more deeply:

continual, verbatim mapping of the database schema to your object model

Am I getting it right, first user need to create table in any way they want, and then they run your codegen and this lib generates model files?

Some other ORMs are doing the opposite: you're writing models and then they generate migrations.

I simply can't understand why - why not to follow simple and reliable RoR approach, I guess the same as in other languages? Write migration, write model, there are scaffolds for lazy people.

"reactive" validation rules - that's interesting, I remember was using it all the time in Rails, if specific column or columns was changed it will run specific validation.

I suggest to make change docs/readme so it's more obvious for the new comer what is special about Joist. "Schema-driven code generation" - new comer doesn't know what is it, you could write it like "Automatically generates code for models out of your database". As a new comer, I don't really see how it's better than others, I see it's similar to MikroORM and has additional features, something graphql specific, but how is it better for everyday use?

"Guaranteed N+1 safe" well, in the page of docs "Avoiding N+1" the first example you show is NOT N+1 safe. And to see how to do it right reader has to scroll to the very bottom of this page.

Fast tests: let me brag about my approach, I patched node-pg so in each tests gets wrapped in a transaction, and it simply rolled back after each test, no need to clean db at all. And this approach worked for 3 of 6 ORM's in this article. Some time later I'm going to write about this and to publish a lib.

Syntax is similar to MikroORM (which I don't like):

const em = newEntityManager();
const a = await em.load(Author, "a:1");

If you're saying ActiveRecord is an inspiration, why not:

const a = Author.find("a:1")
// or
const a = Author.findBy({ id: 'a:1' })
// or
const a = Author.where({ id: 'a:1' }).take()
book.reviews.get;

Looks a bit weird, conventionally we use verbs for functions, here we have property get.

Would be better to do it without property, I believe this should be possible:

book.reviews
S

Thanks for the response Roman!

I like your suggestion about making the docs more directly call out Joist's strengths... I'll work on that.

create table first ... codegen generates model files

Yep! Just like RoR auto-adds first_name getters/setters, Joist does the same for firstName/etc. columns (as well as has many/has ones); the difference is that Joist adds it to a base class, so that TypeScript can see the fields & do type-checking.

But net-net is that you get the same "just an empty model" "class Author {}" when starting out (...okay with an "extends AuthorCodegen" snuck in there :-)) as you do with Rails. I just really loved that low-boilerplate aspect of Rails.

reactive ... in Rails

Yep, it's similar, although AFAIU Rails the rules are only reactive on fields within the current model; in Joist, you can have rules that run when fields on related entities change, i.e. re-run an Author rule whenever it gets a new Book, or one of its Books' titles changes.

First example is not N+1 safe

Ah yeah, I think I focused too much on explaining what N+1 was for new comers. That's fair, that page could do a tldr first.

Running tests in transactions ... then roll them back

Ah yeah! That is also what Rails does, afaiu (by default? I think it's configurable iirc), but that means you can't test any code that commits transactions (or maybe you could use subtransactions?). Which for us is GraphQL mutation resolvers. Afaict the flush database approach is ballpark-same-speed and you don't have that same limitation. Another upshot of Joist's approach is that if your test fails, any test data you've setup will still be in your local db to look at/diagnosis if you need to. But I think both are good; as long as the tests are fast! :-)

book.reviews

Yep! That makes sense, but the wrinkle is: is book.reviews a BookReview[] or Promise<BookReview[]>?

Other ORMs like TypeORM afaiu make you choose one or the other up-front, when writing the model. If you choose Promise<BookReview[]>, it is the safest, but annoying to always .load() it to access. If you choose BookReview[], it is really easy to access, but then you risk accidentally calling it w/o it being loaded yet and getting a runtime error.

I agree Joist's approach looks odd, but it's both: a) really safe; if book.reviews has not be explicitly preloaded with a populate hint, you must call reviews.load(); but it's also b) really ergonomic b/c if you have preloaded book.reviews, then some TypeScript magic automatically makes .get visible to you, and you get really pleasant synchronous access.

In a way, this is similar to Prisma, being able to use "kinda like GQL" snippets of a load hint ({ book: { author: { publisher } }) to preload a bunch of data in 1 await call, but with Joist the hints are based on the combination of your db schema + your custom relations & fields (kinda similar to ActiveRecord scopes) instead of strictly being database tables like they are in Prisma.

Author.find / instead of em.find

Yeah, honestly 80% of this is that we started with MikroORM and its EntityManager API, before writing Joist & migrating our codebase to it. But the other 20% is that, unlike Rails (but the same as Mikro and older ORMs like Hibernate), Joist uses a unit of work pattern to delay/batch SQL statements until a "flush" method is called. So we need some sort of "EntityManager" / "Unit Of Work" "thing" to pass around, so :shrug: I think it's net/net not a big difference. You're right that it is not strictly the Active Record pattern, but I think Joist follows the spirit of the Active Record pattern, which is that models extremely closely/strictly match the database tables, vs. the Data Mapper pattern, which is where the models can drift from the schema in ways that seem nice at first but quickly get complex imo.

1
R
Roman3y ago

Stephen Haberman I'm working on own ORM too and just want to share my ideas/vision on those points:

But net-net is that you get the same "just an empty model" "class Author {}" when starting out

Migration is responsible for changing db schema, not only for creating tables, but also changing, renaming, sometimes we do data updates in migration.

And in the model we define current columns, it doesn't take much time, developer can easily jump to User and see the fields. But I decided to go further, let it be possible to define a validation schema right in the model! If you know Zod library, I mean to define columns in similar way like they do.

Something like:

const User = model({
  schema: (t) => ({
    id: t.serial().primaryKey(),
    email: t.text().email(),
  })
})

And now it's not only db types, but also validation types.

Later in the controller we can reuse this schema to validate payload:

const createUserSchema = User.schema.pick({
  email: true,
  name: true,
  // ...
})

router.post('/user', (req, res) => {
  const data = createUserSchema.parse(req.body)
  data // TS knows exactly what the type is
})

but that means you can't test any code that commits transactions (or maybe you could use subtransactions?)

Yes :)

Just published package for this: https://www.npmjs.com/package/pg-transactional-tests

There is explanation how it handles nested transactions, previously it was a script I copied from project to project and it helped me a lot even on a production project.

Yep! That makes sense, but the wrinkle is: is book.reviews a BookReview[] or Promise<BookReview[]>?

I'm going to implement nested loads with json_agg, i.e postgres can aggregate data to json arrays by it's own.

As for the interface, .include method should be fine:

const result = await Post.include('comments')
result // is an array of posts

const post = result[0]
post.comments // array of comments

Also it should accept sub queries, so we can load filtered comments:

const result = await Post.include({
  comments: Comment.where({ ...conditions }).order({ createdAt: 'DESC' })
})

By the way, is it possible in Joist to load Post with their Comments ordered by comment field or apply some condition?

Regarding whether book.reviews is a BookReview[] or Promise<BookReview[]> - well, I suspect it's tricky to customize result of em.load, but in theory it's possible to return BookReview[] type only if this resource was preloaded, and Promise<BookReview[]> otherwise.

Regarding Unit of Work, I simply don't understand it's benefits, I like the concept of simple structures with fewer magic, when you can see clearly in the code what's going on. Maybe you could suggest an example where it shines?

const user = await User.find(1)
const data: Partial<UserType> = {}

// ...some logic to change user
if (someParam) {
  data.foo = 1
  data.bar = '...'
}

// and in the end:
await User.update(user, data)

Here is quite typical update flow, without Unit of Work so no need for creating some session, track any states, etc. Even without "ActiveRecord" pattern, User.update is a static method, while user is a plain lightweight JS object.

So in my opinion, Unit of Work adds a bit of boilerplate, overhead, complicates things, or am I wrong?

I don't mean to criticize Joist, just have different preferences and beliefs. So maybe node.js has no decent ORM to use with pleasure (yet?), but we have so huge variety of libs which are mixing completely different ideas, and continuing building new ones, there is something good in it (I guess).

S

Late reply, but Roman :

it doesn't take much time

We have ~250 tables at this point, and granted we could/would have incrementally typed out each entity by hand, but it's also nice to run codegen and know that the output is 100% what the schema is.

developer can easily jump to User and see the fields

Yeah, I do like that, we have the same albeit capability in the our UserCodegen files; which I think is basically the same thing.

suspect it's tricky to customize result of em.load, but in theory it's possible to return BookReview[] type only if this resource was preloaded, and Promise<BookReview[]> otherwise.

Yes, this type-system trickiness is exactly what Joist implements (if you go find our Loaded type), but the "trick" is "only sometimes adding .get".

If we changed book.reviews to sometimes-array/sometimes-promise-array, then you can't have helper methods like doSomethingWithBook(b1) that accept both books w/previews loaded & books w/o previews loaded, because the reviews field types are too different.

And that pretty quickly gets annoying (we tried something similar to it). So, yeah it "looks weird", but I assert the combination of reviews.load+reviews.get is net/net the most ergonomic solution.

Unit of Work adds a bit of boilerplate

I tried to lay out the benefits in the docs (just updated):

https://joist-orm.io/docs/features/unit-of-work

email: t.text().email(),

I don't mind that per se; in Joist it would be:

authorConfig.addRule(mustBeEmail("email"))

Which isn't a DSL-pleasant for what you've built-in (like .email()), but I think generalizes to complex/user-driven rules better. And also supports the new field-level validation rules really well (...docs wip...).

[possible with] Joist to load Post with their Comments ordered by comment field or apply some condition

Not via the post1.comments.load(...) method, no; because we cache the loaded-ness of collections + also automatically keep both-side up to date (if you do c1.post.set(p1), then p1.comments.get will automatically has c1 in it), we need them to be the "vanilla" / whole collection.

That said, you could use a dedicated query like:

await em.find(Comment, { post: p1, ...other conditions... }, { orderBy: { title: "ASC" } });

So, it's still doable, just not "via the object graph" (i.e. post.comments.load/get) since we're specifically asking for something that is not "100% the object graph" but instead of custom view of it.

...granted, we could probably add something like post1.comments.load({ ...condition..., ...order... }) that was just syntax sugar for the em.find version, but we don't have that yet.

Filed: https://github.com/stephenh/joist-ts/issues/369

S

Also Roman, you had a great call out here:

in the page of docs "Avoiding N+1" the first example you show is NOT N+1 safe

The code snippet on that page was right, but you're right the copy was pretty confusing; that example actually is N+1 safe in Joist.

I updated the docs today to hopefully be a little more clear:

https://joist-orm.io/docs/goals/avoiding-n-plus-1s

Would love to hear if that's better/not better, and any other thoughts you had.

R
Roman3y ago

Hi Stephen Haberman! I see you are continuing to develop Joist ORM (the last commit is yesterday), great to see such insistence.

I developed my ORM too. And updated this article to include a brief overview. Surely it's completely different from yours, I hope you could find some time to check it out, it may have some interesting ideas. I'd be happy to get feedback from the developer of another ORM (even harsh)!

S

Hey Roman! I took a few looks at Orchid. I like the docs! In terms of feedback, I think my biggest take away is what you also highlight in your docs: personally I wouldn't really call Orchid an ORM, and instead would call it a query builder, ala Knex and Kysely. Granted, "query builder" doesn't really roll off the tongue like "ORM" :-). "Query DSL"? Hm, not sure. Terminology pedanticism aside :-), congrats on shipping!

R
Roman3y ago

Stephen Haberman thank you for checking it out!

I understand your point, so I added an explanation on the very first page of the docs that is not a "traditional" ORM. But it is an ORM like Prisma.

None of the query builders is helping with relations, literally none of them, so if I call it a query builder it's like ignoring the most fun part of it: working with relations. Here is a code example in benchmarks section.

Calling it a DSL or a database SDK would tell nothing to potential users. Everyone knows Prisma, and I guess if they can call it ORM, then mine can be called too.

Anyway, there are no strict clear bounds of what ORM is. Different people have different expectations.

So for me, query builder is abstracting only tables, while ORM is also abstracting relations. And how exactly it is implemented - instantiating class or not - is just an implementation detail.

S

Roman hey!

But it is an ORM like Prisma.

Ah yeah, that's a good point; I suppose I have the same qualm with Prisma as well :-D

query builders is helping with relations

Ah interesting, yeah that is a good point. Thanks for linking to the nested load example. That is a nice way to handle the boilerplate of joins. Looks nice!

With Joist, I've shied away from doing any query building that is not "return me exactly this entity", with the rationale that we'd probably be best off using custom SQL/knex queries for that anyway, but yeah I can see how those "custom"/"not-an-entity" result queries are much nicer in Orchid than doing joins in a query builder. Nice job!

Bookmarking for later. :-)

1
N

Nice in depth article. I've dabbled in Prisma for the past few months. The community is getting pretty big and it is definitely handy at a startup level. There are still a few things they still need to work on like Optimistic Concurrency Control feature updateMany still has a bug (hopefully will be fixed soon), no nested createMany (doesn't make a big deal breaker). With PostgreSQL they released GIN, GiST, SP-GiST, and BRIN indexes support on 3.14.0 my interest is in BRIN for time scheduling. I was trying to avoid decorators so that's why MikoORM was out otherwise it would have been my second choice.

Curious to know what review for code first graphql SDL like Nexus/Nexus-Prisma vs Pothos you would prefer next? IMO Pothos has some really good documentation and the code is readable but the dot notation with Nexus/Nexus-Prisma is smooth (once you can fully master it i think...)

1
R
Roman3y ago

Writing in depth article is so time consuming, because need to take a library, to read it's docs and learn how to use it, then to write bunch of code to see in action. Most of the time spent on fighting with bugs, limitations, digging issues. And it was very pleasant to get a lot of positive comments for my first large post (on reddit many people said it's good written, bookmarked).

I really like the topic of ORMs because that is a tool I work with primarily for many years, and I'm developing own, so I wanted to see what the market has and to get inspiration.

GraphQL solutions overview is a bright idea for an article, but I don't work with GraphQL usually, so not motivated for this.

I enjoyed working with "fastify mercurius" for GraphQL, but it's schema first. After changing schema it generates all the types, and only need to implement resolvers and data loaders.

1
N

Roman it's definitely no easy task writing up multiple ORMs and takes enormous amount of time. I know most people go the REST route but GraphQL it definitely nice to scale. I know for me working in graphql in a work project makes a lot of sense once you understand it because it applicable later on and no major changes are needed. Looking forward to your next article keep it up.

1
R
Roman3y ago

GraphQL is awesome, it's a better approach in almost every way.

The flow: define a GraphQL schema, if it's schema first generate TS types with one command, if it's code first generate schema, implement resolvers and data-loaders and let the types to guide you. I work with React usually, so next step is to generate types and ready to use hooks for the frontend. And you have API docs and playground for free. I love this flow, it saves a lot of time, less code to support, API is more unified.

There is a one question I'm not sure how to solve in GraphQL, what if evil client will load too much of nested resources making server to feel sick, but there must be a way to solve it.

For some reason, GraphQL is still not too popular, perhaps people think it's more complex, or they know some flaws, so in my experience where I worked it was always a REST API, so I'm not very familiar with GQL and only learned it by using on a pet project. If by any chance you'd like to write an article about Nexus vs Pothos, and for me vs Mercurius, there is also typegraphql, apollo server, I'd love to read it!

A

Wow-what a thorough article good one!

1
C

Excellent write-up!

1
D

Very informative Roman thanks!

4