14 ms·

Rdb – a Node.js ORM with transactions, persistence ignorance and promises

39 points by lroal 12y ago

mackwic 12y ago
Seems interesting ! I would definitely like more ORM for node.js technologies. The concurrent are Sequelize (http://sequelizejs.com/ http://sequelizejs.com/) and Bookshelf (http://bookshelfjs.org/ http://bookshelfjs.org/), both very mature libraries (look at the Sequelize test suite, it's quite impressing !). Iroal, any comment on the rdb choices against these two big guys ? How can we expect rdb to evolve ?
- aselzer 12y ago
  And if you just want a flexible ORM without models, knex (http://knexjs.org/ http://knexjs.org/), which bookshelf uses. I found knex + Postgres to be a great combination.
  - whatthemick 12y ago
    Likely also better performance with just using Knex. Any ORM likely adds overhead, Sequelize definitely does, although we keep working on reducing that overhead - But inevitably features === overhead.
  - igl 12y ago
    I really really really like Knex. While Mongo claims to be "easy" and "js-native", It's far easier doing complicated queries with Knex on Postgres than on Mongo with the official client. I work on a mongo project and doing aggregate queries is really ugly and painful. Everyone praising mongo probably never made "advanced" queries like that or dismissed SQL as shitty and unsecure without ever using it.
    - mgkimsal 12y ago
      What would be 'unsecure' about SQL? Not sure I ever heard that argument before.
      - cwmma 12y ago
        unparamatarized queries are the usual issue, so not an issue with sql as much as how people misuse sql.
  - wilmoore 12y ago
    I personally wouldn't consider Knex an ORM. I'd describe it more as: A cohesive set of functions that help you build and execute SQL operations.
- whatthemick 12y ago
  Neither Sequelize nor Bookshelf has persistence ignorance as far as i know.
  - mackwic 12y ago
    Ok, after a lookup, "persistence ignorance" is the ability to have implicit save of the models when things change. Bookshelf has events, which make very easy to wire some kind of persistence ignorance (we implement it in our base Model class).
    - whatthemick 12y ago
      I would expect those events to be triggered by action though, then it wouldn't be persistence ignorance.
      - k__ 12y ago
        Sequelize has the build method, which persists automatically. Couldn't the rest be achieved by simply adding a few (update, delete) methods to the model classes?
        bulkan 12y ago
        The create method on model's persist automatically. Build just creates an instance without saving it to the database.
        k__ 12y ago
        Really? I thought it was the other way around...
- lroal 12y ago
  Rdb is used in commercial products at my employer Timpex (www.timpex.no). So it will definitely be maintained. The code was initially closed source in csharp, we refined it and released it as OSS in javascript. It was created because we needed a solution with persistence ignorance and that does not have any constraints on foreign key naming, columns, table and so on. By persistence ignorance I mean not needing to call model.save() or pass the connection around everywhere - just edit the properties and commit the transaction. Everything in rdb is developed TDD outside-in. So it has a lot of unit tests, but not that many integration tests. There are running examples in the demo repo though that could be considered as kind of integration tests. Choices against sequelize and bookshelf: that is not my mission. If you want a closer integration with express.js, those orms are a better fit than rdb. My main focus on rdb was to keep the API simple and expose as little of the interior as possible - Tell Dont Ask principle. How to expect rdb to evolve ? -domain logic -aggregate functions -order by -support sqlLite
- doublerebel 12y ago
  Great question. To me what Node is missing most is a ORM that isn't tied to a backend -- relationships can easily be expressed without needing to know whether data is fetched through an API or through SQL. This is how ORM is often done in frontend JS. We're working on and using such a backend-agnostic Node lib, but I don't think it's polished enough for release yet.
rattray 12y ago
Not sure I see the advantage over waterline (https://www.npmjs.com/package/waterline https://www.npmjs.com/package/waterline) ?
- onestone 12y ago
  Waterline is a joke, it doesn't even have transactions.
joshuakarjala 12y ago
Is anyone working on auto-generated migrations in node.js ala. Django / South?
- distracteddev90 12y ago
  I believe Sequelize does this: https://sequelize.readthedocs.org/en/latest/docs/migrations/ https://sequelize.readthedocs.org/en/latest/docs/migrations/
  - bulkan 12y ago
    No it does not do this yet. See this issue [1]. The db:create task in the sequelize cli only creates a new stub migration file. 1 - https://github.com/sequelize/cli/issues/8 https://github.com/sequelize/cli/issues/8
bonn1 12y ago
ORMs are not really required in a Node/Mongo/JSON setup anymore: - JSON's nature is already very 'object oriented' and I use in code JSON data like it saved—no translation beetween clunky SQL and objects is necessary (if I have a DB which saves JSON natively like Mongo) - Mongo allows to directly save JSON and stuff like migrations is stuff from the past since tables/collections don't have to be created - Mongo's Native Driver is low level and at the same time fully sufficient, the syntax is not beautiful and tons of libs sit on top to make it beautiful but I still don't see the need for a real ORM So, Mongo's Native Driver is your best friend and validations are often a matter of a few lines. Or did I miss anything which is life saving I get from ORMs in a Node/Mongo setup?
- joshuakarjala 12y ago
  Not everyone is only using Mongo with Node
  - bonn1 12y ago
    Good point and I just checked: rdb seems to be for SQL based DBs. But I am wondering who is using SQL based DBs with Node, feels ancient to me, except you build the next datastore for a bank but even then. Also Postgres with its JSON options does not give the feeling, flexibility and speed as Node/Mongo. I could imagine that the larger part of Node users are working with Mongo, or not?
    - lroal 12y ago
      Rdb (Relational Database) is meant only for Relation Databases (sql). I have no plans on supporting document databases - that would probably a bad compromize. And the api would be affected in a negative way.
    - imjoshholloway 12y ago
      Well it depends on what you're building surely? Anything with relational data would fit in a RDBMS more than it would Mongo. E-Commerce is one area that isn't `ancient` and that would need the reliability of a traditional ACID compliant DB like Postgresql.
distracteddev90 12y ago
Any chance Rdb plans to support a traditional callback style interface for those of us that prefer to stay away from promises?
- jermo 12y ago
  I might be missing something but why would you prefer callbacks over promises? I've heard of 'callback hell' but never heard of 'promise hell'.
  - distracteddev90 12y ago
    Both callbacks and promises are fairly simple interfaces for calling a function after some other function has completed. Both interfaces can be abused to give you an ever growing indent and give the appearance of "callback hell" Both interfaces can be use elegantly to help you reason about your code, make it easy to follow, and handle errors centrally. Only one is supported natively by node.js and is the standard async interface for 90% of node.js's libraries: Callbacks. Also, regarding "callback hell", a straw-man argument against callbacks, I highly suggest reading http://callbackhell.com/ http://callbackhell.com/
    - deleted 12y ago
      [deleted]
    - jermo 12y ago
      Oh ok, was just wondering about your reasons and that clears things up. That's a good read as well. But I think promises give you a way to compose them in a way that callbacks don't. With promises you can call easily call a function when multiple promises are fulfilled.
      - distracteddev90 12y ago
        Actually would argue that Promises are less composable since you're forced to use whatever control flow paradigm the Promise library has provided or add another library to handle control flow. By utilizing callbacks, you are free to use Async.js[0] or Step.js[1] to solve the problem you described. These libraries are great since they give you control over parallel vs series execution of the pre-requisite functions as well as solving more complex control-flow problems such as throttling, etc[2] (See link for more examples). [0] https://github.com/caolan/async https://github.com/caolan/async [1] https://github.com/creationix/step https://github.com/creationix/step [2] https://github.com/caolan/async#control-flow https://github.com/caolan/async#control-flow edit: Yes, you can also use similar control-flow libraries with Promises (that follow the specification) to achieve similar results but then the argument for using promises for the sake of control-flow breaks down.
sehrope 12y ago
Does this allow users to execute custom SQL, in addition to ORM methods, and have it be part of the same transaction?
- lroal 12y ago
  It is not supported today. But it would't be any problem to implement it. Please create an issue if you want it.
k__ 12y ago
Nice. I'm currently checking out ORMs for node. What I found was Bookshelf, Sequelize and Jugglingdb. I think Rdb needs SQLite support for development purposes and smaller projects.
- lroal 12y ago
  Agree

39 points by lroal 12y ago

Seems interesting ! I would definitely like more ORM for node.js technologies. The concurrent are Sequelize (http://sequelizejs.com/ http://sequelizejs.com/) and Bookshelf (http://bookshelfjs.org/ http://bookshelfjs.org/), both very mature libraries (look at the Sequelize test suite, it's quite impressing !). Iroal, any comment on the rdb choices against these two big guys ? How can we expect rdb to evolve ?

aselzer 12y ago
And if you just want a flexible ORM without models, knex (http://knexjs.org/ http://knexjs.org/), which bookshelf uses. I found knex + Postgres to be a great combination.
- whatthemick 12y ago
  Likely also better performance with just using Knex. Any ORM likely adds overhead, Sequelize definitely does, although we keep working on reducing that overhead - But inevitably features === overhead.
- igl 12y ago
  I really really really like Knex. While Mongo claims to be "easy" and "js-native", It's far easier doing complicated queries with Knex on Postgres than on Mongo with the official client. I work on a mongo project and doing aggregate queries is really ugly and painful. Everyone praising mongo probably never made "advanced" queries like that or dismissed SQL as shitty and unsecure without ever using it.
  - mgkimsal 12y ago
    What would be 'unsecure' about SQL? Not sure I ever heard that argument before.
    - cwmma 12y ago
      unparamatarized queries are the usual issue, so not an issue with sql as much as how people misuse sql.
- wilmoore 12y ago
  I personally wouldn't consider Knex an ORM. I'd describe it more as: A cohesive set of functions that help you build and execute SQL operations.
whatthemick 12y ago
Neither Sequelize nor Bookshelf has persistence ignorance as far as i know.
- mackwic 12y ago
  Ok, after a lookup, "persistence ignorance" is the ability to have implicit save of the models when things change. Bookshelf has events, which make very easy to wire some kind of persistence ignorance (we implement it in our base Model class).
  - whatthemick 12y ago
    I would expect those events to be triggered by action though, then it wouldn't be persistence ignorance.
    - k__ 12y ago
      Sequelize has the build method, which persists automatically. Couldn't the rest be achieved by simply adding a few (update, delete) methods to the model classes?
      - bulkan 12y ago
        The create method on model's persist automatically. Build just creates an instance without saving it to the database.
        k__ 12y ago
        Really? I thought it was the other way around...
lroal 12y ago
Rdb is used in commercial products at my employer Timpex (www.timpex.no). So it will definitely be maintained. The code was initially closed source in csharp, we refined it and released it as OSS in javascript. It was created because we needed a solution with persistence ignorance and that does not have any constraints on foreign key naming, columns, table and so on. By persistence ignorance I mean not needing to call model.save() or pass the connection around everywhere - just edit the properties and commit the transaction. Everything in rdb is developed TDD outside-in. So it has a lot of unit tests, but not that many integration tests. There are running examples in the demo repo though that could be considered as kind of integration tests. Choices against sequelize and bookshelf: that is not my mission. If you want a closer integration with express.js, those orms are a better fit than rdb. My main focus on rdb was to keep the API simple and expose as little of the interior as possible - Tell Dont Ask principle. How to expect rdb to evolve ? -domain logic -aggregate functions -order by -support sqlLite
doublerebel 12y ago
Great question. To me what Node is missing most is a ORM that isn't tied to a backend -- relationships can easily be expressed without needing to know whether data is fetched through an API or through SQL. This is how ORM is often done in frontend JS. We're working on and using such a backend-agnostic Node lib, but I don't think it's polished enough for release yet.

rattray 12y ago

Not sure I see the advantage over waterline (https://www.npmjs.com/package/waterline https://www.npmjs.com/package/waterline) ?

onestone 12y ago
Waterline is a joke, it doesn't even have transactions.

joshuakarjala 12y ago

Is anyone working on auto-generated migrations in node.js ala. Django / South?

distracteddev90 12y ago
I believe Sequelize does this: https://sequelize.readthedocs.org/en/latest/docs/migrations/ https://sequelize.readthedocs.org/en/latest/docs/migrations/
- bulkan 12y ago
  No it does not do this yet. See this issue [1]. The db:create task in the sequelize cli only creates a new stub migration file. 1 - https://github.com/sequelize/cli/issues/8 https://github.com/sequelize/cli/issues/8

bonn1 12y ago

ORMs are not really required in a Node/Mongo/JSON setup anymore: - JSON's nature is already very 'object oriented' and I use in code JSON data like it saved—no translation beetween clunky SQL and objects is necessary (if I have a DB which saves JSON natively like Mongo) - Mongo allows to directly save JSON and stuff like migrations is stuff from the past since tables/collections don't have to be created - Mongo's Native Driver is low level and at the same time fully sufficient, the syntax is not beautiful and tons of libs sit on top to make it beautiful but I still don't see the need for a real ORM So, Mongo's Native Driver is your best friend and validations are often a matter of a few lines. Or did I miss anything which is life saving I get from ORMs in a Node/Mongo setup?

joshuakarjala 12y ago
Not everyone is only using Mongo with Node
- bonn1 12y ago
  Good point and I just checked: rdb seems to be for SQL based DBs. But I am wondering who is using SQL based DBs with Node, feels ancient to me, except you build the next datastore for a bank but even then. Also Postgres with its JSON options does not give the feeling, flexibility and speed as Node/Mongo. I could imagine that the larger part of Node users are working with Mongo, or not?
  - lroal 12y ago
    Rdb (Relational Database) is meant only for Relation Databases (sql). I have no plans on supporting document databases - that would probably a bad compromize. And the api would be affected in a negative way.
  - imjoshholloway 12y ago
    Well it depends on what you're building surely? Anything with relational data would fit in a RDBMS more than it would Mongo. E-Commerce is one area that isn't `ancient` and that would need the reliability of a traditional ACID compliant DB like Postgresql.

distracteddev90 12y ago

Any chance Rdb plans to support a traditional callback style interface for those of us that prefer to stay away from promises?

jermo 12y ago
I might be missing something but why would you prefer callbacks over promises? I've heard of 'callback hell' but never heard of 'promise hell'.
- distracteddev90 12y ago
  Both callbacks and promises are fairly simple interfaces for calling a function after some other function has completed. Both interfaces can be abused to give you an ever growing indent and give the appearance of "callback hell" Both interfaces can be use elegantly to help you reason about your code, make it easy to follow, and handle errors centrally. Only one is supported natively by node.js and is the standard async interface for 90% of node.js's libraries: Callbacks. Also, regarding "callback hell", a straw-man argument against callbacks, I highly suggest reading http://callbackhell.com/ http://callbackhell.com/
  - deleted 12y ago
    [deleted]
  - jermo 12y ago
    Oh ok, was just wondering about your reasons and that clears things up. That's a good read as well. But I think promises give you a way to compose them in a way that callbacks don't. With promises you can call easily call a function when multiple promises are fulfilled.
    - distracteddev90 12y ago
      Actually would argue that Promises are less composable since you're forced to use whatever control flow paradigm the Promise library has provided or add another library to handle control flow. By utilizing callbacks, you are free to use Async.js[0] or Step.js[1] to solve the problem you described. These libraries are great since they give you control over parallel vs series execution of the pre-requisite functions as well as solving more complex control-flow problems such as throttling, etc[2] (See link for more examples). [0] https://github.com/caolan/async https://github.com/caolan/async [1] https://github.com/creationix/step https://github.com/creationix/step [2] https://github.com/caolan/async#control-flow https://github.com/caolan/async#control-flow edit: Yes, you can also use similar control-flow libraries with Promises (that follow the specification) to achieve similar results but then the argument for using promises for the sake of control-flow breaks down.

sehrope 12y ago

Does this allow users to execute custom SQL, in addition to ORM methods, and have it be part of the same transaction?

lroal 12y ago
It is not supported today. But it would't be any problem to implement it. Please create an issue if you want it.

k__ 12y ago

Nice. I'm currently checking out ORMs for node. What I found was Bookshelf, Sequelize and Jugglingdb. I think Rdb needs SQLite support for development purposes and smaller projects.

lroal 12y ago
Agree