5 ms·

The presentation was recorded by InfoQ. I hope they will make it available during in the next 1-2 months. There's a lot I said but did not put on the slides. Y

by jrirei 15y ago

The presentation was recorded by InfoQ. I hope they will make it available during in the next 1-2 months. There's a lot I said but did not put on the slides. You are correct in your assumption that each Tile had been a record in a MySQL table (now it is a value i a Redis hash). Actually we considered using a "blob" approach. But in the client we cannot batch requests as we cannot foresee when the user will simply kill the Flash client to go to some other site. So when a user request (i.e. a game event) arrives in the server there is no way to know if another request will follow. So we have to persist that change right away. This is using a stateless server. In a later game called Magic Land we are going for a stateful Erlang server. There we keep the whole user state in RAM while the user plays and persist state changes every minute or so. Here we can do without any database and just use S3 for persistence. Works just great. On the upcoming Erlang User Conference we will give an update on that project and slides will be available at Slideshare next week, too. In the meantime please have a look at this old slide set to explain the concept in detail: http://www.slideshare.net/wooga/erlang-the-big-switch-in-social-games http://www.slideshare.net/wooga/erlang-the-big-switch-in-soc...

Udo 15y ago
Thanks for clearing this up. > Actually we considered using a "blob" approach. But in the client we cannot batch requests as we cannot foresee when the user will simply kill the Flash client to go to some other site. So when a user request (i.e. a game event) arrives in the server there is no way to know if another request will follow. So we have to persist that change right away. < That's an understandable dilemma. I love thinking about stuff like this and see how other people are dealing with these challenges, so please forgive my Sunday morning quarterbacking ;-) Wouldn't the issue have been solvable by creating an relatively simple persistent software layer between the app code manipulating the tiles and the backend storage? I understand that you moved to this model with your Erlang game, but I'd like to know if a persistence/caching layer was considered for the farming game? More generally, somewhere in here is an idea for a great Node.js server project that takes coarse grained datasets from a contentious database and serves as an interface for finer grained portions of that data.
- jrirei 15y ago
  For Monster World we did not consider this as the game is running fine (or good enough) as it is now. During the last weeks we were more focused on reducing RAM consumption of our databases as that is currently the main driver of cost and operation overhead. Regarding your idea: Wouldn't then the Node.js server have to keep the whole user state in memory?
  - Udo 15y ago
    > During the last weeks we were more focused on reducing RAM consumption of our databases as that is currently the main driver of cost and operation overhead. < I can imagine that. > Regarding your idea: Wouldn't then the Node.js server have to keep the whole user state in memory? < Yes, but I think it would have several advantages: (1) The Node.js server code could decide which working sets it keeps in memory based on very simple rules. The details of this would be abstracted away from the application code itself, because the app just issues read and write requests on a user's dataset. So in essence, by splitting up the problem in two, it becomes relatively easy to handle (and optimize) on each end. (2) You just have to keep the active datasets wired in RAM and it wouldn't be necessary for the Node server to know whether a user has disconnected recently or not. All it knows is when the data was last accessed and it can then vacate RAM slots that have become stale. Compare this to Redis, which I believe just keeps everything in memory no matter what. So overall RAM usage would probably be considerably less than what you're doing now. (3) The idea beats "dumb blob caching" such as memcache, because it makes small operations economical. It seems to me that Node is well suited for this kind of task since it makes it very easy to build small server scripts that handle a huge number of small transactions. This would probably mean you need less machines for the same amount of users. (4) I believe it's relatively easy to implement replication and scaling. Anyway, just an idea. I have no clue whether this works in practice ;-)
    - jrirei 15y ago
      Yes, this might work. But I would be careful about replication and scaling - this could make things somewhat complicated. ;-)

by jrirei 15y ago

Udo 15y ago

Thanks for clearing this up. > Actually we considered using a "blob" approach. But in the client we cannot batch requests as we cannot foresee when the user will simply kill the Flash client to go to some other site. So when a user request (i.e. a game event) arrives in the server there is no way to know if another request will follow. So we have to persist that change right away. < That's an understandable dilemma. I love thinking about stuff like this and see how other people are dealing with these challenges, so please forgive my Sunday morning quarterbacking ;-) Wouldn't the issue have been solvable by creating an relatively simple persistent software layer between the app code manipulating the tiles and the backend storage? I understand that you moved to this model with your Erlang game, but I'd like to know if a persistence/caching layer was considered for the farming game? More generally, somewhere in here is an idea for a great Node.js server project that takes coarse grained datasets from a contentious database and serves as an interface for finer grained portions of that data.

jrirei 15y ago
For Monster World we did not consider this as the game is running fine (or good enough) as it is now. During the last weeks we were more focused on reducing RAM consumption of our databases as that is currently the main driver of cost and operation overhead. Regarding your idea: Wouldn't then the Node.js server have to keep the whole user state in memory?
- Udo 15y ago
  > During the last weeks we were more focused on reducing RAM consumption of our databases as that is currently the main driver of cost and operation overhead. < I can imagine that. > Regarding your idea: Wouldn't then the Node.js server have to keep the whole user state in memory? < Yes, but I think it would have several advantages: (1) The Node.js server code could decide which working sets it keeps in memory based on very simple rules. The details of this would be abstracted away from the application code itself, because the app just issues read and write requests on a user's dataset. So in essence, by splitting up the problem in two, it becomes relatively easy to handle (and optimize) on each end. (2) You just have to keep the active datasets wired in RAM and it wouldn't be necessary for the Node server to know whether a user has disconnected recently or not. All it knows is when the data was last accessed and it can then vacate RAM slots that have become stale. Compare this to Redis, which I believe just keeps everything in memory no matter what. So overall RAM usage would probably be considerably less than what you're doing now. (3) The idea beats "dumb blob caching" such as memcache, because it makes small operations economical. It seems to me that Node is well suited for this kind of task since it makes it very easy to build small server scripts that handle a huge number of small transactions. This would probably mean you need less machines for the same amount of users. (4) I believe it's relatively easy to implement replication and scaling. Anyway, just an idea. I have no clue whether this works in practice ;-)
  - jrirei 15y ago
    Yes, this might work. But I would be careful about replication and scaling - this could make things somewhat complicated. ;-)