4 ms·

My concern about that is communication between web and DB servers. Some reports need hundreds of queries to generate - wouldn't the lag severely slow this down?

• sokoloff 16y ago
  Yes, it would, but you might be better served to rethink your app design to return multiple result sets, and/or do most of the relevant joining/filtering on the database, even if your DB is <1ms away from your web farm. We run a good sized e-commerce site (Alexa top 1000; barely) and our ratio of DB queries per dynamic page is right around 2.5:1. You might not be able to get your query-per-page ratio into that range, but hundreds is certainly too many, IMO. Even "dozens" would be cause for concern, IMO. Moving your webfarm to a different datacenter than your DB is going to shine a spotlight on that problem, but even without that spotlight, you still have a problem. PS: Even with pushing that amount of work to the DB, we have almost the inverted ratio of web servers to DB servers as you report.
  • schammy 16y ago
    Well, most of those queries are just to look up the "name" for a given ID. And of course we use memcached to cache almost everything. But it still requires a bunch of round trip requests to either the database or memcached. Before anyone asks "why don't you just use a join?" it's because we have tons of different types of data stored in a couple of summary tables. It's 100x faster to just grab all the IDs and values, and then grab the names one at a time (most of which will always be in memcached) than to do it as a join. Don't believe me? We used to do it that way and the performance sucked.
• KevinMS 16y ago
  Most of our servers are DB anyways. We have 2 load balancers, 6 web servers, and ~40 DB servers. Wow, that's not what I was expecting. I have no idea if those 40 DBs are just mirrors sharing the load of some heavy queries or you are sharding the data in some way. If they are all just replicated maybe move most of them up into the cloud where they talk to your web app. But I don't know how much data you have to replicate up into the cloud or how long that would take.
  • schammy 16y ago
    We're monitoring traffic for 260,000 web sites with over 200 million daily page views. We have almost 10TB of data stored. That requires a lot of servers. Yes, it's sharded. There are no slave servers.
• dedward 16y ago
  It sounds like you need more in terms of actual application architecture & design - whether or not to offsite some of that capacity to AWS is purely a cost measure in the end. You will still ned to deal with systems and sysadmin tasks - and those can always be contracted out. AWS does NOT save you from that.