7 ms·

I think you could get well into the "people are paying money for this service" stage of a hosted Django hosting platform without ever having to touch C. Python

by rcoder 17y ago

I think you could get well into the "people are paying money for this service" stage of a hosted Django hosting platform without ever having to touch C. Python is fast enough to do most of the systems-level programming you'd need for such a beast, especially since so much of the standard library (socket, struct, asyncore, subprocess) sits directly on top of the fast POSIX syscalls you'd be invoking to do the same work in C anyway. Going down to the metal may help squeeze the last 10-20% out of your hardware, but the the really interesting and challenging work in coming up with a scalable hosting platform is elsewhere: security, monitoring, process spawning/reaping, deployment, et. al. If working in Python gives you a time-to-market advantage, then go for it. You can hire a C hacker when you have enough business to make the improvement in your hardware utilization efficiency pay off.

njl 17y ago
Well, some effort needs to go into some plugins for nginx and varnish. nginx needs to route incoming requests to the right varnish, and some custom cache invalidation stuff should happen on varnish as well. Other than that? Lots and lots of python, of course ;) I'm only one guy, and there are three or four big moving parts that need to be written. Something to build eggs and push them, something to monitor and manage processes on individual servers, something to dispatch requests to the appropriate servers, and something to monitor and manage the database servers. Even pulling back and looking at a minimum viable product, I'm still doing most of that work, just far simpler versions. Succinctly, it's too big for me to tackle on my own in the next six months.
- jokull 17y ago
  I'd do without eggs and provide common Python libraries that need compiling on the server (lxml, Crypto etc.). You'd then simply accept Python code over the live for deployments. You'd also want to handle rollback, schema migration, key-value services and a DB per instance, some sort of dumping/loading of data. If you stick with certain django conventions (namely settings-as-a-module and apps/libs on PYTHONPATH) you could get away with a really sweet and simple deployment procedure. A pretty smart client to handle all this is a lot of work too. Have a look at Silver Lining. Just got EC2 working on my fork ;)

by rcoder 17y ago

njl 17y ago

Well, some effort needs to go into some plugins for nginx and varnish. nginx needs to route incoming requests to the right varnish, and some custom cache invalidation stuff should happen on varnish as well. Other than that? Lots and lots of python, of course ;) I'm only one guy, and there are three or four big moving parts that need to be written. Something to build eggs and push them, something to monitor and manage processes on individual servers, something to dispatch requests to the appropriate servers, and something to monitor and manage the database servers. Even pulling back and looking at a minimum viable product, I'm still doing most of that work, just far simpler versions. Succinctly, it's too big for me to tackle on my own in the next six months.

jokull 17y ago
I'd do without eggs and provide common Python libraries that need compiling on the server (lxml, Crypto etc.). You'd then simply accept Python code over the live for deployments. You'd also want to handle rollback, schema migration, key-value services and a DB per instance, some sort of dumping/loading of data. If you stick with certain django conventions (namely settings-as-a-module and apps/libs on PYTHONPATH) you could get away with a really sweet and simple deployment procedure. A pretty smart client to handle all this is a lot of work too. Have a look at Silver Lining. Just got EC2 working on my fork ;)