5 ms·

Nginx design details

151 points by amnigos 14y ago

lnanek2 14y ago
My one experience with this server was quite bad. The web group we contracted used it, but would constantly run into issues us developers could have helped them with had they used Apache. Things like needing to return certain headers in the request, or return certain status codes, or do certain things handled easily by Apache's large ecosystem of modules became very grueling trials full of meetings with the contractors running the thing. Just like Java is sometimes the best language to use because it has huge mind share despite the warts, sometimes Apache is the right server to use just because that's what almost everyone uses and it makes things so much easier than using something more rare.
- asto 14y ago
  Can you give concrete examples of what you couldn't do? Here's the wiki on how to set headers: http://wiki.nginx.org/HttpHeadersModule http://wiki.nginx.org/HttpHeadersModule Here's the wiki on returning a particular http code: http://wiki.nginx.org/HttpRewriteModule#return http://wiki.nginx.org/HttpRewriteModule#return
- harshreality 14y ago
  You would run into the same situation with contractors who used apache, if they didn't know apache very well. Returning status codes from nginx is easy. Adding headers is easy. Perhaps you could elaborate about "certain things handled easily by Apache's large ecosystem of modules". Apache's large ecosystem of modules is built around doing everything in the webserver. More often these days, that stuff is handled by a dynamic application. Nginx talks to those via fastcgi, scgi, uwsgi, or plain http proxying. Nginx does not need to know anything about how the apps do whatever it is they do.
- herewego 14y ago
  I'm curious what you issues you ran into as well. I've used it in production environments for the past few years after moving away form Apache. It was a near seamless transition for my team, myself included. We use it behind some heavy iron load balancers and in front of a dozen web servers and it's been a generally pleasant experience.
  - genwin 14y ago
    Why do you use a web server (Nginx) in front of other web servers? I'm pretty new to this. Thanks.
    - ceejayoz 14y ago
      I've seen situations where nginx sits in front of Apache and serves static files directly, but proxies requests for PHP through to Apache for easier setup.
    - ObnoxiousJul 14y ago
      As described, static or lighttpd for static and reverse proxy for the domain achieves more than one goal: * you can serve page faster; * you can mix on a same domain more than one app from different servers in the DMZ (for sharing domain based mechanism (flash, Cross site ajax) by rewriting the url; * you can have one front (nginx) and several servers, which with heartbeat mechanism can handle failover. * you could (when ssl certificate were only IP based) share one SSL certificate for more than one back (VIP) It is a pretty low cost quite scalable architecture. I guess you could do it with apache, but I dropped apache since its licence is as understandable as its configurations.
- nl 14y ago
  would constantly run into issues us developers could have helped them with had they used Apache I'm not sure that says what you think it does about Ngnix and the developers you deal with...
pooriaazimi 14y ago
This is great! I didn't know that volume II has been published... Also be sure to read the chapter about LLVM compiler family (written by LLVM creator, who is an Apple employee now): http://www.aosabook.org/en/llvm.html http://www.aosabook.org/en/llvm.html It's great. A learnt a lot from that chapter. This book (The Architecture of Open Source Application) is a treasure trove. Just look at the index here: http://www.aosabook.org/en/index.html http://www.aosabook.org/en/index.html and tell me you don't want to skip work (or school, or whatever else is you're doing) for a week to read it all :D
- densh 14y ago
  Thank you for pointing out that other chapters are available for free too. It's indeed an amazing read. (It contains chapters about internal organization of Mercurial, Puppet, Haskell compiler and so much more.)
leeoniya 14y ago
My only quibble with Nginx so far is the magical order in which it evaluates location directives. http://wiki.nginx.org/HttpCoreModule#location http://wiki.nginx.org/HttpCoreModule#location On more than one occasion, you have to unintuitively try to figure out whether a rule at the bottom is getting matched before a rule at the top. The matching algo is quite intricate and easy to to trip up on. It tries to help novices by matching literal strings first for them rather than simply making a note of the performance benefits of putting literal locations at the top, but ends up failing to reveal the true matching order at a glance - you need to check the verbose logs, yuck.
- harshreality 14y ago
  I think the configuration language will eventually have to be reworked to address some of the issues with magical ordering and unintuitive behavior. Making the configuration language lua, for instance, would be a huge step forward, because many complex configs already use the lua 3rd party module. In addition to "location" eval order, "If" statements are another common gotcha.
- mfjordvald 14y ago
  I'll fully grant you that the learning curve for locations is quite steep and it takes some getting used to, but the order is always logical and it's entirely possible to reason your way through it without needing verbose logs.
  - leeoniya 14y ago
    Perhaps you have not been thrown someone else's list of 15-20 locations and tried to determine why one was not matching, and what was being matched before it based on specificity, whether or not it was a literal or regex, whether or not it halted further matching in the order that specific type was defined. The complexity is entirely unneeded, mod_rewrite could def use some minor convenience tweaks, but is far more intuitive and therefore more effective in both understanding and debugging. I assure you that "match in order defined, unless passed through explicitly by a sub-condition" is sufficient and simple for 99.5%. nginx's "match literals, then match everything, then choose most specific, in the order defined by its type" is craziness, "getting used to it" does not make it good, it's simply the first unnecessary step in making it useable.
herewego 14y ago
Not in this book, but worth taking a look at if you're into reading source code for large open source projects, is Redis and Mongrel2. Both are really well written C code-bases.
mef 14y ago
Another problem with the existing worker model is related to limited support for embedded scripting. For one, with the standard nginx distribution, only embedding Perl scripts is supported. For those looking for full featured and robust embedded scripting support in nginx, Yichun Zhang's lua-nginx-module is highly recomended. https://github.com/chaoslawful/lua-nginx-module https://github.com/chaoslawful/lua-nginx-module
- justincormack 14y ago
  The slides from a talk about it at the London Lua group the other day are here http://www.londonlua.org/scripting_nginx_with_lua/slides.html http://www.londonlua.org/scripting_nginx_with_lua/slides.htm... video coming shortly.
genwin 14y ago
I've read several times about people using the Go language's (golang) web server in conjunction with Nginx. Apparently Nginx provides some benefit that Go's web server doesn't, but I haven't seen that thing called out. Can anyone tell me what it is? Thanks. (I'd also love to see a performance comparison between these two.)
- tuxychandru 14y ago
  I do not know about others but I do it because not mixing up web server responsibilities with application server is a good idea. For example, I leave things like gzip compression and TLS handling to nginx without having to deal with them in my app.
  - cluda01 14y ago
    Using HTTP in general as an internal transport protocol has always struck me as odd. But I come from a financial services perspective where latency is the key driver.
    - tuxychandru 14y ago
      I use ZeroMQ or Go's RPC libraries for internal communication and mostly use nginx as reverse proxy and to serve static files on the edge.
- zaphar 14y ago
  I do something like this so that I can serve more than one standalone golang web application behind the same server. It allows me to multiplex several different go binaries and an apache server on the same box with the same address. These are all personal projects that aren't high traffic so one vps is fine and I don't have to remember which app is on which port.
reirob 14y ago
I much appreciated the article. And even more when I discovered from other commenters that http://www.aosabook.org http://www.aosabook.org is a collection of architectural description of a lot of Open-Source projects. I immediately read about sendmail by Eric Allman (http://www.aosabook.org/en/sendmail.html http://www.aosabook.org/en/sendmail.html) and I appreciated it actually more than the article about Nginx. There is a lot of wisdom in it and now I understand the reasons for some architectural decisions that have been taken for sendmail.

151 points by amnigos 14y ago

My one experience with this server was quite bad. The web group we contracted used it, but would constantly run into issues us developers could have helped them with had they used Apache. Things like needing to return certain headers in the request, or return certain status codes, or do certain things handled easily by Apache's large ecosystem of modules became very grueling trials full of meetings with the contractors running the thing. Just like Java is sometimes the best language to use because it has huge mind share despite the warts, sometimes Apache is the right server to use just because that's what almost everyone uses and it makes things so much easier than using something more rare.

asto 14y ago
Can you give concrete examples of what you couldn't do? Here's the wiki on how to set headers: http://wiki.nginx.org/HttpHeadersModule http://wiki.nginx.org/HttpHeadersModule Here's the wiki on returning a particular http code: http://wiki.nginx.org/HttpRewriteModule#return http://wiki.nginx.org/HttpRewriteModule#return
harshreality 14y ago
You would run into the same situation with contractors who used apache, if they didn't know apache very well. Returning status codes from nginx is easy. Adding headers is easy. Perhaps you could elaborate about "certain things handled easily by Apache's large ecosystem of modules". Apache's large ecosystem of modules is built around doing everything in the webserver. More often these days, that stuff is handled by a dynamic application. Nginx talks to those via fastcgi, scgi, uwsgi, or plain http proxying. Nginx does not need to know anything about how the apps do whatever it is they do.
herewego 14y ago
I'm curious what you issues you ran into as well. I've used it in production environments for the past few years after moving away form Apache. It was a near seamless transition for my team, myself included. We use it behind some heavy iron load balancers and in front of a dozen web servers and it's been a generally pleasant experience.
- genwin 14y ago
  Why do you use a web server (Nginx) in front of other web servers? I'm pretty new to this. Thanks.
  - ceejayoz 14y ago
    I've seen situations where nginx sits in front of Apache and serves static files directly, but proxies requests for PHP through to Apache for easier setup.
  - ObnoxiousJul 14y ago
    As described, static or lighttpd for static and reverse proxy for the domain achieves more than one goal: * you can serve page faster; * you can mix on a same domain more than one app from different servers in the DMZ (for sharing domain based mechanism (flash, Cross site ajax) by rewriting the url; * you can have one front (nginx) and several servers, which with heartbeat mechanism can handle failover. * you could (when ssl certificate were only IP based) share one SSL certificate for more than one back (VIP) It is a pretty low cost quite scalable architecture. I guess you could do it with apache, but I dropped apache since its licence is as understandable as its configurations.
nl 14y ago
would constantly run into issues us developers could have helped them with had they used Apache I'm not sure that says what you think it does about Ngnix and the developers you deal with...

pooriaazimi 14y ago

This is great! I didn't know that volume II has been published... Also be sure to read the chapter about LLVM compiler family (written by LLVM creator, who is an Apple employee now): http://www.aosabook.org/en/llvm.html http://www.aosabook.org/en/llvm.html It's great. A learnt a lot from that chapter. This book (The Architecture of Open Source Application) is a treasure trove. Just look at the index here: http://www.aosabook.org/en/index.html http://www.aosabook.org/en/index.html and tell me you don't want to skip work (or school, or whatever else is you're doing) for a week to read it all :D

densh 14y ago
Thank you for pointing out that other chapters are available for free too. It's indeed an amazing read. (It contains chapters about internal organization of Mercurial, Puppet, Haskell compiler and so much more.)

leeoniya 14y ago

My only quibble with Nginx so far is the magical order in which it evaluates location directives. http://wiki.nginx.org/HttpCoreModule#location http://wiki.nginx.org/HttpCoreModule#location On more than one occasion, you have to unintuitively try to figure out whether a rule at the bottom is getting matched before a rule at the top. The matching algo is quite intricate and easy to to trip up on. It tries to help novices by matching literal strings first for them rather than simply making a note of the performance benefits of putting literal locations at the top, but ends up failing to reveal the true matching order at a glance - you need to check the verbose logs, yuck.

harshreality 14y ago
I think the configuration language will eventually have to be reworked to address some of the issues with magical ordering and unintuitive behavior. Making the configuration language lua, for instance, would be a huge step forward, because many complex configs already use the lua 3rd party module. In addition to "location" eval order, "If" statements are another common gotcha.
mfjordvald 14y ago
I'll fully grant you that the learning curve for locations is quite steep and it takes some getting used to, but the order is always logical and it's entirely possible to reason your way through it without needing verbose logs.
- leeoniya 14y ago
  Perhaps you have not been thrown someone else's list of 15-20 locations and tried to determine why one was not matching, and what was being matched before it based on specificity, whether or not it was a literal or regex, whether or not it halted further matching in the order that specific type was defined. The complexity is entirely unneeded, mod_rewrite could def use some minor convenience tweaks, but is far more intuitive and therefore more effective in both understanding and debugging. I assure you that "match in order defined, unless passed through explicitly by a sub-condition" is sufficient and simple for 99.5%. nginx's "match literals, then match everything, then choose most specific, in the order defined by its type" is craziness, "getting used to it" does not make it good, it's simply the first unnecessary step in making it useable.

herewego 14y ago

Not in this book, but worth taking a look at if you're into reading source code for large open source projects, is Redis and Mongrel2. Both are really well written C code-bases.

mef 14y ago

Another problem with the existing worker model is related to limited support for embedded scripting. For one, with the standard nginx distribution, only embedding Perl scripts is supported. For those looking for full featured and robust embedded scripting support in nginx, Yichun Zhang's lua-nginx-module is highly recomended. https://github.com/chaoslawful/lua-nginx-module https://github.com/chaoslawful/lua-nginx-module

justincormack 14y ago
The slides from a talk about it at the London Lua group the other day are here http://www.londonlua.org/scripting_nginx_with_lua/slides.html http://www.londonlua.org/scripting_nginx_with_lua/slides.htm... video coming shortly.

genwin 14y ago

I've read several times about people using the Go language's (golang) web server in conjunction with Nginx. Apparently Nginx provides some benefit that Go's web server doesn't, but I haven't seen that thing called out. Can anyone tell me what it is? Thanks. (I'd also love to see a performance comparison between these two.)

tuxychandru 14y ago
I do not know about others but I do it because not mixing up web server responsibilities with application server is a good idea. For example, I leave things like gzip compression and TLS handling to nginx without having to deal with them in my app.
- cluda01 14y ago
  Using HTTP in general as an internal transport protocol has always struck me as odd. But I come from a financial services perspective where latency is the key driver.
  - tuxychandru 14y ago
    I use ZeroMQ or Go's RPC libraries for internal communication and mostly use nginx as reverse proxy and to serve static files on the edge.
zaphar 14y ago
I do something like this so that I can serve more than one standalone golang web application behind the same server. It allows me to multiplex several different go binaries and an apache server on the same box with the same address. These are all personal projects that aren't high traffic so one vps is fine and I don't have to remember which app is on which port.

reirob 14y ago

I much appreciated the article. And even more when I discovered from other commenters that http://www.aosabook.org http://www.aosabook.org is a collection of architectural description of a lot of Open-Source projects. I immediately read about sendmail by Eric Allman (http://www.aosabook.org/en/sendmail.html http://www.aosabook.org/en/sendmail.html) and I appreciated it actually more than the article about Nginx. There is a lot of wisdom in it and now I understand the reasons for some architectural decisions that have been taken for sendmail.