7 ms·

FastCGI: 30 years old and still the better protocol for reverse proxies

424 points by agwa 5mo ago

tombert 5mo ago
Interesting. Most of the stuff I've done for reverse proxies has been pretty straightforward and just using the stuff built into Nginx, but I have to admit that it wouldn't have even occurred to me to use FastCGI if I needed something more elaborate. I used FastCGI a bit about ten years ago to "convert" some C++ code I wrote to work on the web, but admittedly I haven't used it much since then.
- nzeid 5mo ago
  Also, embedded servers are now much much much more popular. Stuff an HTTP server directly into your application and do whatever you gotta do without gateways.
  - agwa 5mo ago
    That is way! Unfortunately, sometimes you have to do path-based routing to different backends, and now you're back to needing a proxy between your clients and your applications.
    - nostrademons 5mo ago
      This is the way only if you're operating in a trusted environment (eg. homelab, intranet) or you're sticking CloudFlare or some other "reverse proxy as a service" in front of it. If you expose an embedded HTTP app server directly to the Internet you're almost guaranteed to get pwned, as the Internet has now become an extremely hostile place.
      - agwa 5mo ago
        Go's embedded HTTP server can handle it just fine: https://blog.gopheracademy.com/advent-2016/exposing-go-on-the-internet/ https://blog.gopheracademy.com/advent-2016/exposing-go-on-th...
  - winstonwinston 5mo ago
    These are often not enough ‘battle-tested” and come with a warning to never expose to public internet. So then you put a WAF in front of it, and you are back to HTTP reverse proxy setup.
    - nzeid 5mo ago
      I've always chuckled at this. Just don't used bad HTTP server libraries. I wouldn't put something like that on my intranet either. But even if you disagree with me the point is that I can count on only one hand the number of times I went "oh man, I need a FastCGI middle end".
      - winstonwinston 5mo ago
        I agree with your point but this is the reality: F.E. Python stdlib http.server comes with a warning: Warning http.server is not recommended for production. It only implements basic security checks. The `standard` way is then to use WSGI or ASGI, not FastCGI, but it is similar interface implementation.
nostrademons 5mo ago
This is quite an interesting article for its omissions. I remember the great FastCGI vs. SCGI vs. HTTP wars: I was founding a Web2.0 startup right at the time these technologies were gaining adoption, and so was responsible for setting up the frontend stack. HTTP won because of simplicity: instead of needing to introduce another protocol into your stack, you can just use HTTP, which you already needed to handle at the gateway. Now all sorts of complex network topologies became trivial: you could introduce multiple levels of reverse proxies if you ran out of capacity; you could have servers that specialized in authentication or session management or SSL termination or DDoS filtering or all the other cross-cutting concerns without them needing to know their position in the request chain; and you could use the same application servers for development, with a direct HTTP connection, as you did in production, where they'd sit behind a reverse proxy that handled SSL and authentication and abuse detection. It also helped that nginx was lots faster than most FastCGI/SCGI modules of the time, and more robust. I'd initially setup my startup's stack as HTTP -> Lighttpd -> FastCGI -> Django, but it was way slower than just using nginx. The use of HTTP was basically the web equivalent of the End-to-End Principle [1] for TCP/IP. It's the idea that the network and its protocols should be agnostic to what's being transmitted, and all application logic should be in nodes of the network that filter and redirect packets accordingly. This has been a very powerful principle and shouldn't be discarded lightly. The observation the article makes is that for security, it's often better to follow the Principle of Least Privilege [2] rather than blindly passing information along. Allowlist your communications to only what you expect, so that you aren't unwittingly contributing to a compromise elsewhere in the network. And the article is highlighting - not explicitly, but it's there - the tension between these two principles. E2E gives you flexibility, but with flexibility comes the potential for someone to use that flexibility to cause harm. PoLP gives you security, but at the cost of inflexibility, where your system can only do what you designed it to do and cannot easily adapt to new requirements. [1] https://en.wikipedia.org/wiki/End-to-end_principle https://en.wikipedia.org/wiki/End-to-end_principle [2] https://en.wikipedia.org/wiki/Principle_of_least_privilege https://en.wikipedia.org/wiki/Principle_of_least_privilege
- ragall 5mo ago
  The end-to-end principle within a datacenter makes little sense and, as shown in the article, ends up enabling insecure behaviour.
sscaryterry 5mo ago
I've fought many battles with perl + windows + apache + FastCGI in a previous life. No thank you.
- jollyllama 5mo ago
  Indeed. I'm sure that someone will butt in with "it's just a bad implementation!" but the whole bit about allowlisting communications will cause flashbacks in those of us who had all our PUT requests just quit working on an IIS server.
  - inetknght 5mo ago
    > an IIS server There's a reason the internet runs on Linux...
- deleted 5mo ago
  [deleted]
- marcosdumay 5mo ago
  I have already tried to build houses with sand, I'm not falling for your "concrete" idea!
chasil 5mo ago
The PHP/Apache configuration that is distributed in the Red Hat family is "FastCGI Process Manager" (FPM). I don't know if anything else in the RHEL distributions use FastCGI. $ rpm -qi php-fpm | grep ^Summary Summary : PHP FastCGI Process Manager
- agwa 5mo ago
  What you're looking for is mod_proxy_fcgi, not FPM. It's included in Fedora's httpd-core package; I don't know about RHEL: https://packages.fedoraproject.org/pkgs/httpd/httpd-core/fedora-rawhide.html#files https://packages.fedoraproject.org/pkgs/httpd/httpd-core/fed...
  - chasil 5mo ago
    I'm not looking for anything. I use this now, and it works. I don't really know anything about the FastCGI.
marlburrow 5mo ago
[flagged]
shevy-java 5mo ago
I'd love for CGI to be updated, kind of merging what works and not really caring about what does not work. Getting a .cgi file to work on Linux is really easy. Naturally you get more leverage with e. g. rails, but there is also a lot more complexity and I really hate intrinsic complexity.
- somat 5mo ago
  CGI and FastCGI are two different things in two different domains. Well the domains are not that different but enough that CGI solves a real problem and makes sense and FastCGI does not. CGI is the interface between a HTTP transaction and a process. It answers the question "How do we turn a HTTP request into executing a process?". FastCGI answers the question of "How do we turn a HTTP request into a FastCGI request". a convolution that leaves you asking "Why are we jumping through this hoop? Is FastCGI actually bringing anything to the table?, Is it actually more difficult to have a HTTP server instead of a FastCGI server if they are so trivially connected? I am halfway convinced the only reason FastCGI exists is we had got in a mindset that executable code in a HTTP context had to run via the Common Gateway Interface and when we wanted to to change to a persistent process model it had to have the CGI name as well. Well FastCGI to the rescue it does exactly what HTTP does but is not HTTP and most importantly has CGI in the name. As to the articles complaint, "A HTTP relay server had a bug. Therefore HTTP is intrinsically bad". Well.. it failed to convince me. I am not exactly in that domain(backend web development) so my view is not worth much. But I feel that your internal HTTP(application) servers should be built as if they were going directly on the open web. Then you put some relay servers in front in order to block, balance and route requests. But avoid putting too many smarts in the relay servers. A smart network is almost always a bad idea. try and stick with a dumb network and smart edges.
  - lelanthran 5mo ago
    > Is FastCGI actually bringing anything to the table?, Yes; it removes the need for the application server to perform parsing of HTTP, which is notoriously difficult to consistently do safely. The application server can use the safer and simpler FastCGI protocol rather than try to support the full HTTP spec. > Is it actually more difficult to have a HTTP server instead of a FastCGI server if they are so trivially connected? IME, yes. HTTP (the spec) is full of footguns. FastCGI has fewer footguns. HTTP requires a long dependency chain in your application. FastCGI requires maybe a single library. > But I feel that your internal HTTP(application) servers should be built as if they were going directly on the open web. I feel that too, but which developer do you know writes their own HTTP server inside their application? They all use the most popular server via a library or framework, almost all of which warn not to open that to the public internet. What do you propose they do? They use a framework/library and get a warning not to expose it to the public internet, they don't use a framework/library and odds are good that they coded some vulnerability into it.
simonw 5mo ago
As I understand it FastCGI doesn't handle websockets, which is a shame. It should be able to handle SSE though since that's effectively just a regular slow-loading/streaming HTTP response.
max_k 5mo ago
I agree with the article, FastCGI is better than HTTP for these things. Though I'd like to make another protocol known: Web Application Socket (WAS). I designed it 16 years ago at my dayjob because I thought FastCGI still wasn't good enough. Instead of packing bulk data inside frames on the main socket, WAS has a control socket plus two pipes (raw request+response body). Both the WAS application and the web server can use splice() to operate on a pipe, for example. No framing needed. Also, requests are cancellable and the three file descriptors can always be recovered. Over the years, we used WAS for many of our internal applications, and for our web hosting environment, I even wrote a PHP SAPI for WAS. Quite a large number of web sites operate with WAS internally. It's all open source: - library: https://github.com/CM4all/libwas https://github.com/CM4all/libwas - documentation: https://libwas.readthedocs.io/en/latest/ https://libwas.readthedocs.io/en/latest/ - non-blocking library: https://github.com/CM4all/libcommon/tree/master/src/was/async https://github.com/CM4all/libcommon/tree/master/src/was/asyn... - our web server: https://github.com/CM4all/beng-proxy https://github.com/CM4all/beng-proxy - WebDAV: https://github.com/CM4all/davos https://github.com/CM4all/davos - PHP fork with WAS SAPI: https://github.com/CM4all/php-src https://github.com/CM4all/php-src
- assimpleaspossi 5mo ago
  >>FastCGI is better than HTTP for these things. FastCGI and HTTP are at two different levels. HTTP is for data transfer from, say, a browser and a server. FastCGI is for handling that data between the server and an application. Just now I glanced at the article and it seems the author writes in a confusing way to imply that HTTP and FastCGI are interchangeable and they are not. fwiw, I used fcgi for a decade for all our web customers.
  - afavour 5mo ago
    I feel like the author of an alternative protocol probably knows these things. I think the author mentions HTTP because many people use it where they could be using FastCGI and just don’t.
    - assimpleaspossi 5mo ago
nzoschke 5mo ago
I’ve rediscovered plain old CGI as a great way for users to “vibe code” custom pages on our platform. [1] The scenario is we have our first party task lists and data viewers, but often users want to highly customize it. Say build a Kanban view or a custom dashboard with data filters and charts. The box has a coding agent which means the user can code anything vs us building traditional report builder tools. Go’s stdlib has good support on both the server side and user space. The coding agent makes a page-name/main.go that talks CGI and the server delegates requests to it. It’s all “person scale” data and page views so no real need to optimize with fast CGI even. What’s old is new again for agents! 1. https://housecat.com https://housecat.com
- agwa 5mo ago
  Do be aware that CGI, unlike FastCGI, has a pretty big footgun due to the use of environment variables to convey HTTP headers: https://httpoxy.org/ https://httpoxy.org/ Go's CGI server implementation doesn't set $HTTP_PROXY so you're safe from that, but I still don't love how CGI uses environment variables.
  - duskwuff 5mo ago
    > I still don't love how CGI uses environment variables. Neither do I. They really only make sense in the context of a request which was actually to a CGI script resident in a document root - they're an exceptionally awkward way of describing other HTTP requests, especially ones which aren't being served from a document root. And there's a lot of information lost in translation, like the order and original capitalization of HTTP headers. (Not that these things are supposed to matter, but still.)
Tepix 5mo ago
I‘ve had good experiences with FastCGI back when Perl was popular. These days, WebTransport is the new sexy thing. Probably not a real FastCGI replacement.
athrowaway3z 5mo ago
This seems like really bad advice or am i missing something? Using fastcgi requires you write your app to serve fastcgi. The upside of serving http/1.1 instead of fastcgi is that devs can instantly use their browser to test things instead of having to setup a reverse proxy on their machine. The bad parts of http/1.1 are fixed equally well by both http/2.0 and fastcgi. So just use http/2.0 and you get the proper framing as well as browser support.
- agwa 5mo ago
  Please see the section about untrusted headers - this is not fixed by HTTP/2. You're right that being able to point your browser right at the app is very convenient. With Go, you can have a command line flag that switches between http.Serve (for development) and fcgi.Serve (for production).
  - enneff 5mo ago
    In my experience having different serving paths for dev vs production is a recipe for annoying issues. I try to make dev as similar to prod as possible. I’m not sure, I don’t dismiss fcgi outright here, I find the arguments for it compelling (not a huge fan of http for many reasons) but it has to be really worth it to break the consistency of using http everywhere.
    - agwa 5mo ago
      If you want your dev environment to be as similar to prod as possible, and you use a proxy in prod, then you should use a proxy in dev also. I was presenting a solution to someone who doesn't want to do that.
      - enneff 5mo ago
        I think perhaps I was unclear. I don’t mean the entire dev environment should mirror prod (although it’s great if you can do this for end to end testing). I just mean it’s desirable if the process you’re working on operates the same way in dev as in prod.
scotty79 5mo ago
What I'd like to see is someone creating local caching proxy for modern https infested world. I'm fed up with downloading same packages 100 times.
- ebb_earl_co 5mo ago
  I’m in the same boat. Am trying to figure out how to configure Vinyl cache (née Varnish) in my home lab.
- majorchord 5mo ago
  Can't squid do this?
  - scotty79 5mo ago
    Not easily. I always dread setting it up for this. I'd love to have one click solution with nice ui.
- xorcist 5mo ago
  There is no way to do what you ask for without having a local CA. And once you have it, any existing caching proxy will work. Now you have moved all security to the proxy, but for some applications it is worth it.
apitman 5mo ago
> That said, using a vintage technology has some downsides. It was never updated to support WebSockets With widespread browser support for WHATWG streams, it's pretty easy to implement your own WebSockets over long-lived HTTP requests. Basically you just send a byte stream and prepend each message with a header, which can just be a size in many cases. Advantages over WebSockets: * No special path in your server layer like you need for WebSocket. * Backpressure * You get to take advantage of HTTP/2/3 improvements for free * Lower framing overhead Unfortunately AFAIK it's still not supported to still be streaming your request body while receiving the response, so you need a pair of requests for full bidirectional streaming.
- yesbabyyes 5mo ago
  Please be aware that there is a web standard for this since quite some time. See server-sent events and the EventSource interface: https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events https://developer.mozilla.org/en-US/docs/Web/API/Server-sent... https://developer.mozilla.org/en-US/docs/Web/API/EventSource https://developer.mozilla.org/en-US/docs/Web/API/EventSource
  - Oxodao 5mo ago
    That can be used with https://mercure.rocks https://mercure.rocks :)
  - apitman 5mo ago
    Not the same. SSE doesn't support binary, and has higher framing overhead than you can achieve with raw HTTP. If you don't need binary it's a solid option in the browser.
daneel_w 5mo ago
I've built a lot of API backends with Perl and FCGI::ProcManager, letting nginx (and Apache HTTPd in the past) front everything. For me it has been a pleasantly simple, incredibly robust and high-performing setup with no mess to speak of.
Animats 5mo ago
FCGI is also an orchestration system. It launches more server tasks when the load goes up, shuts them down when the load decreases, and launches new copies of tasks if they crash. It's like single-system Kubernetes.
- toast0 5mo ago
  > It launches more server tasks when the load goes up, shuts them down when the load decreases In my experience, this isn't a good feature. It sounds nice, but it can often mean everything runs fine while your load is low, but when your load gets high, you spawn more workers and run out of memory. It's much better to have a static number of workers in my experience. Crash recovery is handy, if needed though.
  - Animats 5mo ago
    It's useful if you have multiple FCGI programs that handle different kinds of requests. Depending on what's being requested, programs start up and shut down.
    - ramses0 5mo ago
      I had to argue about inetd while poking at AI stuff on my home computer. https://en.wikipedia.org/wiki/Inetd https://en.wikipedia.org/wiki/Inetd ... yeah, there's better modern equivalents, but it seems like we're coming full circle with lambdas and cgi. The concept of dynamically scaling has been around a looong time in UNIX, and we lost it for a bit with big honking java monolith servers.
- deleted 5mo ago
  [deleted]
- assimpleaspossi 5mo ago
  This is exactly how we used it.
blipvert 5mo ago
(u)WSGI must surely get a mention here?!
runxiyu 5mo ago
I think there is a lot of merit to this argument, however, FastCGI defers to CGI/1.1 for `PATH_INFO`, etc., which is lossy as it must be URL-decoded and therefore cannot represent encoded slashes, `%2F`. (Some implementations also collapse `//` to `/` in path, but this is an issue in various HTTP implementations too.) It is less expressive than HTTP in ways that may or may not be important to your application; I prefer accurate URL handling.
xp84 5mo ago
> Only if True-Client-IP doesn't exist does it use X-Real-IP. So even if your proxy does the right thing with X-Real-IP, you can still be pwned by an attacker sending a True-Client-IP header. Can we just take a moment to appreciate the absurdity of HTTP headers for a moment? We have X-Forwarded-For, X-Real-IP, each CDN has their own custom flavored one. Some of them are a comma-separated list, and usually ends up having an IP of your own LB uselessly added in there (I know why, it's just not helpful). All of them might be inserted by a malicious user-agent. I guess nobody could agree on how all the various trusted servers in the pipeline should convey the important bit. I guess it fits in quite well with the absurdity of the User-Agent header, which has come so far in absurdity that Apple decided to fully kill it by just sending utterly fake nonsense (false OS version, etc) in the name of "pRiVaCy."
est 5mo ago
Then there's uwsgi protocol. It's also an RPC for basically everything.
teddy_oweh 5mo ago
[flagged]
thayne 5mo ago
The untrusted header problem could potentially be fixed by having the reverse proxy embed all the trusted information in a specific header, and then it just has to make sure that one header is stripped from the request. Unfortunately, there isn't (yet) a standard for that. Or you could use something like haproxy's proxy protocol (although that may not support all the information you want, and doesn't work for multiplexing). Edit: actually the "Forwarded" header kind of fills that niche. Although you may want extensions for things like the client certificate.
- XYen0n 5mo ago
  Unfortunately, it appeared too late, and the relevant support is now far less complete than that for `X-Forwarded-*`.
- max_k 5mo ago
  FastCGI has "parameters" and HTTP headers are special parameters starting with "HTTP_" (mimicking CGI's environment variables). All parameters not starting with "HTTP_" can be trusted because only the web server (= FastCGI client) can construct them.
verifex 5mo ago
I was reading the article and thinking, I wonder what that proxy I'm using in Apache is using, it's really fast and I've had a lot of luck with it. OMG, so I've been using FastCGI all this time and had no idea, well that's awesome. :)
novoreorx 5mo ago
FastCGI is theoretically better does not make it the easier choice in reality, the success of HTTP is just another case of "worse is better"
- tlavoie 5mo ago
  No, in the case of all the desync attacks that James Kettle has found since, HTTP between servers is more like "worse really is worse." At least until they all speak HTTP/2.
xorcist 5mo ago
The mystery is why uWSGI isn't more widely used. Perhaps the name does not help. It has little to do with WSGI just as FastCGI is unrelated to CGI. It is a tiny binary protocol, with frames just as FastCGI. The reference server works with several languages, I've used it over the years mostly with Python but also Ruby and Perl. It is a small C executable with all the practical features one need for web hosting: Draining backends, autoscaling, logging, chrooted backends, everything. Very few FastCGI servers are this mature. Unlike FastCGI, it has been extended to support websockets and async. I have used it in production at several places for many years and have nothing but praise for it. It feels like this weird unknown secret for web operations. Unfortunately, it sees lesser use now in the cloud era, and development seems to have all but stopped. It still works and is still reliable but the writing is probably on the wall. However nothing comes close in terms of speed, simplicity, and features.
LAC-Tech 5mo ago
I am actually implementing a reverse proxy right now... I am doing a typical http thing, but I wonder, has anyone used fastcgi in Caddy? https://caddyserver.com/docs/caddyfile/directives/reverse_proxy#the-fastcgi-transport https://caddyserver.com/docs/caddyfile/directives/reverse_pr...
zokier 5mo ago
Does anyone remember mongrel2? A neat web server that used zeromq for backend communication (based on scgi). I always felt it had huge potential. It was also kinda famous in the early days of HN, back in the days when RoR was in vogue etc. https://hn.algolia.com/?q=mongrel2 https://hn.algolia.com/?q=mongrel2
caycecan 5mo ago
I'm here looking to speed up rendering.
_stiletto_ 5mo ago
[dead]

424 points by agwa 5mo ago

Interesting. Most of the stuff I've done for reverse proxies has been pretty straightforward and just using the stuff built into Nginx, but I have to admit that it wouldn't have even occurred to me to use FastCGI if I needed something more elaborate. I used FastCGI a bit about ten years ago to "convert" some C++ code I wrote to work on the web, but admittedly I haven't used it much since then.

nzeid 5mo ago
Also, embedded servers are now much much much more popular. Stuff an HTTP server directly into your application and do whatever you gotta do without gateways.
- agwa 5mo ago
  That is way! Unfortunately, sometimes you have to do path-based routing to different backends, and now you're back to needing a proxy between your clients and your applications.
  - nostrademons 5mo ago
    This is the way only if you're operating in a trusted environment (eg. homelab, intranet) or you're sticking CloudFlare or some other "reverse proxy as a service" in front of it. If you expose an embedded HTTP app server directly to the Internet you're almost guaranteed to get pwned, as the Internet has now become an extremely hostile place.
    - agwa 5mo ago
      Go's embedded HTTP server can handle it just fine: https://blog.gopheracademy.com/advent-2016/exposing-go-on-the-internet/ https://blog.gopheracademy.com/advent-2016/exposing-go-on-th...
- winstonwinston 5mo ago
  These are often not enough ‘battle-tested” and come with a warning to never expose to public internet. So then you put a WAF in front of it, and you are back to HTTP reverse proxy setup.
  - nzeid 5mo ago
    I've always chuckled at this. Just don't used bad HTTP server libraries. I wouldn't put something like that on my intranet either. But even if you disagree with me the point is that I can count on only one hand the number of times I went "oh man, I need a FastCGI middle end".
    - winstonwinston 5mo ago
      I agree with your point but this is the reality: F.E. Python stdlib http.server comes with a warning: Warning http.server is not recommended for production. It only implements basic security checks. The `standard` way is then to use WSGI or ASGI, not FastCGI, but it is similar interface implementation.

nostrademons 5mo ago

This is quite an interesting article for its omissions. I remember the great FastCGI vs. SCGI vs. HTTP wars: I was founding a Web2.0 startup right at the time these technologies were gaining adoption, and so was responsible for setting up the frontend stack. HTTP won because of simplicity: instead of needing to introduce another protocol into your stack, you can just use HTTP, which you already needed to handle at the gateway. Now all sorts of complex network topologies became trivial: you could introduce multiple levels of reverse proxies if you ran out of capacity; you could have servers that specialized in authentication or session management or SSL termination or DDoS filtering or all the other cross-cutting concerns without them needing to know their position in the request chain; and you could use the same application servers for development, with a direct HTTP connection, as you did in production, where they'd sit behind a reverse proxy that handled SSL and authentication and abuse detection. It also helped that nginx was lots faster than most FastCGI/SCGI modules of the time, and more robust. I'd initially setup my startup's stack as HTTP -> Lighttpd -> FastCGI -> Django, but it was way slower than just using nginx. The use of HTTP was basically the web equivalent of the End-to-End Principle [1] for TCP/IP. It's the idea that the network and its protocols should be agnostic to what's being transmitted, and all application logic should be in nodes of the network that filter and redirect packets accordingly. This has been a very powerful principle and shouldn't be discarded lightly. The observation the article makes is that for security, it's often better to follow the Principle of Least Privilege [2] rather than blindly passing information along. Allowlist your communications to only what you expect, so that you aren't unwittingly contributing to a compromise elsewhere in the network. And the article is highlighting - not explicitly, but it's there - the tension between these two principles. E2E gives you flexibility, but with flexibility comes the potential for someone to use that flexibility to cause harm. PoLP gives you security, but at the cost of inflexibility, where your system can only do what you designed it to do and cannot easily adapt to new requirements. [1] https://en.wikipedia.org/wiki/End-to-end_principle https://en.wikipedia.org/wiki/End-to-end_principle [2] https://en.wikipedia.org/wiki/Principle_of_least_privilege https://en.wikipedia.org/wiki/Principle_of_least_privilege

ragall 5mo ago
The end-to-end principle within a datacenter makes little sense and, as shown in the article, ends up enabling insecure behaviour.

sscaryterry 5mo ago

I've fought many battles with perl + windows + apache + FastCGI in a previous life. No thank you.

jollyllama 5mo ago
Indeed. I'm sure that someone will butt in with "it's just a bad implementation!" but the whole bit about allowlisting communications will cause flashbacks in those of us who had all our PUT requests just quit working on an IIS server.
- inetknght 5mo ago
  > an IIS server There's a reason the internet runs on Linux...
deleted 5mo ago
[deleted]
marcosdumay 5mo ago
I have already tried to build houses with sand, I'm not falling for your "concrete" idea!

chasil 5mo ago

The PHP/Apache configuration that is distributed in the Red Hat family is "FastCGI Process Manager" (FPM). I don't know if anything else in the RHEL distributions use FastCGI. $ rpm -qi php-fpm | grep ^Summary Summary : PHP FastCGI Process Manager

agwa 5mo ago
What you're looking for is mod_proxy_fcgi, not FPM. It's included in Fedora's httpd-core package; I don't know about RHEL: https://packages.fedoraproject.org/pkgs/httpd/httpd-core/fedora-rawhide.html#files https://packages.fedoraproject.org/pkgs/httpd/httpd-core/fed...
- chasil 5mo ago
  I'm not looking for anything. I use this now, and it works. I don't really know anything about the FastCGI.

marlburrow 5mo ago

[flagged]

shevy-java 5mo ago

I'd love for CGI to be updated, kind of merging what works and not really caring about what does not work. Getting a .cgi file to work on Linux is really easy. Naturally you get more leverage with e. g. rails, but there is also a lot more complexity and I really hate intrinsic complexity.

somat 5mo ago
CGI and FastCGI are two different things in two different domains. Well the domains are not that different but enough that CGI solves a real problem and makes sense and FastCGI does not. CGI is the interface between a HTTP transaction and a process. It answers the question "How do we turn a HTTP request into executing a process?". FastCGI answers the question of "How do we turn a HTTP request into a FastCGI request". a convolution that leaves you asking "Why are we jumping through this hoop? Is FastCGI actually bringing anything to the table?, Is it actually more difficult to have a HTTP server instead of a FastCGI server if they are so trivially connected? I am halfway convinced the only reason FastCGI exists is we had got in a mindset that executable code in a HTTP context had to run via the Common Gateway Interface and when we wanted to to change to a persistent process model it had to have the CGI name as well. Well FastCGI to the rescue it does exactly what HTTP does but is not HTTP and most importantly has CGI in the name. As to the articles complaint, "A HTTP relay server had a bug. Therefore HTTP is intrinsically bad". Well.. it failed to convince me. I am not exactly in that domain(backend web development) so my view is not worth much. But I feel that your internal HTTP(application) servers should be built as if they were going directly on the open web. Then you put some relay servers in front in order to block, balance and route requests. But avoid putting too many smarts in the relay servers. A smart network is almost always a bad idea. try and stick with a dumb network and smart edges.
- lelanthran 5mo ago
  > Is FastCGI actually bringing anything to the table?, Yes; it removes the need for the application server to perform parsing of HTTP, which is notoriously difficult to consistently do safely. The application server can use the safer and simpler FastCGI protocol rather than try to support the full HTTP spec. > Is it actually more difficult to have a HTTP server instead of a FastCGI server if they are so trivially connected? IME, yes. HTTP (the spec) is full of footguns. FastCGI has fewer footguns. HTTP requires a long dependency chain in your application. FastCGI requires maybe a single library. > But I feel that your internal HTTP(application) servers should be built as if they were going directly on the open web. I feel that too, but which developer do you know writes their own HTTP server inside their application? They all use the most popular server via a library or framework, almost all of which warn not to open that to the public internet. What do you propose they do? They use a framework/library and get a warning not to expose it to the public internet, they don't use a framework/library and odds are good that they coded some vulnerability into it.

simonw 5mo ago

As I understand it FastCGI doesn't handle websockets, which is a shame. It should be able to handle SSE though since that's effectively just a regular slow-loading/streaming HTTP response.

max_k 5mo ago

I agree with the article, FastCGI is better than HTTP for these things. Though I'd like to make another protocol known: Web Application Socket (WAS). I designed it 16 years ago at my dayjob because I thought FastCGI still wasn't good enough. Instead of packing bulk data inside frames on the main socket, WAS has a control socket plus two pipes (raw request+response body). Both the WAS application and the web server can use splice() to operate on a pipe, for example. No framing needed. Also, requests are cancellable and the three file descriptors can always be recovered. Over the years, we used WAS for many of our internal applications, and for our web hosting environment, I even wrote a PHP SAPI for WAS. Quite a large number of web sites operate with WAS internally. It's all open source: - library: https://github.com/CM4all/libwas https://github.com/CM4all/libwas - documentation: https://libwas.readthedocs.io/en/latest/ https://libwas.readthedocs.io/en/latest/ - non-blocking library: https://github.com/CM4all/libcommon/tree/master/src/was/async https://github.com/CM4all/libcommon/tree/master/src/was/asyn... - our web server: https://github.com/CM4all/beng-proxy https://github.com/CM4all/beng-proxy - WebDAV: https://github.com/CM4all/davos https://github.com/CM4all/davos - PHP fork with WAS SAPI: https://github.com/CM4all/php-src https://github.com/CM4all/php-src

assimpleaspossi 5mo ago
>>FastCGI is better than HTTP for these things. FastCGI and HTTP are at two different levels. HTTP is for data transfer from, say, a browser and a server. FastCGI is for handling that data between the server and an application. Just now I glanced at the article and it seems the author writes in a confusing way to imply that HTTP and FastCGI are interchangeable and they are not. fwiw, I used fcgi for a decade for all our web customers.
- afavour 5mo ago
  I feel like the author of an alternative protocol probably knows these things. I think the author mentions HTTP because many people use it where they could be using FastCGI and just don’t.
  - assimpleaspossi 5mo ago

nzoschke 5mo ago

I’ve rediscovered plain old CGI as a great way for users to “vibe code” custom pages on our platform. [1] The scenario is we have our first party task lists and data viewers, but often users want to highly customize it. Say build a Kanban view or a custom dashboard with data filters and charts. The box has a coding agent which means the user can code anything vs us building traditional report builder tools. Go’s stdlib has good support on both the server side and user space. The coding agent makes a page-name/main.go that talks CGI and the server delegates requests to it. It’s all “person scale” data and page views so no real need to optimize with fast CGI even. What’s old is new again for agents! 1. https://housecat.com https://housecat.com

agwa 5mo ago
Do be aware that CGI, unlike FastCGI, has a pretty big footgun due to the use of environment variables to convey HTTP headers: https://httpoxy.org/ https://httpoxy.org/ Go's CGI server implementation doesn't set $HTTP_PROXY so you're safe from that, but I still don't love how CGI uses environment variables.
- duskwuff 5mo ago
  > I still don't love how CGI uses environment variables. Neither do I. They really only make sense in the context of a request which was actually to a CGI script resident in a document root - they're an exceptionally awkward way of describing other HTTP requests, especially ones which aren't being served from a document root. And there's a lot of information lost in translation, like the order and original capitalization of HTTP headers. (Not that these things are supposed to matter, but still.)

Tepix 5mo ago

I‘ve had good experiences with FastCGI back when Perl was popular. These days, WebTransport is the new sexy thing. Probably not a real FastCGI replacement.

athrowaway3z 5mo ago

This seems like really bad advice or am i missing something? Using fastcgi requires you write your app to serve fastcgi. The upside of serving http/1.1 instead of fastcgi is that devs can instantly use their browser to test things instead of having to setup a reverse proxy on their machine. The bad parts of http/1.1 are fixed equally well by both http/2.0 and fastcgi. So just use http/2.0 and you get the proper framing as well as browser support.

agwa 5mo ago
Please see the section about untrusted headers - this is not fixed by HTTP/2. You're right that being able to point your browser right at the app is very convenient. With Go, you can have a command line flag that switches between http.Serve (for development) and fcgi.Serve (for production).
- enneff 5mo ago
  In my experience having different serving paths for dev vs production is a recipe for annoying issues. I try to make dev as similar to prod as possible. I’m not sure, I don’t dismiss fcgi outright here, I find the arguments for it compelling (not a huge fan of http for many reasons) but it has to be really worth it to break the consistency of using http everywhere.
  - agwa 5mo ago
    If you want your dev environment to be as similar to prod as possible, and you use a proxy in prod, then you should use a proxy in dev also. I was presenting a solution to someone who doesn't want to do that.
    - enneff 5mo ago
      I think perhaps I was unclear. I don’t mean the entire dev environment should mirror prod (although it’s great if you can do this for end to end testing). I just mean it’s desirable if the process you’re working on operates the same way in dev as in prod.

scotty79 5mo ago

What I'd like to see is someone creating local caching proxy for modern https infested world. I'm fed up with downloading same packages 100 times.

ebb_earl_co 5mo ago
I’m in the same boat. Am trying to figure out how to configure Vinyl cache (née Varnish) in my home lab.
majorchord 5mo ago
Can't squid do this?
- scotty79 5mo ago
  Not easily. I always dread setting it up for this. I'd love to have one click solution with nice ui.
xorcist 5mo ago
There is no way to do what you ask for without having a local CA. And once you have it, any existing caching proxy will work. Now you have moved all security to the proxy, but for some applications it is worth it.

apitman 5mo ago

> That said, using a vintage technology has some downsides. It was never updated to support WebSockets With widespread browser support for WHATWG streams, it's pretty easy to implement your own WebSockets over long-lived HTTP requests. Basically you just send a byte stream and prepend each message with a header, which can just be a size in many cases. Advantages over WebSockets: * No special path in your server layer like you need for WebSocket. * Backpressure * You get to take advantage of HTTP/2/3 improvements for free * Lower framing overhead Unfortunately AFAIK it's still not supported to still be streaming your request body while receiving the response, so you need a pair of requests for full bidirectional streaming.

yesbabyyes 5mo ago
Please be aware that there is a web standard for this since quite some time. See server-sent events and the EventSource interface: https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events https://developer.mozilla.org/en-US/docs/Web/API/Server-sent... https://developer.mozilla.org/en-US/docs/Web/API/EventSource https://developer.mozilla.org/en-US/docs/Web/API/EventSource
- Oxodao 5mo ago
  That can be used with https://mercure.rocks https://mercure.rocks :)
- apitman 5mo ago
  Not the same. SSE doesn't support binary, and has higher framing overhead than you can achieve with raw HTTP. If you don't need binary it's a solid option in the browser.

daneel_w 5mo ago

I've built a lot of API backends with Perl and FCGI::ProcManager, letting nginx (and Apache HTTPd in the past) front everything. For me it has been a pleasantly simple, incredibly robust and high-performing setup with no mess to speak of.

Animats 5mo ago

FCGI is also an orchestration system. It launches more server tasks when the load goes up, shuts them down when the load decreases, and launches new copies of tasks if they crash. It's like single-system Kubernetes.

toast0 5mo ago
> It launches more server tasks when the load goes up, shuts them down when the load decreases In my experience, this isn't a good feature. It sounds nice, but it can often mean everything runs fine while your load is low, but when your load gets high, you spawn more workers and run out of memory. It's much better to have a static number of workers in my experience. Crash recovery is handy, if needed though.
- Animats 5mo ago
  It's useful if you have multiple FCGI programs that handle different kinds of requests. Depending on what's being requested, programs start up and shut down.
  - ramses0 5mo ago
    I had to argue about inetd while poking at AI stuff on my home computer. https://en.wikipedia.org/wiki/Inetd https://en.wikipedia.org/wiki/Inetd ... yeah, there's better modern equivalents, but it seems like we're coming full circle with lambdas and cgi. The concept of dynamically scaling has been around a looong time in UNIX, and we lost it for a bit with big honking java monolith servers.
deleted 5mo ago
[deleted]
assimpleaspossi 5mo ago
This is exactly how we used it.

blipvert 5mo ago

(u)WSGI must surely get a mention here?!

runxiyu 5mo ago

I think there is a lot of merit to this argument, however, FastCGI defers to CGI/1.1 for `PATH_INFO`, etc., which is lossy as it must be URL-decoded and therefore cannot represent encoded slashes, `%2F`. (Some implementations also collapse `//` to `/` in path, but this is an issue in various HTTP implementations too.) It is less expressive than HTTP in ways that may or may not be important to your application; I prefer accurate URL handling.

xp84 5mo ago

> Only if True-Client-IP doesn't exist does it use X-Real-IP. So even if your proxy does the right thing with X-Real-IP, you can still be pwned by an attacker sending a True-Client-IP header. Can we just take a moment to appreciate the absurdity of HTTP headers for a moment? We have X-Forwarded-For, X-Real-IP, each CDN has their own custom flavored one. Some of them are a comma-separated list, and usually ends up having an IP of your own LB uselessly added in there (I know why, it's just not helpful). All of them might be inserted by a malicious user-agent. I guess nobody could agree on how all the various trusted servers in the pipeline should convey the important bit. I guess it fits in quite well with the absurdity of the User-Agent header, which has come so far in absurdity that Apple decided to fully kill it by just sending utterly fake nonsense (false OS version, etc) in the name of "pRiVaCy."

est 5mo ago

Then there's uwsgi protocol. It's also an RPC for basically everything.

teddy_oweh 5mo ago

[flagged]

thayne 5mo ago

The untrusted header problem could potentially be fixed by having the reverse proxy embed all the trusted information in a specific header, and then it just has to make sure that one header is stripped from the request. Unfortunately, there isn't (yet) a standard for that. Or you could use something like haproxy's proxy protocol (although that may not support all the information you want, and doesn't work for multiplexing). Edit: actually the "Forwarded" header kind of fills that niche. Although you may want extensions for things like the client certificate.

XYen0n 5mo ago
Unfortunately, it appeared too late, and the relevant support is now far less complete than that for `X-Forwarded-*`.
max_k 5mo ago
FastCGI has "parameters" and HTTP headers are special parameters starting with "HTTP_" (mimicking CGI's environment variables). All parameters not starting with "HTTP_" can be trusted because only the web server (= FastCGI client) can construct them.

verifex 5mo ago

I was reading the article and thinking, I wonder what that proxy I'm using in Apache is using, it's really fast and I've had a lot of luck with it. OMG, so I've been using FastCGI all this time and had no idea, well that's awesome. :)

novoreorx 5mo ago

FastCGI is theoretically better does not make it the easier choice in reality, the success of HTTP is just another case of "worse is better"

tlavoie 5mo ago
No, in the case of all the desync attacks that James Kettle has found since, HTTP between servers is more like "worse really is worse." At least until they all speak HTTP/2.

xorcist 5mo ago

The mystery is why uWSGI isn't more widely used. Perhaps the name does not help. It has little to do with WSGI just as FastCGI is unrelated to CGI. It is a tiny binary protocol, with frames just as FastCGI. The reference server works with several languages, I've used it over the years mostly with Python but also Ruby and Perl. It is a small C executable with all the practical features one need for web hosting: Draining backends, autoscaling, logging, chrooted backends, everything. Very few FastCGI servers are this mature. Unlike FastCGI, it has been extended to support websockets and async. I have used it in production at several places for many years and have nothing but praise for it. It feels like this weird unknown secret for web operations. Unfortunately, it sees lesser use now in the cloud era, and development seems to have all but stopped. It still works and is still reliable but the writing is probably on the wall. However nothing comes close in terms of speed, simplicity, and features.

LAC-Tech 5mo ago

I am actually implementing a reverse proxy right now... I am doing a typical http thing, but I wonder, has anyone used fastcgi in Caddy? https://caddyserver.com/docs/caddyfile/directives/reverse_proxy#the-fastcgi-transport https://caddyserver.com/docs/caddyfile/directives/reverse_pr...

zokier 5mo ago

Does anyone remember mongrel2? A neat web server that used zeromq for backend communication (based on scgi). I always felt it had huge potential. It was also kinda famous in the early days of HN, back in the days when RoR was in vogue etc. https://hn.algolia.com/?q=mongrel2 https://hn.algolia.com/?q=mongrel2

caycecan 5mo ago

I'm here looking to speed up rendering.

_stiletto_ 5mo ago

[dead]