6 ms·

Show HN: Real-time, top-like metrics for Nginx

134 points by lebinh 13y ago

sebslomski 13y ago
Cool! Why is this not on pypi?
- lebinh 13y ago
  Thanks, I'll put it on pypi soon, just put it on github today
  - HeyImAlex 13y ago
    Just a heads up, you're going to want to translate your readme to rst if you want it to render right on pypi.
korzun 13y ago
Am I the only one that bothered when tools like this do not give you some insight on resource consumption/limit? Even if it's just parsing logs via sqlite, I still want to see the limitations and X per Y usage metrics to see if it can handle monitoring high traffic domain without impacting performance.
- lebinh 13y ago
  The performance impact really depend on your server and traffics. Currently with high traffic server it will take quite some amount of CPU if running for awhile, after 15-20 mins or so, but I'm working on reduce that. So now it mostly useful to know what's going on with your nginx at the moment but not over a long period.
- evilduck 13y ago
  You're seriously bothered about the robustness of a 2 hour old project?
  - jamra 13y ago
    Well, if you're announcing a project that monitors a performance critical part of your infrastructure, it seems fairly appropriate to performance test your product.
    - w-ll 13y ago
      There is a difference between a project and a product.
- clubhi 13y ago
  You aren't the only one. I often am bothered by the limitations of my own project. I rarely need outsiders to point out the major issues because I know of so many myself. Some people consider it a talent to blurt out obvious observations though.
yeukhon 13y ago
Side question: I saw a project similar to this, but display top in your terminal/browser with a beautiful visual dashboard. I remember seeing it here on HN recently but I can't find it. I mention that because it has a non-ascii visual component. I am interested in looking at the two source codes and see if I could make pretty dashboard out of it... EDIT: Thanks for finding the, HNers. I also think it is possible to look at IPythoNoteBook integration.
- bearlikelion 13y ago
  The only other top I know of is htop http://hisham.hm/htop/ http://hisham.hm/htop/ but I don't think that's what you're looking for
- dewey 13y ago
  Maybe that one? https://github.com/Unitech/pm2 https://github.com/Unitech/pm2
- nullwagon 13y ago
  Fulfills the browser aspect of the question...this? http://scoutapp.github.io/scout_realtime/ http://scoutapp.github.io/scout_realtime/
  - yeukhon 13y ago
    Yes. This one. Thanks. Sorry for hijacking the thread. But trying to gather a list of metric tools I can try.
- brickcap 13y ago
  could it be go access? http://goaccess.prosoftcorp.com/ http://goaccess.prosoftcorp.com/
KevinBongart 13y ago
Looks cool! Installation instructions would be a nice plus
- lebinh 13y ago
  Thanks, I'll definitely add that, maybe put this on pypi asap. For now you can clone the repo and then run pip install -r requirements.txt to install required dependencies.
  - diziet 13y ago
    Doesn't seem to like to read the logs : ./ngxtop.py -l /opt/nginx/logs/access.log ## just pretends like the log is not being written to For some reason the access log file is defaulted to logs/access.log too~
    - bndr 13y ago
      There was a bug that was overwriting --access-log (-l) option. Should be fixed now in master
jmngomes 13y ago
This is nice, congrats! Would also like some performance metrics, or at least exec time per script
- lebinh 13y ago
  If what you mean by exec time is request_time, i.e., serving time of a request, then it's pretty close on my agenda as long as you have it in your access log :)
  - jmngomes 13y ago
    Yes, an average serving time per resource request would definitely be nice :)
scrollaway 13y ago
Very cool tool. It seems to be clearing the screen every time it runs though which isn't great, especially when it crashes that hides the backtrace. Consider using ncurses? On a different note... developers, please use Python 3. Several Linux distributions are now using Py3 as their main python and those scripts with an "env python" hashbang will not work. Porting your codebase to be python 2+3 compatible is dead simple, and most of the time it is just a matter of using print("foo") instead of print "foo". So please be considerate.
- lebinh 13y ago
  Unfortunately most of the servers I'm working with are still on python2 by default. The latest Ubuntu LTS, 12.04, only has python2 by default so python2 is still my priority. But in this case, a port should be trivia so I'll look into python3 as soon as possible. Yes, ncurses would be much better but I haven't had time for it yet. Clearing screen is simple enough and work pretty well for me so far so here we are :)
  - mappu 13y ago
    It's a release goal for 14.04 (due in about 30 days) to have no python2 software in the default desktop/server installs. Seems like there's still lots to be done though. https://wiki.ubuntu.com/Python/3 https://wiki.ubuntu.com/Python/3
- stefantalpalaru 13y ago
  > Several Linux distributions are now using Py3 as their main python Only Arch Linux does it.
kureikain 13y ago
How about porting it to LUA and embed directly in nginx?
ProAm 13y ago
Does it matter which version of python we use?
- sc00ty 13y ago
  Seems like it was made with 2. Will not run on 3.
cheald 13y ago
Nice. You can get similar for any HTTP interface via Varnish and Varnishtop, if you're interested, as well.
babuskov 13y ago
For those who didn't know about it, please check Apachetop: https://github.com/JeremyJones/Apachetop https://github.com/JeremyJones/Apachetop I use it for both Apache and Nginx.
dougcorrea 13y ago
Very cool put it on pypi it will be useful with my blogs in aws
jedicoffee 13y ago
I think this is a great idea. Very useful, thank you.

134 points by lebinh 13y ago

sebslomski 13y ago

Cool! Why is this not on pypi?

lebinh 13y ago
Thanks, I'll put it on pypi soon, just put it on github today
- HeyImAlex 13y ago
  Just a heads up, you're going to want to translate your readme to rst if you want it to render right on pypi.

korzun 13y ago

Am I the only one that bothered when tools like this do not give you some insight on resource consumption/limit? Even if it's just parsing logs via sqlite, I still want to see the limitations and X per Y usage metrics to see if it can handle monitoring high traffic domain without impacting performance.

lebinh 13y ago
The performance impact really depend on your server and traffics. Currently with high traffic server it will take quite some amount of CPU if running for awhile, after 15-20 mins or so, but I'm working on reduce that. So now it mostly useful to know what's going on with your nginx at the moment but not over a long period.
evilduck 13y ago
You're seriously bothered about the robustness of a 2 hour old project?
- jamra 13y ago
  Well, if you're announcing a project that monitors a performance critical part of your infrastructure, it seems fairly appropriate to performance test your product.
  - w-ll 13y ago
    There is a difference between a project and a product.
clubhi 13y ago
You aren't the only one. I often am bothered by the limitations of my own project. I rarely need outsiders to point out the major issues because I know of so many myself. Some people consider it a talent to blurt out obvious observations though.

yeukhon 13y ago

Side question: I saw a project similar to this, but display top in your terminal/browser with a beautiful visual dashboard. I remember seeing it here on HN recently but I can't find it. I mention that because it has a non-ascii visual component. I am interested in looking at the two source codes and see if I could make pretty dashboard out of it... EDIT: Thanks for finding the, HNers. I also think it is possible to look at IPythoNoteBook integration.

bearlikelion 13y ago
The only other top I know of is htop http://hisham.hm/htop/ http://hisham.hm/htop/ but I don't think that's what you're looking for
dewey 13y ago
Maybe that one? https://github.com/Unitech/pm2 https://github.com/Unitech/pm2
nullwagon 13y ago
Fulfills the browser aspect of the question...this? http://scoutapp.github.io/scout_realtime/ http://scoutapp.github.io/scout_realtime/
- yeukhon 13y ago
  Yes. This one. Thanks. Sorry for hijacking the thread. But trying to gather a list of metric tools I can try.
brickcap 13y ago
could it be go access? http://goaccess.prosoftcorp.com/ http://goaccess.prosoftcorp.com/

KevinBongart 13y ago

Looks cool! Installation instructions would be a nice plus

lebinh 13y ago
Thanks, I'll definitely add that, maybe put this on pypi asap. For now you can clone the repo and then run pip install -r requirements.txt to install required dependencies.
- diziet 13y ago
  Doesn't seem to like to read the logs : ./ngxtop.py -l /opt/nginx/logs/access.log ## just pretends like the log is not being written to For some reason the access log file is defaulted to logs/access.log too~
  - bndr 13y ago
    There was a bug that was overwriting --access-log (-l) option. Should be fixed now in master

jmngomes 13y ago

This is nice, congrats! Would also like some performance metrics, or at least exec time per script

lebinh 13y ago
If what you mean by exec time is request_time, i.e., serving time of a request, then it's pretty close on my agenda as long as you have it in your access log :)
- jmngomes 13y ago
  Yes, an average serving time per resource request would definitely be nice :)

scrollaway 13y ago

Very cool tool. It seems to be clearing the screen every time it runs though which isn't great, especially when it crashes that hides the backtrace. Consider using ncurses? On a different note... developers, please use Python 3. Several Linux distributions are now using Py3 as their main python and those scripts with an "env python" hashbang will not work. Porting your codebase to be python 2+3 compatible is dead simple, and most of the time it is just a matter of using print("foo") instead of print "foo". So please be considerate.

lebinh 13y ago
Unfortunately most of the servers I'm working with are still on python2 by default. The latest Ubuntu LTS, 12.04, only has python2 by default so python2 is still my priority. But in this case, a port should be trivia so I'll look into python3 as soon as possible. Yes, ncurses would be much better but I haven't had time for it yet. Clearing screen is simple enough and work pretty well for me so far so here we are :)
- mappu 13y ago
  It's a release goal for 14.04 (due in about 30 days) to have no python2 software in the default desktop/server installs. Seems like there's still lots to be done though. https://wiki.ubuntu.com/Python/3 https://wiki.ubuntu.com/Python/3
stefantalpalaru 13y ago
> Several Linux distributions are now using Py3 as their main python Only Arch Linux does it.

kureikain 13y ago

How about porting it to LUA and embed directly in nginx?

ProAm 13y ago

Does it matter which version of python we use?

sc00ty 13y ago
Seems like it was made with 2. Will not run on 3.

cheald 13y ago

Nice. You can get similar for any HTTP interface via Varnish and Varnishtop, if you're interested, as well.

babuskov 13y ago

For those who didn't know about it, please check Apachetop: https://github.com/JeremyJones/Apachetop https://github.com/JeremyJones/Apachetop I use it for both Apache and Nginx.

dougcorrea 13y ago

Very cool put it on pypi it will be useful with my blogs in aws

jedicoffee 13y ago

I think this is a great idea. Very useful, thank you.