16 ms·

Although logstash does have a built in elasticsearch, I wouldn't really say anyone uses logstash itself to provide search for the logs. Logstash itself just pr

by adient 14y ago

Although logstash does have a built in elasticsearch, I wouldn't really say anyone uses logstash itself to provide search for the logs. Logstash itself just provides a way to move events from one place to another, that's all.

kordless 14y ago
Right! Speaking of that, have you looked at Grok (field extractor)? It's pretty awesome.
- adient 14y ago
  Grok, grep, multiline, date, etc. Logstash changes my events prior to reaching their destination, but logstash itself is not the end destination.
sciurus 14y ago
"I wouldn't really say anyone uses logstash itself to provide search for the logs" Huh? The front page of http://logstash.net/ http://logstash.net/ suggests that one of the primary uses! "logstash is a tool for managing events and logs. You can use it to collect logs, parse them, and store them for later use (like, for searching). Speaking of searching, logstash comes with a web interface for searching and drilling into all of your logs. All your logs from all over your infrastructure in one place - with searching and graphing. Since we can easily parse text-based logs, you can query for more precise things like, all 404 http errors, nagios critical alerts in hard state, or mail server faults - all without accidentally finding logs with the word ‘404’ or ‘critical’ in the wrong place."
- adient 14y ago
  Elasticsearch, the recommended backend for making your logs searchable, is a separate project from logstash. Logstash does come with a built in elasticsearch, designed to get people up and running very quickly, but if you are considering any serious use of elasticsearch you would set it up yourself as a standalone service. Logstash does come with a simple web interface, and kibana is a slightly better but still simple interface being ported into logstash. Again this is geared towards getting people up and running quickly, and at the end of the day it's just a pretty curl wrapper for elasticsearch. You can also use logstash without elasticsearch/kibana, which we do for a good bit of our logs. I think logstash intentionally blurs the lines of what it is or isn't so people don't get caught up in trying to figure out how to get it running. Give it a try and see for yourself exactly what it is or isn't.

by adient 14y ago

kordless 14y ago

Right! Speaking of that, have you looked at Grok (field extractor)? It's pretty awesome.

adient 14y ago
Grok, grep, multiline, date, etc. Logstash changes my events prior to reaching their destination, but logstash itself is not the end destination.

sciurus 14y ago

"I wouldn't really say anyone uses logstash itself to provide search for the logs" Huh? The front page of http://logstash.net/ http://logstash.net/ suggests that one of the primary uses! "logstash is a tool for managing events and logs. You can use it to collect logs, parse them, and store them for later use (like, for searching). Speaking of searching, logstash comes with a web interface for searching and drilling into all of your logs. All your logs from all over your infrastructure in one place - with searching and graphing. Since we can easily parse text-based logs, you can query for more precise things like, all 404 http errors, nagios critical alerts in hard state, or mail server faults - all without accidentally finding logs with the word ‘404’ or ‘critical’ in the wrong place."

adient 14y ago
Elasticsearch, the recommended backend for making your logs searchable, is a separate project from logstash. Logstash does come with a built in elasticsearch, designed to get people up and running very quickly, but if you are considering any serious use of elasticsearch you would set it up yourself as a standalone service. Logstash does come with a simple web interface, and kibana is a slightly better but still simple interface being ported into logstash. Again this is geared towards getting people up and running quickly, and at the end of the day it's just a pretty curl wrapper for elasticsearch. You can also use logstash without elasticsearch/kibana, which we do for a good bit of our logs. I think logstash intentionally blurs the lines of what it is or isn't so people don't get caught up in trying to figure out how to get it running. Give it a try and see for yourself exactly what it is or isn't.