Organizational Research By

Surprising Reserch Topic

Experts Most Trusted Topic


not able to filter ganglia udp data into elasticseach using -'elasticsearch,logstash,ganglia'


not able to filter ganglia udp data into elasticseach  using -'elasticsearch,logstash,ganglia'

The objective of this exercise is to visualise ganglia gmond output in kibana.
I am trying to filter out certian lines from ganglia gmond though logstash.

If i try listening to udp port in logstash and apply the filter, no messages are filtered. However my filter is correct, because when i try using logstash generated text file output of gmond, instead of directly listening to gmond on udp, i am able filter the message.

To solve this issue, i have used to conf files.
1. Logstash listens  to gmond on upd port and output the resluts  to a file
2. Logstash uses the generated file as input, applies filters on it and then sends the output to elasticsearch.
This would make it more clear
Conf1

input {  
      ganglia {
            port => "8686"
            type => "ganglia"
           }
}
output {
    file {
            path => "/tmp/elastic_gmond-log.txt"
    }
 }


Conf2

input {
      file {
          path => "/tmp/elastic_gmond-log.txt"
          }
      }
filter {
       # bacsically filtering all the string values so that i can visusalize other float values in kibana
         if  ([message] =~ /machine_type/ or [message] =~ /os_name/ or [message] =~ /location/ or [message] =~ /os_release/ ) {
         drop{}
       }
      }
output {
       file {
            path => "/tmp/gmond-log.txt"
       }
       elasticsearch {
                     cluster => elasticsearch
                     port => 9300
                     index => "ganglialog-%{+YYYY.MM.dd}"
                    }
      }


However my logstash is dying as soon as i restart the gmond

Logstash startup completed
Exception in thread ">output" java.lang.UnsupportedOperationException
    at java.lang.Thread.stop(Thread.java:869)
    at org.jruby.RubyThread.exceptionRaised(RubyThread.java:1221)
    at org.jruby.internal.runtime.RubyRunnable.run(RubyRunnable.java:112)
    at java.lang.Thread.run(Thread.java:745)


The command used is

./logstash  --config /etc/logstash/conf.d/


How do get this thing working? Is there an  easlier/better alternative?
    

asked Sep 7, 2015 by rajesh
0 votes
19 views



Related Hot Questions



Walkin Jobs Opening



Government Jobs Opening


...