[graylog2] how to resolve issue with indexer

[Jiří Kolb]

Hello,
I have following architecture Logstash -> RabbitMQ ->Graylog

I have issue in indexer with parsing:
17 minutes ago graylog_0 d58fe350-f41b-11e6-8407-000c29438b97
MapperParsingException[failed to parse [level]]; nested:
NumberFormatException[For input string: "information"]

In logstash debug I can see message like this:

{
"devid" => "FGVMEV0000000000",
"raw_message" => "date=2017-02-16 time=00:55:46
devname=FortiGate-VM64 devid=FGVMEV0000000000 logid=0100040704 type=event
subtype=system level=notice vd=root logdesc=\"System performance
statistics\" action=\"perf-stats\" cpu=0 mem=47 totalsession=3 disk=0
bandwidth=0/0 setuprate=0 disklograte=0 fazlograte=0 msg=\"Performance
statistics: average CPU: 0, memory: 47, concurrent sessions: 3,
setup-rate: 0\"",
"fazlograte" => "0",
"type" => "fortigate",
"mem" => "47",
"syslog5424_pri" => "189",
"@version" => "1",
"host" => "10.0.0.90",
"action" => "perf-stats",
"disklograte" => "0",
"device_id" => "FortiGate-VM64",
"level" => "notice",
"bandwidth" => "0/0",
"cpu" => "0",
"<189>date" => "2017-02-16",
"message" => "<189>date=2017-02-16 time=00:55:46
devname=FortiGate-VM64 devid=FGVMEV0000000000 logid=0100040704 type=event
subtype=system level=notice vd=root logdesc=\"System performance
statistics\" action=\"perf-stats\" cpu=0 mem=47 totalsession=3 disk=0
bandwidth=0/0 setuprate=0 disklograte=0 fazlograte=0 msg=\"Performance
statistics: average CPU: 0, memory: 47, concurrent sessions: 3,
setup-rate: 0\"",
"vd" => "root",
"logdesc" => "System performance statistics",
"disk" => "0",
"@timestamp" => 2017-02-16T07:55:39.383Z,
"setuprate" => "0",
"logid" => "0100040704",
"time" => "00:55:46",
"totalsession" => "3"
}



Can you please help? I do not understand why level is not parsed it is
string like others that are parsed correctly.

Thank you!

Jiri

[Jochen Schalanda]

Hi Jiří,

the "level" message field has to be a numeric value, i. e. the numeric
severity level of syslog
messages: https://en.wikipedia.org/wiki/Syslog#Severity_level

You can use message processor pipeline rules to change that in
Graylog: http://docs.graylog.org/en/2.2/pages/pipelines.html

Although in your specific case, I'd recommend converting the "level" field
in Logstash.

Cheers,
Jochen

Post by Jiří Kolb
Hello,
I have following architecture Logstash -> RabbitMQ ->Graylog
17 minutes ago graylog_0 d58fe350-f41b-11e6-8407-000c29438b97
NumberFormatException[For input string: "information"]
{
"devid" => "FGVMEV0000000000",
"raw_message" => "date=2017-02-16 time=00:55:46
devname=FortiGate-VM64 devid=FGVMEV0000000000 logid=0100040704 type=event
subtype=system level=notice vd=root logdesc=\"System performance
statistics\" action=\"perf-stats\" cpu=0 mem=47 totalsession=3 disk=0
bandwidth=0/0 setuprate=0 disklograte=0 fazlograte=0 msg=\"Performance
statistics: average CPU: 0, memory: 47, concurrent sessions: 3,
setup-rate: 0\"",
"fazlograte" => "0",
"type" => "fortigate",
"mem" => "47",
"syslog5424_pri" => "189",
"host" => "10.0.0.90",
"action" => "perf-stats",
"disklograte" => "0",
"device_id" => "FortiGate-VM64",
"level" => "notice",
"bandwidth" => "0/0",
"cpu" => "0",
"<189>date" => "2017-02-16",
"message" => "<189>date=2017-02-16 time=00:55:46
devname=FortiGate-VM64 devid=FGVMEV0000000000 logid=0100040704 type=event
subtype=system level=notice vd=root logdesc=\"System performance
statistics\" action=\"perf-stats\" cpu=0 mem=47 totalsession=3 disk=0
bandwidth=0/0 setuprate=0 disklograte=0 fazlograte=0 msg=\"Performance
statistics: average CPU: 0, memory: 47, concurrent sessions: 3,
setup-rate: 0\"",
"vd" => "root",
"logdesc" => "System performance statistics",
"disk" => "0",
"setuprate" => "0",
"logid" => "0100040704",
"time" => "00:55:46",
"totalsession" => "3"
}
Can you please help? I do not understand why level is not parsed it is
string like others that are parsed correctly.
Thank you!
Jiri

[Jiří Kolb]

Thank you! Resolved with renaming level to log_level in logstash
configuration

rename => [ "level" , "log_level" ]

Regards,
Jiri

Post by Jochen Schalanda
Hi Jiří,
the "level" message field has to be a numeric value, i. e. the numeric
https://en.wikipedia.org/wiki/Syslog#Severity_level
http://docs.graylog.org/en/2.2/pages/pipelines.html
Although in your specific case, I'd recommend converting the "level" field
in Logstash.
Cheers,
Jochen

Post by Jiří Kolb
Hello,
I have following architecture Logstash -> RabbitMQ ->Graylog
17 minutes ago graylog_0 d58fe350-f41b-11e6-8407-000c29438b97
NumberFormatException[For input string: "information"]
{
"devid" => "FGVMEV0000000000",
"raw_message" => "date=2017-02-16 time=00:55:46
devname=FortiGate-VM64 devid=FGVMEV0000000000 logid=0100040704 type=event
subtype=system level=notice vd=root logdesc=\"System performance
statistics\" action=\"perf-stats\" cpu=0 mem=47 totalsession=3 disk=0
bandwidth=0/0 setuprate=0 disklograte=0 fazlograte=0 msg=\"Performance
statistics: average CPU: 0, memory: 47, concurrent sessions: 3,
setup-rate: 0\"",
"fazlograte" => "0",
"type" => "fortigate",
"mem" => "47",
"syslog5424_pri" => "189",
"host" => "10.0.0.90",
"action" => "perf-stats",
"disklograte" => "0",
"device_id" => "FortiGate-VM64",
"level" => "notice",
"bandwidth" => "0/0",
"cpu" => "0",
"<189>date" => "2017-02-16",
"message" => "<189>date=2017-02-16 time=00:55:46
devname=FortiGate-VM64 devid=FGVMEV0000000000 logid=0100040704 type=event
subtype=system level=notice vd=root logdesc=\"System performance
statistics\" action=\"perf-stats\" cpu=0 mem=47 totalsession=3 disk=0
bandwidth=0/0 setuprate=0 disklograte=0 fazlograte=0 msg=\"Performance
statistics: average CPU: 0, memory: 47, concurrent sessions: 3,
setup-rate: 0\"",
"vd" => "root",
"logdesc" => "System performance statistics",
"disk" => "0",
"setuprate" => "0",
"logid" => "0100040704",
"time" => "00:55:46",
"totalsession" => "3"
}
Can you please help? I do not understand why level is not parsed it is
string like others that are parsed correctly.
Thank you!
Jiri