Discussion:
[graylog2] rsyslogd Structured data
Lily Chadha
2015-03-05 12:00:35 UTC
Permalink
Hi,

i am new to syslog module.i am trying to log STRUCTURED DATA in log
file.But when i am using this parameter in template it is showing null
value.i am using rsyslogd 5.8.11.
this below template i am using to log messages.

"<%PRI%>%TIMESTAMP:::date-rfc3339%%HOSTNAME%%syslogtag%%APP-NAME%%PROCID%%MSGID% %msg% %STRUCTURED-DATA%\n”

i am getting message like this:

<142> 2015-03-05T06:55:39.816659-05:00 host login[1986]: login 1986 - [1986 : 1986 INFO]SERIAL Login from IP:127.0.0.3 user:sysadmin -but i want message with structure data
--
You received this message because you are subscribed to the Google Groups "graylog2" group.
To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+***@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Jochen Schalanda
2015-03-09 09:05:15 UTC
Permalink
Hello Lily,

Graylog currently supports structured syslog messages according to RFC 5424
<https://tools.ietf.org/html/rfc5424#section-6.3>. As far as I can see
there are several things missing or not according to the spec in the syslog
message format you're using.

Please try using the syslog message format described at
http://docs.graylog.org/en/1.0/pages/sending_data.html#rsyslog to get
structured syslog data from rsyslog into Graylog.


Cheers,
Jochen
Post by Lily Chadha
Hi,
i am new to syslog module.i am trying to log STRUCTURED DATA in log
file.But when i am using this parameter in template it is showing null
value.i am using rsyslogd 5.8.11.
this below template i am using to log messages.
"<%PRI%>%TIMESTAMP:::date-rfc3339%%HOSTNAME%%syslogtag%%APP-NAME%%PROCID%%MSGID% %msg% %STRUCTURED-DATA%\n”
<142> 2015-03-05T06:55:39.816659-05:00 host login[1986]: login 1986 - [1986 : 1986 INFO]SERIAL Login from IP:127.0.0.3 user:sysadmin -but i want message with structure data
--
You received this message because you are subscribed to the Google Groups "graylog2" group.
To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+***@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Lily Chadha
2015-05-27 14:08:23 UTC
Permalink
Hi,

Can you tell me ,is the structured-data related things are sent by syslog
API as an argument or these are automatically generated by daemon ?
Post by Lily Chadha
Hi,
i am new to syslog module.i am trying to log STRUCTURED DATA in log
file.But when i am using this parameter in template it is showing null
value.i am using rsyslogd 5.8.11.
this below template i am using to log messages.
"<%PRI%>%TIMESTAMP:::date-rfc3339%%HOSTNAME%%syslogtag%%APP-NAME%%PROCID%%MSGID% %msg% %STRUCTURED-DATA%\n”
<142> 2015-03-05T06:55:39.816659-05:00 host login[1986]: login 1986 - [1986 : 1986 INFO]SERIAL Login from IP:127.0.0.3 user:sysadmin -but i want message with structure data
--
You received this message because you are subscribed to the Google Groups "graylog2" group.
To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+***@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Jochen Schalanda
2015-05-27 14:14:33 UTC
Permalink
Hi Lily,

the structured data fields inside the syslog messages should be
generated/populated by rsyslog.


Cheers,
Jochen
Post by Lily Chadha
Hi,
Can you tell me ,is the structured-data related things are sent by syslog
API as an argument or these are automatically generated by daemon ?
Post by Lily Chadha
Hi,
i am new to syslog module.i am trying to log STRUCTURED DATA in log
file.But when i am using this parameter in template it is showing null
value.i am using rsyslogd 5.8.11.
this below template i am using to log messages.
"<%PRI%>%TIMESTAMP:::date-rfc3339%%HOSTNAME%%syslogtag%%APP-NAME%%PROCID%%MSGID% %msg% %STRUCTURED-DATA%\n”
<142> 2015-03-05T06:55:39.816659-05:00 host login[1986]: login 1986 - [1986 : 1986 INFO]SERIAL Login from IP:127.0.0.3 user:sysadmin -but i want message with structure data
--
You received this message because you are subscribed to the Google Groups "graylog2" group.
To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+***@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Lily Chadha
2015-05-28 05:33:14 UTC
Permalink
Hi Jochen,

Thanks for your quick reply.Do i need to define these things in some
rsyslog source file?Can i get some idea how these value will be generated?

Thanks,
--Lily
Post by Lily Chadha
Hi,
i am new to syslog module.i am trying to log STRUCTURED DATA in log
file.But when i am using this parameter in template it is showing null
value.i am using rsyslogd 5.8.11.
this below template i am using to log messages.
"<%PRI%>%TIMESTAMP:::date-rfc3339%%HOSTNAME%%syslogtag%%APP-NAME%%PROCID%%MSGID% %msg% %STRUCTURED-DATA%\n”
<142> 2015-03-05T06:55:39.816659-05:00 host login[1986]: login 1986 - [1986 : 1986 INFO]SERIAL Login from IP:127.0.0.3 user:sysadmin -but i want message with structure data
--
You received this message because you are subscribed to the Google Groups "graylog2" group.
To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+***@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Jochen Schalanda
2015-05-28 08:48:36 UTC
Permalink
Hi Lily,

please refer to the rsyslog documentation for examples:

-
http://www.rsyslog.com/doc/v8-stable/configuration/modules/mmnormalize.html
-
http://www.rsyslog.com/doc/v8-stable/configuration/modules/mmjsonparse.html
- http://www.rsyslog.com/tag/structured-data/


Cheers,
Jochen
Post by Lily Chadha
Hi Jochen,
Thanks for your quick reply.Do i need to define these things in some
rsyslog source file?Can i get some idea how these value will be generated?
Thanks,
--Lily
Post by Lily Chadha
Hi,
i am new to syslog module.i am trying to log STRUCTURED DATA in log
file.But when i am using this parameter in template it is showing null
value.i am using rsyslogd 5.8.11.
this below template i am using to log messages.
"<%PRI%>%TIMESTAMP:::date-rfc3339%%HOSTNAME%%syslogtag%%APP-NAME%%PROCID%%MSGID% %msg% %STRUCTURED-DATA%\n”
<142> 2015-03-05T06:55:39.816659-05:00 host login[1986]: login 1986 - [1986 : 1986 INFO]SERIAL Login from IP:127.0.0.3 user:sysadmin -but i want message with structure data
--
You received this message because you are subscribed to the Google Groups "graylog2" group.
To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+***@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Lily Chadha
2015-05-28 11:55:35 UTC
Permalink
Hi Jochen,

After loading mmjsonparse,i am still getting structured data empty.Here is
my debug log:

3818.712254754:main Q:Reg/w0 : ACTION 0 [mmjsonparse::mmjsonparse:]
3818.712262545:main Q:Reg/w0 : executing action 0
3818.712267008:main Q:Reg/w0 : Called action, logging to mmjsonparse
3818.712271907:main Q:Reg/w0 : wti 0x99ef6e0: we need to create a new
action worker instance for action 0
3818.712293355:main Q:Reg/w0 : DDDD: writing data to table spot 0
3818.712297580:main Q:Reg/w0 : wti 0x99ef6e0: created action worker
instance 1 for action 0
3818.712301394:main Q:Reg/w0 : Action 0 transitioned to state: itx
3818.712305432:main Q:Reg/w0 : entering actionCalldoAction(), state: itx,
actionNbr 0
3818.712314391:main Q:Reg/w0 : mmjsonparse: no JSON cookie: '[origin
software="rsyslogd" swVersion="8.9.0" x-pid="7098"
x-info="http://www.rsyslog.com"] start'

Thanks,
--Lily
Post by Lily Chadha
Hi,
i am new to syslog module.i am trying to log STRUCTURED DATA in log
file.But when i am using this parameter in template it is showing null
value.i am using rsyslogd 5.8.11.
this below template i am using to log messages.
"<%PRI%>%TIMESTAMP:::date-rfc3339%%HOSTNAME%%syslogtag%%APP-NAME%%PROCID%%MSGID% %msg% %STRUCTURED-DATA%\n”
<142> 2015-03-05T06:55:39.816659-05:00 host login[1986]: login 1986 - [1986 : 1986 INFO]SERIAL Login from IP:127.0.0.3 user:sysadmin -but i want message with structure data
--
You received this message because you are subscribed to the Google Groups "graylog2" group.
To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+***@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Jochen Schalanda
2015-05-28 12:46:35 UTC
Permalink
Hi Lily,

please consult the official rsyslog support channels for detailed questions
about rsyslog configuration: http://www.rsyslog.com/doc/free_support.html

Cheers,
Jochen
Post by Lily Chadha
Hi Jochen,
After loading mmjsonparse,i am still getting structured data empty.Here is
3818.712254754:main Q:Reg/w0 : ACTION 0 [mmjsonparse::mmjsonparse:]
3818.712262545:main Q:Reg/w0 : executing action 0
3818.712267008:main Q:Reg/w0 : Called action, logging to mmjsonparse
3818.712271907:main Q:Reg/w0 : wti 0x99ef6e0: we need to create a new
action worker instance for action 0
3818.712293355:main Q:Reg/w0 : DDDD: writing data to table spot 0
3818.712297580:main Q:Reg/w0 : wti 0x99ef6e0: created action worker
instance 1 for action 0
3818.712301394:main Q:Reg/w0 : Action 0 transitioned to state: itx
3818.712305432:main Q:Reg/w0 : entering actionCalldoAction(), state: itx,
actionNbr 0
3818.712314391:main Q:Reg/w0 : mmjsonparse: no JSON cookie: '[origin
software="rsyslogd" swVersion="8.9.0" x-pid="7098" x-info="
http://www.rsyslog.com"] start'
Thanks,
--Lily
Post by Lily Chadha
Hi,
i am new to syslog module.i am trying to log STRUCTURED DATA in log
file.But when i am using this parameter in template it is showing null
value.i am using rsyslogd 5.8.11.
this below template i am using to log messages.
"<%PRI%>%TIMESTAMP:::date-rfc3339%%HOSTNAME%%syslogtag%%APP-NAME%%PROCID%%MSGID% %msg% %STRUCTURED-DATA%\n”
<142> 2015-03-05T06:55:39.816659-05:00 host login[1986]: login 1986 - [1986 : 1986 INFO]SERIAL Login from IP:127.0.0.3 user:sysadmin -but i want message with structure data
--
You received this message because you are subscribed to the Google Groups "graylog2" group.
To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+***@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Lily Chadha
2015-05-28 12:52:17 UTC
Permalink
Hi,

Okay thank you
Post by Lily Chadha
Hi,
i am new to syslog module.i am trying to log STRUCTURED DATA in log
file.But when i am using this parameter in template it is showing null
value.i am using rsyslogd 5.8.11.
this below template i am using to log messages.
"<%PRI%>%TIMESTAMP:::date-rfc3339%%HOSTNAME%%syslogtag%%APP-NAME%%PROCID%%MSGID% %msg% %STRUCTURED-DATA%\n”
<142> 2015-03-05T06:55:39.816659-05:00 host login[1986]: login 1986 - [1986 : 1986 INFO]SERIAL Login from IP:127.0.0.3 user:sysadmin -but i want message with structure data
--
You received this message because you are subscribed to the Google Groups "graylog2" group.
To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+***@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Loading...