[graylog2] Graylog Docker Logging/ Java Stacktrace Splitter

[ivan morozov]

Hi @all,

first of all props to the Graylog team for the docker log support! This
great feature makes a lot of things easier!

My situation is as follow: my java application runs into a docker container
and log into STDOUT. The content splitter split each line as a new log
message, that makes the monitoring of java exceptions with stack trace
pretty hard.

Is there a workaround to solve that? Is there a way to integrate a custom
content splitter into the docker log collector?

Thanks in advance!

Ivan

[Jochen Schalanda]

Hi Ivan,

collapsing multiple individual log lines/messages into one is unfortunately
not possible with Graylog at the moment.

If you can modify the Docker container or the Java application running in
it, respectively, you could use one of the existing GELF logging appenders
(see https://www.graylog.org/resources/data-sources/) for the logging
framework being used in the application and send log messages directly (and
stack traces in one piece) to Graylog and maybe even add additional
information using something like the Logback MDC (
http://logback.qos.ch/manual/mdc.html).


Cheers,
Jochen

Post by ivan morozov
first of all props to the Graylog team for the docker log support! This
great feature makes a lot of things easier!
My situation is as follow: my java application runs into a docker
container and log into STDOUT. The content splitter split each line as a
new log message, that makes the monitoring of java exceptions with stack
trace pretty hard.
Is there a workaround to solve that? Is there a way to integrate a custom
content splitter into the docker log collector?
Thanks in advance!
Ivan

[shiva jawanjal]

Hi All,

First thing, *thanks for creating such an amazing tool for log analysis++ !*

I have set-up Graylog2 for a Java application. We are using Logback for
logging ( Please understand that I am not a Java developer. Hope you get
me.)

I am getting INFO & DEBUG logs properly, but when there's error in Java log
file, I get each line in Java stack trace as a separate log.

My* /etc/rsyslog.d/90-graylog2.conf* file is

$template GRAYLOGRFC5424,"<%pri%>%protocol-version%
%timestamp:::date-rfc3339% %HOSTNAME% %app-name% %procid% %msg%\n"
*.* @1xxx.xx.xxx.xx:514;GRAYLOGRFC5424

And in *rsyslog.conf*, I have done configuration as





*$InputFileName /path/to/java-log.log$InputFileTag
app-logs:$InputFileStateFile$InputFileFacility local6$InputRunFileMonitor*

For each file.

Can you please suggest me,

1. any way to get Java stacktrace properly in Graylog Web Server ?

2. Can I use Collector here ?

3. Is there any work around available?

I will really appreciate your help.

Thanks,
Aniruddha

[Jochen Schalanda]

Hi Aniruddha,

the easiest way to get complex logs from Java applications into Graylog is
by using one of the existing GELF log appenders (see
https://www.graylog.org/resources/data-sources/ for a list of projects).
There are several GELF appenders for Logback.

Cheers,
Jochen

Post by shiva jawanjal
Hi All,
First thing, *thanks for creating such an amazing tool for log analysis++ !*
I have set-up Graylog2 for a Java application. We are using Logback for
logging ( Please understand that I am not a Java developer. Hope you get
me.)
I am getting INFO & DEBUG logs properly, but when there's error in Java
log file, I get each line in Java stack trace as a separate log.
My* /etc/rsyslog.d/90-graylog2.conf* file is
$template GRAYLOGRFC5424,"<%pri%>%protocol-version%
%timestamp:::date-rfc3339% %HOSTNAME% %app-name% %procid% %msg%\n"
And in *rsyslog.conf*, I have done configuration as
*$InputFileName /path/to/java-log.log$InputFileTag
app-logs:$InputFileStateFile$InputFileFacility local6$InputRunFileMonitor*
For each file.
Can you please suggest me,
1. any way to get Java stacktrace properly in Graylog Web Server ?
2. Can I use Collector here ?
3. Is there any work around available?
I will really appreciate your help.
Thanks,
Aniruddha

[shiva jawanjal]

Thanks a lot, Jochen!

I will check for that.


Best regards,
Aniruddha

Post by Jochen Schalanda
Hi Aniruddha,
the easiest way to get complex logs from Java applications into Graylog is
by using one of the existing GELF log appenders (see
https://www.graylog.org/resources/data-sources/ for a list of projects).
There are several GELF appenders for Logback.
Cheers,
Jochen

Post by shiva jawanjal
Hi All,
First thing, *thanks for creating such an amazing tool for log analysis++ !*
I have set-up Graylog2 for a Java application. We are using Logback for
logging ( Please understand that I am not a Java developer. Hope you get
me.)
I am getting INFO & DEBUG logs properly, but when there's error in Java
log file, I get each line in Java stack trace as a separate log.
My* /etc/rsyslog.d/90-graylog2.conf* file is
$template GRAYLOGRFC5424,"<%pri%>%protocol-version%
%timestamp:::date-rfc3339% %HOSTNAME% %app-name% %procid% %msg%\n"
And in *rsyslog.conf*, I have done configuration as
*$InputFileName /path/to/java-log.log$InputFileTag
app-logs:$InputFileStateFile$InputFileFacility local6$InputRunFileMonitor*
For each file.
Can you please suggest me,
1. any way to get Java stacktrace properly in Graylog Web Server ?
2. Can I use Collector here ?
3. Is there any work around available?
I will really appreciate your help.
Thanks,
Aniruddha
--

You received this message because you are subscribed to a topic in the
Google Groups "Graylog Users" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/graylog2/Sqkm9t0ShSk/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
To view this discussion on the web visit
https://groups.google.com/d/msgid/graylog2/2e1dd18a-576f-42b3-a088-88c1345a1b00%40googlegroups.com
<https://groups.google.com/d/msgid/graylog2/2e1dd18a-576f-42b3-a088-88c1345a1b00%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.

[shiva jawanjal]

Hi All,

I am trying to create a stream that will contain all Java Exceptions except
a specific type of exception that contains a string as
* "Failed to get SFDC Access token from refresh token for user" *
i.e. it should contain all exceptions except following type of exception

*2015-09-10T15:27:28.663+0000 c.s.i.a.LoadAccountsUsenBolt [ERROR]
LoadAccountsUsenBolt EXCEPTION for username *****@gmail.com
<http://gmail.com> IS java.lang.RuntimeException: Failed to get SFDC Access
token from refresh token for user : ***@gmail.com
<***@gmail.com>*

I get the proper results when I search for *"Exception" AND NOT "Failed to
get SFDC Access token from refresh token for user" *in Graylog search box*,
*but, while creating stream for this rule

Stream Rule:



*message must match regular expression "Exception" AND NOT "Failed to get
SFDC Access token from refresh token for user"*
I face an issue that, I don't get other exceptions after filtering out
"Failed to get SFDC --- token for user" line.

Can you please help me? (Just tell me if I am being bit confusing here)

Best regards,
Aniruddha
<http://52.10.81.177:9000/streams/55f17d7be4b0261d30f2ce70/rules#>
<http://52.10.81.177:9000/streams/55f17d7be4b0261d30f2ce70/rules#>

Post by shiva jawanjal
Thanks a lot, Jochen!
I will check for that.
Best regards,
Aniruddha

Post by Jochen Schalanda
Hi Aniruddha,
the easiest way to get complex logs from Java applications into Graylog
is by using one of the existing GELF log appenders (see
https://www.graylog.org/resources/data-sources/ for a list of projects).
There are several GELF appenders for Logback.
Cheers,
Jochen

Post by shiva jawanjal
Hi All,
First thing, *thanks for creating such an amazing tool for log analysis++ !*
I have set-up Graylog2 for a Java application. We are using Logback for
logging ( Please understand that I am not a Java developer. Hope you get
me.)
I am getting INFO & DEBUG logs properly, but when there's error in Java
log file, I get each line in Java stack trace as a separate log.
My* /etc/rsyslog.d/90-graylog2.conf* file is
$template GRAYLOGRFC5424,"<%pri%>%protocol-version%
%timestamp:::date-rfc3339% %HOSTNAME% %app-name% %procid% %msg%\n"
And in *rsyslog.conf*, I have done configuration as
*$InputFileName /path/to/java-log.log$InputFileTag
app-logs:$InputFileStateFile$InputFileFacility local6$InputRunFileMonitor*
For each file.
Can you please suggest me,
1. any way to get Java stacktrace properly in Graylog Web Server ?
2. Can I use Collector here ?
3. Is there any work around available?
I will really appreciate your help.
Thanks,
Aniruddha
--

You received this message because you are subscribed to a topic in the
Google Groups "Graylog Users" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/graylog2/Sqkm9t0ShSk/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
To view this discussion on the web visit
https://groups.google.com/d/msgid/graylog2/2e1dd18a-576f-42b3-a088-88c1345a1b00%40googlegroups.com
<https://groups.google.com/d/msgid/graylog2/2e1dd18a-576f-42b3-a088-88c1345a1b00%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.