[graylog2] Spaces between characters!

[Tom Collins]

Hi all - I was wondering if anyone could help.
I've been using Graylog successful, in production for several months now.

Today, I've run in to my first real problem.

I'm sending in some FSLogix log files, from a Windows machine, using NXLog.

They're getting to Graylog just fine, and at first they appear fine,
however when searching I noticed that I couldn't return any results against
content I knew was there. Even when searching against extracted fields.
After clicking on search terms, I've noticed that all of the fields seem to
have (what looks like) spaces between each character. They look perfectly
normal until you try actually try to search etc.

Here is what I'm talking about;

<Loading Image...>


<Loading Image...>


Weirdly, if I copy the text from field terms (above), in to, say
notepad...there are no spaces.

Does anyone have any idea what might be causing this?! It's been driving me
crazy all day.
This is a sample of the log that is being fed via nxlog

---------------------------------------------------------------

[2017-02-08 02:05:59.779][pid:0f1c][tid:20fc] =====

Begin: Unload profile: vannup =====
vannup. SID: S-1-5-21-2000128468-286259493-1166484339-21833.

[2017-02-08 02:05:59.779][pid:0f1c][tid:20fc] [INFO :0x00000000]

Configuration setting not found: ConcurrentUserSessions. Using default: 0

[2017-02-08 02:05:59.779][pid:0f1c][tid:20fc] [INFO :0x00000000] No

teardown required

[2017-02-08 02:05:59.779][pid:0f1c][tid:20fc] [INFO :0x00000000]

Configuration setting not found: ShutdownOnUserLogoff. Using default: 0

[2017-02-08 02:05:59.779][pid:0f1c][tid:20fc] [INFO :0x00000000]

Configuration setting not found: RebootOnUserLogoff. Using default: 0

[2017-02-08 02:05:59.779][pid:0f1c][tid:20fc] [INFO :0x00000000]

S-1-5-21-2000128468-286259493-1166484339-21833.

[2017-02-08 02:05:59.779][pid:0f1c][tid:20fc] [INFO :0x00000000]

unloadProfile time: 0 milliseconds

[2017-02-08 02:05:59.779][pid:0f1c][tid:20fc] =====

End: Unload profile: vannup =====

[2017-02-08 08:17:56.911][pid:0f1c][tid:1b98] =====

Begin: LoadProfile: USJOLNETPC14 =====

[2017-02-08 08:17:56.911][pid:0f1c][tid:1b98] [INFO :0x00000000]

Configuration Read (DWORD): SOFTWARE\FSLogix\Profiles\Enabled. Data: 0

Here's my nxlog config. I've tried with everything I can think off (GELF,
json etc etc) - this works with other plain-text files;

<Extension syslog>
Module xm_syslog
</Extension>

<Input in>
Module im_file
File 'D:\\FSLogix\\FSLogix\\Logs\\Profile\\Profile-*.log'
SavePos TRUE
ReadFromLast TRUE
PollInterval 1
InputType LineBased
Exec $fullMessage = $raw_event;
</Input>

<Output out>
Module om_udp
Host 10.50.8.114
Port 12204
Exec to_syslog_bsd();
</Output>


<Route 1>
Path in => out
</Route>

[Jochen Schalanda]

Hi Tom,

this looks like an encoding problem (UTF-16 vs. UTF-8),
see https://github.com/Graylog2/graylog2-server/issues/3130 for a related
issue with a potential fix.

Cheers,
Jochen

Post by Tom Collins
Hi all - I was wondering if anyone could help.
I've been using Graylog successful, in production for several months now.
Today, I've run in to my first real problem.
I'm sending in some FSLogix log files, from a Windows machine, using NXLog.
They're getting to Graylog just fine, and at first they appear fine,
however when searching I noticed that I couldn't return any results against
content I knew was there. Even when searching against extracted fields.
After clicking on search terms, I've noticed that all of the fields seem
to have (what looks like) spaces between each character. They look
perfectly normal until you try actually try to search etc.
Here is what I'm talking about;
<https://lh3.googleusercontent.com/-yZVp1Swh7tw/WKXBj0_fKEI/AAAAAAAAAQI/3n_G-LtPL8cQOESy0FBzEbDk6tHBFy8jwCLcB/s1600/gl1.png>
<https://lh3.googleusercontent.com/-aADTb8AXxOI/WKXB0gy7QyI/AAAAAAAAAQM/s741npQMT8UHZO6GYKZUwGo_TbKc9vcvACLcB/s1600/gl2.png>
Weirdly, if I copy the text from field terms (above), in to, say
notepad...there are no spaces.
Does anyone have any idea what might be causing this?! It's been driving
me crazy all day.
This is a sample of the log that is being fed via nxlog
---------------------------------------------------------------

[2017-02-08 02:05:59.779][pid:0f1c][tid:20fc] =====

Begin: Unload profile: vannup =====
vannup. SID: S-1-5-21-2000128468-286259493-1166484339-21833.

[2017-02-08 02:05:59.779][pid:0f1c][tid:20fc] [INFO :0x00000000]

Configuration setting not found: ConcurrentUserSessions. Using default: 0

[2017-02-08 02:05:59.779][pid:0f1c][tid:20fc] [INFO :0x00000000] No

teardown required

[2017-02-08 02:05:59.779][pid:0f1c][tid:20fc] [INFO :0x00000000]

Configuration setting not found: ShutdownOnUserLogoff. Using default: 0

[2017-02-08 02:05:59.779][pid:0f1c][tid:20fc] [INFO :0x00000000]

Configuration setting not found: RebootOnUserLogoff. Using default: 0

[2017-02-08 02:05:59.779][pid:0f1c][tid:20fc] [INFO :0x00000000]

S-1-5-21-2000128468-286259493-1166484339-21833.

[2017-02-08 02:05:59.779][pid:0f1c][tid:20fc] [INFO :0x00000000]

unloadProfile time: 0 milliseconds

[2017-02-08 02:05:59.779][pid:0f1c][tid:20fc] =====

End: Unload profile: vannup =====

[2017-02-08 08:17:56.911][pid:0f1c][tid:1b98] =====

Begin: LoadProfile: USJOLNETPC14 =====

[2017-02-08 08:17:56.911][pid:0f1c][tid:1b98] [INFO :0x00000000]

Configuration Read (DWORD): SOFTWARE\FSLogix\Profiles\Enabled. Data: 0

Here's my nxlog config. I've tried with everything I can think off (GELF,
json etc etc) - this works with other plain-text files;
<Extension syslog>
Module xm_syslog
</Extension>
<Input in>
Module im_file
File 'D:\\FSLogix\\FSLogix\\Logs\\Profile\\Profile-*.log'
SavePos TRUE
ReadFromLast TRUE
PollInterval 1
InputType LineBased
Exec $fullMessage = $raw_event;
</Input>
<Output out>
Module om_udp
Host 10.50.8.114
Port 12204
Exec to_syslog_bsd();
</Output>
<Route 1>
Path in => out
</Route>

[Tom Collins]

Wonderful, thank you.In my case, the encoding was UCS-2LE. Added the
following to my nxlog config, and everything is now working correctly;

Exec convert_fields('UCS-2LE','UTF-8'); if $raw_event == '' drop();

Post by Tom Collins
Hi all - I was wondering if anyone could help.
I've been using Graylog successful, in production for several months now.
Today, I've run in to my first real problem.
I'm sending in some FSLogix log files, from a Windows machine, using NXLog.
They're getting to Graylog just fine, and at first they appear fine,
however when searching I noticed that I couldn't return any results against
content I knew was there. Even when searching against extracted fields.
After clicking on search terms, I've noticed that all of the fields seem
to have (what looks like) spaces between each character. They look
perfectly normal until you try actually try to search etc.
Here is what I'm talking about;
<https://lh3.googleusercontent.com/-yZVp1Swh7tw/WKXBj0_fKEI/AAAAAAAAAQI/3n_G-LtPL8cQOESy0FBzEbDk6tHBFy8jwCLcB/s1600/gl1.png>
<https://lh3.googleusercontent.com/-aADTb8AXxOI/WKXB0gy7QyI/AAAAAAAAAQM/s741npQMT8UHZO6GYKZUwGo_TbKc9vcvACLcB/s1600/gl2.png>
Weirdly, if I copy the text from field terms (above), in to, say
notepad...there are no spaces.
Does anyone have any idea what might be causing this?! It's been driving
me crazy all day.
This is a sample of the log that is being fed via nxlog
---------------------------------------------------------------

[2017-02-08 02:05:59.779][pid:0f1c][tid:20fc] =====

Begin: Unload profile: vannup =====
vannup. SID: S-1-5-21-2000128468-286259493-1166484339-21833.

[2017-02-08 02:05:59.779][pid:0f1c][tid:20fc] [INFO :0x00000000]

Configuration setting not found: ConcurrentUserSessions. Using default: 0

[2017-02-08 02:05:59.779][pid:0f1c][tid:20fc] [INFO :0x00000000] No

teardown required

[2017-02-08 02:05:59.779][pid:0f1c][tid:20fc] [INFO :0x00000000]

Configuration setting not found: ShutdownOnUserLogoff. Using default: 0

[2017-02-08 02:05:59.779][pid:0f1c][tid:20fc] [INFO :0x00000000]

Configuration setting not found: RebootOnUserLogoff. Using default: 0

[2017-02-08 02:05:59.779][pid:0f1c][tid:20fc] [INFO :0x00000000]

S-1-5-21-2000128468-286259493-1166484339-21833.

[2017-02-08 02:05:59.779][pid:0f1c][tid:20fc] [INFO :0x00000000]

unloadProfile time: 0 milliseconds

[2017-02-08 02:05:59.779][pid:0f1c][tid:20fc] =====

End: Unload profile: vannup =====

[2017-02-08 08:17:56.911][pid:0f1c][tid:1b98] =====

Begin: LoadProfile: USJOLNETPC14 =====

[2017-02-08 08:17:56.911][pid:0f1c][tid:1b98] [INFO :0x00000000]

Configuration Read (DWORD): SOFTWARE\FSLogix\Profiles\Enabled. Data: 0

Here's my nxlog config. I've tried with everything I can think off (GELF,
json etc etc) - this works with other plain-text files;
<Extension syslog>
Module xm_syslog
</Extension>
<Input in>
Module im_file
File 'D:\\FSLogix\\FSLogix\\Logs\\Profile\\Profile-*.log'
SavePos TRUE
ReadFromLast TRUE
PollInterval 1
InputType LineBased
Exec $fullMessage = $raw_event;
</Input>
<Output out>
Module om_udp
Host 10.50.8.114
Port 12204
Exec to_syslog_bsd();
</Output>
<Route 1>
Path in => out
</Route>