SC4S Custom Filter For Windows Event Log in Syslog Format (NXLog)

GitHub - Zeroska/SC4S-All-You-Need: Repo about to guide you custom filter and docker lab for testing sc4s configurationGitHub

Here are the repo contain an example of a custom filter I made for this Windows Event Log in Syslog Format

Why am I sending Windows Event Log in Syslog Format? 😂

Because I can, long story short, my use case or company policy or requirement doesn't want me to use Splunk UF to act as an intermediate forwarder to forward Windows Event Logs but wants Windows Machine to send logs using nxlog and deliver logs in Syslog format. So here we are sending Windows Event Log in Syslog format😂.

Overview Topology

This is the same as the Splunk Test Lab I made, so you can see the SC4S container and the NXLog

The setup

• I'm using NXLog CE to forward logs from the Windows endpoint

• By using NXLog I convert Windows Event Logs into JSON after that I wrap it with Syslog Format

About the nxlog configuration file, you have to read the documentation for configuration on your own. I use this configuration line to convert logs to JSON and wrap it with Syslog format:

Exec $Message = to_json(); to_syslog_bsd();

Nxlog Documentation: https://docs.nxlog.co/userguide/integrate/windows-eventlog.html#forwarding-windows-logs-in-json-format

Log example: this is BSD Syslog Format

<14>Jul  3 02:39:18 example.com Microsoft-Windows-Security-Auditing[740]: {
    "EventTime": "2023-07-03 02:39:18",
    "Hostname": "example.com",
    "Keywords": -9214364837600034816,
    "EventType": "AUDIT_SUCCESS",
    "SeverityValue": 2,
    "Severity": "INFO",
    "EventID": 464
    "SourceName": "Microsoft-Windows-Security-Auditing",
    "ProviderGuid": "{54849625-5478-4994-A5BA-3E3B0328C30D}",
    "Version": 0,
    "Task": 12544,
    "OpcodeValue": 0,
    "RecordNumber": 8621827,
    "ActivityID": "{3A8FE162-56BE-0002-6EE1-8F3ABE56D901}",
    "ProcessID": 740,
    "ThreadID": 14228,
    "Channel": "Security",
    "Message": "",
    "Category": "Logon",
    "Opcode": "Info",
    "SubjectUserSid": "S-1-0-0",
    "SubjectUserName": "-",
    "SubjectDomainName": "-",
    "SubjectLogonId": "0x9446",
    "LogonGuid": "{00000000-0000-0000-0000-000000000000}",
    "TargetUserName": "service",
    "TargetLogonGuid": "{00000000-0000-0000-0000-000000000000}",
    "TargetServerName": "example.com",
    "TargetInfo": "example.com",
    "ProcessName": ".service.exe",
    "IpAddress": "-",
    "IpPort": "-",
    "EventReceivedTime": "2023-07-03 02:39:19",
    "SourceModuleName": "eventlog",
    "SourceModuleType": "im_msvistalog"
}

What is SC4S and how to create a custom Parser/Filter?

SC4S is Splunk Connect for Syslog, a Syslog intermediate for Splunk that parses and indexes some known vendor products using Syslog.

😕 The question is if you have a custom application that is using Syslog and you want to support it then now what, or does your SC4S keep sending you event sc4s:fallback

sc4s:fallback is when you don't have a parser or a filter for that type of log, product, or vendor, and most likely you will have to develop or create one for that

Where to create an SC4S custom parser?

You navigate to /opt/sc4s/config/app_parser/syslog/

This folder is where SC4S stores your custom parser configuration

On the SC4S GitHub repo: https://github.com/splunk/splunk-connect-for-syslog/tree/main/package/etc/conf.d/conflib this store the built-in parser/filter configuration of SC4S (which is very useful to use this repo as a guideline or configuration reference)

---

Sharing my experience

• The configuration file I made

• How do I approach this

• Takeaway and my to-do

The Configuration File

The JSON configration file

block parser app-syslog-winevent-json() {
 channel {
        # In the sc4s documentation don't mention this at all you need to read the GitHub repo to know
        # This exist: json-parser
        parser {
            json-parser(
                prefix('.values.')
            );  
        };
        rewrite {
            #set defaults these values can be overridden at run time by splunk_metadata.csv
            r_set_splunk_dest_default(
                index("main")
                source("os_win_syslog")
                sourcetype('os_win_syslog')
                #this value is used to lookup runtime settings such as index from splunk_metadata.csv
                vendor("Microsoft")
                product("Windows")
                template("t_msg_only")
            );
        };

   };
};
application app-syslog-winevent-json[sc4s-syslog] {
    parser {  app-syslog-winevent-json(); };
};

Or Xml Configration file which I'll never get to use

block parser app-syslog-winevent-xml() {
 channel {
        # In the sc4s documentation don't mention this at all you need to read the GitHub repo to know
        # This exist: json-parser also xml (parser)
        parser {
            xml(
                prefix('.values.')
            );  
        };
        rewrite {
            #set defaults these values can be overridden at run time by splunk_metadata.csv
            r_set_splunk_dest_default(
                index("main")
                source("os_win_syslog")
                sourcetype('os_win_syslog')
                #this value is used to lookup runtime settings such as index from splunk_metadata.csv
                vendor("Microsoft")
                product("Windows")
                template("t_msg_only")
            );
        };

   };
};
application app-syslog-winevent-xml[sc4s-syslog] {
    parser {  app-syslog-winevent-xml(); };
};

How do I approach this?

In the beginning, I didn't know how to do this because the documentation was quite limited, but luckily the open source SC4S so I can take a look at their code and find out how they create a parser and hopefully I can't reverse engineer the process and create one of my own.

• Reading Reddit post and their GitHub repo helps a lot -> found out create a custom parser is a normal thing in SC4S -> now I know I can do it

• Copy other parser configurations and testing them using the local lab is super helpful -> I create a guide repo for you to create an SC4S test lab above

Takeaway and my to-do

Because the NXLog Windows Event Log Schema is different from Splunk UF and I also create different source types for the Windows Event Log in Json via Syslog

There are like 3 field that are different

Because I named the sourcetype is os_win_syslog -> the log I forwarded is not CIM compatible (Not only the sourcetype is the only problem)-> So it is not good because in Splunk world CIM 😂 Compatible is everything (it normalize everything and make a standard in Splunk)

• Also sending windows event log in JSON is fine but if you want to leverage the CIM, I think I should send it in XML rather than JSON (Only support in NXLog EE)

Aug 11 2023

---

NXLog has a different schema -> changing the source type will not help and sending it using XML format will also not help because of NXLog, the only way is that you have a transformer between the NXLog and the Splunk (right now is sc4s -> but it is not going to do a great job to act as a Transformer)

The alternative solution could be Cribl Stream as an SC4S replacement (because it can parse and transform logs) or you buy the NXLog EE -> which will allow you to do Splunk Windows Event Log schema

About Syslog Format

There are 3 type of Syslog Format:

• BSD

• IETF

• Snare

Conclusion

Fun ride, please future developer or god of technology please make the all the format in one or create a god parser or something 😂