I have been using Crowdsec for some time now on all my Enhance instances, but so far I have only been able to enable detection for: SSH, Postfix, Dovecot. All of these services log inside /var/local/enhance in their native format.
I have always had problems trying to enable detection for web server access and error logs since Enhance sets custom log formats for these services.
I have been working in a custom parser to be able to read logs that live in: /var/local/enhance/webserver_logs/. And though I believe all supported webservers log in the same custom format I could only confirm with OLS and Apache.
I'll describe the steps to enable this. It would be nice if some could confirm if this also works for Nginx and Litespeed.
Add the log folders to your /etc/crowdsec/acquis.yaml file:
---
# enhance-webserver logs
filenames:
- /var/local/enhance/webserver_logs/*.log
labels:
type: enhance-webserver
Create a file in /etc/crowdsec/parsers/s01-parse, I called it enhance-webserver-logs.yaml, with the following content:
filter: "evt.Parsed.program == 'enhance-webserver'"
onsuccess: next_stage
name: crowdsecurity/enhance-webserver-logs
description: "Parse enhance webserver access logs"
nodes:
- grok:
#access log
pattern: '"%{IP:remote_addr}" "%{DATA:time_local}" "%{WORD:verb} %{DATA:request} HTTP/%{NUMBER:http_version}" "%{NUMBER:status}" "%{NUMBER:input_size}" "%{NUMBER:body_bytes_sent}" "%{NOTDQUOTE:http_referer}" "%{NOTDQUOTE:http_user_agent}"'
apply_on: message
statics:
- meta: log_type
value: http_access-log
- target: evt.StrTime
expression: evt.Parsed.time_local
statics:
- meta: service
value: http
- meta: source_ip
expression: "evt.Parsed.remote_addr"
- meta: http_status
expression: "evt.Parsed.status"
- meta: http_path
expression: "evt.Parsed.request"
- meta: http_verb
expression: "evt.Parsed.verb"
- meta: http_user_agent
expression: "evt.Parsed.http_user_agent"
Install the collections for your current webserver from crowdsec hub, since that will install many detection rules and scenarios that will work in tandem with this, specially the base-http-scenarios.
For example I installed the following collection:
cscli collections install crowdsecurity/litespeed
I believe OLS and LiteSpeed log in the same format.
I also added, the default access and error logs for OpenLiteSpeed into the acquis.yaml file, which live in /var/local/enhance/openlitespeedlogs. This varies depending on the webserver you're using, you should check which ones are available, and enable them accordingly. There are examples in crowdsec hub for nginx, apache and so on, you just need to find where Enhance saves the logs.
---
# ols default logs
filenames:
- /var/local/enhance/openlitespeedlogs/access.log
- /var/local/enhance/openlitespeedlogs/error.log
labels:
type: litespeed
Reload or restart crowdsec.
After a few seconds you can check the metrics cscli metrics and see if the detection is working.
I'm no expert on Crowdsec, please use this carefully and if you have more experience, correct me if I'm wrong or if I'm missing something.
