=====Overview=====
GeoIP was COMPLETELY NUTS to set up, at least for me it was impossible with logstash, so I used directly a beat agent to set it directly to the elasticsearch.
For some reason, my grok pattern wasn't recognized...even tough according the grok debugger WAS WORKING!!! :) So I used the traefik module of Filebeats as follows:
=====Install Filebeats=====
The installation of file beats is rather simple, just download the version (or use yum repo) and install it:
[root@dokuwiki ~]# ls -alrt
-rw-r--r--. 1 root root 12699052 May 15 04:46 filebeat-6.2.4-x86_64.rpm
[root@dokuwiki ~]# rpm -Uvh filebeat-6.2.4-x86_64.rpm
After the installation we have to configure 2 things:
* Configuration
* Modules
As I said, I was not able to set up logstash for my httpd access log, even though my GROK WAS CORRECT !!! :)
So I just used the already done module by filebeat: traefik
====Configuration====
#=========================== Filebeat prospectors =============================
filebeat.prospectors:
- type: log
enabled: true
paths:
- /etc/httpd/logs/access.log
tags: [apache_access]
paths:
- /etc/httpd/logs/ssl_access_log
tags: [ssl_apache_access]
fields:
level: debug
#============================= Filebeat modules ===============================
filebeat.config.modules:
# Glob pattern for configuration loading
path: ${path.config}/modules.d/*.yml
# Set to true to enable config reloading
reload.enabled: true
#==================== Elasticsearch template setting ==========================
setup.template.settings:
index.number_of_shards: 3
#============================== Kibana =====================================
setup.kibana:
host: "10.132.0.4:5601"
#================================ Outputs =====================================
#-------------------------- Elasticsearch output ------------------------------
output.elasticsearch:
hosts: ["10.132.0.4:9200"]
protocol: "http"
username: "elastic"
password: "password"
#----------------------------- Logstash output --------------------------------
#output.logstash:
# The Logstash hosts
hosts: ["10.132.0.4:9200"]
xpack.monitoring.enabled: true
xpack.monitoring.elasticsearch:
xpack.monitoring.enabled: true
xpack.monitoring.elasticsearch.url: 10.132.0.4:9200
xpack.monitoring.elasticsearch.username: kibana
xpack.monitoring.elasticsearch.password: password
xpack.monitoring.elasticsearch.hosts:
====Modules====
I have enabled 2 modules:
[root@dokuwiki modules.d]# ls -alrt
-rw-r--r--. 1 root root 440 Jul 19 13:19 apache2.yml
-rw-r--r--. 1 root root 229 Jul 19 13:27 traefik.yml
drwxr-xr-x. 2 root root 4096 Jul 23 13:28 .
drwxr-xr-x. 3 root root 4096 Jul 23 13:30 ..
====Modules Config====
[root@dokuwiki modules.d]# cat apache2.yml
- module: apache2
# Access logs
access:
enabled: true
# Set custom paths for the log files. If left empty,
# Filebeat will choose the paths depending on your OS.
var.paths:
- /etc/httpd/logs/access_log
# Error logs
error:
enabled: true
# Set custom paths for the log files. If left empty,
# Filebeat will choose the paths depending on your OS.
var.paths:
- /etc/httpd/logs/error_log
[root@dokuwiki modules.d]# cat traefik.yml
- module: traefik
# Access logs
access:
enabled: true
# Set custom paths for the log files. If left empty,
# Filebeat will choose the paths depending on your OS.
var.paths:
- /etc/httpd/logs/access_log
=====Done=====
After that the filebeat, should start sending the necessary data to the elasticsearch and from there you can check it via Kibana :)
{{ :geoip_kibana.jpg?600 |}}