Contents

Using Logstash to add WhoIS information to IP addresses

In my previous post, I talked about how I used Logstash and Memcached to enrich IP addresses collected from HoneyPotDB with confidence scores from AbuseIPDB, which worked great! In this post I’ll quickly go over how I used the same methods to add data Whois from https://ip-api.com/ to enrich IP addresses with:

  • GeoLocation Data
  • ISP
  • Organsation
  • ASN Number
  • ASN Name
  • Mobile IP status
  • VPN/Tor/Proxy IP status
  • Datacenter IP status

Creating a logstash pipeline

Here is my Logstash pipeline to do this, I’d recommend taking a look at my previous post to understand more about how this works. The below will take IP addresses, and query a Memcached instance to try and pull a cached value. If no value is cached, the IP-API API (:P) is queried for information, added to the event and then stored in the cache for 7 days.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
input { pipeline { address => filter_whois } }

filter {
        # Try and pull IP info from cache
        if [src_ip] {
                        memcached {
                                        hosts => ["192.168.1.25:11211"]
                                        namespace => "ipapicache"
                                        get => {
                                        "%{[src_ip]}" => "[ipapi]"
                                        }
                                        add_tag => ["ipapi_from_cache"]
                                        id => "memcached-ipapi-get"
                        }
        }
                if ! [ipapi] {
                http {
                        id => "ipapi-http-01"
                        url => "http://ip-api.com/json/%{[src_ip]}?fields=status,message,isp,org,as,asname,reverse,mobile,proxy,hosting,query"
                        verb => "GET"
                        connect_timeout => 15
                        headers => { "Accept" => "application/json" }
                        target_body => "[ipapi]"
                        target_headers => "[@metadata][ip-api_response_headers]"
                }
                if [ipapi] {
                        mutate { convert => { "[ipapi]" => "string" } }
                        memcached {
                                hosts => ["192.168.1.25:11211"]
                                namespace => "ipapicache"
                                set => {
                                "[ipapi]" => "%{[src_ip]}"
                                }
                                ttl => 604800
                                add_tag => ["ipapi_cached"]
                                id => "memcached-ipapi-set"
                        }
                }
        }
        if [ipapi] {
                mutate { convert => { "[ipapi]" => "string" } }
        }
}
output {
        if [@metadata][source_type] == "filebeat_honeypot" {
                pipeline { send_to => [filter_abuseipdb] }
        }
}