With more and more application moving to the cloud, web based applications have become ubiquitous. They are ideal for providing access to applications sitting on the cloud (over HTTP through a standard web browser). This has removed the need to install specialized application on the client system, the client just needs to install is a fairly modern browser.
While this is good for reducing load on the client, the job of the firewall has become much tougher.
Traditionally firewall rules look at the Layer 3 and Layer 4 attributes of a packet to identify a flow and associate it with applications generating the traffic. To a traditional firewall looking at L3/L4 headers all the traffic between the client and different web apps looks like http communication. Without proper classification of traffic flows the firewall is not be able to apply a security policy.
It has now become important to look at the application layer to identify the traffic associated with a web service or web application and enforce effective security and bandwidth allocation policy.
In this blog, I will look at features provided by IPTables that can be used to classify packets by application Layer header and how this can be used to implement security and other network policy.
Looking in to the application layer
IPTables are the de-facto choice for implementing firewall on Linux. It provides extensive packet matching, classification, filtering and many more facilities. Like any traditional firewall the core features of IPTables allow packet matching with Layer3 and Layer4 header attributes. These features as we discussed in the introduction may not be sufficient to differentiate between traffic from various web apps.
While researching for a solution to provide APP Firewall on Linux I came across an IPTables extensions called NFQUEUE.
The NFQUEUE extension provides a mechanism to pass a packet to a user-space program which can run some kind of test on the packet and tell IPTables what action(accept/drop/mark) to perform for the packet.
This gives a lot of flexibility for the IPTables user to hook up custom tests for the packets before it is allowed to pass through the firewall.
To understand how NFQUEUE can help classify and filter traffic based on application layer headers, let’s try to implement a web app filter providing URL based access control. In this test we will extract the request URL from the HTTP header and the filter will allow access based on this URL
A simple web app
For this experiment, we will use python bottle to deploy two application. Access will be allowed for the first app(APP1) while access to the second app(APP2) will be denied.
We will use the following code to deploy the sample Apps
from bottle import Bottle app1 = Bottle() app2 = Bottle() @app1.route('/APP1/') def app1_route(): return 'Access to APP1!\n' @app2.route('/APP2/') def app2_route(): return 'Access to APP2!\n' if __name__ == '__main__': app1.merge(app2) app1.run(server='eventlet', host="192.168.121.22", port=8081)
The web application will bind to port 8081 and local IP of 192.168.121.22.
NOTE: we need a eventlet based bottle server else the application hangs after a deny from the app filter(connections are not closed and the next request is not processed)
To access the web apps use the curl commands
Configuring the IPTables NFQUEUE
The next step is to configure IPTables to forward the client traffic accessing the web apps to our user space web-app filter.
The NFQUEUE IPTables extension works by adding a new target to IPTables called NFQUEUE. This target allows IPTables to put the matching packet on a queue. These packets can then be read from this queue by a filter application in user space. The filter application can then perform custom tests and provide a verdict to allow or deny the packet.
The NFQUEUE extension provides 65535 different queues. It also provides fail-safe options like what action IPTables should take if a queue is created but no filter is attached to it, load balancing of packets across multiple queues. Also, there are knobs in the /proc filesystem to control how much of the packet data will be copied to user space. A complete list of options can be found in the iptables extensions man page
To enable NFQUEUE for the web-app traffic we will add the following rule to IPTables.
iptables -I INPUT -d 192.168.121.22 -p tcp --dport 8081 -j NFQUEUE --queue-num 10 --queue-bypass
The –queue-num option selects the NFQUEUE number to which the packet will be queued. The –queue-bypass option allows the packet to be accepted if no custom filter is attached to queue number 10, without this option if no filter is attached to the queue, packets will be dropped.
Implementing a simple APP filter
With the above IPTables rule the packets destined for our sample web app will be pushed into NFQUEUE number 10. I am going to use the python bindings for NFQUEUE called nfqueue-bindings to develop the filter. Let’s run a simple print and drop filter.
#!/usr/bin/python # need root privileges import struct import sys import time from socket import AF_INET, AF_INET6, inet_ntoa import nfqueue from dpkt import ip def cb(i, payload): print "python callback called !" payload.set_verdict(nfqueue.NF_DROP) return 1 def main(): q = nfqueue.queue() print "setting callback" q.set_callback(cb) print "open" q.fast_open(10, AF_INET) q.set_queue_maxlen(50000) print "trying to run" try: q.try_run() except KeyboardInterrupt, e: print "interrupted" print "unbind" q.unbind(AF_INET) print "close" q.close() if __name__ == '__main__': main()
Now we have tested that the packets trying to access our web app are passing through a app filter implemented in user space. Next we need to unpack the packet and look at the HTTP header to extract the URL that the user is trying to access. For unpacking the headers we will use python dpkt library. The following code will let us access to APP1 and deny access to APP2
#!/usr/bin/python # need root privileges import struct import sys import time from socket import AF_INET, AF_INET6, inet_ntoa import nfqueue import dpkt from dpkt import ip count = 0 def cb(i, payload): global count count += 1 data = payload.get_data() pkt = ip.IP(data) if pkt.p == ip.IP_PROTO_TCP: # print " len %d proto %s src: %s:%s dst %s:%s " % ( # payload.get_length(), # pkt.p, inet_ntoa(pkt.src), pkt.tcp.sport, # inet_ntoa(pkt.dst), pkt.tcp.dport) tcp_pkt = pkt.data app_pkt = tcp_pkt.data try: request = dpkt.http.Request(app_pkt) if "APP1" in request.uri: print "Allowing APP1" payload.set_verdict(nfqueue.NF_ACCEPT) elif "APP2": print "Denying APP2" payload.set_verdict(nfqueue.NF_DROP) else: print "Denying by default" payload.set_verdict(nfqueue.NF_DROP) except (dpkt.dpkt.NeedData, dpkt.dpkt.UnpackError): pass else: print " len %d proto %s src: %s dst %s " % ( payload.get_length(), pkt.p, inet_ntoa(pkt.src), inet_ntoa(pkt.dst)) sys.stdout.flush() return 1 def main(): q = nfqueue.queue() print "setting callback" q.set_callback(cb) print "open" q.fast_open(10, AF_INET) q.set_queue_maxlen(50000) print "trying to run" try: q.try_run() except KeyboardInterrupt, e: print "interrupted" print "%d packets handled" % count print "unbind" q.unbind(AF_INET) print "close" q.close() if __name__ == '__main__': main()
Here are the result of the test on the client
The output from the filter on the firewall
What else can be done with App based traffic classification
Firewall is just one use-case of the advance packet classification. With the flows identified and associated to different applications we can apply different routing and forwarding policy. NFQUEUE based filter can be used to set different firewall marks on the classified packets. The firewall marks can then be used to implement policy based routing in Linux.