I’ve seen a particular kind of vulnerability in a few different applications but I’m not sure of an appropriate name for it. So I thought I’d write about it, and informally call it the “DNS defeated IP address check”. Basically, if you have an application that can be used as a proxy, or can be instructed to make web request, you don’t want it fetching files from internal services.

For example, there is a simple proxy called Camo, which is used to fetch third party images when you need to display them on a SSL secure site. (Read more about Camo on the GitHub blog).

This kind of application can be incorrectly setup such that the application has access to internal servers and resources that wouldn’t normally be exposed to the Internet. This make the proxy application a good way a hacker could gain information about a private network. However Camo tries to address this issue by forbidding URLs that contain private IP addresses. It does a check like so:

RESTRICTED_IPS = /^((10\.)|(127\.)|(169\.254)|(192\.168)|(172\.((1[6-9])|(2[0-9])|(3[0-1]))))/

if (url.host.match(RESTRICTED_IPS))
  return four_oh_four(resp, "Hitting excluded hostnames")

This code (written for Node.js in CoffeeScript) is taking a url object and checking the hostname doesn’t match a restricted address. This works great against URLs such as http://127.0.0.1/, or http://10.0.0.1/, however this check can easily be defeated. If you create a domain name, such as localhost.bramp.net, which resolves to 127.0.0.1, and ask the proxy to fetch http://localhost.bramp.net/, then it won’t be caught by that check. Now the proxy will continue to try and fetch a resource from 127.0.0.1.

The solution to this problem is to do that IP address check after the DNS name has been resolved. This can also be problematic if you use a standard library for making web requests, as they will do the DNS lookup for you, and don’t give you the fine grain control you need. For example, I’ve seen this be a problem for a Java application using the Apache HTTP Client.

One might naively assume they could do a DNS check, and then hand the processing to a HTTP library to make the actual request. The issue here is that the DNS record the HTTP library uses might not be the same as the one you checked against with the DNS check. For example, many domains have multiple A records, and some DNS servers can be configured to round robin DNS records. If you can’t be sure the HTTP library will do another DNS requests, then you’d be vulnerable.

Luckily, in Camo’s case the fix was relatively easy (see my pull request).

# We do DNS lookup ourselves
Dns.lookup url.host, (err, address, family) ->
  if address.match(RESTRICTED_IPS)
    return four_oh_four(resp, "Hitting excluded hostnames")

  # We connect to the IP address, not hostname
  src = Http.createClient url.port || 80, address

  # We add a host header, so the request will work
  headers = 
    "Host' : url.host

  # Boom, we make the request
  srcReq = src.request 'GET', query_path, headers

The above code was simplified a little from the real code, but basically we do the DNS lookup, check the returned address is good, and then make a HTTP request to that IP address with a Host: header to ensure the request will work.

Really though, the correct solution to this is to configure a suitably paranoid firewall to stop requests from the proxy machine to anything internal. However, as with all security, the more layers of protection you have the better, and you should never depend on just one.

I was using Wireshark to capture an exchange of HTTP packets, however, some of the HTTP responses were using “content-encoding: gzip”, which meant I couldn’t view them decompressed in the “Follow TCP Stream”. Wireshark does decompress them in Packet Details view, but it is hard to follow the full stream like this.

The solution was to write some Python which made use of the dpkt library. My code naively reassembles the TCP flow and then assumes traffic on port 80 is HTTP. Therefore there is much room for improvement, but here is the code anyway.

#!/usr/bin/env python
# Turns a pcap file with http gzip compressed data into plain text, making it
# easier to follow.

import dpkt

def tcp_flags(flags):
	ret = ''
	if flags & dpkt.tcp.TH_FIN:
		ret = ret + 'F'
	if flags & dpkt.tcp.TH_SYN:
		ret = ret + 'S'
	if flags & dpkt.tcp.TH_RST:
		ret = ret + 'R'
	if flags & dpkt.tcp.TH_PUSH:
		ret = ret + 'P'
	if flags & dpkt.tcp.TH_ACK:
		ret = ret + 'A'
	if flags & dpkt.tcp.TH_URG:
		ret = ret + 'U'
	if flags & dpkt.tcp.TH_ECE:
		ret = ret + 'E'
	if flags & dpkt.tcp.TH_CWR:
		ret = ret + 'C'

	return ret

def parse_http_stream(stream):
	while len(stream) > 0:
		if stream[:4] == 'HTTP':
			http = dpkt.http.Response(stream)
			print http.status
		else:
			http = dpkt.http.Request(stream)
			print http.method, http.uri
		stream = stream[len(http):]

def parse_pcap_file(filename):
	# Open the pcap file
	f = open('market.pcap', 'rb')
	pcap = dpkt.pcap.Reader(f)
	
	# I need to reassmble the TCP flows before decoding the HTTP
	conn = dict() # Connections with current buffer
	for ts, buf in pcap:
		eth = dpkt.ethernet.Ethernet(buf)
		if eth.type != dpkt.ethernet.ETH_TYPE_IP:
			continue
	
		ip = eth.data
		if ip.p != dpkt.ip.IP_PROTO_TCP:
			continue
	
		tcp = ip.data
	
		tupl = (ip.src, ip.dst, tcp.sport, tcp.dport)
		#print tupl, tcp_flags(tcp.flags)
	
		# Ensure these are in order! TODO change to a defaultdict
		if tupl in conn:
			conn[ tupl ] = conn[ tupl ] + tcp.data
		else:
			conn[ tupl ] = tcp.data
	
		# TODO Check if it is a FIN, if so end the connection
	
		# Try and parse what we have
		try:
			stream = conn[ tupl ]
			if stream[:4] == 'HTTP':
				http = dpkt.http.Response(stream)
				#print http.status
			else:
				http = dpkt.http.Request(stream)
				#print http.method, http.uri
	
			print http
			print

			# If we reached this part an exception hasn't been thrown
			stream = stream[len(http):]
			if len(stream) == 0:
				del conn[ tupl ]
			else:
				conn[ tupl ] = stream
		except dpkt.UnpackError:
			pass

	f.close()

if __name__ == '__main__':
	import sys
	if len(sys.argv) <= 1:
		print "%s <pcap filename>" % sys.argv[0]
		sys.exit(2)

	parse_pcap_file(sys.argv[1])

Please note, I had to make a couple of changes to the dpkt library, which I have submitted back for review. Those changes can be found in the following patches 1 2 3. I will update this code if/when the patches get accepted.