Wire shark can decrypt SSL traffic, how?

Wire shark can decrypt SSL traffic, as documented here on their wiki:

http://wiki.wireshark.org/SSL

It mentions "..This only works for RSA key exchange if the RSA keys can be provided."

That's fine. But why?

The short answer is that if you were to use a certificate signed with a DSA signature, generally speaking, the SSL session will also use Diffie Hellman ephemeral mode for key exchange, (which incidently will grant you perfect forward secrecy), and therefore the private key of the certificate won't help you to decrypt traffic.

That is fair enough but not really a satisfying technical answer.

Here we'll talk about specifically why this is true.

In order to do so, first we will investigate why specifically we are able to eventually decrypt SSL connections when using an RSA key exchange.

To do so first let's review what happens in SSL secure communications between a Web server and a web browser.

During the initial handshake phase of SSL, the client issues a ClientHello record, and within this initial record (among other things) is a random integer presented to the server for use later on:



The server responds with a ServerHello and also sends to the client a random integer, also for use later on.



The Web Server also sends its SSL certificate in a Certificate record, and within this certificate is contained the public key of the Web server.

After the client has has verified the signature of the Certificate Authority within the SSL certificate, one of the steps it needs to perform is to generate something called the pre-master secret, which is a random integer that will also be used later on to derive the secret encryption key(s).

Specifically the client encrypts this pre-master secret using the RSA public key of the Web server. The Web server takes this value and decrypts the pre-master secret using its corresponding private key.

At this point the client and the server both know three values:
The client random value, the server random value and the client's generated pre-master secret.

The first two are known publicly and have not been transmitted in encrypted format. The only piece of information at this point that has been transmitted encrypted is the pre-master secret.

At this point both client and server take the pre-master secret, client random, and server random, and use them to run through a key derivation function, first to derive a master secret for the session, and then once again to derive the encryption keys, mac keys and other values that they may need.

The encryption key that we just derived is the key used for bulk data encryption. So as you can see this is not the private key that we use with Wireshark.

So if you have been capturing an SSL session with a tool like Wireshark or tcpdump, at key exchange time, it will be able to recover the initial client random and server random values, just like anyone who was listening to the network at that time.

The additional detail that Wireshark provides the user to input is the RSA private key of the Web server. Recall that this private key is the counterpart of the public-key (that was used to encrypt the pre-master secret by the client).

So now Wireshark has enough information to derive the master secret, and the other keys for the session, including the bulk encryption key(s), since it can decrypt the pre-master secret using the RSA private key. Thus we are now able to decrypt SSL traffic that has been encrypted when using an RSA key exchange.

If we talk now about a DSA based certificate, we need to remember that DSA technology is used for digital signature and verification, but not encryption, or key exchange.

We won't be able to encrypt the pre-master secret with a DSA public key in a key exchange like we did with the RSA key above, because again it can only be used for signatures and verifications.

In SSL, DSA technologies are generally coupled with Diffie-Hellman for key exchange, the full details you can read about in RFC 2631 or read a lighter version.

On a very, very, high abstract level, each party generates a Diffie-Hellman public and private key, and then exchanges its public key with the other side, and then uses its own private key, the other party's public-key and some other integers and numbers to eventually derive a shared secret value, that both parties know about.

Suffice it to say once the Diffie-Hellman key exchange has been performed each side is aware of a secret. This secret is the pre-master secret as we saw above. Both client and server go ahead and derive both the master secret and the eventual encryption keys for the connection.

So now what does Wireshark have to use if it was listening to that traffic? It doesn't have enough information to derive the pre-master secret. It doesn't know about the secret keys of either party, as those are never sent across the network, only the public keys, and other Diffie-Hellman parameters have been sent across the wire. So again, it can't derive the pre-master secret and ultimately derive the encryption key(s).

Also, those private keys are considered ephemeral, that is they are not stored anywhere, so after the key exchange and the session is over they will likely (hopefully) be destroyed never to be used again, and, in this case you'll gain Perfect Forward Secrecy.

Keep in mind now that even if you had the DSA private key of the Web server, Wireshark still could not decrypt this traffic.

Although it was used in signing the Diffie-Hellman parameters that have been sent to the other side, that private key doesn't factor in the computation of the Diffie- Hellman key exchange at all.

A great reference for anyone interested in learning more about SSL is Eric Rescorla's book, SSL and TLS: Designing and Building Secure Systems. I highly recommend it as it is an extremely detailed, complete view of SSL.

Also, check out the RFCs for:

TLS
http://www.ietf.org/rfc/rfc2246.txt

Diffie-Hellman Key Agreement Method
http://www.ietf.org/rfc/rfc2631.txt

Piercing your Firewall with Fwknop

Recently I had been looking for a solution to the problem where I would like to grant access to users to an SSH server which is externally available on the Internet, but add sensible firewall rules to protect the service.

I have hardened the SSH server and I am allowing my IP address to get to port 22, however when a user has a dynamic IP address, it makes it very difficult to write a firewall rule.

Relying on DNS for a firewall rule wasn't a great solution as many IPs may come from the same ISP I'd like to block (or allow), not to mention DNS can be spoofed.

There are many solutions to this problem, but fwknop is a very interesting one.

The author states:

"fwknop stands for the "FireWall KNock OPerator", and implements an authorization scheme called Single Packet Authorization (SPA). This method of authorization is based around a default-drop packet filter (fwknop supports both iptables on Linux systems and ipfw on FreeBSD and Mac OS X systems) and libpcap"


fwknop basically allows a user to set up a dynamic firewall rule after they have been authenticated.


Here's how it works

First, set your iptables ruleset such that all packets are dropped on port 22 (DENY).

Next, you install the fwknop server package on an SSH server you would like to protect and install the fwknop client on a client machine that wishes to connect.

The actual installation and configuration of the server didn't take me more than 15 minutes or so. The documentation is great.

When I ran the client it prompted me for my passphrase:





fwknop sent a single UDP packet to the server which was then consumed, decrypted the contents of the packet, confirmed my passphrase and then opened up a firewall rule for my IP address for 30 seconds as you see here in /var/log/messages:


Aug 7 15:03:49 Myhost fwknopd: received valid Rijndael encrypted packet from: 127.0.0.1 , remote user: j, client version: 1.9.11 (SOURCE line num: 26)
Aug 7 15:03:50 Myhost fwknopd: add FWKNOP_INPUT 127.0.0.1 -> 0.0.0.0/0(tcp/22) ACCEPT rule 30 sec

Note the new rule referencing the FWKNOP_INPUT chain now:

#iptables -L -n
Chain INPUT (policy DROP)
target prot opt source destination
FWKNOP_INPUT all -- 0.0.0.0/0 0.0.0.0/0
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0
BLACKLIST all -- 0.0.0.0/0 0.0.0.0/0
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 state RELATED,ESTABLISHED
ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 tcp dpt:22
ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 tcp dpt:80
ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 tcp dpt:443


Chain FWKNOP_INPUT (1 references)
target prot opt source destination
ACCEPT tcp -- 127.0.0.1 0.0.0.0/0 tcp dpt:22


The rule expired after 30 seconds:

Aug 7 15:03:50 Myhost fwknopd: add FWKNOP_INPUT 127.0.0.1 -> 0.0.0.0/0(tcp/22) ACCEPT rule 30 sec
Aug 7 15:04:21 Myhost fwknop(knoptm): removed iptables FWKNOP_INPUT ACCEPT rule for 127.0.0.1 -> 0.0.0.0/0(tcp/22), 30 sec timeout exceeded


That's a crude and quick story, but what's great about this is that I can deny all IP addresses on the Internet for SSH access and not worry about any particular scripts that are run against the server.


Before a user can even attempt to validate their username and password (or public key,etc) they need to authenticate with this fwknop server.

This severely reduces the possibility of an attack, because at this point you have only about 30 seconds to exploit the server.


I highly recommend this if this suits your environment.

(Tcp)traceroute

Tcptraceroute per the author's website is "a traceroute implementation using TCP SYN packets, instead of the more traditional UDP or ICMP ECHO packets. In doing so, it is able to trace through many common firewall filters."

Before we get into tcptraceroute, let's get started first with traceroute, which has been the standard historical network tool to trace a packet's route across a network. It's a great tool but a lot of sites may simply block that traffic minimizing it's usefulness.

Traceroute works using a few principals:

1) A low, but increasing time to live (TTL), usually starting at 1. (This can be set with the -f flag).

2) An invalid destination port (generally UDP port 33434 for Unix systems, Microsoft tends to use ICMP)

As each router on the network processes an IP packet, it will decrement the TTL by 1 and forward the packet on. However, an IP packet with a TTL of 0 is discarded and then an ICMP packet with a message of "TTL Expired in Transit" is sent to the source IP sending the packet. This reduces the chance of infinite routing loops.

So that's how the Traceroute program builds the route list, by listing the IPs of the router that sent the message as shown here in a traceroute to yahoo.com.

Note that the Source IPs that are associated with the TTL Expired in Transit messages are changing. Those are the routers sending these messages:



The Traceroute program knows it's reached the host because since UDP port 33434 is generally invalid, the remote host will reply with an ICMP packet saying "destination unreachable" (port unreachable).

Here's a network capture showing a Linux host tracing a route:



Note that the initial UDP port is 33434 and it increases for each packet sent and also that the remote host we are tracing to sent us back a "port unreachable" ICMP packet.

Now as we mentioned tcptraceroute uses TCP SYN packets and defaults to port 80, which is generally open on the remote host if it is a web server or at least generally are all allowed across routers.

Compare difference between traceroute and tcptraceroute:


http://michael.toren.net/code/tcptraceroute/examples.txt


Wow, all the way through!

Here's how a network capture looks when we tcptraceroute to pages.ebay.com:



Note when tcptraceroute finally reaches the remote host it sends a packet with the SYN bit set, the remote server responds with SYN and ACK bits set, and tcptraceroute resets the connection with RST.

Pretty cool stuff when you're looking to trace a route!

Deny Hosts with..........DenyHosts!

While I was looking for some security tools recently I had come across DenyHosts.

DenyHosts is a Python script that will allow you to monitor a Linux system for failed SSH logins and add to their IP address to /etc/hosts.deny based on the number of failed attempts. ( This assumes you have set up your SSH install to be protected by TCP wrappers. )

One of the nice features about this script is that you can set different limits for the number of allowed failed logins for the root user, users that do not exist on this system, and users that do exist.

Installing this software was very easy as it was available as "apt-get install denyhosts".

Configuring the script was also very easy and didn't take more than about 10 minutes or so to read through the configuration file and set a few parameters that the author documents very well.

After that I just set up the script to be run from Cron every 5 minutes or so:


and Denyhosts was all ready to go.

As an example, you can see in on the secure log ( /var/log/auth.log on my system), the users Devil, bob, hacz0r, root, and loot have failed login attempts from 1.2.3.4:

Jun 11 15:51:05 RuleNumber1 sshd[1113]: Failed none for invalid user Devil from 1.2.3.4 port 35722 ssh2
Jun 11 15:51:06 RuleNumber1 sshd[1113]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=1.2.3.4
Jun 11 15:51:08 RuleNumber1 sshd[1113]: Failed password for invalid user loot from 1.2.3.4 port 35722 ssh2
Jun 11 15:51:18 RuleNumber1 sshd[1117]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=1.2.3.4 user=root
Jun 11 15:51:20 RuleNumber1 sshd[1117]: Failed password for root from 1.2.3.4 port 46446 ssh2
Jun 11 15:51:26 RuleNumber1 sshd[1117]: Failed password for root from 1.2.3.4 port 46446 ssh2
Jun 11 15:51:30 RuleNumber1 sshd[1117]: PAM 1 more authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=1.2.3.4 user=root
Jun 11 15:51:41 RuleNumber1 sshd[1121]: Failed password for invalid user hacz0r from 1.2.3.4 port 41470 ssh2
Jun 11 15:51:53 RuleNumber1 sshd[1121]: PAM 1 more authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=1.2.3.4
Jun 11 16:05:21 RuleNumber1 sshd[1177]: pam_unix(sshd:auth): authentication failure; logname=


and a few minutes later we see that deny hosts has done its job and added the IP address to our deny file:

cmd#cat /etc/hosts.deny
sshd: 1.2.3.4


Another nice feature of DenyHosts is that you can use synchronization mode, where you are accepting information about IP addresses that have been blocked for other users' hosts and/or (it's configurable) upload the IP addresses you have blocked, in a sense collaborating on a large black list of IP addresses.

All in all this was very easy to install and configure, there is one main configuration file this easy to understand and it is well commented.

Deny hosts makes a nice addition to add another level of security to your SSH setup, and I'd suggest adding this to your toolkit.

RST - Who are you?

So the other day I was chatting with some technical (geeky!) friends of mine and the conversation turned to TCP flags. So the question was posed in the group, "What happens if you connect to a remote host and you get a connection refused message?"

Well *one* short answer is the remote side reset the connection, most likely because the particular application or port is not listening on that particular port.

That is to say that when you connect to a Web server on port 80 the connection (the operating system's TCP/IP stack that is running the web browser, or a command line utility, etc) will go through the standard TCP handshake to connect to that well-known port number 80.

Say you connect to port 73, and there is no application running at that IP address and port, the correct thing for that remote host to do is to reply with a packet with the RST flag set, resetting the connection, which is better than the other side just waiting until it times out and gives up.

In layman's terms is basically saying "There is no one here, please go away and try somewhere else".

This is simply one example, more details on TCP (and the RST flag) can be found in this RFC:

Well, so what, why is this important?

Well in the course of troubleshooting it will become important as the following example illustrates:

I was on my customer's network troubleshooting a network connectivity problem.

Basically they were trying to index content on their Web server secured with SSL (https) and they're having a heck of a time with connections from their internal network to the host.

I didn't have any thorough knowledge of their network topology, nor did the customer at the time, however I did have my Linux host and the tcpdump command at the ready so I was able to do a packet capture and then some analysis on the results.

But before that I did some basic tasks:

I did a simple telnet to port 80 to see if there was any Web server running.

Sure enough there was - and when I gave it an HTTP GET request for the / document, it redirected me to port 443. Good to know.

Port 80 is not necessarily directly related to the Web server and its associated port 443 but at least I can confirm to basic IP and TCP connectivity to that host.

So off now to port 443:

First I also issued a telnet to port 443.

That basically didn't give me any real information, nor will it, generally, because the telnet command doesn't really understand the SSL program or protocol. Actually it doesn't at all. For example:

#telnet www.blogger.com 443
Trying 74.125.19.191...
Connected to www.blogger.com (74.125.19.191).
Escape character is '^]'.
Connection closed by foreign host.

All this did was check to see if port 443 was open and listening, which it was, or if it quickly reset, or gave any other information. (plus it's the shortest command to type and remember).

So the next step in troubleshooting would be to use something like the OpenSSL utilty, which is basically an SSL enabled telnet. It's actually quite a bit more than that but for our purposes we're going to connect to the host on port 443 and see what the responses, and then issue some HTTP commands.

However what I found was this response from OpenSSL:

#openssl s_client -connect customerweb:443
CONNECTED(00000003)
write:errno=104

Curl didn't fair much better:

#curl -i "https://customerweb/"
curl: (35) SSL: error:00000000:lib(0):func(0):reason(0)

As you can see neither output was very helpful and clearly doesn't indicate success.

What I would be expecting is something more like this from openSSL:



As you can see the OpenSSL command has connected to the secure blogger website and retrieved some certificate information. This confirms that's a secure Web server at the other end is able to be contacted and we can read the certificate.

So I forwarded the errors and information to the customer and suggested for their network team to look into the network hardware in between these two devices, the first one which incidentally was a Google Search Appliance, which I work often times with.

That's fine and helpful but not really getting to the heart of the issue.

We need to once again check in with TCPdump and really see what's going on a packet level. So I did and this is what I found:

The Google search appliance was making a valid connection to port 443 however the remote host was waiting for a few seconds and then coming back with an RST flag set in the packet, basically resetting the connection, as if to say note is no Web server listening here as shown in the packet trace:



I saw this using Wireshark, which is a great tool for analyzing tcpdump files. It is actually able to resolve the first 3 bytes of the Mac address to the the vendor who has been assigned that block of MAC addresses, which can be really helpful and a time saver.

In this case the default gateway was in fact a device manufactured by the F5 Corporation. Given that, our picture looks a little more clear:

"GSA"->"F5/LB"->"HTTPS Customer WebServer".

Again without even knowing about the full network topology it was clear that the path the Google Search Appliance was going through over network there definitely was F5 device (most likely a firewall or load balancer), just based on the MAC address of the default gateway for the GSA.

The resolution of this in fact was a problem with the F5 load balancer, which taken care of with an update to its configuration.

But here is a good example of starting from what you know, what is available to you, and just knowing a little bit about TCP flags, and how they're used to get closer to a solution.

Rate Limiting and DOS protection in Mod-Security

Using Mod-Security we can deploy some Rate Limiting and DOS (denial of service) protection for Apache.

You'll use 2 scripts, one called httpd-guardian, which will use the second script, called blacklist, which will then use either iptables or the OpenBSD Packet Filter (pf) to deny IPs.

You can get both scripts here: http://apache-tools.cvs.sourceforge.net/apache-tools/.

As per the website, "..By default httpd-guardian will defend against clients that send more than 120 requests in a minute, or more than 360 requests in five minutes."

Setting this up was relatively easy. After installing Mod-Security and getting it running, we'll configure as shown below:


Fist, I downloaded httpd-guardian into /etc/apache2/httpd-guardian.

Next, I added this line:

SecGuardianLog "|/etc/apache2/httpd-guardian"

to /etc/apache2/apache2.conf.

I edited httpd-guardian and followed the directions, basically just uncommenting and setting these lines, the first to call the blacklist script:

my $PROTECT_EXEC = "/sbin/blacklist block %s 3600";

and the second lowered the maximum number of requests that one IP address can issue in a one minute window:

my $THRESHOLD_1MIN = 1; # 60 requests in a minute

Note: there is a 5 min threshold as well, but I left that default:
my $THRESHOLD_5MIN = 1; # 360 requests in 5 minutes

I also then downloaded the blacklist script into /sbin/blacklist and since I am using ipchains, I made sure this line was set in the httpd-guardian script:

my $FWCMD = "iptables";

I did as requested by the great instructions in the file itself and created a new RULE set:

iptables -N BLACKLIST
iptables -A INPUT -p tcp --dport 80 -j BLACKLIST

To make sure apache agreed with my setup, I issued the "apache2ctl graceful" command.

Then for testing, I hit the site with a bunch of requests:

#while [ 1 ]; do wget 'http://my-site.com';done

I confirmed 60+ hits (IP was changed to protect the innocent) were established in minute 2009:14:21 in my web server logs:

# grep '1.2.3.4' /var/log/apache2/access.log | grep '17/May/2009:14:21' | wc -l
140

And the next minute later I saw this in syslog:

May 17 14:22:13 blacklist[18464]: Blocking 1.2.3.4/32 for 3600 seconds

I couldn't connect or do a thing from my IP with my browser to my web server - it worked!

Mod-Security - Real Time Web Filtering

Mod-Security is a "real-time" application firewall for Apache. It's able to detect, log, deny, and take action on web attacks, and security related events for your Apache web server.

Installation is a breeze on a Debian system running Apache2:

#apt-get install libapache2-mod-security
#a2enmod mod-security
#/etc/init.d/apache2 force-reload

From there you can use the great documentation to start adding rules.

Just add a new section like so in the apache2.conf file:

<IfModule mod_security.c>

...new rules....

</IfModule>



Here's a quick entry to alert your would-be attackers that you know of their path traversal attempt for /etc/passwd:

I.E the user here requested: http://myserver/test/../../etc/passwd and we redirect them to /busted.html.


Here's the rule:
SecFilter /etc/passwd "deny,log,status:406,redirect:/busted.html

and the results:




This is a mere fraction of the capabilites with Mod-security. From there you can investigate a whole range of options, including denying DOS attempts using httpd-guardian.

Happy Filtering!