Recently, an Apple iBook found its way into our home. Rather than spend all our time tethered to an Ethernet switch, we decided to go wireless between the laptop and our Internet gateway. However, given the woeful security record of WEP, the decision was made not to rely upon WEP to secure our data from prying eyes. While I used to be a very adept Macintosh user and admin, years or avoiding the platform (for a variety of reasons), and a major OS change on Apple's part had conspired to render most of my Macintosh knowledge rusty and best and obsolete at worst. So, when choosing a good data encryption platform, my choices were between OpenVPN and IPSec. Having used OpenVPN in the past, it was my first choice. However, it would take some doing to get the OpenVPN tunnel up between the two systems and most of that doing would be on the Mac. I generally viewed this option as an extremely shallow to nonexistant VPN learning curve with a (possibly) very steep OS X learning curve. The other option was to use the IPSec support built in to both operating systems. Since Mac OS X draws much of its lineage from FreeBSD, it has a KAME derived IPSec support under the hood. If I went the IPSec route, I knew that I would get quick response since the encryption/decryption is all done in the kernel, high-quality crypto, and the hassle that comes with setting up an IPSec tunnel. However, that option would avoid the possibly steep Mac OS X learning curve. In the end, I decided to go with the IPSec option, at least for the short term. To help others avoid some of the hassle that I went through, I'm posting some notes about my experience below. On my wireless LAN, the FreeBSD system acts as both a firewall and gateway to the Internet and my wired LAN. So, all my laptop traffic needed to pass through the BSD system at some point, regardless of its destination. In addition, I could not use the DHCP daemon on the wireless access point since it always handed out DHCP addresses with an incorrect default router (i.e., the access point always said that it was the default gateway when the FreeBSD system was the proper gateway). Therefore, it was important that whatever I used, it needed to allow DHCP packets to exist outside of the encrypted datastream. The other option was to use the IPSec support built in to both operating systems. Since Mac OS X draws much of its lineage from FreeBSD, it has a KAME derived IPSec support under the hood. If I went the IPSec route, I knew that I would get quick response since the encryption/decryption is all done in the kernel, high-quality crypto, and the hassle that comes with setting up an IPSec tunnel. However, that option would avoid the possibly steep Mac OS X learning curve. In the end, I decided to go with the IPSec option, at least for the short term. To help others avoid some of the hassle that I went through, I'm posting some notes about my experience below. On my wireless LAN, the FreeBSD system acts as both a firewall and gateway to the Internet and my wired LAN. So, all my laptop traffic needed to pass through the BSD system at some point, regardless of its destination. In addition, I could not use the DHCP daemon on the wireless access point since it always handed out DHCP addresses with an incorrect default router (i.e., the access point always said that it was the default gateway when the FreeBSD system was the proper gateway). Therefore, it was important that whatever I used, it needed to allow DHCP packets to exist outside of the encrypted datastream. On the FreeBSD system, I setup the DHCP server daemon to always give the laptop the same IP address. I'm reasonably certain that the FreeBSD/Mac OS X IPSec implementation cannot function with dynamic addresses as tunnel endpoints. By giving the laptop the same IP address, I always know where to look for it on the wireless LAN. Both systems will need to have racoon installed on them to manage the IKE process. You may need to use the port system on the FreeBSD system to install racoon if you have not previously done so. The Mac OS X system ships with racoon installed already. You will also need to enable IPSec in your FreeBSD kernel. Do so by adding the following lines to your custom kernel configuration:options IPSEC #IP securityoptions IPSEC_ESP #IP security (crypto; define w/IPSEC)Many FreeBSD focused web pages say that you need to uncomment the gif device as well and spend time mucking around with the resulting devices. Whether or not that is true remains to be seen. I did uncomment the gif device in my kernel, but spent zero time mucking around with it.

Once you have your customer kernel configured, compile it via the instructions in the FreeBSD Handbook.

Now comes the interesting part: configuring IPSec.

There are two nodes that we will reference in this example. The first is the FreeBSD system. It has the IP address 172.18.1.1. The second is the Mac OS X system. It has the IP address 172.18.1.7.

Decide what you want to use as a pre-shared key on both systems to start the encryption process. For the sake of argument, I'm going to use 'pskDemo' as my key. On the Mac OS X system, put the following line in /etc/racoon/psk.txt:172.18.1.1 pskDemoOn the FreeBSD system, put the following into /usr/local/etc/racoon/psk.txt:172.18.1.7 pskDemoNow, make sure that the psk.txt file is read/write only for root on both systems by using a chmod command something like the following:chmod 0600 /etc/racoon/psk.txtNow, you'll want to ensure that both systems are using the same racoon arguments for various negoiation and encryption processes. On the Macintosh, edit the file /etc/racoon/remote/anonymous.conf. My version of the file looks like the following:remote anonymous{ #exchange_mode main,aggressive; exchange_mode aggressive,main; doi ipsec_doi; situation identity_only; my_identifier address; nonce_size 16; lifetime time 24 hour; # sec,min,hour initial_contact on; support_mip6 on; proposal_check obey; # obey, strict or claim proposal { encryption_algorithm 3des; hash_algorithm sha1; authentication_method pre_shared_key ; dh_group 2 ; lifetime time 24 hour; }}sainfo anonymous{ lifetime time 60 min; encryption_algorithm rijndael, blowfish, 3des ; authentication_algorithm hmac_md5, hmac_sha1; compression_algorithm deflate ;}Now, you will need a matching set of directives on the FreeBSD system. For the sake of better organization, I changed some of how the racoon port is installed. My /usr/local/etc/racoon/racoon.conf looks like this:path include "/usr/local/etc/racoon" ;include "anonymous.conf" ;path pre_shared_key "/usr/local/etc/racoon/psk.txt" ;path certificate "/usr/local/etc/cert" ;log notify;padding{ maximum_length 20; # maximum padding length. randomize off; # enable randomize length. strict_check off; # enable strict check. exclusive_tail off; # extract last one octet.}listen{ #isakmp ::1 [7000]; #isakmp 202.249.11.124 [500]; #admin [7002]; # administrative's port by kmpstat. #strict_address; # required all addresses must be bound.}timer{ counter 5; # maximum trying count to send. interval 20 sec; # maximum interval to resend. persend 1; # the number of packets per a send. phase1 30 sec; phase2 15 sec;}I then created the file /usr/local/etc/racoon/anonymous.conf that contains the following:remote anonymous{ exchange_mode aggressive,main; my_identifier address; send_cert off; send_cr off; verify_cert off; passive off; lifetime time 24 hour; proposal { encryption_algorithm 3des; hash_algorithm sha1; authentication_method pre_shared_key; lifetime time 24 hour; dh_group 2; }}sainfo anonymous{ lifetime time 60 min; encryption_algorithm rijndael, blowfish, 3des ; authentication_algorithm hmac_md5, hmac_sha1 ; compression_algorithm deflate ;}Now that we have identical (or nearly identical) configurations on both ends of the tunnel, we can create the IPSec policy database directives. On the FreeBSD system, these go in /etc/ipsec.conf:#delete all existing entries from the SAD and SPD databasesflush;spdflush;#add the policy to the SPD databasespdadd 172.18.1.1 0.0.0.0/0 any -P out ipsecesp/tunnel/172.18.1.1-172.18.1.7/require ;spdadd 0.0.0.0/0 172.18.1.1 any -P in ipsecesp/tunnel/172.18.1.7-172.18.1.1/require ;spdadd 172.18.1.7 0.0.0.0/0 any -P in ipsecesp/tunnel/172.18.1.7-172.18.1.1/require ;spdadd 0.0.0.0/0 172.18.1.7 any -P out ipsecesp/tunnel/172.18.1.1-172.18.1.7/require ;spdadd 172.18.1.1[500] 0.0.0.0/0[500] udp -P out none;spdadd 0.0.0.0/0[500] 172.18.1.1[500] udp -P in none;spdadd 0.0.0.0/0[67] 0.0.0.0/0[68] udp -P out none;spdadd 0.0.0.0/0[68] 0.0.0.0/0[67] udp -P in none;The first two lines ensure that the IKE packets are dropped onto the wireless LAN outside of the IPSec data stream. While I did not have trouble with that myself, others in the world have, so it seemed prudent to avoid the problems they had encountered.

The next two lines ensure that DHCP packets are also dropped onto the wireless LAN outside of the IPSec data stream. When these were stuck inside the IPSEC data stream, the laptop would always fail to renew its DHCP address which dropped the laptop off the LAN after roughly fifteen minutes.

The next lines all dictate how the tunnel is created. Some resources online will tell you that you only need the following configuration and that everything will "just work":spdadd 172.18.1.1 0.0.0.0/0 any -P out ipsecesp/tunnel/172.18.1.1-172.18.1.7/require ;spdadd 0.0.0.0/0 172.18.1.1 any -P in ipsecesp/tunnel/172.18.1.7-172.18.1.1/require ;That was very much not the case with me. When I used just those lines, I got all sorts of very helpful error messages from racoon like "pre-process failed" or "no policy found". In the end, I needed all of the above lines on the FreeBSD side of the tunnel.

On the Mac OS X side of the tunnel, I put my SPD rules in /etc/ipsec.rules. Why the different name? No idea. Anyway, the contents of that file look like this:flush;spdflush;spdadd 172.18.1.7[500] 172.18.1.1[500] udp -P out none;spdadd 172.18.1.1[500] 172.18.1.7[500] udp -P in none;spdadd 0.0.0.0/0[68] 0.0.0.0/0[67] udp -P out none;spdadd 0.0.0.0/0[67] 0.0.0.0/0[68] udp -P in none;spdadd 172.18.1.7 0.0.0.0/0 any -P out ipsecesp/tunnel/172.18.1.7-172.18.1.1/require;spdadd 0.0.0.0/0 172.18.1.7 any -P in ipsecesp/tunnel/172.18.1.1-172.18.1.7/require;Again, the top four lines serve to drop IKE and DHCP packets outside of the IPSec data stream. The bottom four lines serve to establish the data encryption policy between the client and the server.

Once I had those files in place, I did the following on the FreeBSD system:# /usr/local/etc/rc.d/racoon.sh start# setkey -f /etc/ipsec.confOn the Mac OS X system, I did the following:# /usr/sbin/racoon# setkey -f /etc/ipsec.rulesIn addition, you'll need to adjust your firewall rules to accomodate the wireless LAN and the IPSec traffic. I use ipfilter, my Internet traffic flows out tun0, and my wireless LAN sits off of interface dc0, so my ipfilter rules look like the following: /etc/ipf.conf:block in on dc0pass in quick on dc0 proto udp from any port = 68 to any port = 67pass out quick on dc0 proto udp from any port = 67 to any port = 68pass in quick on dc0 proto udp from any port = 500 to any port = 500pass out quick on dc0 proto udp from any port = 500 to any port = 500pass in quick on dc0 proto esp from any to anypass out quick on dc0 proto esp from any to any/etc/ipnat.conf:map tun0 172.18.1.0/24 -> 0/32 proxy port ftp ftp/tcpmap tun0 172.18.1.0/24 -> 0/32 portmap tcp/udp 10000:20000map tun0 172.18.1.0/24 -> 0/32Once I did that, everything worked for me.

Obviously, I glazed over some details and processes while covering portions of the above. However, there should be enough here to give people an idea of what I did to get everything working between the two systems.