Vonage & PF: Prioritizing VoIP Traffic

For the impatient: sample /etc/pf.conf.

Overview

I used to use a little Netgear RP114 router. It's a fine product, but my networking needs outgrew its capabilities. One feature I wanted was the ability to prioritize my voice over IP traffic over other data transfer activity. I use Vonage's VoIP service, which ordinarily provides nice, clear connections over my relatively slow (768 kbps down/384 kbps up) connection. However, I found it all too easy to saturate the upload pipe and cause choppy, unintelligible phone calls.

PF with ALTQ is the perfect combination of tools for solving this problem with its Quality of Service (QoS) features. While there is plenty of well-written documentation, specific even to FreeBSD, I had a hard time finding complete information regarding VoIP traffic prioritization. A number of examples I ran across discussed prioritizing traffic based on the phone adapter's IP address, but I'm interested in a more general solution than a hard-coded address. This article attempts to fill that gap by detailing how I finally got things to work as expected.

The Vonage Service

Vonage's bandwidth saver control.

Additionally, Vonage works with some analog telephone adapters with built-in routers that include QoS capabilities. The Linksys RT31P2 is one of these. However, I wasn't much interested in an "all-in-one" device: I already had a router/firewall and wanted a stand-alone phone adapter. The RT31P2 is a great device (though I don't really like the Linksys router/firewall feature set) and is perfect for folks not interested in the contortions I detail here to get QoS (like my dad). Note, however, that multiple all-in-one devices sometimes won't mix: if you've got a router with a built-in wireless access point or print server and want to add the RT31P2 or similar device to the mix, you might end up having to sacrifice either QoS features or wireless/printer connectivity to your wired devices. Hence my preference for stand-alone devices...

The Linksys RT31P2 router & phone adapter.

So, I ended up with the Linksys PAP2. It's a fairly small device, making it nicely portable for use on the road. The power supply, while external, is compact, light, and supports up to 240 volts. (I should note that a friend of mine in Thailand uses one with Vonage -- 7-digit dialing to Bangkok!) My only complaint thus far is that it's got four gratuitously bright blue LEDs. I don't believe Vonage offers the PAP2 when you sign up with them directly. However, they're usually available essentially free after rebates at BestBuy, Office Depot, etc.

The Linksys PAP2 phone adapter.

PF under FreeBSD

To get started with PF and ALTQ, the FreeBSD Handbook has some good information. Specifically, the section titled "The OpenBSD Packet Filter (PF) and ALTQ." Additionally, OpenBSD's PF FAQ is invaluable to anyone getting started with PF. It includes a number of helpful examples and tips.

To use ALTQ, you're going to have to recompile your kernel. The handbook section from above covers this pretty well, but following are the additions I made to mine:

    # ALTQ support
    options ALTQ            # Enable ALTQ.
    options ALTQ_CBQ        # "Class Based Queuing" discipline.
    options ALTQ_RED        # "Random Early Detection" extension.
    options ALTQ_RIO        # "Random Early Drop" for input and output.    
    options ALTQ_HFSC       # "Hierarchical Packet Scheduler" discipline.
    #options ALTQ_CDNR      # Traffic conditioner.  (Meaningless right now.)
    options ALTQ_PRIQ       # "Priority Queuing" discipline.
    #options ALTQ_NOPCC     # Required for SMP builds.
    options ALTQ_DEBUG      # Enable additional debugging facilities.

The ALTQ(4) man page explains the options a little. I thought about trying to figure out exactly what each one does and if I needed them all for my application, but that thought didn't last very long and I just commented-out the "meaningless" one and the SMP one. Note that you don't have to add the pf, pflog, or pfsync devices to your kernel configuration file. They'll be loaded as modules automatically.

With console access available, configure your /etc/rc.conf as detailed in the handbook. PF should be ready to go after a reboot, though in a rather restrictive state.

Configuring PF

There are a couple of additional "features" in my PF configuration that don't directly apply to the VoIP project: interoperability with IPSec (Cisco's VPN Client) and PPTP VPNs, prioritized TCP acknowledgment packets, and port forwarding/redirection for externally available services. I explain all of these below, but feel free to ignore them if all you're interested in is voice traffic.

Getting started, pretty much all the configuration you have to do is contained in /etc/pf.conf. The pf.conf(5) man page is pretty useful, as is the man page for the utility used to load a configuration, pfctl(8). To validate your pf.conf file without loading it, use the command:

    # pfctl -nf /etc/pf.conf

To load your configuration, you don't have to stop and restart PF or even interrupt open network connections; just use the command:

    # pfctl -f /etc/pf.conf

First, a little background about my configuration:

    > ifconfig -a
    xl0: flags=8843 mtu 1500
            options=9
            inet 10.99.1.3 netmask 0xffffff00 broadcast 10.99.1.255
            inet 10.99.1.4 netmask 0xffffffff broadcast 10.99.1.4
            inet 10.99.1.5 netmask 0xffffffff broadcast 10.99.1.5
            ether 00:01:03:be:65:9a
            media: Ethernet autoselect (100baseTX )
            status: active
    xl1: flags=8843 mtu 1500
            options=9
            inet XX.XX.XX.195 netmask 0xffffff00 broadcast XX.XX.XX.255
            ether 00:01:02:c1:76:30
            media: Ethernet autoselect (100baseTX )
            status: active
    lo0: flags=8049 mtu 16384
            inet 127.0.0.1 netmask 0xff000000 
    pflog0: flags=141 mtu 33208

Interface xl0 is my internal network, 10.99.1.0/24. The firewall's IP address on this subnet is 10.99.1.3, which all other machines on the internal network use as their default gateway. It's also got two aliased addresses: 10.99.1.4 and 10.99.1.5. These aliases are unimportant to our task at hand; suffice it to say that I need to them to run multiple instances of various services.

Interface xl1 is connected directly to my ADSL modem and is assigned a static IP address (represented as XX.XX.XX.195). I've obfuscated the address for this article, but rest assured that the firewall configuration doesn't depend on the address. If you've got a dynamically assigned address, things might be a little different; please consult the PF documentation for details.

The internal 10.99.1.0/24 network is mapped to the static Internet IP using Network Address Translation (NAT). Additionally, I've got some external ports redirected to internal IPs so I can provide services from machines other than the firewall.

So, let's get to the meat of the issue. Following is a line-by-line explanation of the sample pf.conf file. Getting started, I've got:

    # Interfaces.
    #
    intif = "xl0"
    extif = "xl1"


    # Define our hosts and networks.
    #
    mars = "10.99.1.3"
    saturn = "10.99.1.20"
    jupiter = "10.99.1.21"
    neptune = "10.99.1.100"

These macros define my internal and external interfaces and a couple of hosts on the internal network with static IP addresses. Next is:

    # Work VPN hosts.
    vpnhosts = "{ XX.XX.XX.2, YY.YY.YY.2, ZZ.ZZ.ZZ.2 }"

This part is irrelevant to our VoIP project, but I'm including it for the sake of completeness. I use Cisco's VPN Client to connect to three different networks using IPSec over UDP. The line above simply defines a list of (again, obfuscated) IP addresses to my VPN servers. The firewall treats these sorts of connections a little differently than normal network traffic, as you'll see below.

    # Vonage traffic ports (SIP and RTP).
    voipports = "{ 5060, 5061, 10000:20000 }"

Here I've got a list of UDP ports my Vonage phone adapter wants to use for the Session Initiation Protocol (SIP) and the Real-time Transport Protocol (RTP). Vonage details which ports its service uses in an FAQ article. Note that other VoIP providers might use different ports.

    icmp_types = "echoreq"
    nonroutable = "{ 192.168.0.0/16, 127.0.0.0/8, 172.16.0.0/12, 10.0.0.0/8,
        0.0.0.0/8, 169.254.0.0/16, 192.0.2.0/24, 204.152.64.0/23, 224.0.0.0/3,
        255.255.255.255/32 }"

The icmp_types macro defines allowed ICMP packet types and is used below in my "ping" filter. The nonroutable macro defines a list of address blocks from which we should never see traffic on the external interface. These include:

• Loopback addresses (0.0.0.0/8, 127.0.0.0/8);
• RFC 1918 addresses (10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16);
• DHCP and IPv4 auto configuration, "link local" block, RFC 3330 (169.254.0.0/16);
• IPv4 multicast, RFC 3171 (224.0.0.0/3);
• Sun Microsystems private cluster interconnects (204.152.64.0/23);
• "TEST-NET" addresses, RFC 3330 (192.0.2.0/24); and
• Broadcasts (255.255.255.255/32).

    # Set some runtime options.
    #
    set block-policy return
    set loginterface $extif

These are some PF runtime options. The block-policy return option tells PF to be nice and let the sending host know it has blocked a TCP packet. Otherwise, it'll just silently drop a blocked packet. The loginterface option specifies that we want to gather traffic statistics for our external interface. (Only one interface may be specified at a time.) This'll be useful below for verifying that our configuration works.

    # Scrub packets.
    #
    scrub on $extif reassemble tcp no-df random-id

Here we're normalizing packets as they come in. The PF documentation has specifics on each option; this set works well for me and doesn't seem to interfere with VoIP traffic.

    # Fire up ALTQ.  We'll prioritize empty TCP ACK traffic, VoIP traffic
    # next, and then everything else.
    #
    altq on $extif priq bandwidth 350Kb queue { std, voip, tcpack }
    queue std priq(red default)
    queue voip priority 10 priq(red)
    queue tcpack priority 15 priq(red)

ALTQ lets us do packet queuing and prioritization. It's important to note that the router can only prioritize outgoing traffic; we can't really do much with incoming traffic. On an asymmetric connection (e.g., my ADSL service) where the download speed is typically a couple times faster than the upload, this shouldn't be too much of an issue. But, be aware that this setup won't prevent a saturated download pipe from affecting VoIP call quality.

I've chosen to use the Priority Queue (priq) queuing scheduler. It seems to be a little simpler to configure than the alternative, Class Based Queuing (cpq). The PF documentation gives detailed information about the differences between the two, but for my home network priq seems to work well. It's setup to run on the external interface, and I've given it 350 kbps to work with. Now, my upload bandwidth is closer to 384 kbps; the PF documentation explains that setting the queue's bandwidth a little lower than the maximum can result in better performance. I did some very informal testing (timing FTP uploads and downloads) that seemed to agree with this suggestion. You'll probably have to experiment a little to find the value that works best for your configuration.

For my network, I've configured three queues:

• std: The default queue for all normal outgoing traffic. It's using Random Early Detection (RED) for congestion avoidance which should improve performance in high traffic situations.
• voip: The queue for outgoing voice over IP traffic. It's got a higher priority than our default traffic and is also using RED. Note that the PF documentation mentions that UDP traffic is usually not well-suited for RED. Since I've only got one Vonage line running traffic through this queue, using RED should be OK. However, further experimentation with this option is left as an exercise for the reader.
• tcpack: The queue for TCP acknowledgment packets, given the highest priority. This trick can improve performance in cases where you're running concurrent downloads and uploads. Again, it's using RED, which may not be necessary for such traffic.

    # Now, setup some NAT action for the internal network.
    #
    # First NAT is for the Cisco VPN client.
    nat on $extif inet proto { tcp, udp } from $intif:network port isakmp \
        to any -> $extif:0 port isakmp
    nat on $extif from $intif:network to any -> $extif

The second NAT rule is the more important one: it maps your internal network to your Internet connection's address. Note that if your external IP is dynamic (assigned via DHCP), you might want to put parenthesis around the last $extif macro reference so that the rule is automatically updated.

    # Configure some port forwarding.
    # Note that all of our services on mars are being redirected for now.
    # I suppose they could be passed directly, but this standardizes the
    # configuration.
    #
    tcpservices = "{ 22, 25, 53, 80, 2401, 3389, 8080 }"
    udpservices = "{ 53 }"

These macros define lists of ports through which I offer services on the external interface. As noted in the comments, though the firewall machine is offering some of these services itself, I've chosen to redirect traffic to its internal address, primarily for ease of maintenance.

    # SSH
    rdr on $extif proto tcp from any to any port 22 -> $mars
    # SMTP
    rdr on $extif proto tcp from any to any port 25 -> $mars
    # DNS
    rdr on $extif proto { tcp, udp } from any to any port 53 -> $mars
    # HTTP
    rdr on $extif proto tcp from any to any port 80 -> $mars
    # CVS
    rdr on $extif proto tcp from any to any port 2401 -> $mars
    # RDP
    rdr on $extif proto tcp from any to any port 3389 -> $saturn
    rdr on $extif proto tcp from any to any port 8080 -> $neptune

Here, my traffic redirection rules forward the ports I defined in the macros above. (None of these are applicable to the Vonage VoIP service.) We'll use those macros later in some filter rules. Note that I'm redirecting two different ports for the Remote Desktop Protocol (RDP). TCP port 3389 is the standard port for this protocol, but since I've got two Windows computers I'd like to reach from the Internet, I changed the listen port on one of them. Now, I could have simply redirected traffic to TCP port 8080 to the default RDP port on the 2nd Windows machine, but I wanted access to the machine to be consistent both internally and externally.

    # Filter rules.
    #
    block log all
    pass quick on lo0 all

    block drop in quick on $extif from $nonroutable to any
    block drop out quick on $extif from any to $nonroutable

At this point, we're actually starting to firewall something. By default, I'm blocking everything. It's only by exception (the pass rules) that traffic flows. The first exception is traffic on the firewall machine's loopback interface: it flows freely, and isn't subject it to any other filter rules (the quick option). Subsequently, I explicitly block, again quickly, any traffic on the external interface coming in or going out for the special address blocks defined above.

    # Let in the services we offer.
    pass in on $extif inet proto tcp from any to $intif:network port \
        $tcpservices flags S/SA keep state
    pass in on $extif inet proto udp from any to $intif:network port \
        $udpservices keep state

This snippet is where I use the macros defined above for the redirected TCP and UDP ports. All these rules do is allow traffic in from the external interface destined for the specified ports. Note, however, the keep state directive: this important feature allows packets associated with an already open connection to bypass ruleset evaluation. Additionally, for the TCP traffic, I only want to create a state entry for packets indicating the start of a new connection. The TCP flag S/SA does just that.

    # Pings.
    pass in inet proto icmp all icmp-type $icmp_types keep state

I'm not a big fan of firewalls that totally block ICMP traffic, or "pings". The ping and traceroute utilities are two of the networking tools I use the most -- they're invaluable for troubleshooting connectivity problems. They're also useless when a firewall administrator blindly blocks ICMP packets. So, the above rule lets ICMP traffic pass through (using the icmp_types macro defined earlier), again keeping state for each session.

    # Internal network.
    #
    # Note that in order for our special Vonage queue to work, we need to
    # tag incoming traffic so we can identify it on its way out.
    #
    pass in on $intif from $intif:network to any keep state
    pass in quick on $intif proto udp from $intif:network to any \
        port $voipports tag VONAGE_OUT keep state
    pass out on $intif from any to $intif:network keep state

Now that inbound traffic on the external interface has been taken care of, let's turn our attention to the internal interface. The first rule is pretty straightforward: I let all traffic from my internal network into the firewall. I trust all the users on my network, and since the firewall also acts as a file server, among other things, there's little value in trying to lock things down further. If you've got a dedicated firewall or a more open internal network (e.g., a publicly accessible wireless access point), you'll probably want to be more restrictive with internal interface traffic.

The second rule is a little more interesting: here I use my voipports macro defined earlier to tag Vonage traffic with the VONAGE_OUT identifier. This means that all traffic from the internal network destined for UDP ports that Vonage uses is given a special tag, which I'll refer to again shortly.

The third rule simply lets traffic out on the internal interface. This can include both traffic from the firewall machine itself (remember, it's also a file server) and traffic from the outside world that's already been vetted by my earlier rules involving the external interface.

    # Now, let our traffic out.
    # 1. Prioritize empty TCP ACKs (and give other TCP traffic default priority).
    # 2. Prioritize VoIP traffic on the specified UDP ports.
    # 3. Let out all other eligible traffic (GRE for PPTP connections).
    #
    pass out on $extif inet proto tcp all modulate state flags S/SA \
        queue(std, tcpack)
    pass out on $extif inet proto { udp, icmp, gre } all keep state
    pass out on $extif tagged VONAGE_OUT keep state queue(voip, tcpack)

And, finally, these rules take care of outbound traffic on the external interface. Recall that ALTQ only queues outgoing traffic. Consequently, these rules are where I assign packets to the appropriate queues for prioritization. The first rule lets TCP packets out: normal packets are assigned to the std (default) queue, but "low-delay" and TCP acknowledgment packets we talked about earlier are assigned to the highest-priority tcpack queue. The additional modulate state option is an additional security measure for TCP packets to protect lesser operating systems that don't properly randomize their Initial Sequence Numbers (ISNs).

Next, I let out regular UDP, ICMP, and GRE traffic. Again, letting ICMP traffic flow allows the likes of ping and traceroute to work. Allowing GRE, or Generic Route Encapsulation protocol, packets to pass is necessary for PPTP VPNs to work properly. Since I don't specify an alternative, this traffic just goes in the default queue.

Last, but not least, the third rule looks for my specially-tagged VONAGE_OUT packets -- the voice traffic. These packets are placed in the voip queue for appropriate prioritization. Since the last matching filter rule is the one that counts (unless quick is specified), this rule will override the previous and more generic UDP rule.

    # Some Cisco IPSec VPN shiznit.
    pass out on $extif proto esp all
    pass in on $extif inet proto { tcp, udp } from $vpnhosts to \
        $intif:network port isakmp keep state

To cap things off, I had to add two more rules for IPSec VPN packets. The Encapsulating Security Payload (ESP) protocol is used for both tunneling and encryption. Also, I need to explicitly open up the ISAKMP port on the external interface for key exchange, despite already having the NAT rule above.

Conclusion

Nevertheless, whether you use my approach or experiment with your own, you'll want to test things to verify that your configuration works. Fortunately, PF lets you look at statistics for each ALTQ queue defined:

    # pfctl -s queue -v
    queue std priq( red default ) 
      [ pkts:       1393  bytes:     508271  dropped pkts:      0 bytes:      0 ]
      [ qlength:   0/ 50 ]
    queue voip priority 10 priq( red ) 
      [ pkts:        674  bytes:     173463  dropped pkts:      0 bytes:      0 ]
      [ qlength:   0/ 50 ]
    queue tcpack priority 15 priq( red ) 
      [ pkts:      12470  bytes:     674076  dropped pkts:      0 bytes:      0 ]
      [ qlength:   0/ 50 ]

If you find any glaring errors, opportunities for optimization, or just want to comment, please let me know!