{"id":843,"date":"2015-08-08T05:55:53","date_gmt":"2015-08-08T12:55:53","guid":{"rendered":"https:\/\/dreness.com\/blog\/?p=843"},"modified":"2019-12-07T17:15:56","modified_gmt":"2019-12-08T00:15:56","slug":"limit-network-bandwidth-used-by-an-os-x-process","status":"publish","type":"post","link":"https:\/\/dreness.com\/blog\/archives\/843","title":{"rendered":"Traffic micro-management: limit network bandwidth used by an OS X process"},"content":{"rendered":"

Problem: you have some silly uploader app that only knows one speed: maximum. You would love a way to make that app back off so it doesn’t saturate your uplink and badly impact the other things using your network. Previously\u00a0in OS X, you would accomplish this using ipfw, but these days you’d use\u00a0pfctl<\/a>\u00a0and\u00a0dnctl.<\/p>\n

None of that is breaking news. The reason for this post is that I thought of an easy way to make the limits apply only to a specific app’s traffic, and all of that traffic. Traditionally you’d have to identify the traffic using some combination of source or destination IP address or port, which can get quite cumbersome. PF also supports matching packets by uid and gid. Let’s use gid, and then run the target app with a custom gid :)<\/p>\n

First, make a new unix group called throttle:<\/p>\n

sudo dseditgroup -o create throttle<\/pre>\n
Next, give yourself permission to run things with the group id of the throttle group. To do this, edit \/etc\/sudoers to add a line such as this, substituting your actual user name.<\/p>\n
username ALL=(ALL:throttle) ALL\r\n<\/code><\/pre>\n
To test\u00a0the new group and sudo configuration, run ‘id -g’, and then again with a custom group id (using\u00a0sudo’s -g option). The results should be different.<\/p>\n
$ id -g\r\n20 \r\n$ sudo -g throttle id -g\r\n502\r\n<\/code><\/pre>\n
Now we can instantiate\u00a0the\u00a0requisite PF and dummynet setup. Here’s a little script that wants to be run as root.<\/p>\n
#!\/bin\/bash\r\n\r\n# Reset dummynet to default config\r\ndnctl -f flush\r\n\r\n# Compose an addendum to the default config to create a new anchor\r\nread -d '' -r PF <<EOF\r\ndummynet-anchor \"throttle\"\r\nanchor \"throttle\"\r\nEOF\r\n\r\n# Reset PF to default config and apply our addendum\r\n(cat \/etc\/pf.conf && echo \"$PF\") | pfctl -q -f -\r\n\r\n# Configure the new anchor\r\ncat <<EOF | pfctl -q -a throttle -f -\r\nno dummynet quick on lo0 all\r\ndummynet out proto tcp group throttle pipe 1\r\ndummynet out proto udp group throttle pipe 1\r\nEOF\r\n\r\n# Create the dummynet queue - adjust speed as desired\r\ndnctl pipe 1 config bw 1Mbit\/s\r\n\r\n# Show new configs\r\nprintf \"\\nGlobal pf dummynet anchors:\\n\"\r\npfctl -q -s dummynet\r\nprintf \"\\nthrottle anchor config:\\n\"\r\npfctl -q -s dummynet -a throttle\r\nprintf \"\\ndummynet config:\\n\"\r\ndnctl show queue\r\n\r\n# Enable PF\r\npfctl -E<\/pre>\n
Finally, start the target app as follows:<\/p>\n
sudo -g throttle open -a \"Whatever\"<\/pre>\n
To watch the counters on the bandwidth limiting queue:<\/p>\n
sudo dnctl show queue<\/pre>\n
To clear all PF config \/ state and reset PF to defaults:<\/p>\n
sudo pfctl -F all ; sudo pfctl -q -f \/etc\/pf.conf<\/pre>\n
NB: prior to 10.11.2, use of “pfctl -F all” might kernel panic your machine. This post was made public in\u00a0December, but I wrote it\u00a0back in August and filed the kernel panic bug then, which is fixed as of 10.11.2. Unfortunately, the\u00a0bug<\/a> goes back at least as far as\u00a012F45…<\/em><\/p>\n
To wrap up, keep in mind that this is a hack. What we’re really trying to accomplish requires both a privileged position on the network and also more expressive and fine-grained traffic controls than are provided by dummynet. As both of those things are not always available, doing dumb rate limiting on an individual host as documented above can still be useful.<\/p>\n
While we’re here, let’s\u00a0take a moment to illustrate\u00a0the difference between dumb host-level rate limits (which I am derisively referring to as “traffic micro-management”) and proper traffic shaping at the network edge. Two\u00a0primary goals of traffic shaping are: 1) avoid congestion at the bottleneck(s), which is typically your internet connection, and 2) maximize network utilization. Congestion happens when\u00a0packets traversing the bottleneck get piled up (‘queued’) and have to wait a relatively long time to get through. This results in\u00a0high ping times (known on Twitch as: “high ms”), in-game lag, and generally sluggish performance of any software that directly or indirectly uses the Internet. The way to avoid congestion at the bottleneck is to avoid sending more traffic than can pass through the bottleneck without incurring substantial additional delay. We also want to utilize as much of the bottleneck’s capacity as possible. These are somewhat opposed goals.<\/p>\n
As an analogy for the two\u00a0goals of avoiding congestion and maximizing throughput, imagine\u00a0pouring stuff\u00a0in a funnel as fast as possible\u00a0without allowing any stuff\u00a0to accumulate in the funnel.<\/p><\/blockquote>\n
Now imagine performing the above experiment again, but with multiple people pouring stuff in the funnel, each starting and stopping at random times with no coordination between them. Chances are good that the multi-user experiment\u00a0won’t do as well at avoiding congestion\u00a0and \/or\u00a0maximizing throughput. For the same reason, the only way to reliably do traffic shaping\u00a0to avoid network congestion is to do it\u00a0at the edge of the network (such as a router), where traffic shaping policies can account for and apply to all traffic that traverses the bottleneck.<\/p>\n","protected":false},"excerpt":{"rendered":"
Problem: you have some silly uploader app that only knows one speed: maximum. You would love a way to make that app back off so it doesn’t saturate your uplink and badly impact the other things using your network. Previously\u00a0in … Continue reading →<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[4,6,13,10],"tags":[],"class_list":["post-843","post","type-post","status-publish","format-standard","hentry","category-os-x","category-scripts","category-the-more-you-know","category-tutorials"],"_links":{"self":[{"href":"https:\/\/dreness.com\/blog\/wp-json\/wp\/v2\/posts\/843","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/dreness.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/dreness.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/dreness.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/dreness.com\/blog\/wp-json\/wp\/v2\/comments?post=843"}],"version-history":[{"count":14,"href":"https:\/\/dreness.com\/blog\/wp-json\/wp\/v2\/posts\/843\/revisions"}],"predecessor-version":[{"id":1350,"href":"https:\/\/dreness.com\/blog\/wp-json\/wp\/v2\/posts\/843\/revisions\/1350"}],"wp:attachment":[{"href":"https:\/\/dreness.com\/blog\/wp-json\/wp\/v2\/media?parent=843"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/dreness.com\/blog\/wp-json\/wp\/v2\/categories?post=843"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/dreness.com\/blog\/wp-json\/wp\/v2\/tags?post=843"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}