Caching Apple Downloads

14th July 2015

The worst part of working with iPads is waiting for apps to download, watching your internet connection strain at 40+ devices downloading the 599MB Garage Band update. The 10Mb/s connection in the school this article was written in is spread over nearly 200 devices so these iPad updates are causing us a noticeable headache.

Apple do offer a caching server as part of OSX Server which sounds on paper like a good solution to the problem but schools are often behind firewalls outside of their control on large networks that Apple never planned for. There are quite a few issues at the moment within Lancashire due to schools being told to use a caching server at another school that they can’t access.

These issues rule out an Apple caching server, for me they have too much potential for wierdness and to just not work at all.

So I looked at squid and weather it could be used as a cache and as it turns out it can! I found a great article by Luke Arms over on his blog which explains how to ignore Apple’s HTTP headers and cache the downloads.

That article assumes that you already use squid which we don’t and more importantly for us our LEA uses a filtering system that uses the devices IP to apply filtering rules and log usage etc… a proxy would mess that right up. This can be solved but first I need to setup squid.

For another iPad issue I already have a debain box running which I can easily add squid to so I quickly installed the squid3 package and changed /etc/squid3/squid.conf to this:

1http_port 3128
3hierarchy_stoplist cgi-bin ?
4cache_mem 1024 MB
5cache_dir aufs /squid_cache/cache 81920 16 256
6maximum_object_size 5120 MB
8cache_store_log /squid_cache/store.log
9coredump_dir /var/spool/squid3
11refresh_pattern ^ftp: 1440 20% 10080
12refresh_pattern ^gopher: 1440 0% 1440
13refresh_pattern -i (/cgi-bin/|\?) 0 0% 0
14refresh_pattern . 0 20% 4320
16refresh_pattern -i appldnld\.apple\.com 129600 100% 129600 ignore-reload ignore-no-store override-expire override-lastmod ignore-must-revalidate
18refresh_pattern -i phobos\.apple\.com 129600 100% 129600 ignore-reload ignore-no-store override-expire override-lastmod ignore-must-revalidate
20quick_abort_min -1 QB
22read_ahead_gap 1 MB
24positive_dns_ttl 30 seconds
25negative_dns_ttl 1 second
27minimum_expiry_time 600 seconds
28chunked_request_body_max_size 4096 KB
30acl manager proto cache_object
32acl localhost src ::1
33acl to_localhost dst ::1
35acl curric src
37http_access allow manager localhost
38http_access deny manager
40http_access deny to_localhost
42http_access allow curric
43http_access allow localhost
Language js

This file is specific to my network and will need changing if you copy it.

cache_dir is the directory squid saves its cache to, I used /squid_cache which a separate partition mounted to that point. I also threw cache_store_log into the same folder.

acl curric src is the IP range the iPads are on.

Everything else should be pretty uniform across networks.

At this point squid is running as a cache box and can be used by any device, if used as a device’s proxy server it would work great and cache everything. I don’t want that, not only will it ruin the system we get from the LEA, it might result in Apple cached content being dropped to make room for something new.

To only use this proxy server for Apple downloads I used a PAC file hosted on the IIS server that hosts the school intranet page.

1function FindProxyForURL(url,host){
2 appleURLS= ["*phobos*","*appldnld*"];
4 for(i=0; i<appleURLS.length; i++){
5 if(shExpMatch(host, appleURLS[i])){
6 return "PROXY;";
7 }
8 }
10 return "DIRECT";
Language javascript

Once configured to use this file our iPads will go DIRECT to any URL unless it contains or which it will send to the cache box.

I saw an improvement straight away and the cache logs confirmed that the first iPad caused the cache box to cache the download and then the following iPads were served from the cache instead.

I will keep this article up to date with any improvements I make over time but for now it seems pretty good.