Cross-application URL cache searching!

I was out taking a (filesystem) walk, and wandered into /var/folders. Not remembering where / when this came from, I was pleased to find that the hier(7) manpage is aware of this directory, describing it as ‘per-user temporary files and caches’. More research reveals that this directory structure is created as defined by some code in Apple’s Libc as of Leopard. Some of the constants are defined in dirhelper_priv.h, and the corresponding _dirhelper.c does the creating and locating of directories – note the fancy two-level hashing.

Walking around more, I find that I’ve got a directory for myself (as observed via the file permissions) with the following contents:

drwx------  35 andre  staff  1190 Oct 27 19:34 -Caches-
drwx------  12 andre  staff   408 Oct 27 21:03 -Tmp-
drwxrwxrwx@  3 andre  staff   102 Oct 27 20:42 Cleanup At Startup
drwxr-xr-x   2 andre  staff    68 Oct 27 19:26 TemporaryItems

The -Caches- folder contains application directories in the reverse domain notation, e.g., or or com.blizzard.worldofwarcraft. All of these directories seem to have at least one file: Cache.db. hmm… ok, database. fun. Let’s see what’s in it!

{56} andre@gyro [-Caches-/] % file Cache.db
Cache.db: SQLite database (Version 3)

Cool! I can do sqlite3…

{74} andre@gyro [-Caches-/] % sqlite3 Cache.db
SQLite version 3.4.0
Enter ".help" for instructions
sqlite> .tables
cfurl_cache_blob_data       cfurl_cache_schema_version
sqlite> .schema cfurl_cache_response

(this produces output with really long lines, so let’s hop back to the shell right quick)

sqlite3 Cache.db ".schema cfurl_cache_response" | fold -s
UNIQUE, 		     version INTEGER, hash_value INTEGER,
storage_policy INTEGER, request_key TEXT UNIQUE, 		     time_stamp
CREATE INDEX request_key_index ON cfurl_cache_response(request_key);
CREATE INDEX time_stamp_index ON cfurl_cache_response(time_stamp);

ok, back in sqlite3 interactive mode now:

sqlite> .mode line
sqlite> select * from cfurl_cache_response where entry_ID=1;
      entry_ID = 1
       version = 0
    hash_value = -1475899696
storage_policy = 0
   request_key =
    time_stamp = 2008-10-28 01:55:10
sqlite> select * from cfurl_cache_response where entry_ID<4;
      entry_ID = 1
       version = 0
    hash_value = -1475899696
storage_policy = 0
   request_key =
    time_stamp = 2008-10-28 01:55:10

      entry_ID = 2
       version = 0
    hash_value = -763438389
storage_policy = 0
   request_key =
    time_stamp = 2008-10-28 01:55:10

      entry_ID = 3
       version = 0
    hash_value = 1822950361
storage_policy = 0
   request_key =
    time_stamp = 2008-10-28 01:55:10

Neat! Ok ok, here’s the cross-app url cache searching code:

# Here's a little ditty that does keyword searching across all the URLs in your
# /var/folders caches directory - the one returned by
# /usr/bin/getconf DARWIN_USER_CACHE_DIR

# Search term comes in as first command line argument
# -v is an optional second argument to list databases are they are searched

if [ ! $SEARCH ]
then echo "Usage: $0 [search term] [-v]"

if ( [ -n "$2" ] && [ $2 == "-v" ] )
then VERBOSE=1


ls -1 $CACHE/*/Cache.db | while read db
	echo -n ""
	if [ $VERBOSE == "1" ] ; then echo "   Searching $db" ; fi
	sqlite3 $db "select request_key from cfurl_cache_response where \
		request_key LIKE'%$SEARCH%';"

Probably safe to not depend on any of this being available or staying the way it is in future versions of Mac OS X… but for now at least, it’s here and contains data. In case you were wondering, Safari doesn’t log here when private browsing is enabled :)

About dre

I like all kinds of food.
This entry was posted in OS X, scripts, The More You Know. Bookmark the permalink.

3 Responses to Cross-application URL cache searching!

Leave a Reply