Tiger Server servermgrd library for Python

motivation
I finally found a reason to write some Python. Being something of a monitoring and data junkie, I’ve had a fair amount of experience with snmp, data mining scripts, etc. After writing this post to the server list, I figured I’d make some templates for snmp that published interesting pieces of data about the server. A lot of good stuff can be retrieved through servermgrd, the ‘Mac OS X Server administrative daemon’, which is basically a little web service that uses xml plists to do request / response transactions. Usually the only software that talks to servrmgrd is Apple’s Server Admin utility, but the enterprising sysadmin can strike up his or her own conversations. This post documents my python baby steps, as well as the birth of a tiny python library I wrote for simplifying servermgrd interactions.

If you happen to have a Mac OS X Server handy, point your web browser at . Accept the ol’ SSL warnings, and then you’ll be presented with a list of servermgrd modules. Each module has its own html / cgi wrapper which provides a menu of request templates, e.g.

<?xml version="1.0" encoding="UTF-8"?>
<plist version="0.9">
<dict>
    <key>command</key>
    <string>getState</string>
    <key>variant</key>
    <string>withDetails</string>
</dict>
</plist>

Different modules support different commands, some with optional arguments (e.g. variant, timescale). Clicking ‘Send Command’ will do just that, and you’ll see the results as returned by servermgrd. Not surprisingly, the result is also an xml plist.

We’ll definitely want some sort of plist parsing library to let us grab ahold of this data in a fairly painless way. This was ultimately why I chose Python for the task; because Mac OS X ships with a little Python library called plistlib. plistlib is pretty basic; you give it a plist and it will hand over a data structure with all the stuff in it.

Dive in: the python interpreter, arrays, and dicts in like 3 minutes
First let’s find a plist we can use to test. We’ll need to convert it to xml format, as many plists are stored on disk in a biniary format these days. Let’s take our Dock plist and make an xml copy of it at ~/test.plist

plutil -convert xml1 -o test.plist ~/Library/Preferences/com.apple.dock.plist

Now let’s try plistlib through an interactive python session.

{29} andre@donk [~] % python
Python 2.3.5 (#1, Aug 12 2006, 00:08:11)
[GCC 4.0.1 (Apple Computer, Inc. build 5363)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import plistlib
>>> pl = plistlib.Plist.fromFile('test.plist')
>>>

Cool, no errors! (right? ;) Simply say the name of the object to see its contents:

>>> pl

Woof, lots of output. Here’s where it gets fun. This is a dict, which a collection of property / value pairs. let’s iterate through the items.

>>> for item in pl : print item
...
wvous-br-corner
orientation
mod-count
tilesize
trash-full
persistent-apps
pinning
wvous-br-modifier
version
launchanim
autohide
persistent-others
checked-for-dashboard

Those are properties. Properties have values.

>>> pl['trash-full']
True
>>> pl['launchanim']
False

Iterate through all the items and display them as key –> value pairs:

>>> for item in pl : print item, ' --> ', pl[item]
...

Let’s drill down into that persistent-apps item. It appears to be a bunch of nested stuff, all wrapped in a single list (array). Let’s list the properties of one of the apps; we’ll pick the first one (list index 0).

>>> for item in pl['persistent-apps'][0] : print item
...
tile-data
tile-type
GUID

That’s a bit more managable. Let’s examine these.

>>> pl['persistent-apps'][0]['GUID']
289528741
>>> pl['persistent-apps'][0]['tile-type']
'file-tile'
>>> pl['persistent-apps'][0]['tile-data']
Dict(**{'parent-mod-date': 3264304302L, 'file-label': 'System Preferences', 'file-data': ...

tile-data has more nested stuff, but now we can easily pick out file-label as the human readable name of the thing. Remember how we got here? Iterate through all the persistent-apps, look into the tile-data dict, then print the value of the file-label property

>>> for app in pl['persistent-apps'] : print app['tile-data']['file-label']
...
System Preferences
iChat
Mail
iTunes
Safari
Terminal
Quake 4
TextEdit
World of Warcraft
Hex Fiend

Pretty much everything that comes out of plistlib is either a dict or an array, and there’s often some nesting, so you sorta have to grope the format a bit to figure out how you want to access the data. Definitely beats the crap out of hand parsing :)

what about servermgrd?
servermgrd can be harnessed either via http or a shell. We’ll use a shell. In that mode, the request is delivered via standard input, and the result is… well ya know. output. Let’s build a request. After playing with the web interface for a while, its obvious that the various requests are formated the same way. The value of the ‘command’ property is the most significant part. Some commands have additional parameters, such as ‘variant’ or ‘timescale’. Our request-building function shall be called buildXML.

def buildXML ( command, variant, timescale ) :
  request = """<?xml version="1.0" encoding="UTF-8"?>
<plist version="0.9">
<dict>
        <key>command</key>
        <string>"""
  request = request + command
  request = request + '</string>'
  if timescale != '' :
    request = request + """
        <key>timeScale</key>
        <integer>"""
    request = request + timescale
    request = request + '</integer>'
  if variant != '' :
    request = request + """
        <key>variant</key>
        <string>"""
    request = request + variant
    request = request + '</string>'
  request = request + """
</dict>
</plist>"""
  return request

buildXML is called like this:

request = buildXML('getHistory', 'v1+v2', '60')

Sometimes you need not specify anything more than command. In these cases, supply a null value for any unused parameters using empty single quotes.

Now that we have a request, we can send it to servermgrd by opening a pipe. We’ll use popen2 so we can grab both STDIN and STDOUT. The filesystem path we use depends on the name of the module we’re targetting. Here is our sendXML function, which is called with the name of the servermgrd module and the xml request.

def sendXML ( servermgrdModule, request ) :
  modulePath = '/usr/share/servermgrd/cgi-bin/'+servermgrdModule
  pipeIn, pipeOut = os.popen2(`modulePath`)
  print >>pipeIn, request
  pipeIn.close()
  xmlresult = pipeOut.read(20480)
  pipeOut.close()

We now have xmlresult, which is a string containing the entire result body from servermgrd. Plistlib is accustomed to parsing plists from files, but we don’t want to write this data to the filesystem because we don’t want to keep it. Instead, we’ll finish this function by creating a file-like object (like a file, but without any of that annoying disk access) which contains the result, using the StringIO library, then hand that file-like object to plistlib. plistlib parses the xml into native data structures (dicts, arrays), and we return the result.

  xmlFauxFile = StringIO.StringIO(xmlresult)
  return plistlib.Plist.fromFile(xmlFauxFile)

Examples
I recommend exploring the XML plist you can get from servermgrd through the eyes of the python interpreter using the techniques demonstrated with the Dock example. To get started, simply download the servermgrd.py library, then enter the python interpreter as root on a Tiger Server, while your current directory is the same as servermgrd.py’s directory (or else the import fails).

{131} root@tiny [~] # python
Python 2.3.5 (#1, Jul 25 2006, 00:38:48)
[GCC 4.0.1 (Apple Computer, Inc. build 5363)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import srvrmgrdIO
>>> request = srvrmgrdIO.buildXML('getHistory', 'v1', '60')
>>> pl = srvrmgrdIO.sendXML('servermgr_info', request)
>>> pl
Plist(**{'v2Legend': 'NETWORK_THROUGHPUT', 'v5Legend': 'NETWORK_THROUGHPUT_EN0', ...

Here’s some quick examples of how this library can be used. There is much more available data than is being shown.

#!/usr/bin/python
# We require the srvrmgrdIO module to prepare the request and talk to servermgrd
import re
import srvrmgrdIO
import time

print 'network bytes / second over the last 15 minutes:'
request = srvrmgrdIO.buildXML('getHistory', 'v1+v2', '900')
pl = srvrmgrdIO.sendXML('servermgr_info', request)
for s in pl['samplesArray'] :
  print s['v1'], 'at', time.ctime(s['t'])
print ""

# dns - this one's real slow for some reason...
#request = srvrmgrdIO.buildXML('getStatistics', '', '')
#pl = srvrmgrdIO.sendXML('servermgr_dns', request)

#print "DNS: success / fail / recursive / referral / nxdomain"
#print `pl['success']` + ' /',
#`pl['failure']` + ' /',
#`pl['recursion']` + ' /',
#`pl['referral']` + ' /',
#`pl['nxdomain']`
#print ""

# afp connected users
request = srvrmgrdIO.buildXML('getConnectedUsers', '', '')
pl = srvrmgrdIO.sendXML('servermgr_afp', request)
print "AFP Users:"
for u in pl['usersArray'] :
  print u['ipAddress'] + " ==> " + u['name']
print ""

# dirserv
print "Directory Services"
request = srvrmgrdIO.buildXML('getState', 'withDetails', '')
pl = srvrmgrdIO.sendXML('servermgr_dirserv', request)

for s in pl :
  if re.search("stat", s, re.I) : print s," ==> ",`pl[s]`

When executed on my server:

{4} root@tiny [~] # ./satest.py
network bytes / second over the last 15 minutes:
38 at Mon Jun 11 19:56:14 2007
35 at Mon Jun 11 19:55:14 2007
35 at Mon Jun 11 19:54:14 2007
35 at Mon Jun 11 19:53:14 2007
38 at Mon Jun 11 19:52:14 2007
35 at Mon Jun 11 19:51:14 2007
43 at Mon Jun 11 19:50:14 2007
43 at Mon Jun 11 19:49:14 2007
33 at Mon Jun 11 19:48:14 2007
38 at Mon Jun 11 19:47:14 2007
40 at Mon Jun 11 19:46:14 2007
36 at Mon Jun 11 19:45:14 2007
35 at Mon Jun 11 19:44:14 2007
26 at Mon Jun 11 19:43:14 2007
36 at Mon Jun 11 19:42:14 2007

AFP Users:
10.0.1.201 ==> andre
10.0.1.6 ==> andre

Directory Services
timState  ==>  'STOPPED'
ldapdState  ==>  'RUNNING'
kdcStatus  ==>  'RUNNING'
lookupdState  ==>  'RUNNING'
passwordServiceState  ==>  'RUNNING'
state  ==>  'RUNNING'
netinfodState  ==>  'RUNNING'
netinfodParentState  ==>  'STOPPED'