Difference between revisions of "TigerPage"

From Wikifications
Jump to: navigation, search
(revert)
 
(16 intermediate revisions by 3 users not shown)
Line 2: Line 2:
  
 
== Metadata / Spotlight ==
 
== Metadata / Spotlight ==
 +
===Compound queries with logic===
 
The "Smart Folder" UI does not let you create compound queries with nested logic or "OR". Everything is at the same 'level' of evaluation (in the 'order of operations' sense), and everything is ANDed.
 
The "Smart Folder" UI does not let you create compound queries with nested logic or "OR". Everything is at the same 'level' of evaluation (in the 'order of operations' sense), and everything is ANDed.
  
Line 15: Line 16:
  
 
(note also the use of & to get a literal ampersand in html, heh)
 
(note also the use of & to get a literal ampersand in html, heh)
 +
 +
... and here's the official documentation on the raw query syntax:
 +
 +
===External references===
 +
* Apple's [http://developer.apple.com/documentation/Carbon/Conceptual/SpotlightQuery/Concepts/QueryFormat.html Raw query documentation].
 +
* [http://homepage.mac.com/superpixel/highlight/ Highlight], a GUI tool for doing some SpotLight operations, such as front end for mdls and mdimport.
 +
* [http://www.fluffy.co.uk/spotmeta/ SpotMeta], a front end for easily adding your own custom SpotLight attributes / values. Also, a google cached [http://72.14.207.104/search?q=cache:LEJU59GjsN0J:typo.stateful.net/articles/2006/01/27/spotmeta-is-fb-gtd-bliss+spotmeta+bliss&hl=en&gl=us&ct=clnk&cd=1 writeup].
 +
 +
== networking ==
 +
 +
=== ipfw / dummynet ===
 +
* bandwidth limiting based on UID of socket owner
 +
* dynamically viewing / adjusting xfer rates in realtime
 +
 +
==== Basic Rate Limiting ====
 +
Scenario: I want to limit outgoing bandwidth from a specific service port so it does not saturate my link and lag my ssh sessions. Add the following ipfw rules on the machine hosting the service.
 +
 +
ipfw add pipe 10 ip from any pdps to any
 +
ipfw pipe 10 config bw 40Kbyte/s queue 50KBytes
 +
 +
Where: "pdps" is the service port I'm using (defined in /etc/services); 40Kbyte/s is the rate limit, and 50Kbytes is the size of the queue
 +
 +
Observe your queues as follows:
 +
root@idio[~]ipfw queue show
 +
00010: 320.000 Kbit/s    0 ms  50 KB 1 queues (1 buckets) droptail
 +
    mask: 0x00 0x00000000/0x0000 -> 0x00000000/0x0000
 +
BKT Prot ___Source IP/port____ ____Dest. IP/port____ Tot_pkt/bytes Pkt/Byte Drp
 +
  0 tcp      72.1.149.40/1314    17.216.21.158/50768 11547 16745978 15 22500  5
 +
 +
I'm just getting start on this stuff, the possibilities are endless :)
 +
 +
== sys administration ==
 +
 +
=== launchd ===
 +
The One True Solution to process management.
 +
 +
==== Adding additional listeners ====
 +
Let's say we also want sshd to listen on port 1314.
 +
 +
'''Duplicate the ssh config file'''
 +
root@idio[Library/LaunchDaemons]pwd
 +
/System/Library/LaunchDaemons
 +
root@idio[Library/LaunchDaemons]cp ssh.plist ssh2.plist
 +
root@idio[Library/LaunchDaemons]
 +
 +
'''Edit the new config file as follows'''
 +
 +
First we change the Label, which is the name we use to refer to the service (all Lables must be unique)
 +
<key>Label</key>
 +
        <string>com.openssh.sshd</string>
 +
 +
becomes
 +
 +
<key>Label</key>
 +
        <string>com.openssh-alt.sshd</string>
 +
 +
Then we edit the SockServiceName, which determines which listener port is used
 +
<key>SockServiceName</key>
 +
                        <string>ssh</string>
 +
 +
becomes
 +
 +
<key>SockServiceName</key>
 +
                        <string>pdps</string>
 +
 +
'''Load the new config file'''
 +
root@idio[Library/LaunchDaemons]launchctl load ./ssh2.plist
 +
 +
'''Verify that the new port is open'''
 +
root@idio[Library/LaunchDaemons]netstat -na | grep 1314   
 +
tcp4      0      0  *.1314                *.*                    LISTEN
 +
tcp6      0      0  *.1314                *.*                    LISTEN
 +
 +
You're done!
 +
 +
==== Adjusting resource limits ====
 +
Launchd now superceeds sysctl as far as setting resource limits such as kern.maxfiles, kern.maxfilesperproc, kern.maxproc and kern.maxprocperuid. The defaults are:
 +
 +
<pre>andre@werk[~]launchctl
 +
launchd% limit maxfiles
 +
        maxfiles    256            unlimited     
 +
launchd% limit maxproc
 +
        maxproc    100            532</pre>
 +
 +
To increase these, create and populate /etc/launchd.conf as follows, then reboot.
 +
 +
limit maxproc 500 1000
 +
limit maxfiles 512 1024
 +
 +
Launchd will take care of raising the corresponding sysctl variables, which act as upper limits in the kernel. Even though launchd supplies only kern.maxfilesperproc and kern.maxprocperuid, it still adjusts the other two sysctl values.
 +
 +
==== Sample Configs ====
 +
===== Keeping an app running with a specifc document =====
 +
<pre>
 +
<?xml version="1.0" encoding="UTF-8"?>
 +
<!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
 +
<plist version="1.0">
 +
<dict>
 +
<key>Label</key>
 +
<string>safari</string>
 +
<key>OnDemand</key>
 +
<false/>
 +
<key>UserName</key>
 +
<string>andre</string>
 +
<key>ProgramArguments</key>
 +
<array>
 +
<string>/Applications/Safari.app/Contents/MacOS/Safari</string>
 +
<string>/Users/andre/Desktop/test.webarchive</string>
 +
</array>
 +
<key>ServiceIPC</key>
 +
<false/>
 +
</dict>
 +
</plist>
 +
</pre>
 +
 +
==Mail==
 +
 +
Here's a reference of the items in a tiger mail folder, along with information about whether they can be 'regenerated' to their previous contents after a fresh install:
 +
 +
'''CANNOT be regenrated'''
 +
 +
* Default Counts, LSMMap2 - junk mail training
 +
 +
* Mailboxes - this folder contains local mailboxes
 +
 +
* POP-xyz - contains POP mailboxes, probably CANNOT be regenerated unless you are sure POP was configured to leave messages on server (that is default behavior in Tiger)
 +
 +
* ~/Library/Mail Downloads - mail attachments you have downloaded / accessed - probably CANNOT be regenerated by itself, but of course all the attachments would still be in any IMAP messages.
 +
 +
'''ONLY regenerated with .Mac syncing'''
 +
 +
* ~/Library/Preferences/com.apple.mail.plist - ONLY account settings can be regenerated; other settings are lost
 +
 +
* MessageRules.plist - mail rules
 +
 +
* Signatures - email signatures
 +
 +
* SmartMailboxes.plist - smart mailbox definitions
 +
 +
'''CAN be regenerated'''
 +
 +
* IMAP-xyz - cached IMAP messages
 +
 +
* Mac-xyz - cached IMAP mailboxes from .Mac
 +
 +
* Exchange-xyz - cached IMAP mail from Exchange
 +
 +
* Envelope Index - the SQLite database of the messages (takes lots of time for big mailboxes)
 +
 +
'''Other'''
 +
 +
* OpenedAttachments.plist - ?

Latest revision as of 17:16, 28 July 2007

This will be populated slowly with my own observations / tips, etc about Tiger.

Metadata / Spotlight

Compound queries with logic

The "Smart Folder" UI does not let you create compound queries with nested logic or "OR". Everything is at the same 'level' of evaluation (in the 'order of operations' sense), and everything is ANDed.

However, the underlying APIs fully support such queries, e.g.: 

mdfind -onlyin /Users/andre/foodle "kMDItemFSOwnerUserID == 501 || kMDItemFSOwnerGroupID == 18010"

It is also possible to edit the .savedSearch file that Finder creates when saving a Smart Folder.

Here is an example of an updated "string" attribute from a .savedSearch file that uses an OR:

<string>((kMDItemFSContentChangeDate >= $time.today) &#124;&#124; (kMDItemFSCreationDate >= $time.today)) &amp;&amp; (kMDItemContentType != com.apple.mail.emlx) && (kMDItemContentType != public.vcard)</string>

(note also the use of &#38; to get a literal ampersand in html, heh)

... and here's the official documentation on the raw query syntax:

External references

  • Apple's Raw query documentation.
  • Highlight, a GUI tool for doing some SpotLight operations, such as front end for mdls and mdimport.
  • SpotMeta, a front end for easily adding your own custom SpotLight attributes / values. Also, a google cached writeup.

networking

ipfw / dummynet

  • bandwidth limiting based on UID of socket owner
  • dynamically viewing / adjusting xfer rates in realtime

Basic Rate Limiting

Scenario: I want to limit outgoing bandwidth from a specific service port so it does not saturate my link and lag my ssh sessions. Add the following ipfw rules on the machine hosting the service. 

ipfw add pipe 10 ip from any pdps to any
ipfw pipe 10 config bw 40Kbyte/s queue 50KBytes

Where: "pdps" is the service port I'm using (defined in /etc/services); 40Kbyte/s is the rate limit, and 50Kbytes is the size of the queue

Observe your queues as follows: 

root@idio[~]ipfw queue show
00010: 320.000 Kbit/s    0 ms  50 KB 1 queues (1 buckets) droptail
    mask: 0x00 0x00000000/0x0000 -> 0x00000000/0x0000
BKT Prot ___Source IP/port____ ____Dest. IP/port____ Tot_pkt/bytes Pkt/Byte Drp
  0 tcp      72.1.149.40/1314    17.216.21.158/50768 11547 16745978 15 22500   5

I'm just getting start on this stuff, the possibilities are endless :)

sys administration

launchd

The One True Solution to process management.

Adding additional listeners

Let's say we also want sshd to listen on port 1314.

Duplicate the ssh config file 

root@idio[Library/LaunchDaemons]pwd
/System/Library/LaunchDaemons
root@idio[Library/LaunchDaemons]cp ssh.plist ssh2.plist
root@idio[Library/LaunchDaemons]

Edit the new config file as follows

First we change the Label, which is the name we use to refer to the service (all Lables must be unique) 

<key>Label</key>
       <string>com.openssh.sshd</string>

becomes 

<key>Label</key>
       <string>com.openssh-alt.sshd</string>

Then we edit the SockServiceName, which determines which listener port is used 

<key>SockServiceName</key>
                       <string>ssh</string>

becomes 

<key>SockServiceName</key>
                       <string>pdps</string>

Load the new config file 

root@idio[Library/LaunchDaemons]launchctl load ./ssh2.plist

Verify that the new port is open 

root@idio[Library/LaunchDaemons]netstat -na | grep 1314     
tcp4       0      0  *.1314                 *.*                    LISTEN
tcp6       0      0  *.1314                 *.*                    LISTEN

You're done!

Adjusting resource limits

Launchd now superceeds sysctl as far as setting resource limits such as kern.maxfiles, kern.maxfilesperproc, kern.maxproc and kern.maxprocperuid. The defaults are: 

andre@werk[~]launchctl
launchd% limit maxfiles
        maxfiles    256            unlimited      
launchd% limit maxproc
        maxproc     100            532

To increase these, create and populate /etc/launchd.conf as follows, then reboot. 

limit maxproc 500 1000
limit maxfiles 512 1024

Launchd will take care of raising the corresponding sysctl variables, which act as upper limits in the kernel. Even though launchd supplies only kern.maxfilesperproc and kern.maxprocperuid, it still adjusts the other two sysctl values.

Sample Configs

Keeping an app running with a specifc document
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
	<key>Label</key>
	<string>safari</string>
	<key>OnDemand</key>
	<false/>
	<key>UserName</key>
	<string>andre</string>
	<key>ProgramArguments</key>
	<array>
	<string>/Applications/Safari.app/Contents/MacOS/Safari</string>
	<string>/Users/andre/Desktop/test.webarchive</string>			
	</array>
	<key>ServiceIPC</key>
	<false/>
</dict>
</plist>

Mail

Here's a reference of the items in a tiger mail folder, along with information about whether they can be 'regenerated' to their previous contents after a fresh install:

CANNOT be regenrated

  • Default Counts, LSMMap2 - junk mail training
  • Mailboxes - this folder contains local mailboxes
  • POP-xyz - contains POP mailboxes, probably CANNOT be regenerated unless you are sure POP was configured to leave messages on server (that is default behavior in Tiger)
  • ~/Library/Mail Downloads - mail attachments you have downloaded / accessed - probably CANNOT be regenerated by itself, but of course all the attachments would still be in any IMAP messages.

ONLY regenerated with .Mac syncing

  • ~/Library/Preferences/com.apple.mail.plist - ONLY account settings can be regenerated; other settings are lost
  • MessageRules.plist - mail rules
  • Signatures - email signatures
  • SmartMailboxes.plist - smart mailbox definitions

CAN be regenerated

  • IMAP-xyz - cached IMAP messages
  • Mac-xyz - cached IMAP mailboxes from .Mac
  • Exchange-xyz - cached IMAP mail from Exchange
  • Envelope Index - the SQLite database of the messages (takes lots of time for big mailboxes)

Other

  • OpenedAttachments.plist - ?
Retrieved from "https://dreness.com/wikimedia/index.php?title=TigerPage&oldid=1224"