Fixing a Mac OS X Spotlight Index, that Doesn't

After a spate of Mail.app problems probably related to having too damn many mail accounts and messages, I had to do the "Mail.app Reimport Samba" taking hours to let Mail.app reimport the entire mail store. After that got resolved, I found out that my Mail.app "Entire Message" selection in Search was greyed out. This turned out to be a symptom of a problem with the Spotlight index, since this search function of Mail.app is dependent upon the Spotlight index of the hard drive where your mail is stored. Generally speaking, the Solution was to delete the Spotlight index, and then re-index the drive. However, it was not a simple process (is anything?!), and as such was very "un-Apple", so I thought I would take the time to document it.

How to Make Spotlight Re-index Your Drive, Even When She Says No!

A.k.a., How to Fix Greyed Out "Entire Message" in Mail.app Search. As I was learning to do this procedure in my OS X 10.6.3 system, looking at the Apple forums and elsewhere, I observed that Spotlight was taking "forever" to re-index, while it should re-index a 500GB drive in an hour or so. When I clicked the "pulsating" Spotlight icon in the upper right of the screen (it pulsates or winks while re-indexing), I saw a message that Spotlight was calculating the time required to index the drive with a barber pole progress bar, but it was just stuck in this state for days. This prompted me to try various things to fix it, and no one method from any one forum ever worked for me, in practice.

So, let me try to explain what did work.

Remove the Extraneous

Disconnect any external drives and shut down any unneeded programs.

Remove Unneeded Spotlight Importers

Spotlight importers which allow programs to get their output files indexed are sometimes the cause of Spotlight crashes. You can find them in /Library/Spotlight and ~/Library/Spotlight (where ~ is your home directory, of course). If you have third-party importers, especially in ~/Library/Spotlight, you can move them to another folder for safekeeping, then move them back one by one to see if the "indexing forever" problem recurs. I deleted ones that were associated with programs I never use, but kept the ones I do use.

You can search the entire drive for the .mdimporter files, using this command. Hat tip to "Hal Itosis" (lol) on the MacFixit Forum.

# find -x / -iname \*.mdimporter -exec ls -lndotT {} +

Repair Disk Permissions

I use the excellent Cocktail for this but you can do it in 3rd Party OnyX or Apple's included Disk Utility as well. In Cocktail, use the Disks menu, Permissions tab. Choose to Reset Permissions and ACLs for Home Directories for All Users, then click Repair. This takes about 30 mins to execute.

Prepare the Index

Using Terminal, use launchctl to unload the indexer by controlling the appropriate launchd command, use mdutil to turn off indexing for the root folder (the /) of the local hard drive, use rm to delete the index itself (.Spotlight-V100 in the root of the drive), and finally trash Spotlight's plist.

$ sudo bash

Password: *******

# launchctl unload -w /System/Library/LaunchDaemons/com.apple.metadata.mds.plist

# mdutil -i off -E /

# cd /

# rm -rf .Spotlight-V100

# rm -rf ~/Library/Preferences/com.apple.spotlight.plist

# exit

$

To explain, note the sudo bash and exit lines, bookending the procedure. This gets you a root prompt (the #) after you enter the admin password and then exits the root prompt at the end (returning to the $ prompt). You can also use sudo before every command, to be extra safe.

Clear All Caches

Noting that this part will force a reboot, again in Cocktail Files, Caches, choose Options and select all caches you can. Executing this resets the system to a state close to what you get when you install anew or do a major upgrade. Note that the system will take longer to restart because it is rebuilding caches, and applications will be sluggish the first time they restart, because they are rebuilding font caches.

Click Clear to execute the cleaning procedure and wait while it finishes. Choose to let the system restart, but when you hear the startup chime, hold down Shift so the system restarts in Safe Mode.

Safe Mode Index Rebuild

The system will take a while to get into Safe Mode. After you press and hold Shift while starting up, once you see the progress bar during startup (which is not present in normal mode), you can let go of Shift and go get coffee while it starts up. The login prompt will have a red Safe Mode in it, to alert you to the difference.

Again in Terminal, issue some commands to rebuild.

$ sudo bash

Password: *******

# launchctl load -w /System/Library/LaunchDaemons/com.apple.metadata.mds.plist

# mdutil -i on -E /

/:

Indexing enabled.

# exit

$

After a few minutes you should see the Spotlight icon start to pulsate, but this time, the "Calculating Indexing Time" should change to some value, and a blue progress bar, which means the indexing is actually working. You can restart into regular mode, and the indexing just picked up where it left off, for me. However, do not connect any additional drives until it is done indexing the main drive.

Observing and Confirming

There are a few things you can do to observe and confirm Spotlight's activities, in Terminal. This assumes you are at a root prompt.

Check the size of the Spotlight index using du (directory usage):

# du -hsc /.Spot*

1.1G /.Spotlight-V100

1.1G total

Note that this should be a fairly large file. If it is only a megabyte or so, something is broken or your Spotlight index is off.

Check Spotlight-related processes with ps (process lister):

# ps axcru |sed '1p;/ md/!d'

USER PID %CPU %MEM VSZ RSS TT STAT STARTED TIME COMMAND

root 352 0.0 2.8 3696728 117196 ?? Ss 8:44AM 24:25.73 mds

_spotlight 6677 0.0 0.2 2532748 10292 ?? SNs 3:19AM 0:00.33 mdworker

rcogley 6664 0.0 1.2 2638620 48240 ?? SNs 3:19AM 0:02.95 mdworker

Check index status with mdutil.

# mdutil -as

/:

Indexing enabled.

/Volumes/COGLEY-WD:

Indexing enabled.

/Volumes/COGLEY-WD/Backups.backupdb:

Indexing enabled.

/Volumes/COGLEY-WD2:

Indexing enabled.

/Volumes/COGLEY:

Indexing enabled.

List open files in /System/Library related to Spotlight, using lsof (list open files).

# lsof -c md |grep -v /System/Library |grep -v Spotlight

Get general system information with df and diskutil:

# df -lh

# diskutil info /

Show commands you have entered in the bash shell:

# history

Note that if you use an alternative shell like fish, some of the above commands will not work. From fish, just do sudo bash to get a bash-based root prompt. Be sure also to check the Console GUI application in /Applications/Utilities. This will show crash information for the mdworker program, which may give you hints to what is going on.

I hope this article helps someone, especially since this problem seems to be pretty prevalent, these days. Enjoy!

Opening a Tab-Delimited "CSV" in Apple iWork Numbers

I like Numbers, from Apple's competent and beautiful iWork suite, but there are still things where it is different from Microsoft Excel, and hence it feels somewhat unfamiliar at times. For example, although Excel has never been good at handling text files saved in UTF-8 format (a big deal for those of us who work in Asia), it does have a nice Wizard for importing CSV or TSV text files.

Some applications export text files with tab-separated values, and put a CSV extension on them. Actually, CSV stands for "Comma Separated Values" where these are really "TSV" or "Tab Separated Values." When you try to import such a CSV file, Numbers will mash all the fields into one cell in the left-most column. That makes sense, because it's looking for commas judging by the CSV extension. Finding none, it just lets the data pile on.

How to Import a Tab-Separated "CSV" in Numbers

If you have a tab-separated CSV, here's how to open it:

1. Save the file somewhere you can find it.
2. Rename the file in Finder, so that it has a .TXT extension.
3. Ctrl-click the file and choose "Open With" and "Numbers".

Now the data will open correctly, with each field getting its due, and you'll get the added benefit of Numbers not munging any Japanese or Chinese characters.

Hope this tip helps someone. Enjoy!

Corrupt Apple Leopard Server Open Directory Services

I had a Leopard Server crash and burn so that nothing was responding, and when I forced the server to reboot (as well as rebooting a bunch of other ancillary servers and services just in case), I found an ominous sign in Server Admin, along with no user accounts in Workgroup Manager. Eek! Server Admin's Open Directory showed:

LDAP Server is: stopped

Password Server is: running

Kerberos is: stopped

Not good. Never fear, though.

How to Fix a Corrupted Open Directory

First, don't panic. Apple's forums show you can use "

slapd -Tt

" to check the configuration.

myhost:~ administrator$ sudo bash

Password: ********

bash-3.2# /usr/libexec/slapd -Tt

overlay_config(): warning, overlay "dynid" already in list

overlay_config(): warning, overlay "dynid" already in list

overlay_config(): warning, overlay "dynid" already in list

overlay_config(): warning, overlay "dynid" already in list

overlay_config(): warning, overlay "dynid" already in list

bdb(dc=myhost,dc=mydomain,dc=com): PANIC: fatal region error detected; run recovery

bdb_db_open: Database cannot be opened, err -30978. Restore from backup!

bdb(dc=myhost,dc=mydomain,dc=com): DB_ENV->lock_id_free interface requires /

an environment configured for the locking subsystem

backend_startup_one: bi_db_open failed! (-30978)

slap_startup failed (test would succeed using the -u switch)

The "run recovery" here means to run the

db_recover

command (a.k.a.

slapd_db_recover

on other *nix LDAPs). Use the -v switch to make the result verbose.

bash-3.2# db_recover-v -h /var/db/openldap/openldap-

openldap-data/ openldap-slurp/

bash-3.2# db_recover -v -h /var/db/openldap/openldap-data/

db_recover: Finding last valid log LSN: file: 6 offset 4190936

db_recover: Recovery starting from [6][4190795]

db_recover: Recovery complete at Thu Oct 15 21:57:41 2009

db_recover: Maximum transaction ID 80000225 Recovery checkpoint [6][4190936]

Ah, that looked nice. Then run

slapd -Tt

again to test, and if all is well, exit out of the sudo'ed bash shell.

bash-3.2# /usr/libexec/slapd -Tt

overlay_config(): warning, overlay "dynid" already in list

overlay_config(): warning, overlay "dynid" already in list

overlay_config(): warning, overlay "dynid" already in list

overlay_config(): warning, overlay "dynid" already in list

overlay_config(): warning, overlay "dynid" already in list

config file testing succeeded

bash-3.2# exit

myhost:~ administrator$

After a few minutes

launchd

should kickstart the Open Directory services again so that you see:

LDAP Server is: running

Password Server is: running

Kerberos is: running

A couple of tests shows I once again have Wiki Server, iCal Server, Jabber Chat etc, all the Open Directory and Kerberos-based services back on line. Breathe a sigh of relief if this helped you and let me know in the comments!

Linking File Types and Apps in OS X

If you are an OS X user, and you find files of a certain type, say PDFs, are opening in one applications but you want them to open in a different one, you can easily change the association using Finder.

How to Re-associate File Types with Applications in OS X

Here's how:

1. Select a file in Finder and ctrl-click it.
2. Select "get info" from the context menu.
3. Find the "open with" section in the "get info" menu that appears.
4. Select your desired application from the drop down list.
5. Click "change all" to set the association between that file type and the application you selected.

This has worked in OS X Tiger, Leopard and Snow Leopard. You can use this method to, say, open all PDFs in the native OS X "Preview", Adobe's Acrobat, or Skim, for instance. Please leave a comment if this helped you. Enjoy!

iPhone OS 3.0 Tips on Parade

Apple released yesterday an updated iPhone operating system, iPhone OS 3.0, with a number of useful improvements. For me, the update itself was seamless, and took about 15 minutes after clicking the update button in iTunes. I upgraded first thing in the morning, but colleagues who tried later in the day had some delays. I guess it's to be expected when so many people try to download at the same time.

Some Tips for iPhone OS 3.0

I took some screenshots of the obvious and not-so-obvious (by pressing the home and power buttons simultaneously in case you aren't aware how), so I'll introduce those to you now.

If you have MobileMe you can enable Find My iPhone and track where the phone is if it is stolen, via GPS. You can also remote-wipe and send messages and sounds to your iPhone. Maybe you won't get the iPhone back, but you can at least antagonize the thief! The sounds and messages will play even if it's locked, so this will help if the iPhone is in "manner" mode.

I suspect it will be useful for when it's in manner mode but lost somewhere in the nether regions of your sofa.

iPhone OS 3.0 now finally has select, copy and paste, and it works in and across various applications. Even the new TweetDeck for iPhone that was just released supports it already. Press and hold to get the selection "handles" and various commands.

Once something is on the clipboard you can paste. You can see I copied from TweetDeck to Mail. The new clipboard also honors rich text, which means you can copy something like formatted text from a webpage in Safari, to a Mail Message, and it will paste it in just as you saw it on the web. Slick!

You can also select in mail to "quote" part of a message before replying. It works more like Leopard OS X.

iPhone OS 3.0 now has Spotlight. You can swipe the left-most home screen right to get spotlight search, and you'll see that the page icon (the dots at the bottom) is actually a magnifying glass. Spotlight search works across apps, and like big sister OS Leopard OS X, also works to find apps so you can launch them. Check your settings as well - you can link "home button double-press" to Spotlight search if you like.

Lastly, you'll find that many apps with lists, like Address Book and Mail have a search box at the top. This is Spotlight, but as you scroll down through the entries, you can double-tap the very top of the screen (near the clock) to jump back to the top. Try it!

In Address Book you can now send an Address Card as a vCard entry via Email or MMS (which Softbank Japan supports). Scroll down to the bottom of a contact to find the share buttons.

Here's what a shared vCard in .vcf format via Email looks like. Note also another new feature: the Landscape keyboard is available in more apps, negating the need for apps like EasyWriter.

While we're on Mail, you can move messages instead of just deleting them in the Edit screen. Convenient for stuff like spam.

Neat stuff on the podcast front, too. In iPod, you can control podcast playback better, with the "jump back 30 seconds" and the speed changer buttons. There's also a button for emailing the actual link to the podcast so you can share it with friends.

When you click the email icon from the podcast, you get a pre-formatted email with a link to the podcast.

There are plenty of tiny new tweaks, including this almost incomprehensible one, showing you a very small thumbnail of the last photo you snapped. Maybe it's fuzzy because I merely have a 3G and not a shiny new 3GS.

A couple more things:

• MS Exchange integration is much better, but note that the interface in Address books allows you to "view all", so it might look like you have duplicates if you keep your MobileMe and Exchange address books synced.
• iTunes now lets you buy & download whatever's in the iTunes store.
• iPod lets you "shake to shuffle".
• There's a new basic Voice Recorder app that lets you record and share voice memos. Nice.
• They've improved the keyboard "secret" hold-to-see shortcuts. Check it out by going to the numbers and symbols keyboard, then holding down various keys.
• While browsing in Safari, you can press and hold a link to get a popup that lets you open the link in another tab.
• You can shake to undo in the main applications like Mail or Messaging.

If you own an iPhone, you can upgrade for free, and there's a small fee to upgrade if you have an iPod Touch. Try it, I'm sure you'll like it. It really makes the iPhone an even more attractive platform. Enjoy!

Disc Burnin' with Disco

Disco is a polished, inexpensive (USD 29.95, currently 10 dollars off as of 13 May 2009) alternative to Toast Titanium. I've had mixed results with Toast, and did not want to pay the upgrade fee, so I started looking for alternatives. I've done a few burns in Disco so far, and it seems to work quite well. No problems or "coasters" yet, at this time.

Here's some highlights:

• Disco supports spanning of a backup set to multiple discs, and lets you know how many discs you will need to complete the backup. Very convenient, and you just feed it the disc media when prompted, while it works.
• Disco stores a discography for created media or disc images, so you can easily search for files on your media.
• Straightforward workflow, assisted by what the publisher calls "Crossroads" technology. It's unnoticeable, and I had no problems following, so in a way I guess it works!
• Creation of CDs or DVDs with Hybrid, HFS+, UDF, PC Joliet, or ISO 9660 file systems, which means you'll be able to read the media most anywhere.
• Burn to all Apple-supported in- and external drives, and enjoy CD multi-session support, burn and erase for rewritable media, and support for dual-layer DVDs.
• If your audio is supported by Quicktime, you can create an audio CD or easily switch to a MP3 data CD. Also, you can drag-and-drop to change track order.
• You can create disc images from files and discs in the CDR, ISO or DMG formats, as well as create CDs from ISO, DMG, IMG or CUE/BIN.

Oh, I almost forgot that it "smokes" when you burn, provided a new-enough Mac. Pretty cool effect!

Assessment after Some Usage

I used Disco for a few burns (the demo allows you 7, gratis), and I found the program to be a tag buggy here and there. There's times where restarting Disco will help it detect discs, or, where it mis-detects the burner you want to use. I have a San Disk Cruzer USB stick, which for some reason Disco thinks is a burner, and it gets "stuck" on it until I go into Disco Preferences and change it.

Path Finder - Collapsing All Folders

I could not find this information in the help file or on the CocoaTech forum. If you use CocoaTech's Path Finder, the Finder replacement that has a lot of extra functionality that is missing in the normal OS X Finder, you may be browsing around in a folder with many subfolders, and wish to somehow "collapse" the folders you opened, to clean up the view.

Clicking the disclosure triangle of 100 open subfolders is rather tedious, so isn't there an easier way? I stumbled on a method. Here it is.

How to Collapse All Subfolders in Path Finder

1. In list view, select all. Command-A.
2. Press Option-Left Arrow.
3. Sit back, amused, but don't do anything boneheaded like pressing Command-Delete.

And voila! Collapsed folders.

Mac OS X Leopard 10.5.7 Update

I updated my MacBook Pro's OS X from 10.5.6 to 10.5.7 today, and there was a little hiccup. Here's what happened, but see UPDATES below:

• I noticed there was an update available, so I shut down all the user applications and things like ExpanDrive and Plaxo, then tried Software Update from the Apple menu. Software update started to work, then stalled at around 150 MB of 450 MB downloaded, claiming to take another 8 hours to download. It's not my ISP because everything else is fine. Everyone must be hitting the server at once. Stopped the update during the download.
• Intending to use the "combo updater", which is just the full update download, I found the update on Apple's download site. There are two updaters: the Delta Updater, and the Combo Updater. The 450+ MB Delta is just labeled as "Update" and is for updating from 10.5.6 to 10.5.7. The 700+ MB Combo is for any 10.5, to 10.5.7. Decided to try the Delta.
• Apple Installer offers to run the updater when it downloads, so I chose to do that. It seemed to work, then offered to Restart so I did that.
• After the restart, the process froze on a blue screen with a mouse pointer.
• I force restarted, then after the restart, the system auto-rebooted.
• After the second restart, the expected normal grey Apple Logo screen with the progress spinner appeared. It took a long time, but it finally came to the login screen, and login was normal.
• I confirmed that the system did indeed it had been upgraded, by checking Apple menu, About this Mac. It's indeed updated to 10.5.7.

When I started Mail and iCal after the upgrade, there were no "please wait while we update your database" type of messages, as there have been in the past when Apple has made database changes to these applications.

I'm going to do an OnyX run, to clean things up and reset permissions and so on as a matter of maintenance, and I'll report back here if there are any other problems related to the update.

UPDATES

13 May 2009, 23:55 - the system worked snappily all day, but this evening I got the grey kernel crash screen. The "You need to restart" in multiple languages one. Ugh. Apparently, Apple recognizes the problem and will issue a fix. Recommend waiting.

14 May 2009, 11:41

The 10.5.7 update is definitely much snappier. A lot of things are running very fast. This is nice. Now, just don't crash on me again, please!

15 May 2009, 10:30

Crashed after waking from sleep. Had to force reset. System worked well after that.

16 May 2009, 9:30

Flock crashed and could not be force-quit. Activity Monitor would not start. Terminal kill command would not kill.

What's in the 10.5.7 Update

FYI, here's what Apple says is in the update:

Address Book

• Improves reliability of Address Book syncing with iPhone and other devices and applications.

AirPort

• Improves the reliability of AirPort connections, including improvements when roaming in large wireless networks with an Intel-based Mac.

Client management

• Improves reliability of synchronizing files on a portable home directory.

• Fixes an issue in Mac OS X 10.5.4 and 10.5.5 in which managed users may not see printers that use the Generic PPD.

• Client computers that use UUID-based ByHost preferences now respect managed Screen Saver settings.

iChat

• Addresses an issue that could cause an encryption alert to appear in the chat window.

• Setting your iChat status to "invisible" via AppleScript no longer logs you out of iChat.

• Resolves an issue in which pasting text from a Microsoft Office document could insert an image rather than text.

Graphics

• Includes general improvements to gaming performance.

• Includes graphics improvements for iChat, Cover Flow, Aperture, and iTunes.

• Includes fixes for possible graphics distortion issues with certain ATI graphics cards.

Mail

• Includes overall performance and reliability fixes.

• Improves Connection Doctor accuracy.

• Fixes an issue that could cause messages identified as junk to remain in the inbox.

• Fixes an issue that could cause Mail to append a character to the file extension of an attachment.

• Addresses an issue that could prevent Mail from quitting.

• Improves reliability when printing PDF attachments.

MobileMe

• Contacts, calendars, and bookmarks on a Mac automatically sync within a minute of the change being made on the computer, another device, or the web at me.com.

Networking

• Improves Apple File Service performance, especially when using a home directory hosted on an AFP server. Important: If you are using Mac OS X 10.5.6 (client) to connect to a • • Mac OS X Server 10.4-based server, it is strongly recommended that you update the server to Mac OS X Server version 10.4.11.

• Improves the performance and reliability of TCP connections.

• Improves reliability and performance for AT&T 3G cards.

• Updates the ssh Terminal command for compatibility with more ssh servers.

Printing

• Improves printing for the Adobe CS3 application suite.

• Improves printing for USB-based Brother and Canon printers.

Parental Controls

• Addresses an issue in which a parentally-controlled account could be unable to access the iTunes Store.

• Includes general fixes for time limits.

• Resolves an issue that prevented adding allowed websites from Safari via drag and drop.

Time Machine

• Fixes issues that could cause Time Machine to state the backup volume could not be found.

• Improves Time Machine reliability with Time Capsule.

Safari

• Improves compatibility with web proxy servers.

General

• Includes Mac OS X security improvements. See this website for more information.

• Addresses inaccuracies with Calculator when the Mac OS X language is set to German or Swiss German.

• Improves the performance and reliability of Chess.

• Improves DVD Player performance and reliability.

• Performance improvements for iCal are included.

• Fixes an issue when running the New iCal Events Automator action as an applet.

• Adds a Trackpad System Preference pane for portable Macs.

• Improves compatibility with smart cards such as the U.S. Department of Defense Common Access Card.

• Updates time zone data and Daylight Saving Time rules for several countries.

Mac OS X 10.5.7 is available via Software Update and also via standalone installers.

Does my SuperDrive do "dual-layer"?

If you are wondering whether the SuperDrive in your Mac is dual-layer capable, or whether it can store 8+ GB of data on a data DVD rather than 4+ GB, have a look at the System Profiler app (you can Spotlight it to easily find it if you use Leopard), and check Hardware, Disc Burning, DVD-Write. If you see the tell-tale DLs in the description, you're good to go. Just buy the appropriate media.

Migrating User Folders in OS X Leopard Server

I did an experiment on my test Leopard Server to migrate User Folders from the default /Users to another direct-attached volume based on some questions that came up.

The problem is, OS X Leopard Server defaults to using the system disk as the Users partition, and this disk is not often your largest disk. It can fill up quickly if users start storing their photos or music. Long story short, the trick is to use Workgroup manager to auto-create the User directories, but, before that there's some things you need to do for prep.

How to Migrate OS X Leopard Users Folder from Default to External

In Terminal, you can either sudo bash or use sudo in front of every command. I like to use sudo bash, but remember to exit out of your sudo'ed bash shell later, to avoid risk. Don't use sudo bash if you don't know your way around a Terminal. If you are not sure, don't use it, and just go for sudo in front of each command you want to use.

sudo bash
Password:

Create the containing Users folder on the external drive, which we'll call "EXTHD" here. You can confirm its POSIX permissions using ls -la.

cd /Volumes/EXTHD
mkdir Users
ls -la

Start Server Admin, select the server in the left-hand list, then select the File Sharing section from the toolbar. Copy down all the attributes of your existing Users folder Share Point, for example to auto-mount as "User home folders", protocol options to share via AFP, SMB, FTP or NFS, and base ACL and POSIX Permissions with whether to inherit or not.

Now, you can Unshare the Share Point in Server Admin. Click Unshare and Save. Click another Share Point to make the list refresh.

After you confirm the Users Share Point is gone you can create a new Users Share Point on /Volumes/EXTHD/Users, your external disk. Reset all the settings you wrote down for your original Users, then Propagate Permissions using the Gear dropdown.

Start Workgroup Manager, and specify the new User share for each user. Multi-select users, select your new share, and click "Create Home Now" to create.

Now you can migrate the contents, which is probably the most tedious part because there's unfortunately no automated way to do it. Use the mv command, and you may need to reset permissions after the move. Users can try connecting and accessing their files. Here's one way, which involves using the su command from root's shell to mimic any given user, in this case John Allen Smith or "jas".

bash-3.2# su - jas
mysrv:~ jas$ whoami
jas
mysrv:~ jas$ ls
Desktop Documents Downloads Library Movies Music Pictures Public Sites
mysrv:~ jas$ pwd
/Network/Servers/mysrv.mydomain.com/Volumes/EXTHD/Users/jas
mysrv:~ jas$ cd Documents/
mysrv:Documents jas$ ls
About Stacks.pdf
mysrv:Documents jas$ mv /Users/jas/Documents/* .
mv: ./myfile.txt: set owner/group (was: 1034/80): Operation not permitted

If you get errors like the "Operation not permitted" above, deal with those permissions separately. Use the exit command to exit to root again, and use it again to get out of the "sudo bash" root shell. I hope this helps get you started migrating user folders to an external hard drive, in OS X Leopard Server. Enjoy!

Color Management in Flock (and Firefox)

If you use anything but IE 7 you are still out of luck, but Safari users have had this functionality for quite a while. I refer to "color management", which is the concept of adjusting the color properties of devices, like displays, monitors, printers or scanners, so that colors mean the same thing across devices, and more importantly look the same. The adjustment can be done in the operating system itself, which is what Mac OS X does (and one reason it is superior to Windows), or via International Color Consortium "ICC" profiles and other software trickery.

Let me keep it very simple for this post, though. If you use Flock, which is at the time of this writing based on Firefox 3, or Firefox 3 itself, color management is disabled by default. However, you can easily enable color management, so that photos on photo sharing sites like Flickr or SmugMug will render with the color profile embedded in the photo, which usually means they will look much better than the default, using no profile. In my opinion, these days you will get far better results by turning color management on, in your Flock or Firefox 3 browser.

How to Enable Color Management in Flock or Firefox 3

The easiest way to enable the settings is by using the Color Management Addon for Firefox which will work in Firefox and should work in Flock. Alternatively, you can enter the secret "about:config" code in your address bar (where you type the URL), and enable color management yourself:

about:config

Be very careful about what you change in here, because you can break your browser easily. That said, scroll down until you find:

gfx.color_management.enabled

gfx.color_management.display_profile

You can double click gfx.color_management.enabled to enable Firefox to use whatever profile is embedded in the photo you are viewing, and, you can specify an ICC profile by entering its path in the gfx.color_management.display_profile parameter. If you are not sure what that means, just enable gfx.color_management.enabled.

Restart Flock or Firefox, and you should get much better colors on sites like Flickr or SmugMug. Finally, please check out Deb Richardson's excellent introductory post on this topic, for a clear depiction of the difference of using profiles or not, and how different applications render color. Don't waste time with color management turned off - turn it on now and enjoy much richer color.

Splitting Long Videos for YouTube in iMovie

iMovie, included among other excellent consumer applications in Apple's iLife 09, now provides an easy way to share your videos on Google's YouTube video sharing service. You just prep your video, and then use the Share menu to share it in various ways. However, YouTube has a 10 minute, 1GB limit.

Using iMovie 09 with YouTube

Here's a few tips to work with those YouTube limits:

• Confirm the total video time at the bottom of the project window, and the length of a clip by hovering the mouse over the clip and looking at the time-stamp appearing at its beginning.
• Use Share menu, YouTube. If the movie is over the YouTube limit, the dialog box that appears will tell you so.
• Make a clip smaller by positioning the playhead, ctrl-clicking and selecting "Split Clip." Once clips are split you can delete parts that are not needed.
• If need be, split up your Project into two Projects. Let's say you have a 14 minute, 600 MB AVI file you have imported into your project, that you have split into two 7 minute chunks. Ctrl-click on the Project in Project Library and choose Duplicate. Rename each project to indicate which half it will be in the end. Open each half and delete the unneeded half.

I trust someone will find this useful when working with iMovie and YouTube. Enjoy!

URL for MobileMe iDisk Public Folder

MobileMe iDisk users can share files Publicly, but it's less than obvious how to do it. Further, the whole MobileMe experience is still a bit buggy as well, in my experience. Apple announced back in the summer when MobileMe launched, that easy sharing of public files via email would be "coming soon", and their site still reflects that notion. The MobileMe launch was not quite Apple, actually, and felt very poorly handled. Maybe if we're good we'll get a nice MobileMe upgrade Christmas present and be able to share Public file URLs easily via email. That aside, here's how you can do it now:

How to Share Files via your MobileMe iDisk Public Folder

First, let's assume your MobileMe login name is awako.kaneyama@me.com. You would simply connect to your iDisk in OS X or Windows, and copy files you wish to share into the Public folder. Then, tell your friends the URL:

http://idisk.mac.com/awako.kaneyama/Public

They can visit the URL and download the AVIs of junior tearing up the yard.

But wait, there's a caveat.

I tried this in Flock, which is built using the Firefox 3 engine, and the Public site just stalls out, with a handsome progress spinner, spinning endlessly. It works fine with Safari, though. Ironic, no? (Please, Apple, don't become Microsoft.) I hope Apple's MobileMe team will work these bugs out soon, but until then, Enjoy!

iPhone Tap Tips

After (finally!) getting a new iPhone and contract from Softbank here in Japan, just like I did with Mac Keyboard Shortcuts, I struggled a bit with the "tap sequences" one needs to learn to be able to input text or navigate efficiently in this superior smartphone.

How to Manipulate the iPhone with Taps and Button Presses

For convenience, I compiled a list of iPhone "Tap Tips", or shortcuts for tapping and pressing the buttons and interface of the iPhone, which I hope will be useful to iPhone users.

For instance, do you know what these sequences do?

• Triple-click the mic clicker.
• Double-press the home button.
• Double-tap space when entering text.

If not, take a peek at my "iPhone tap tips" list for the answers and more, and most of all, Enjoy!

Word 2007 Performance Optimization Tips

A Windows user colleague asked about optimizing Word 2007 documents because he is having no end of problems working with 100 or 200 page documents in Word. If you are pushing the envelope in Word use, you might like to keep in mind several recommendations.

Here are some Optimization Hints for Word 2007

Defragment your hard disk regularly. This is not necessary on a Mac but for Windows, you definitely need to do it. (There are other maintenance tasks you have to do for a Mac, of course and you can use OnyX for those.)

Use the Microsoft "DOCX" format which is the default for Office 2007. If you are collaborating with a person who only has Office 2003, there is a free filter they can download and install, which allows them to use the latest and more efficient format. The DOC format is binary, while DOCX is XML-based and much better for efficiency. The new DOCX format is 75% smaller than the old DOC format, and is less prone to corruption. Your colleagues using Office 2003 will need to install the Microsoft compatibility kit, that allows them to open DOCX.

Do not keep all revisions of your document, for the life of the document. Instead, "accept" revisions in, at certain times during the edit cycle. Having 100's of revisions in a Word document will degrade performance.

If you are using a lot of graphics in your Word 2007 document, do not simply paste in BMP files because these are large and inefficient. Instead, use PNG and adjust the PNG to be as light as possible before you paste it in. If you are taking a lot of screen prints / screen shots, you can do this easily using Techsmith's excellent utility SnagIt. It is much, much easier than using the print screen button and MS Paint, and the results are light and better for long documents. In addition, you can use "picture placeholders" instead of the actual graphics, to save memory. Click the Office Button, Word Options, Advanced, Show Document Content. Select the "Show picture placeholders" checkbox. Or you can simply use the Draft view to display graphics as empty boxes.

Use fewer fonts in the document, and uninstall fonts you do not use. Fonts use up system resources.

Disable background saves which Word 2007 has enabled by default, and which take up memory. You can disable them by clicking the Office Button, then in Word Options, click Advanced, and in Save, clear the "Allow Background Saves" checkbox. If you do this, remember to save frequently.

Disable automatic grammar and spell checking, which constantly checks text in the document. Click the Office Button, then Word Options, then Proofing, and clear the "check spelling as you type" and "check grammar with spelling" checkboxes.

It goes without saying, but confirm that the PC you are using has enough system resources to complete the task, including disk space and RAM. If you have a desktop, you can install a fast secondary hard disk, such as a SAS or SATA drive, and put a page file on this drive. Further, you can help performance by moving your My Documents and work files to this faster hard drive.

In my experience, Word has never been an excellent platform for long, complex or heavy documents. I too had a lot of trouble writing manuals for technical training courses 10 years ago. In the end, the best thing is to use a proper DTP program if you are doing long documents, such as Adobe InDesign and Quark Express, or, if you are working large documents with a team, Adobe InCopy. If you must use Word, then you might give the above a try.

Save Local Disk with Gmail "Advanced IMAP Controls"

I outlined "Setting up Leopard Mail.app with Google Gmail IMAP" previously, and Google Labs has released a new feature for regular and Google Applications Premier Edition Gmail - Advanced IMAP Controls. This tip shows you how to avoid having to sync the GMail "All Mail" folder, thus saving a considerable amount of local hard disk space in many cases. Before enabling Google Labs features, note the caveat mentioned by Google on this FAQ.

At any rate, here's how to enable Advanced IMAP Controls:

1. Close Mail.app.
2. In Gmail settings, Labs tab, turn on "Advanced IMAP Controls" by enabling it.
3. Delete the locally sync'ed All Mail Folder.
4. Start Mail.app and confirm. You can press Cmd-0 to check activity in Mail.app.

You can easily delete your local copy of the All Mail folder from Terminal. Use the sudo command to run a bash shell within your shell, and feed the password prompt your administrator password. You can check the size of the various folders using du, and rm -rf to force-delete the folder. Exit will get you out of the sudo'ed bash prompt and back to your normal prompt.

jrc $ cd ~/Library/Mail/IMAP-rick.cogley\@imap.gmail.com/[Gmail]
jrc $ sudo bash
Password: *********
bash-3.2 # ls
All Mail.imapmbox Drafts.imapmbox Reference.imapmbox Starred.imapmbox
Expiring.imapmbox	Sent Mail.imapmbox	Trash.imapmbox OmniFocus.imapmbox Spam.imapmbox
bash-3.2 # du -h -d 1
bash-3.2 # rm -rf All \Mail.imapmbox/
bash-3.2 # exit
jrc $ 

When you restart Mail.app, the folders for which you had sync enabled which are now disabled, will take a moment to disappear. I saved myself 2 GB of disk space by enabling this.

Finding Photos within a Date Range in Aperture

A friend using Apple's powerful Aperture photo management application asked me how to sort photos into date ranges, such as an entire year, a calendar month, or by day. Here's how to do it with Aperture's Smart Album feature.

First, create the Smart Album by ctrl-clicking on the Library folder and choosing "New Smart" then "Album". The same thing is available from the File menu.

A grey-black semi-transparent Heads-Up Display or "HUD" will appear, and from the HUD's plus drop down in its upper right corner, select "date". A Date line is added to the HUD, and you can choose "Image Date" "Is In The Range" and enter your date range in the boxes. Note also the "Calendar" selector, which seems only to do a single day.

Notice a couple more things about the HUD. Each criteria line has a checkbox which serves to enable or disable the criteria. There is a "Include if" line with a dropdown, from which you can choose "Any" or "All" from, and whether to check for matches or no matches. This covers the selection logic, and acts upon all the selected criteria below. You can also choose just to filter if the photo was a stack pick, using the checkbox for that in the lower left of the HUD.

When you are done entering criteria, click the X in the HUD's upper left corner to close it. You can reopen the HUD by clicking on the little magnifying glass to the right of the Smart Album itself, to reopen the HUD. I'm using Aperture 2.1.1 at the time of this writing so your mileage may vary.

Flushing DNS

When accessing websites or mailservers, your DNS resolver caches the request so that future lookups are fast. However, sometimes when a zone on DNS is being changed while you work, you want to be able to clear the cache to force the resolver to get the latest name:IP address pair mappings. Here are some command line commands for flushing the DNS cache on your local machine: On Mac OS X Tiger:

lookupd -flushcache

On Mac OS X Leopard:

dscacheutil -flushcache

On Windows:

ipconfig /flushdns

Update: I noticed that this cache seems to survive reboots, on Leopard.