Parsing URL query parameters in Python

June 2nd, 2008

I’ve been dabbling with Python for several months now, but I’m not quite as proficient with it as I’d like.

I was hacking on some stuff recently, and needed to parse the query parameters in a URL. Python has URL parsing, but it doesn’t include querystring parsing.

This was the pleasantly easy solution:

from urlparse import urlparse
url = urlparse('http://www.google.com/search?hl=en&safe=off&q=atomized&btnG=Search')
params = dict([part.split('=') for part in url[4].split(’&')])

What’s going on here

Let’s break the example down a bit.

  1. from urlparse import urlparse

    This pulls in Python’s URL parser.

  2. url = urlparse('http://www.google.com/search?hl=en&safe=off&q=atomized&btnG=Search')

    This passes the URL through urlparse, leaving us with a tuple of URL components.

  3. url[4].split(’&')

    This splits the querystring on “&” characters, leaving us with a list of “key=val” scalars. The output of this would be:

    ['hl=en', 'safe=off', 'q=atomized', 'btnG=Search']

  4. [part.split('=') for part in url[4].split(’&')]

    This is Python’s list mapping, also known as list comprehension. It allows us to map a function over a list - in this case, part.split('=') on the list of “key=value” pairs from the previous step. This leaves us with:

    [['hl', 'en'], ['safe', 'off'], ['q', 'atomized'], ['btnG', 'Search']]

    That is, an array of arrays, where the first member of the child array is the key and the second is the value.

  5. params = dict([part.split('=') for part in url[4].split(’&')])

    The last part of this is dict(), which turns the array structure above into a dictionary:

    {'q': 'atomized', 'safe': 'off', 'btnG': 'Search', 'hl': 'en'}
    

    Thix is roughly equivalent to a hash map or associative array. It allows us to access specific keys, such as params['q'].

The correct way to disable Spotlight

May 30th, 2008

A while back, I was looking for a way to remove the Spotlight icon from my menubar. I rarely if ever use menu icons, and wanted to reduce the clutter.

The only guides I was able to find recommended chmodding Spotlight.app to 000, to prevent it from being run. I did this, and was happy for a while, but this approach has a number of problems.

It’s unreliable. After OS updates or running repair permissions1, it’s back. But the main problem is more severe: launchd tries to relaunch it continuously. This is a much larger problem than it might seem. While I don’t think the system resources this consumes are significant, it does cause launchd to complain incessantly in your log files - every few seconds - which can eat up your disk space at an alarming rate.

I knew there had to be a better way, so I went looking. It turns out that launchd has normal .plist configuration files which tells it what services to launch. The relevant file for Spotlight is in /System/Library/LaunchAgents/com.apple.Spotlight.plist.

To completely disable Spotlight, this is what you need to do:

  1. In Finder, select Go ▸ Go To Folder (⌘⇧G)
  2. Enter /System/Library/LaunchAgents; click Go.
  3. Download my copy of com.apple.Spotlight.plist.
  4. Drag the file from wherever you downloaded it to the /System/Library/LaunchAgents window.
  5. Click “Authenticate” when prompted; enter your password; confirm that you want to replace the old file.
  6. Reboot.

Spotlight is gone for good, and I noticed a significant reduction in the time it takes to log in.

  1. Repair permissions is voodoo, and never fixes anything, but
    people do it anyway.

Keybinding irritations

May 27th, 2008

⌘T in nearly any app which uses tabs opens a new tab. In apps without tabs, it opens the Show Fonts. This is a floating window, which cannot be closed with ⌘W, the usual binding to close a tab or window.

While it can be closed in most cases, you have to use a different keystroke, which can be problematic.

Example:

  1. Attempt to open a new tab in Firefox, but Adium has the focus.
  2. Press ⌘T, get the Show Fonts window.
  3. Press ⌘W, close the tab you’re talking in, the unwanted Show Fonts window is still open.
  4. Scrabble for the mouse in frustration.

This happens to me several times a day. I’ve noticed that since Leopard enabled the ability to scroll background windows, I do it more. While I love that ability, I think it removes a subtle cue as to which app has the focus. The Firefox 3 betas exacerbated the problem, since they appeared active even when in the background.

Something I did not know

May 13th, 2008

Using tar to back up a disk which contains large numbers of hard links (e.g. a Time Machine volume) is extremely slow.

Slow as in, I started the backup of this 300gb disk last night, and it’s just over halfway done this morning.

Unicode wins

May 5th, 2008

The Unicode family of encodings has just become the dominant encoding on the web.

Good news for all.