Gentoo "emerging" pains of upgrading to Python 2.5

And I thought it's time for me to upgrade to Python 2.5, partly due to a particular masked package and partly due to the fact that I had the itch to upgrade that particular package. So I gone ahead to doing that, happy that it's successful, and sort of left it there for a few days.

A few days later, I do the oh-so-routine of "emerge -uD world" and expect it to run smoothly, only to be proved wrong when emerge start to act up on me. One of the most prominent errors being "ACCESS VIOLATION SUMMARY" in big red text before the emerge fails. Then some of them seemed to have problems with xml2po.

Apparently when python is upgraded, python packages and a couple of programs that relies on it needs to be updated as well. This can be done by executing "python-updater" right after you do the release update of python. And later do the re-emerging of gnome-doc-util, just to be on the safe side.

Currently these steps weren't officially documented in Gentoo and the mention of it is in the GNOME 2.12 upgrading guide. But then again, Python 2.5 is still under the unstable flag, so such inconveniences was expected. Hopefully by the time when the maintainers believe Python 2.5 is ready for prime time (i.e. become "stable") they will issue a notice in the Gentoo newsletter or something.

[P/S: Still haven't got around unmerging Python 2.4 though, but probably I'd do that later]

Annoyances: Firefox on FAT, Rhythmbox with iPods

There had been two minor annoyances that I had faced until now:

Mozilla Firefox downloading problems on FAT32 partitions

The Problem: Personally I used a FAT32 partition where I placed all my data that is to be shared, so naturally all downloads will go there. However, downloading with Firefox had been a pain to me: it tends to give me zero bytes zip file downloads as well as the behaviour of appending something extra at the back of the filename (for instance, music.mp3 will become music.mp3.bin, or that package.tar.gz will become package.tar.gz.tar).

Honestly speaking, that's extremely annoying.

But apparently I'm not alone, and the reason for that happening, according to one of the analysis being done "In a way it seems to be a permission problem... As soon as the Fat32 partition is mounted with permissions rwx on files and directories, aLocalFile is always executable. When downloading a regular file (something not a binary executable), Firefox seems to set the mode of targetFile to 600, so the expression is true and I get a aborted download and a 0 byte file."

The Fix: So in a way, this problem highlights one of the biggest mistake I had made to my current Linux setup: I set the mount parameters (/etc/fstab) for my FAT32 partition wrongly. I set the fstab entry as "umask=0007", which is blatantly wrong because that means that all files and directory are set as executable. This can prove to be a great security risk as I might accidentally execute something nasty without me knowing it.

The more correct way of setting the /etc/fstab entry is to utilize the fmask (file permission mask) and dmask (directory permission mask) parameters, which should be set to something like this "fmask=117,dmask=007", equivalent to allowing read and write (but not execute) to files and all permissions to directories. And that other users whom are not the owner or within the designated group are not able to access it.

After that is done, remount your partitions and Mozilla Firefox will now download files into FAT32 without problems. (other users reported that this is also the solution for the same problem in NTFS)

However, it'd be nice if Firefox can acknowledge and fix this minor pet peeve. Or at least throw some sort of a rejection or warning message would be better.

Rhythmbox doesn't actually "delete" files from iPod

The Problem: Having an iPod myself, the ability of synchronizing songs from Rhythmbox is definitely a convenience to myself without having to rely on iTunes. However, easy synchronization is only half of the story; deleting them from your iPod is completely different story altogether.

Now, as a user, I came to expect that I can free up some disk space by right-clicking the songs that I don't want anymore from my iPod's playlist and select the "Move to wastebasket". End of story... or is it? The actual situation wasn't completely so: Yes, the playlist entries were removed, but it doesn't free up any disk space on my iPod at all! Not to mention that my desktop Trash icon doesn't show any indication that the files had been there for me to cleared away.

Now, that means that the files were dangling somewhere in my iPod and search for these orphans are a complete pain. And it just doesn't make any sense to format my iPod completely because of that either. Double whammy! ARGH~~~!!!

The Fix: Searching online, apparently this issue has been in Rhythmbox's Bugzilla for quite some time. And worse, the maintainers can't seem to agree the behaviour of "Move to wastebasket" option... for more than one year!

Although someone has made a patch which does delete the files in iPod from Rhythmbox, if you are having some reason that patching it on your own is not an option (to me it's just because I hate to make a local portage overlay ebuild of Rhythmbox every time the software is updated), there's another workaround to it.

The workaround is that after you mounted your iPod, go to the root directory of your iPod and add the ".Trash-[username]" directory, where [username] is the username which you usually use in your Linux instance. Which now the "Move to wastebasket" option will move the files that you chose to removed there. And to free up iPod's disk space, just delete the files manually.

I know, I know, it's a two steps process. But it's better than formatting my iPod completely. And until the time that the maintainers agree that deleting files from the iPod is perfectly OK (or at least have some other intelligible solutions to the problem), that's the safest way to do it.

Probably it might be a good time for me to search/write a Rhythmbox plugin that does just that.

Creating your own OpenSearch plugin for Firefox

Personally, I liked the search bar feature which is now a staple for all major web browsers already. However there are times when you found out that people may not have created a search plugin for that one site you frequent? For my case, I needed a search plugin for BitEx, a Chinese-Japanese dictionary but I can't find it at Mycroft. (the default place which Firefox points for users to find custom search plugins)

So basically you can either write it manually using a normal text editor or the automated way provided by Mycroft.

To do it automatically (Recommended!), you can use Mycroft's Submit plugins link, which will show you a simple form for you to fill up. The more important part is to fill in the "Search URL" field. Basically you can either view the HTML source or just do a normal search in order for you to get the search string being used, and then replace the parameters containing your keywords into {searchTerms}.

For instance, after I submitted a search at BitEx, the address bar now shows "http://www.bitex-cn.com/search_result.php?deal_type=jp2cn&keywords=%E6%97%A5%E6%9C%AC". Knowing that the "keyword" parameter is used for my search term, the "Search URL" field should be filled as "http://www.bitex-cn.com/search_result.php?deal_type=jp2cn&keywords={searchTerms}".

Now, make sure you put a meaningful plugin name and description and make sure what encoding that the webpage is using, especially if you are dealing with sites that was in a foreign language. Some of them aren't using Unicode yet so it's important to make sure that it's set correctly or your search plugin may be rendered useless.

Now, once everything is done, press the "Generate plugin" button and the textfield below will show the xml code of your plugin. I greatly recommend that you test your plugin once by saving the xml as a normal xml file and copy it to your search plugin folder. In Linux, it'd typically be stored in "~/.mozilla/firefox/[some random string]/searchplugins" folder. Note that [some random string] differs from each user profile. Restart your Firefox browser and test your plugin. Once confirmed that it's working, please do others a favour by submitting it to Mycroft. That'd be extremely helpful to other users.

To do it manually, you can refer to the "Creating OpenSearch plugins for Firefox" tutorial at Mozilla Development Center. Here are the steps that I used:

1. Open a new xml file, which in this case I named it as bitex.xml
   
2. Copy the template from the tutorial above
   
3. (Optional) Download the icon file from BitEx and edit it. The reason for me doing that because the icon file is extremely large, so I have to trim the fat myself before proceeding to the next step
   
4. Using the data: URI kitchen, get the xml data representation of the icon. Remember to tick the base64 checkbox located on top of the form, right next to "Title" text field. Then you either input the URL of the icon or upload your own (if you have edited one yourself) and press the "Generate" button. Copy the link which is presented to you and paste it between the <Image> tags. (see template)
   
5. Now, substitute all necessary fields shown in bold in the template file. Here's my quick explanation of how to fill in the values:
   • engineName: The name of the search engine
     
   • engineDescription: Your description of the search engine
   • inputEncoding: The input encoding used, typically UTF-8
   • imageData: The image data that you have generated in the previous step
   • method: Either GET or POST. Typically GET
   • searchURL: The base URL of the search engine, its the same as my previous example (see "To do it automatically" section)
   • (Not required) paramNameN and paramValueN: Used to pass parameters in the search requests. Typically not required unless the engine uses the POST method, in which you have to fill in such data
   • (Not required) suggestionURL: If the search engine supports the suggestion feature ala Google Suggest, then it'd be nice to fill this in
   • searchFormURL: The link to the search page, useful if users want to access the search page directly
     
6. Once you are done, save the xml file and copy it into your search plugins folder (see "To do it automatically" section)

Rsync from Windows to Linux

With the mounting documentations that I'm having at work, there's always the insecurity that losing them, especially those "critical work-in-progress" documents. To most of us, having some sort of an easy backup solution is a must, but "making things easy" is one of the biggest hurdles that I ended up delaying to find the solution until recently.

And so I bumped into rsync, which is a fast and secure way to perform backups. And having it installed by default in our office's development server (a Redhat Enterprise Linux 4), I sort of gave it a go.

So the idea is to have a shortcut at windows so that it can mirror all important documents into Linux, which goes through rsync. Since Redhat have rsync daemon setup to run via xinetd, so the first thing is to setup rsync client for Windows, which is a piece of cake if you install Cygwin packages. Gaztronics.net provides a good tutorial for it (See 1. Cygwin and 3. Client section).

All is good but now the problem is that everytime when rsync is executed the server always asks for a password. Reason being the rsync that came with Redhat using SSH as the transport protocol, which means you can't simply do a "rsync --password-file=[file]". To automate the authentication step (i.e. not needing to key in the password every time), Troy Johnson had a tutorial which explains how to do it.

Just for your quick reference, this is the big picture on what I have done ([text in square brackets] denote fill in the blank fields :) ):

1. Install rsync and ssh Cygwin packages in Windows
2. Perform a rsync transfer to make sure that it works (basically just a "rsync [local file] [linux-username]@[linuxserver-address]:[remote directory]" would do)
3. Perform a ssh-keygen at Windows to get a public-private key pair (*Note: You'd have to use "ssh-keygen -t dsa -b 1024 -f [key filename]" command)
4. Transfer the public key (the file with the .pub extension) to Linux
5. Copy the contents of the public key into ~/.ssh/authorized_keys and make the necessary modifications (See Troy Johnson's tutorial above)
6. Perform a rsync transfer test again to make sure that it works and without requiring a password from you ("rsync -e "ssh -i [private key file]" [local file] [linux-username]@[linuxserver-address]:[remote directory])
7. Write a batch script (refer to Gastronics' tutorial) which automates the whole backup process
8. Add the shortcut on your desktop and your done :)

Note that you might face some problems if you have directory names having spaces (especially if you want to backup documents in your My Documents folder. Basically here's my working script for your reference:
@cls
@echo off

rem Rsync job control file

set DOCROOT=/cygdrive/c/Documents and Settings/shleong/My Documents

C:\Cygwin\bin\rsync -e 'ssh -i rsynckey' -vrtz --delete --include-from='%DOCROOT%/bin/filelist' '%DOCROOT%/' shleong@linuxserver:docstore