Sunday, March 21, 2010

Get offlineimap working with non ASCII characters.

Introduction

I recently started using offlineimap [1] to manage my GMail accounts locally on my machine.

With offlineimap all my email is local so the navigation is very fast and also I am a little more relieved that I have a local copy of all my email.

The only problem is that offlineimap does not work well with non english characters. This makes it difficult to use for those with labels in Russian, Chinese, Japanese etc [2].

Fortunately offlineimap offers some nice options that allows us to modify it's behavior. Using the "pythonfile" and "nametrans" options I was able to get Japanese working flawlessly.

Problem Description


IMAP4 uses a modified UTF-7 coding for all folder names but offlineimap does not convert the folder names to UTF-8 or something readable before creating local repositories that results in very cryptic folder names like "&MMYwuTDI-".

Some research (googling) on the topic resulted in a very nice code to add IMAP4 UTF-7 encoding capabilities to Python [3]. So with a little time and some copy/paste skills I added this code to offlineimap and ban! I got Japanese working correctly in my folders.

Fixing unicode issues in offlineimap


First we must add the code to convert from IMAP4 UTF7 encoding support to offlineimap. Thanks to Dominic LoBue (offlineimap developer) I learned that we can add our own python code to offlineimap. To do this we created a simple python script (e.g. ~/.utf7.py) that contains the following code (copy/paste from [3]:



Then in our offlineimap configuration file, in the "[general]" section we add a line to load this python script like:



Finally in our remote repository configuration add a nametrans option to convert all foldernames from "imap4-utf-7" encoding to your encoding of preference. My Ubuntu installation is all UTF-8 so I convert all folder names to this encoding:



The "imap4-utf-7" encoding is added by our "utf7.py" script. With this configuration I can now see the correct Japanese names for all the folders (labels) in my GMail accounts inside Mutt [4].

Resources


[1] http://software.complete.org/software/wiki/offlineimap
[2] http://software.complete.org/software/issues/show/102
[3] IMAP4 UTF-7 Encoding/Decoding
[4] Best email client