Skip to main content

Archiving email : Thunderbird tags not ready for primetime yet

Thunderbird stores email in plain text files (mbox format) that have names like INBOX and mysweetiesemails (files with no extensions). Riding with these mbox files are other plaintext files with corresponding names INBOX.msf and mysweetiesemails.msf . These .msf files are mail summary files used by thunderbird to speed up certain actions that would otherwise require repeated trawls through a large text file. The .msf files are autogenerated as required and can be deleted.

The good thing about mbox, I think, is that in the end it is not proprietary and it is plain text. Thunderbird has enough of a user base that whatever comes after Thunderbird will have a way to import mbox mail (if not use mbox directly).

I wanted to archive my email off my university's server onto local folders on my compuer. Thunderbird's "Local Folders" can be reconfigured to any folder on your computer, by going to tools->account settings.

I had organized my mail into different folders on my pop server by year, and some separate folders for special classes of emails (e.g. rejection notices from journals...). When I moved the emails, I decided to move them, by year, into archive folders on my computer (local folders).

Now, dumping all the mail into one archive "folder" (which would translate to one Thunderbird mbox file) would be fine, but I believe that this single file would balloon too much in size and eventually become unmanageable, so I settled on having a folder (mbox file) for each year.

The transfer from pop to local archive folder went fine, but of course, I lost my special organization, so the mail from an editor of a magazine complimenting me on my work (important to me) in 2005 got mixed in with several hundred emails between me and my colleagues trying to figure out some trivial, daily work related stuff (not so important) also in 2005.

This highlights, to me, the need to have some kind of categorization that travels with each individual email, so it won't get lost if you move it around - like ratings and tags and captions for images, which are now stored within the jpg as metadata, so you don't have separate files that need to travel with your pictures, storing captions and ratings etc.

Thunderbird's tag (formerly label) system covers this. But be aware that not only are there no standards for labels in emails, the mbox storage format, though popularly supported is not a formal standard and comes in different inconsistent forms. So tagging, recognized by Thunderbird 2.0 may not be by Thunderbird 3.0 or FutureMailClient 1.0. Heck, people have had problems when mixing labels (TB1.5) and tags (TB2.0).

Anyhow, Thunderbird supposedly stores tag information in the message header with the "X-Mozilla-Keys" header and I got all excited until I found out that for local folders the tags get stored in the .msf and not the header [Bug 378973]. So, for the moment I've given up on tagging or labeling until the dust settles. I may even go back to separate folders.

Incidentally is a good place to browse mozilla source code. Thunderbird stuff is found here. The code for handling tags is here.


  1. Where does it store tag info for IMAP Maildirs? I had to manipulate a user's messages directly on the mailserver and let Tbird reindex, which wiped out her existing tags. I've now retagged a few messages in order to have something to grep, but I cannot figure out where they're stored, so I can't use the backup to try to restore her previous tag data.

  2. Okay, found it in the local .msf, but I can't make heads or tails out of it. I made copies of two versions of the INBOX.msf, one where a single message was tagged and one where it wasn't, then diffed them. The msf header lines are fairly straightforward; we find
    (8A=label) but nothing for tag except (C6=current-view-tag), which isn't it.

    The diff is an additional 13 lines but the only instance of 8A is (^8A=0), which appears for every message's entry. I'd sure like to find an msf -> human-readable parser.

  3. Hi Ludwig,

    Unfortunately, I didn't go too deep into the .msf files.

    Why are you looking to manipulate the .msf files yourself?

  4. Also, TBird can be forced to store labels in the mbox see this post.


Post a Comment

Popular posts from this blog

Flowing text in inkscape (Poster making)

You can flow text into arbitrary shapes in inkscape. (From a hint here).

You simply create a text box, type your text into it, create a frame with some drawing tool, select both the text box and the frame (click and shift) and then go to text->flow into frame.


The omnipresent anonymous asked:
Trying to enter sentence so that text forms the number three...any ideas?
The solution:
Type '3' using the text toolConvert to path using object->pathSize as necessaryRemove fillUngroupType in actual text in new text boxSelect the text and the '3' pathFlow the text

Pandas panel = collection of tables/data frames aligned by index and column

Pandas panel provides a nice way to collect related data frames together while maintaining correspondence between the index and column values:

import pandas as pd, pylab #Full dimensions of a slice of our panel index = ['1','2','3','4'] #major_index columns = ['a','b','c'] #minor_index df = pd.DataFrame(pylab.randn(4,3),columns=columns,index=index) #A full slice of the panel df2 = pd.DataFrame(pylab.randn(3,2),columns=['a','c'],index=['1','3','4']) #A partial slice df3 = pd.DataFrame(pylab.randn(2,2),columns=['a','b'],index=['2','4']) #Another partial slice df4 = pd.DataFrame(pylab.randn(2,2),columns=['d','e'],index=['5','6']) #Partial slice with a new column and index pn = pd.Panel({'A': df}) pn['B'] = df2 pn['C'] = df3 pn['D'] = df4 for key in pn.items: print pn[key] -> output …

Drawing circles using matplotlib

Use the pylab.Circle command

import pylab #Imports matplotlib and a host of other useful modules cir1 = pylab.Circle((0,0), radius=0.75, fc='y') #Creates a patch that looks like a circle (fc= face color) cir2 = pylab.Circle((.5,.5), radius=0.25, alpha =.2, fc='b') #Repeat (alpha=.2 means make it very translucent) ax = pylab.axes(aspect=1) #Creates empty axes (aspect=1 means scale things so that circles look like circles) ax.add_patch(cir1) #Grab the current axes, add the patch to it ax.add_patch(cir2) #Repeat