Posts

Showing posts from 2012

Recover music files from iPod (Windows)

Problem: The computer with the original music is out of operation, leaving the only copy of music on the iPod. How to get the music back

On windows:
Connect the iPodDouble click on the iPod icon to open the folderIt will list four folders none of which contain music. The music folder is hiddenPress ALT+T to expose the tools menuClick on tools and show hidden filesEnter the iPod_Controller folderCopy all the subfolders under this into a temporary directory (say /OldMusic) on the new computer (the music files are here)Select all music directories under  /OldMusic and uncheck hidden (the folders are hidden)Open up iTunesClick ALT to bring up the menu barClick File->Add folder to add the folders to the library

Python: re (regexp): index of matches: difference between findall and finditer

using the findall method of a regexp will return all matches to the regular experession as a listusing finditer will return an iterator which you can use to interate through the matches if m is the result of iterator.next() then m.span() will give you the start and end indexes of the match and m.group() gives you the matchIf you have groups in the regexp e.g r"a(.*?)b" then m.group() will return the WHOLE matched expression and m.span() will return the corresponding start and stop indexes. To get just the particular group, for the expression given above, you have to do m.span(1) and m.group(1)

Python: using a regexp to do pattern matching on a byte stream

Python regular expressions are a very concise and efficient way of performing pattern matching on strings. Many computing problems involves a similar kind of pattern matching, but on arbitrary data. For my particular application I have a long sequence of one byte digital codes that indicate a sequence of runs for an experiment. Each run starts with the codes 9,9,9 followed by some codes telling me what happened during the experiment and ends with 18,18,18. I need to split up this long sequence of codes into runs and then parse the events in each run.

In the past I would have written a state machine to do this, but I thought, that's a waste: the regexp module already implements logic of this kind. So I came up with the following:

ec = array.array('B', [ev & 0xff for ev in event_sequence]).tostring()
separate = re.compile(r"\x09\x09\x09(.*?)\x12\x12\x12")
trials = separate.findall(ec)

array.array converts the sequence of bytes into a fake string and the r…

apsw + cifs (samba) on ubuntu

Problem: a python program using apsw on MacOSX runs fine and fast using c.executemany to do inserts. When the same program is run on Ubuntu 12 the program runs very slow.

Solution:  Use transactions explicitly. Use a 'BEGIN' and 'END' statements to bracket the list of inserts. For some reason on Mac OS X this does not make a difference, but in Ubuntu it does

Problem: On Ubuntu a python program using apsw can create a database just fine on a local drive but on a samba (smb) network drive it gives the error: apsw.BusyError: BusyError: database is locked

Solution: use the nobrl option to mount. eg. in the /etc/fstab the line should look like

//server/path   /mnt/LabServer cifs nobrl,credentials=/my/home/credfile,iocharset=utf8,file,uid=1000,gid=1000

LaTeX import

Problem: You need to break up a large LaTeX document into smaller files (eg. chapters) each in its own folder with figures. LaTeX include requires you to specify the path of the figures with respect to the main tex document, rather than the subdocument  which is illogical.

Solution: Use the import package and instead of \include{dir/file}, use \import{dir/}{file} which will handle subdirectories.

Less well known Python libraries

(Based on this thread from Hacker news)

docopt - does the job of argparse but lets you specify the options in the doc stringfuzzywuzzy - fuzzy string matching

Getting a bottle.py application to work with launchd (mac os x)

Just as I had learned enough about SystemStarter to build a startup script to let Chotha run as a mac os service, I learn that SystemStarter has been deprecated and we should all now use launchd.

Here are some tips that might help others. The name of my launchd plist is com.bengalbionics.chotha.plist

plutil -lint com.bengalibionics.chotha.plist  will scan the plist file for any errors

sudo launchctl log level debug  will enable you to view some errors on the system.log

sudo launchctl log level error   will set the log level back

I could not get things to work by setting the Program to /path/to/python and /path/to/chotha.py as ProgramArguments

I had to set both as ProgramArguments.

The full plist is given below:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>Label</key>
…

Skim and skimpdf

I've been using skim to read pdfs instead of preview. Preview on OS X Lion annoyed me because of huge startup times and some other really annoying behavior of windows and grouping.

However, I missed being able to merge multiple pdfs which was quite easy with preview. Then I discovered that skim supplies a command line tool called skimpdf that allows us to do this! It is located in Skim.app/Contents/SharedSupport/skimpdf

Setting numpy (pylab) printing options

Numpy (and pylab which uses numpy) can be instructed to change how it prints out arrays and other objects. The one I find most useful is:

pylab.set_printoptions(threshold=5)
If there are more than threshold elements in an array then pylab/numpy will use the ... (elipses) notation to print, which reduces clutter.

Python: Paths, packages and directory organization

Adding a path (/a) to a .pth file placed in a directory already on python's search path will add /a to the search path.

If we have a subdirectory b under a (/a/b) modules in thsi subdirectory will not be included in the path. A second line /a/b needs to be included in the .pth file

If we add a file __init__.py to b/ (the file can be empty) then python treats the structure as part of a package and we can then do (assuming mod.py is a module in /a/b)

import b.mod

Logging in python

It's just a wee bit tricky to set up python logging such that each module has its own logger and the logger level is controlled by a main script. The way to do that is illustrated below. The crucial line is

logging.basicConfig(level=logging.DEBUG)

in the main script. This line sets up all the loggers from root onwards and sets them to a uniform level.

main.py

import logging import module as m logging.basicConfig(level=logging.DEBUG) logger = logging.getLogger(__name__) logger.warning('This is a warning from the main script') m.f1()

module.py
import logging logger = logging.getLogger(__name__) def f1(): logger.info('A warning from the module')

Mr. Cheng and the Insurance salesmen

Starting some time in the first week of June I began to receive calls every five minutes, during working hours, on my prepaid cellphone. These numbers where not in my phonebook. Sometimes when I would pick up the phone would simply be disconnected. Most of the time, however, the caller would ask for Mr. Cheng.

Upon asking I would be informed that a Mr. Cheng had filled out an online form for insurance information and they were calling back about that.

Of course a cynical guy like me thinks, that's a shady story. I highly doubt that a Mr. Cheng would fillout at least 20 online forms with my cellphone number for them to contact me.

I think it is more likely this is some kind of dodgy move to get around the do not call list legalities.

I have filed a complaint with the FTC, but here is a list of the shady phone numbers so you can add them to your blocklist if you so wish.

Calling these numbers goes straight to voicemail and they are all insurance companies whom I will not name and gi…

Get rid of blank page at end of document after a table (MS Word and Libreoffice)

From a hint here we see that when we get a mysterious blank page at the end of a document, after a table that fills a page we can get rid of it by moving the cursor to just after the table and then setting the font to as small as possible.

Mount a file server on Linux (OpenSUSE)

First create a folder (say LabServer) where you have permissions.

Then do:

sudo mount -t cifs //xx.xxx.xx/ServerPath /home/kghose/LabServer -o username=user123

If you add the noperm option as

sudo mount -t cifs //xx.xxx.xx/ServerPath /home/kghose/LabServer -o username=user123,noperm

Then you don't have to deal with annoying permissions issues that might prevent you from writing to the disk

Googlesites: filecabinet = folder with files

From a tip here, creating a new page with the File Cabinet template is equivalent to creating a folder on the site with whatever files you upload. You can link to these files from other pages on the site.

Change document font (latex)

\usepackage{helvet} %Use Arial
\renewcommand{\familydefault}{\sfdefault}

Change page margins (latex)

\usepackage[top=.5in, bottom=.5in, left=.5in, right=.5in]{geometry}

(http://www.latex-community.org/forum/viewtopic.php?f=4&t=3245) \usepackage{titlesec}
\titleformat{\section}{\large\bfseries}{\thesection}{1em}{}

Bolding author name in CV (latex)

If you use latex and bibtex to create your CV then you can use this trick to bold your name in the author list of the bibliography. There are several other methods shown here.:
Use plain.bst, copy it over to the location where your cv tex file is. Rename it (eg. to cvplain.bst)Find the format.names function and change the line
{ s nameptr "{ff~}{vv~}{ll}{, jj}" format.name$'t := to { "\FormatName{" s nameptr "{ff~}{vv~}{ll}{, jj}" format.name$ * "}" * 't :=In your cv tex file add the following lines:
\def\FormatName#1{%
\def\myname{My Name}%
\edef\name{#1}%
\ifx\name\myname
\textbf{#1}%
\else
#1%
\fi
}

ffmpeg on openSUSE

From the opensuse docs we see that ffmeg is found in the packman repository.

From the instructions here we can see how to use yast2 from the commandline to add the repo (I used ftp://ftp.uni-erlangen.de/pub/mirrors/packman/suse/openSUSE_12.1/) and then install the package.

Shooting yourself down

From this story on futility closet I came to know about Thomas W. Attridge's F11F 'Tiger' flight where, after firing off two volleys of 20mm cannon he put his aircraft into a supersonic dive and ran into the shells damaging his aircraft enough that he had to ditch it. The full story is here.

ffmpeg to create a movie from images

I've switched over to ffmpeg from mencoder. ffmpeg can be obtained from here for mac.

The ffmpeg documentation on creating video slideshows is great.

An example command line for such a use is:

ffmpeg -r 1/.5 -i "0%05d.jpg" -vcodec mjpeg -r 25 drop.avi

Important: if the numbering of your images doesn't start from 1 you should use the -start_number to indicate the first image (from here):
ffmpeg -start_number 2724 -r 1/.075 -i "DSC_%04d.jpg" -vcodec mjpeg -r 25 advair.avi
If there is a break in the file name numbering (eg. you have image01.jpg ... image05.jpg then image08.jpg) ffmpeg will stop at the break and finish the movie. You can use a short bash script to generate a copy of the files numbered as you want them. ls *.JPG| awk 'BEGIN{ a=0 }{ printf "cp %s image_%04d.jpg\n", \$0, a++ }' | bash
You should add options to ls to sort the files as you wish
Here, the first rate command (-r 1/.5) indicates how long each frame should take (here …

Increasing the nesting of subheadings in LaTeX

LaTeX lets you go \section, \subsection and then \subsubsection. It is possible to (easily) go a little bit further by putting \setcounter{secnumdepth}{5} in the preamble and then using \paragraph and \subparagraph

From here.

IRS Free File Tip

To add a W-2 click on the second tab (E-file your tax forms). This is counter intuitive. This should be in the "Add forms" tab.

Mail.app + MS Echange server 2010 = memory leak

Mail.app when accessing an account on MS Exchange Server 2010 steadily increases is memory footprint past 1.3 GB with about an equal amount of virtual memory.

This problem does not occur when the account is deactivated and normal IMAP accounts are operative.

SVG:Firefox:Multiline text

Problem: Flowing text generated in Inkscape, saved as svg, does not work in firefox
Solution:  Use a regular text box and use carriage returns (or hard line breaks) to get the line breaks we want.

Details:
From the discussion here, the answer from Erik Dahlstrom and others, tells us that flow elements haven't been accepted even in the 1.2 specification. So Firefox etc won't display flowing text.

CSS empty div collapses

Problem: A div with a width but no content is collapsing (not visible)
Solution: Either put an &nbsp character, or explicitly set a div height

CS:S map making with Hammer editor

Valve's definitive guide is here.A nice set of tutorials here. Entity guide Navigation point helper guide.Notes

CT/T spawn
Insert entities "info_player_counterterrorist" or "info_player_terrorist". The number of entities you place is the max number of player spawnValve suggests about 20 points per team 128 units apart Placing lights
Do this in the end - debugging is harder with all those fancy lights Make sure they are not recessed into ceilings/walls - use the 3D camera view to place light.Do not name lights (if you do you should know what you are doing).Buy Zones/Bomb plant zones
Select tools/toolstrigger for texture, make a block with this texture where you want the buy zone. Click "toEntity", select "func_buyzone" from the dropdown list. Set the appropriate team name from the drop down. For bomb zones, do the same first step, then select "func_bomb".

Texture one face of a solid
Press shift + A to bring up texture applicatio…

Python : struct.upack format data justification

Problem:
While reading data from a binary data file, when mixing data formats with different lengths (e.g. 'h' and 'f') python will assume the file has pad bytes that align all the data.

Solution
Use the '=' qualifier to prevent weird justification.

Mac OS Lion PPoE bug

In system preferences->Network create a new PPoE connection. Check "show modem status in menu bar"A symbol "<...>" appears in the menu barDelete the PPoE connectionThe symbol remains in the menu bar and it is not possible to remove it until ...... you create a new PPoE connection, uncheck "show modem status..." and then delete it

Mac OS Lion, external display and sleep

When I close the Macbook (Mac OS Lion) lid it usually goes to sleep. But with an external display attached and a usb mouse attached the computer just switches to using the external display, rather than sleeping when the lid is closed.

From here, we see that we should use the apple menu to explicitly send the computer to sleep.