Monday, December 17, 2012

Recover music files from iPod (Windows)

Problem: The computer with the original music is out of operation, leaving the only copy of music on the iPod. How to get the music back

On windows:
  1. Connect the iPod
  2. Double click on the iPod icon to open the folder
  3. It will list four folders none of which contain music. The music folder is hidden
  4. Press ALT+T to expose the tools menu
  5. Click on tools and show hidden files
  6. Enter the iPod_Controller folder
  7. Copy all the subfolders under this into a temporary directory (say /OldMusic) on the new computer (the music files are here)
  8. Select all music directories under  /OldMusic and uncheck hidden (the folders are hidden)
  9. Open up iTunes
  10. Click ALT to bring up the menu bar
  11. Click File->Add folder to add the folders to the library

Monday, December 10, 2012

Python: re (regexp): index of matches: difference between findall and finditer

  • using the findall method of a regexp will return all matches to the regular experession as a list
  • using finditer will return an iterator which you can use to interate through the matches
  • if m is the result of iterator.next() then m.span() will give you the start and end indexes of the match and m.group() gives you the match
  • If you have groups in the regexp e.g r"a(.*?)b" then m.group() will return the WHOLE matched expression and m.span() will return the corresponding start and stop indexes. To get just the particular group, for the expression given above, you have to do m.span(1) and m.group(1)

Sunday, December 9, 2012

Python: using a regexp to do pattern matching on a byte stream

Python regular expressions are a very concise and efficient way of performing pattern matching on strings. Many computing problems involves a similar kind of pattern matching, but on arbitrary data. For my particular application I have a long sequence of one byte digital codes that indicate a sequence of runs for an experiment. Each run starts with the codes 9,9,9 followed by some codes telling me what happened during the experiment and ends with 18,18,18. I need to split up this long sequence of codes into runs and then parse the events in each run.

In the past I would have written a state machine to do this, but I thought, that's a waste: the regexp module already implements logic of this kind. So I came up with the following:

  ec = array.array('B', [ev & 0xff for ev in event_sequence]).tostring()
  separate = re.compile(r"\x09\x09\x09(.*?)\x12\x12\x12")
  trials = separate.findall(ec)

array.array converts the sequence of bytes into a fake string and the regexp does its usual magic.

Friday, December 7, 2012

apsw + cifs (samba) on ubuntu

Problem: a python program using apsw on MacOSX runs fine and fast using c.executemany to do inserts. When the same program is run on Ubuntu 12 the program runs very slow.

Solution:  Use transactions explicitly. Use a 'BEGIN' and 'END' statements to bracket the list of inserts. For some reason on Mac OS X this does not make a difference, but in Ubuntu it does

Problem: On Ubuntu a python program using apsw can create a database just fine on a local drive but on a samba (smb) network drive it gives the error: apsw.BusyError: BusyError: database is locked

Solution: use the nobrl option to mount. eg. in the /etc/fstab the line should look like

//server/path   /mnt/LabServer cifs nobrl,credentials=/my/home/credfile,iocharset=utf8,file,uid=1000,gid=1000


Friday, November 16, 2012

LaTeX import

Problem: You need to break up a large LaTeX document into smaller files (eg. chapters) each in its own folder with figures. LaTeX `include` requires you to specify the path of the figures with respect to the main tex document, rather than the subdocument  which is illogical.

Solution: Use the import package and instead of \include{dir/file}, use \import{dir/}{file} which will handle subdirectories.


Monday, November 12, 2012

Less well known Python libraries

(Based on this thread from Hacker news)

  1. docopt - does the job of argparse but lets you specify the options in the doc string
  2. fuzzywuzzy - fuzzy string matching

Saturday, September 15, 2012

Getting a bottle.py application to work with launchd (mac os x)

Just as I had learned enough about SystemStarter to build a startup script to let Chotha run as a mac os service, I learn that SystemStarter has been deprecated and we should all now use launchd.

Here are some tips that might help others. The name of my launchd plist is com.bengalbionics.chotha.plist

plutil -lint com.bengalibionics.chotha.plist  will scan the plist file for any errors

sudo launchctl log level debug  will enable you to view some errors on the system.log

sudo launchctl log level error   will set the log level back

I could not get things to work by setting the Program to /path/to/python and /path/to/chotha.py as ProgramArguments

I had to set both as ProgramArguments.

The full plist is given below:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
        <key>Label</key>
        <string>Chotha</string>
        <key>ProgramArguments</key>
        <array>
                <string>/Library/Frameworks/Python.framework/Versions/current/bin/python</string>
                <string>/path/to/Chotha/chotha.py</string>
        </array>
        <key>RunAtLoad</key>
        <true/>
        <key>WorkingDirectory</key>
        <string>/path/to/Chotha/</string>
</dict>
</plist>


Saturday, August 4, 2012

Skim and skimpdf

I've been using skim to read pdfs instead of preview. Preview on OS X Lion annoyed me because of huge startup times and some other really annoying behavior of windows and grouping.

However, I missed being able to merge multiple pdfs which was quite easy with preview. Then I discovered that skim supplies a command line tool called skimpdf that allows us to do this! It is located in Skim.app/Contents/SharedSupport/skimpdf

Thursday, August 2, 2012

Setting numpy (pylab) printing options

Numpy (and pylab which uses numpy) can be instructed to change how it prints out arrays and other objects. The one I find most useful is:

pylab.set_printoptions(threshold=5)

If there are more than threshold elements in an array then pylab/numpy will use the ... (elipses) notation to print, which reduces clutter.

Tuesday, July 31, 2012

Python: Paths, packages and directory organization

Adding a path (/a) to a .pth file placed in a directory already on python's search path will add /a to the search path.

If we have a subdirectory b under a (/a/b) modules in thsi subdirectory will not be included in the path. A second line /a/b needs to be included in the .pth file

If we add a file __init__.py to b/ (the file can be empty) then python treats the structure as part of a package and we can then do (assuming mod.py is a module in /a/b)

import b.mod

Tuesday, July 17, 2012

Logging in python

It's just a wee bit tricky to set up python logging such that each module has its own logger and the logger level is controlled by a main script. The way to do that is illustrated below. The crucial line is

logging.basicConfig(level=logging.DEBUG)

in the main script. This line sets up all the loggers from root onwards and sets them to a uniform level.

main.py

import logging
import module as m
logging.basicConfig(level=logging.DEBUG)

logger = logging.getLogger(__name__)
logger.warning('This is a warning from the main script')
m.f1()


module.py 
import logging
logger = logging.getLogger(__name__)

def f1():
  logger.info('A warning from the module')

Saturday, June 16, 2012

Mr. Cheng and the Insurance salesmen

Starting some time in the first week of June I began to receive calls every five minutes, during working hours, on my prepaid cellphone. These numbers where not in my phonebook. Sometimes when I would pick up the phone would simply be disconnected. Most of the time, however, the caller would ask for Mr. Cheng.

Upon asking I would be informed that a Mr. Cheng had filled out an online form for insurance information and they were calling back about that.

Of course a cynical guy like me thinks, that's a shady story. I highly doubt that a Mr. Cheng would fillout at least 20 online forms with my cellphone number for them to contact me.

I think it is more likely this is some kind of dodgy move to get around the do not call list legalities.

I have filed a complaint with the FTC, but here is a list of the shady phone numbers so you can add them to your blocklist if you so wish.

Calling these numbers goes straight to voicemail and they are all insurance companies whom I will not name and give free publicity for. I would not buy insurance from them either.

360 474 3298
954 332 1713
615 776 1490
866 329 3468
240 308 7005
831 515 8394
754 223 9752
954 623 6599
503 457 1826
925 627 8350
323 761 2612
347 756 5059



Wednesday, June 13, 2012

Get rid of blank page at end of document after a table (MS Word and Libreoffice)

From a hint here we see that when we get a mysterious blank page at the end of a document, after a table that fills a page we can get rid of it by moving the cursor to just after the table and then setting the font to as small as possible.

Tuesday, June 12, 2012

Mount a file server on Linux (OpenSUSE)

First create a folder (say LabServer) where you have permissions.

Then do:

sudo mount -t cifs //xx.xxx.xx/ServerPath /home/kghose/LabServer -o username=user123

If you add the noperm option as

sudo mount -t cifs //xx.xxx.xx/ServerPath /home/kghose/LabServer -o username=user123,noperm

Then you don't have to deal with annoying permissions issues that might prevent you from writing to the disk

Tuesday, May 29, 2012

Googlesites: filecabinet = folder with files

From a tip here, creating a new page with the File Cabinet template is equivalent to creating a folder on the site with whatever files you upload. You can link to these files from other pages on the site.

Wednesday, May 23, 2012

Change document font (latex)

\usepackage{helvet} %Use Arial
\renewcommand{\familydefault}{\sfdefault}

Change page margins (latex)

\usepackage[top=.5in, bottom=.5in, left=.5in, right=.5in]{geometry}

Change section heading size (latex)

(http://www.latex-community.org/forum/viewtopic.php?f=4&t=3245)

\usepackage{titlesec}
\titleformat{\section}{\large\bfseries}{\thesection}{1em}{}

Sunday, May 20, 2012

Bolding author name in CV (latex)

If you use latex and bibtex to create your CV then you can use this trick to bold your name in the author list of the bibliography. There are several other methods shown here.:
  1. Use plain.bst, copy it over to the location where your cv tex file is. Rename it (eg. to cvplain.bst)
  2. Find the format.names function and change the line
    { s nameptr "{ff~}{vv~}{ll}{, jj}" format.name$ 't :=
    to
    { "\FormatName{" s nameptr "{ff~}{vv~}{ll}{, jj}" format.name$ * "}" * 't :=
  3. In your cv tex file add the following lines:
    \def\FormatName#1{%
    \def\myname{My Name}%
    \edef\name{#1}%
    \ifx\name\myname
    \textbf{#1}%
    \else
    #1%
    \fi
    }

Tuesday, May 15, 2012

ffmpeg on openSUSE

From the opensuse docs we see that ffmeg is found in the packman repository.

From the instructions here we can see how to use yast2 from the commandline to add the repo (I used ftp://ftp.uni-erlangen.de/pub/mirrors/packman/suse/openSUSE_12.1/) and then install the package.

Shooting yourself down

From this story on futility closet I came to know about Thomas W. Attridge's F11F 'Tiger' flight where, after firing off two volleys of 20mm cannon he put his aircraft into a supersonic dive and ran into the shells damaging his aircraft enough that he had to ditch it. The full story is here.

Monday, May 14, 2012

ffmpeg to create a movie from images

I've switched over to ffmpeg from mencoder. ffmpeg can be obtained from here for mac.

The ffmpeg documentation on creating video slideshows is great.

An example command line for such a use is:

ffmpeg -r 1/.5 -i "0%05d.jpg" -vcodec mjpeg -r 25 drop.avi


Important: if the numbering of your images doesn't start from 1 you should use the -start_number to indicate the first image (from here):
ffmpeg -start_number 2724  -r 1/.075 -i "DSC_%04d.jpg" -vcodec mjpeg -r 25 advair.avi

If there is a break in the file name numbering (eg. you have image01.jpg ... image05.jpg then image08.jpg) ffmpeg will stop at the break and finish the movie. You can use a short bash script to generate a copy of the files numbered as you want them.
ls *.JPG| awk 'BEGIN{ a=0 }{ printf "cp %s image_%04d.jpg\n", $0, a++ }' | bash

You should add options to ls to sort the files as you wish
Here, the first rate command (-r 1/.5) indicates how long each frame should take (here 0.5 of a second) and the second rate command (-r 25) indicates the movie frame rate.

Another example command to compile images into a movie is:
ffmpeg  -i test%06d.png -vcodec libx264 -x264opts keyint=123:min-keyint=20 -an sudoku.mkv


Friday, April 27, 2012

Increasing the nesting of subheadings in LaTeX

LaTeX lets you go \section, \subsection and then \subsubsection. It is possible to (easily) go a little bit further by putting \setcounter{secnumdepth}{5} in the preamble and then using \paragraph and \subparagraph

From here.

Saturday, April 14, 2012

IRS Free File Tip

  1. To add a W-2 click on the second tab (E-file your tax forms). This is counter intuitive. This should be in the "Add forms" tab.

Sunday, March 25, 2012

Mail.app + MS Echange server 2010 = memory leak

I have not found a solution for this problem

Mail.app when accessing an account on MS Exchange Server 2010 steadily increases is memory footprint past 1.3 GB with about an equal amount of virtual memory.

This problem does not occur when the account is deactivated and normal IMAP accounts are operative.

Tuesday, March 13, 2012

BRL-CAD

  1. It installs under /usr/brlcad
  2. Binaries are in /usr/brlcad/relxxxx/bin/mged and /archer

Monday, March 12, 2012

SVG:Firefox:Multiline text

Problem: Flowing text generated in Inkscape, saved as svg, does not work in firefox
Solution:  Use a regular text box and use carriage returns (or hard line breaks) to get the line breaks we want.

Details:
From the discussion here, the answer from Erik Dahlstrom and others, tells us that flow elements haven't been accepted even in the 1.2 specification. So Firefox etc won't display flowing text.

Monday, February 20, 2012

CSS empty div collapses

Problem: A div with a width but no content is collapsing (not visible)
Solution: Either put an &nbsp character, or explicitly set a div height

Sunday, February 5, 2012

CS:S map making with Hammer editor

Links
  • Valve's definitive guide is here.
  • A nice set of tutorials here.
  • Entity guide
  • Navigation point helper guide.
Notes


CT/T spawn
  1. Insert entities "info_player_counterterrorist" or "info_player_terrorist". 
  2. The number of entities you place is the max number of player spawn
  3. Valve suggests about 20 points per team 128 units apart
Placing lights
  • Do this in the end - debugging is harder with all those fancy lights
  • Make sure they are not recessed into ceilings/walls - use the 3D camera view to place light.
  • Do not name lights (if you do you should know what you are doing).
Buy Zones/Bomb plant zones
  1. Select tools/toolstrigger for texture, make a block with this texture where you want the buy zone. 
  2. Click "toEntity", select "func_buyzone" from the dropdown list. 
  3. Set the appropriate team name from the drop down.
For bomb zones, do the same first step, then select "func_bomb".  

Texture one face of a solid
  1. Press shift + A to bring up texture application tool 
  2. Select your texture and click on the surface you want textured
Making ramps
  1. Select the object
  2. Use the vertex tool to move the sides as needed
Props
  1. Select entity
  2. Select prop_dynamic, prop_static or prop_physics depending on the model you want
  3. When you select the model verify on the "info" tab that it's type is the same as the entity.
  4. Mismatch between the model type and prop type will cause the model not to appear on the map
Sounds
The way sounds work is that there are triggers (env_soundscape) that start the playing of a sound when a player gets within their radius and has line of sight with them. The sound then keeps playing for ever (for that player) until the player trips a different soundscape. The way to set up soundscapes, then, is to set them at doorways and other entrance/exits. 
  • soundscape descriptions
  • List of CS:S soundscapes 
Navigation
  1. Only edit the mesh once you are done laying out the map
  2. Load the map in CS:S, go into spectate and bring up the console
  3. sv_cheats 1
  4. nav_edit 1 (to see the mesh)  
  5. Usually upstairs areas only connected by ladders/ramps/stairs will not be connected by the automatically generated mesh
  6. Go to the highest navigable point, point at a spot and type nav_make_walkable. A blue pyramid will appear
  7. Go to each ladder in turn, point to it and type nav_make_ladder
  8. Now type nav_generate to regenerate the mesh. The top walkable should propagate everywhere
  9. Sometimes some ledges may not be included in the jump calculations: you can forcibly join the ledge: point at the top part, type nav_mark, then point at the landing point and then type nav_connect.
  10. Type nav_save to save the mesh  
Mini-Map
Follow instructions here.

Introtext
Type your intro text into a text file. Name the text file as map_name.txt (map_name being your exact map name) and save it in the same directory as the map file.


Debugging tips
  • sv_cheats 1
  • mat_wireframe 1
  • cl_leveloverview 8 (or whatever scale factor works)
  • nav_edit 1 (To see nav meshes, helpful to see jumps etc.)
  • mat_fullbright 1 (To remove lighting and see everything) 
  • compile with "rad" set to "no" - faster compilation with no lightmaps
Misc stuff
  • Night Vision does not amplify red
  • Grouping lights lets you change their properties together

Friday, February 3, 2012

Python : struct.upack format data justification

Problem:
While reading data from a binary data file, when mixing data formats with different lengths (e.g. 'h' and 'f') python will assume the file has pad bytes that align all the data.

Solution
Use the '=' qualifier to prevent weird justification.

Mac OS Lion PPoE bug

  1. In system preferences->Network create a new PPoE connection. Check "show modem status in menu bar"
  2. A symbol "<...>" appears in the menu bar
  3. Delete the PPoE connection
  4. The symbol remains in the menu bar and it is not possible to remove it until ...
  5. ... you create a new PPoE connection, uncheck "show modem status..." and then delete it

Tuesday, January 24, 2012

Mac OS Lion, external display and sleep

When I close the Macbook (Mac OS Lion) lid it usually goes to sleep. But with an external display attached and a usb mouse attached the computer just switches to using the external display, rather than sleeping when the lid is closed.

From here, we see that we should use the apple menu to explicitly send the computer to sleep.