Sunday, December 6, 2009

Splinter Cell: Chaos Theory - disabling lights

Well, the right click (alt fire) on the suppressed pistol activates some magic device that can temporarily knock out a light - for about 10s or so. Why did I have to search on the internet and track down vague clues until I hit this page and found out for sure? Shouldn't there  be a manual somewhere? UPDATE: I gotta read the notes on the equipment screen more carefully...

UPDATE2: The manual is useful. Save trees/animal skins, don't print it out.

Friday, December 4, 2009

Max os x service start up : StartupItem

A good guide is available here

File:  /Library/StartupItems/RRiki/StartupParameters.plist:

 Description = "RRiki server";
 Provides    = ("RRiki");
 Uses        = ("Ruby");
 OrderPreference = "Last";
 Messages =
  start = "Starting RRiki server";
  stop = "Stopping RRiki server";
  restart = "Restarting RRiki server";

File: /Library/StartupItems/RRiki/RRiki


# RRiki service startup script

. /etc/rc.common

StartService ()
    ConsoleMessage "Starting RRiki"
    cd /Users/kghose/Source/Rriki2/
    /usr/local/bin/thin start -eproduction -d

StopService ()
    ConsoleMessage "Stopping RRiki"
    cd /Users/kghose/Source/Rriki2/
    /usr/local/bin/thin stop 

RestartService ()
    ConsoleMessage "Restarting RRiki"
    cd /Users/kghose/Source/Rriki2/
    /usr/local/bin/thin restart -eproduction -d

RunService "$1"

To manipulate the service:
sudo SystemStarter restart RRiki

Thursday, December 3, 2009

Moved from mongrel to thin

Now using thin rather than mongrel. Seems to be fast and responsive and most importantly installs without errors.

Calling an action when a selection box changes

in partial (view)

<%= select('history', 'sel', history_list, {:selected => selected}, {:class => 'historyselect'}) %>
<%= link_to_remote 'x',
      :url => {:controller => 'rriki',
               :action => 'clear_history'},
      :html => {:title => 'Clear history'},
      :method => :get %>
<%= observe_field 'history_sel',
    :url => {:controller => 'rriki',
             :action => 'select_item_in_history'},
    :with => 'history_sel' %>

changing the select will call the action select_item_in_history with params['history_sel'] set to the selected value.

Testing to see if a variable exists in ruby

Partials can take parameters (called locals), but can crash your app if a caller does not pass a local and you try and access it in the partial. You can use the following construct to initialize a variable if it does not exist

var = nil unless defined?var

where var is your variable

Rails: Debugging form helpers generated with remote calls (AJAX)

When parts of a webpage are updated using AJAX calls (e.g remote functions) these updated parts do not show up when you ask Firefox for the webpage html (using CTRL+U). A hack to see the generated html from a suspect rails call is to simply print out the html on the webpage instead of executing it by doing:

<%=h ...rails command... %> Instead of

<%= ...rails command... %>

html_options for rails select tag

I had a select tag that I wanted to mess with a little to apply some styles etc. to. The select tag has an html_options hash and I was trying to use that. My initial attempt was

select('history', 'sel', history_list, html_options => {:class => 'historyselect'}) WRONG

This is not python, and even though the rails docs setout the syntax as:

select(object, method, choices, options = {}, html_options = {})

ruby can not accept keyword arguments. Only positional, so you have to do:

select('history', 'sel', history_list, {}, {:class => 'historyselect'})

The empty braces signify the empty 'options' argument, and the next set of braces signify the html_options hash

Tuesday, November 24, 2009


At NIH when you look at a grant they all have a Program Announcement number (PA). When you go to they ask for the Funding Opportunity Announcement number (FOA). The first test of eligibility to apply for the grant is to realize that they both mean the same thing...

Wednesday, November 11, 2009

Running a script from the regular python shell

regular python shell: execfile('')
ipython: run
           : run -i (if you want the script to have access to the workspace variables)

Tuesday, November 10, 2009

Installing brian

  1. nose
  2. sympy

Comparison on neural network simulators

A tabular wiki page of GPLd or otherwise open source code that can be used to run neural network simulations.

Notable among them is Brian, which is written in Python. The venerable NEURON also has hooks into python.

Tuesday, November 3, 2009

affixa = the new gattach

gattach was a handy tool to use for webmail clients like gmail. I couldn't find it anymore, but then from this discussion I found out that gattach is now affixa.

Sunday, November 1, 2009

Form contents after a crash

Firefox crashes semi-occasionally. On the plus side, when it recovers it repopulates any form fields I have been filling out. All is not lost...

Saturday, October 31, 2009

Neurojobs and Firefox 3.5.3 on Mac

The multi select does not work, unless you use user agent switcher to pretend you are IE6

Wednesday, October 28, 2009

Wednesday, October 21, 2009

Adjusting focus on d40

There are people, apparently, who use hex keys and whatnot to mess with the focus calibration of their nikon camera. I prefer to take pictures with mine.

Thursday, October 8, 2009

HP Pavillion Dv2700 intermittent fan

Problem: Laptop (HP Pavillion DV2700) has an intermittent fan. Fan goes on for a second and then stops, goes on again, stops. While this may be OK for the hardware (but who knows) it drives the user nuts.
Cause: Laptop bios upgraded to F.2E (latest)
My Fix: Downloaded and installed F.25 (older version) and this stopped the intermittent fan. I hope that this does not lead to overheating problems, I should monitor CPU temps.

Wednesday, October 7, 2009

50mm f/1.8 AF-D

I'm talking about this lens.
Its made in China, it costs about $130 now, bumped up from about $100 a year back. The focusing ring is dampened and easy to grip (Its a MF on the D40). The mount is metal, but not machined accurately - mine is a tight fit on the camera (and others have commented on this).
The image quality is excellent. This is a set of images of a deck of playing cards shot at the nearest range and at different apertures. The images are 100% crops from center (left) to edge (right). The DOF at f/1.8 is really shallow. The cards look washed out at f/1.8, but if you look at the red diamond and the edge of the middle card, they are sharp, indicating that the DOF is shallow enough (.1" or less) that the card goes out of focus by the time you go from center to edge. At f/2.5 the sharpness and contrast are remarkable:

Sunday, October 4, 2009

The spelling is 'coArse'

So here I was, sitting at a class in MIT and the instructor puts up a slide and there is a legend that says 'course coding'. It is clear from the concept that the meaning is 'coarse coding', with 'coarse' being the opposite of fine. I keep my big mouth shut, because we are here for the science, and I don't want to side track people pointing out pipifax things like typos on a slide. A few slides later we come across coarse coding, and it is again spelled 'course'. I keep quiet. Again, and again repeatedly on different slides. Now I know its not a typo. The instructor thinks that's the right spelling. I keep quiet.

Some days later I come across the misspelling on some website. Then again on slashdot, and now I have to open my big mouth to relieve the pressure.

Its 'coarse'. 'COARSE'. Meaning...well, not fine. 'course' means a path, like a race course, or the course of a river.

Buyer beware

A month or so ago, my spouse got me something from Brookstone. It was a great idea, but I realized that the product did not work out for me. On the receipt there was a note saying that returns could be made up to 60 days. We went a month or so after purchase and were told that Brookstone's policy was to refund the money before 30 days were up and to give store credit after that.

I personally think this is misleading. That store policy should be prominently displayed on the receipt where it says 60 days to return. Otherwise its a shady move - probably false advertising. A little bit on the range of mail in rebates - the whole idea with the mail in rebate is that a certain fraction of people will forget to mail it in, so you get more people to buy (they think they are paying less for it) but your actual revenues are more (because the average price is more than you say it is with rebate).

Brookstone's policy is in the similar vein. A certain fraction of customers (like us) will see 'Returns in 60 days' and assume its the regular return policy as you get in most stores (money back), while really after 30 days, they actually have your money and you have no merchandise. Clever.

I found nothing in the store I liked, but they have my spouse's money, so I need to get something. But never again.

Saturday, October 3, 2009

iTunes 9 : Aaargh

It just works. Not. Updated iTunes to 9, now it freezes, uses 100% CPU and locks up, randomly. Why do I mess with something that was working?

Thursday, October 1, 2009

Slashdot QOD

Q: Choosing a Personal Printer For the Long Haul (Writer is asking for advice on what printer to buy that will last him for ever and ever and cost nothing and will take direct hits from meteorites and keep working)
A: The best laser printer is the one that uses the same toner cartridges as the one at work. (by Anonymous Coward on Thursday October 01, @10:38AM)


VisionEgg compiles and runs fine on Mac OS X using sudo python install, just make sure you have Pyro if you want to use the experiment server and GUI

Saturday, September 26, 2009

Spotting photographic equipment scammers

  1. Go to nexttag or some other site and figure out the price range of the product
  2. Go to several different resellers and check the prices.
  3. Do a search with the reseller's name along with the phrase 'bait and switch'
  4. Find the product on amazon, adorama, ritzcamera and
If the
  1. Price of the product is much below (say 30%) the well know retailers' prices
  2. Accessories are correspondingly more costly
  3. Extended warranties are correspondingly more costly
  4. website sells the same product at a higher price and it looks like one is body only and the other is kit (but the website does not tell you)
Then be wary. They could be
  1. Trying to bait and switch you (sell you another product after saying out of stock)
  2. Trying to sell you over priced accessories and warranties to make up
  3. Selling a used or refurb kit as new
  4. Just plain old scam you - take your money and not deliver the product.

Tuesday, September 22, 2009

Macro lens

A lens of a given focal length has a certain field of view. If you can focus the lens close enough, you can get large reproductions of objects. Such lenses are called macro lenses.

Maximum reproduction ratio

1:6.6 maximum reproduction ratio -> 10mm object will take up 1.515151515151 (sorry had to do that) mm on the sensor.

DX sensor on D40 23.7mm -> 3008 pixels so 1.51 mm -> 192 pixels -> about 1in for a 200dpi print.

So with the 50mm 1.8 AF-D lens if you shoot at the closest distance (about .5m) a 1cm object can be printed at 1in decently. Not that you would want to use this lens that way...

Latex math: Vertical bar

Like that used for indicating the evaluation of integrals between limits:


as in


from a hint here from robphy

Friday, September 18, 2009

50mm f/0.7

Its billed as the world's fastest lens. A limited number were built for NASA (glamor portraits in outer space - astronauts photographed only by the light of their own instruments) and Stanley Kubrik snagged a few to make some obscure movie shot mostly by candle light (yep, he spent all his money on those fancy high speed lenses and couldn't afford real lighting).

Here is a picture of one. And here is a page by the guy who modified the lenses for Kubrik.

PS. You've probably figured out I was kidding about some of the stuff about astronaut portraits and Kubrik running out of money...

Thursday, September 17, 2009

How the D40 handles a multi dimensional problem

There are three dimensions to exposure:
  1. Aperture
  2. Shutter speed
  3. Sensitivity
In film days adjusting sensitivity on the fly was not practical, so you basically had a two dimensional plane once you loaded film.

The life of the camera computer was easy when you set the camera to A or S modes: all it had to do was adjust the free variable to maintain correct exposure. In M mode the camera merely informed you of exposure letting you roam freely in this 2D space.

The D40 can be setup exactly this way if you select fixed ISO. If you select auto ISO, however, the camera now is handed two free parameters (for A and S) and one free parameter (for M). How does the D40 deal with this?

For A mode the computer minimizes ISO and maximizes shutter speed. You can set a lower limit for Shutter speed and upper limit for ISO and the camera will drop shutter speed until it hits the lower limit and the start to bump ISO.

For S mode the computer minimizes ISO. It will maximize the aperture until it reaches the lens' limit. The it bumps ISO.

This all sounds sensible, but what about M mode? This is funny. In M mode you no longer have the freedom to mess with exposure - the camera runs loose with ISO changing it to give you correct exposure regardless of your A and S combination. Eventually it hits the camera ISO limits and starts to show you over- and under-exposure.

M mode with auto-ISO will enable you to play with particular shutter (motion capturing) and aperture (DOF) combinations for your subject that would normally be inaccessible to you (cumbersome with manual ISO, impractical with film).

Everything you wanted to know about flashes

(But couldn't find on google because it thought you were looking for compact flash memory cards)

Hopefully you find this link if you are looking for
  1. Photographic flashes - AKA:
  2. Strobe
  3. Strobelight
  4. Flash
  5. Camera flash
  6. Compact flash (Hell why not)

Wednesday, September 16, 2009

IR Remote for Nikon

I didn't buy this for my Film SLR because I cost too much. Currently Nikon's brand (ML-L3) costs $15 on Amazon/Adorama etc. and on Nikon Mall costs $20. However, if you look here (bestofferbuy) you can get an 'alternate version', called the YN ML-L3, for $4.80 and free shipping. I was a little wary at first and wondered if this was a scam, but I decided to try it out and ordered two (aha, see how this works, they price it less and that gets the sucker customer to buy more). I guessed it would take a long time to get here, but it took about a week and a half. I tried out the first one and it works fine. The unit comes with battery installed but with a plastic tab to keep it from discharging. These shady Chinese operators are moving upscale. (Actually it ships from Hong Kong. Don't know if that's any different anymore).

I have plans for the other one. An intervalometer costs an insane amount of money. Say like $140. What I want to do is order a 555 and some pots from Digikey and modify the second remote for time lapse photography. Who knows, I may even get fancy and use a crystal and a binary decade counter with BCD encoding for accurate timing...

Tuesday, September 15, 2009


Any Nikon camera with lens info contacts (e.g. any DSLR and any newer Nikon) needs the BR2A reverser, rather than the BR2 to reverse lenses.

Monday, September 7, 2009

Rounded borders in Firefox

CSS3 supports divs with rounded borders, but for now Firefox lets you use something like
-moz-border-radius: 2em;

Grouping results in rails

I needed to group records by year. From this post and the enumerate module docs we end up with
   entries = Entry.find :all, :order => 'date DESC'
entries.group_by do |entry|
The group_by returns a hash with keys corresponding to the result of the block (i.e. the year)

Sunday, September 6, 2009

What will 6MP get you?

So, how big a print can I make with a 6MP (3000x2000pixel) image, if I assume that the image is meant to be seen all at once?

From wikipedia (eye acuity) we find that a human with 20/20 vision can resolve 2 arcminute per line pair. This means two pixels separated by 2/60 = .033 degrees they can be seen separately

Then a 1000 pixels can subtend 1000/2 * .033 = 16.66 degrees, so a 6MP image can subtend 50x33 degrees.

At a distance of .3m (arm's length) this works out to .358m x .195m (14.3"x7.8")

At a distance of 1m this works out to 1.192m x .65m (47.5"x25.87")

I don't think this is too shabby.

Thursday, September 3, 2009

D40 quick controls

  1. Set Fn button to control ISO
  2. Set AE/AF lock to AE only
In A/S (Aperture/Shutter priority) modes
  1. Wheel alone controls Aperture(Shutter)
  2. Wheel + function controls ISO
  3. Wheel + (+/-) controls exposure compensation (I keep forgetting this)
In M (Full manual) modes
  1. Wheel controls shutter
  2. Wheel + (+/-) controls aperture
  3. Wheel + function controls ISO
  4. Exposure is indicated on the bar graph
In Auto mode you only get to control ISO

Wednesday, September 2, 2009

Digital vs FIlm and Point and Shoot vs SLR

I'm sure Google crashes when you do this search. But here's what I think:
  1. The only way to get photographs you attach to is to take a lot of them as often as possible, look at the results and link it with what you did.
  2. Digital makes this process cheaper and faster. Digital gives you instant feedback. From the instant LCD display, to the home viewing on the big monitor to the prints at the dime store.
  3. More pictures, more fun.
  4. I think the guys who say 'all you need is a pinhole' are right. Really. It ain't the camera. I have photos I treasure from my Dad's Exakta RTL (with a busted exposure meter), from point and shoot Yashikas and Kyoceras, and from my Canon A510. From my D40 so far I have perhaps two photos I treasure for artistic reasons (out of may be 500). The camera and the lens just don't matter. Just take a lot of photos. And have fun.
  5. UPDATE: One thing though - the D40 has really good low light capability. I took several indoor shots with room lighting and they came out gorgeous handheld. My point of comparison is with a Canon A510 so current P&S may have improved.

Tuesday, September 1, 2009

The moon and six pence

Photography increases my admiration for the human visual system. I look up into the night sky and I see the full moon, I see the stars around it and the houses below. Then I put my camera on a tripod and try to capture the same scene. WOW! The moon is BRIGHT! I mean you could shoot it hand-held (1/30th even at f13). But wait, at that exposure the stars are gone, not to mention the house. I go down to 1" at f13. Now I get the stars, not really the house and the moon is one big halo with no internal structure.

Shutter mechanisms

I got fascinated by the camera shutter mechanisms and started to do some reading:
  1. A great set of images of a shutter opening and closing. Read the tech details of how it was done.
  2. A very nice writeup with attractive hand drawn sketches of the principle of the focal plane shutter and some funny effects that occur with fast moving objects or flash use.
I wanted to know more about the shutter mechanism.
  1. Nikon has two pages with tiny drawings [1],[2]
  2. A search on google images with the word "shutter mechanism" and filter by line art gives interesting results
  3. An example of how shutter speed knobs interact with the actual shutter is given here.

Monday, August 31, 2009

Unhappy with shutterfly photo scans

In 2006 I had taken a bunch of film photos and had them processed by shutterfly. Shutterfly also scanned these photos and put them in my account so I was happy. Finally this year I decided I wanted the scans on my computer and I ordered a DVD of them (you can't download the hi-res versions from the website). I was unpleasantly surprised when I got the scans back. They are 270 ppi or so, amounting to 1536x1024 from 35mm film. I would have expected something more detailed from them such as 2000 ppi or so. But it was a bad surprise as some of the photos were landscapes and I was looking forward to enlargements. I guess I'll have to dig out the negatives from the boxes.

Saturday, August 29, 2009

D40 : Burst mode test

Kingston SDGC 4GB class 6 (SD6/4GB) card. Auto focus (Prefocused by half shutter press before burst). Auto mode. Large/Fine Jpeg. Off white wall with a few objects on it.
23 pictures in 11 seconds. Buffer sat at around r4/r5 during burst. After burst the card activity light flickered for about 4s.

Friday, August 28, 2009

The D40 and the F65

My own first SLR is the Nikon F65 which I got in 2001 for about $400 (more than my D40!). It was billed as an entry level SLR for beginners. That's me, a beginner for about a decade now. I've used a digital compact (Canon A510) since 2005 and details of the F65 slowly faded from my mind as it stayed stashed in the closet. But after I got the D40 I began to remember things - didn't it have a DOF preview, didn't it have this? That? So I got it out of the closet and lookee here, the F65 (Nikon entry level film) has:
  1. Depth of field preview
  2. Exposure bracketing
  3. Five (5) AF areas
  4. Pentaprism
  5. Top LCD
It doesn't have two control wheels, but has the control wheel + modifier buttons.

Hmm... methinks the Nikon engineers are losing to the Nikon marketing people (C'mon DOF preview, exposure bracketing, 5 AF areas these don't add to cost - they probably COST money to take away, the only reason to take these small things away is to make people buy the more expensive one isn't it?). The pentaprism is cost and weight, and the top LCD is redundant.


I was reading an essay by a photographer who shoots 4x5 format for landscapes. Its very entertaining to read about how he focuses on a ground glass screen and then uses a loupe to check focus. But here's a sentence that I found entertaining because its technically wrong, but I think I know what he means:

"Fourth, the inverted image on the ground glass creates a more direct visual impression because the brain does not have to flip the image upside down. This last remark is based on the fact that our eyes act as lenses and thus project an upside-down image of the world to the brain, which then has to flip it right side up. Because the ground glass image is inverted it is projected to the brain right side up thereby nullifying the need for the brain to rectify the image."

Its hard to explain why this is wrong but basically the brain is used to how it gets the images and it is harder for the brain to process an upside down image. Consider the empirical results that we find it harder to recognize upside down faces, and read upside down writing.

But I think what the author (Alain Briot) experiences is his brain reacting to the novelty of the upside down image. Because the image is upside down, and is harder to recognize, his brain starts to analyze the picture as individual components (note his use of the loupe to focus). At this stage of the artistic process he is trying to extract detail from little pieces of the image (hence his need for large format). The upside down image forces him to do this.

Thursday, August 27, 2009

D40 : Manual lens

I have a F65 which came with a 28-80mm lens. I used this lens on the D40. It required manual focus, the red focus dot did not work as a focus indicator and I could not select a focus area. (I had gotten the impression that the focus circuitry would still be working so that I could use the focus indicator to judge focus). The dot works. It just goes off when you are way off focus, flashes when you are close and is steady when you are focused. Now you know why my moon shot was out of focus and I was complaining that there was no red dot :P

The exposure meter still gave readings, and they looked sensible. Here are two shots of the moon. One with the kit lens at 55m and the other with the manual lens at 80mm with a UV filter on it. The 80mm shot is blurry, either because of my poor focus, or because of a cloud infront of the moon. I'm going to try some more shots later at night.

Kit lens at 55mm

Film camera lens at 80mm
This is the appropriate 1.5x magnification you expect from the differences between the DSLR sensor size and 35mm film.

UPDATE: I finally made a better exposure and got my manual focusing hands back:
1/250s, f6.3, 80mm (film equivalent 120mm), hand held

Tuesday, August 25, 2009

D40 on flikr

It was interesting that the D40 even beat out compact cameras from Nikon...
The link to the live page is here.

How fast a card should I get for my dSLR?

I couldn't find a scientific answer to this one. Funnily enough I couldn't even find online how fast the D40 writes out its data. Perhaps its in the manual and I should just look. And what's with all these different speed metrics for SD cards !?

Anyway, here's how I went about it. From yesterday's shoot I have 101 pictures totaling 204.6 MB (Large Fine Jpegs. I ain't no RAWer) = 2Mb/picture

The D40 can shoot bursts of 2.5 frames a second = 5Mb/s. If we are to keep up with that we should have a card that is capable of at least 5Mb/s write speed.

Thanks to wikipedia's page on SD cards, we learn that class 6 SDHC cards write at 6 MB/s and above and the 'x' notation e.g. 40x, 50x means 40x150kB/s.

So we will need a class 6 card doing 5MB/s or more or a 33x card.

Lets see how that works out for an amateur shooting like 10 pictures or so at a go.

The cost of things...

In 2005 I bought a 512MB SD card for $37. Now, in 2009, from the same company you can get a 4GB SD card for $9.

Sunday, August 23, 2009

D40 : Kit lens (18-55mm Nikkor Af-S G)

You need to watchout for depth of field at 55mm. I have to reprogram myself after 4 years of using a compact. I keep thinking f5.6 is 'Ok for things at inifinity, like that building over there'

But it ain't. You have to do tricks like 'hyper focus', i.e. focus at infinity (like a building behind this one) and then recompose, or stop down the lens more.

Lets actually work this out:
Circle of confusion (Zeiss formula)
c = d/1730
where d = sqrt(23.7^2 + 15.6^2) = 28.37mm
c = 0.0164mm

Hyper focal distance (Wikipedia)
H = f^2/Nc
For the shot I had
f = 55mm
N = 5.6

H = 32937.71 mm = 32 m

The building was probably 10-15m away, and I focused on that, reducing my DOF. I should have kept my aperture at around

N = f^2/Hc = 55^2/(15000*.0164) = f12 for a good shot.

Alternatively, I should have focused on the farthest part of the building, locked focus and recomposed.

Stop down the lens Kaushik, stop it down. This 'ere's a REAL camera :)

D40 : Spot metering and exposure lock

Metered on the sun reflection on the water and recomposed

Metered on the cobblestones and recomposed

Saturday, August 22, 2009

The D40 : Summary

The D40 is a great camera. It has spot metering, exposure lock and focus lock. These are important things for me because I like to play with pictures taken against light, which are challenging even for film. It's evaluative metering is superb, which is for everything else. I have no problems with the controls, though some people say they are cumbersome. At most, you have to hold down a modifier button while spinning the thumb wheel. The exposure lock button is the only inconvenient thing - you have to jam your thumb next to your eye as you look through the viewfinder. I haven't made prints yet. Theoretically 6 MP will yield 10"x 6.6" prints at 300dpi and 20"x 13" at 150dpi. We'll see how that turns out. The D40 is as light as my film F65 and lighter than the fancier older film Nikons. It feels solid in the hand, more solid than the F65. The kit lens (18-55 Nikkor G) is great, its quiet and has great depth of field. Be aware that you need to stop down quite a bit (f9, f11) if you want to keep a bunch of things in focus.

My Settings
  1. Setup->File No. Sequence->ON (other wise you will reset image numbering to zero each time you erase pictures from your card)
  2. Setup->Auto Image Rotation->ON
  3. Setup->USB->PTP (For Mac, to use image capture)
  4. Shooting->Optimize Image->VI (Vivid)
  5. Shooting->Image Quality->Fine jpeg
  6. Shooting->Image Size->L
Preferred shooting mode
When I have time to compose (e.g. a few seconds) I put it on spot metering, aperture priority and then play with aperture for depth of field, and spot meter at different image locations, lock exposure and recompose to get different exposure effects. I set focusing to single area (Custom Setting->AF-area mode->Single area) and enable focus lock (Nikon's famous half shutter press - Custom Setting-> Focus mode->AF-S Single-servo AF)
When I have to be quick, I put metering on evaluative, and focus on closest subject.

Specific Bravos
  1. Auto mode with no flash. The camera adjusts shutter speed to be handheld safe, opens up the aperture and then boosts the ISO as necessary to take the shot. Beautiful for cases when you don't want flash and are too lazy/have no time to adjust everything for a handheld shot.
  2. Low (and I mean LOW) sensor noise up to 1600ISO
  3. On camera image enhancement can be custom tweaked
  4. Spot metering
  5. Matrix metering is great (just like for the film Nikon)

Specific Annoyances

  1. Exposure mode needs to be changed through the menu
  2. Exposure mode not indicated in the heads up display

What I would have liked
  1. Exposure bracketing
  2. Time lapse timer
  3. Grid lines on viewfinder
  4. Mirror lockup
  5. Exposure lock button somewhere else
  6. An analog focusing meter OR a split prism focusing screen
  7. A million dollars
Did I mention, the D40 is great. I'm sure you can get cameras with the above 3 features and more. For a lot more money. I'm not a professional, the challenge of leveling the horizon, the challenge of playing with exposure, the challenge of playing with depth of field IS the fun, and the D40 is all I need.


D40 : Depth of field

Having shot almost exclusively with a compact digital (Canon A510) for the last four years I forgot how much fun and challenging optics is.

The small sensor and the short focal length of the compact, much like older, rangefinder point and shoots (e.g. yashika mg-1) gave almost pinhole camera like behavior. Basically with these cameras infinity was about an arms length or so away and I would be more causal with the focus.

With the D40 its back to the F65 habits. Depth of field is back down (which is what I wanted). But I'm not used to it any more.

For this shot (click for full size crop) I thought "I'll just focus on them clouds there, the trees are at infinity, just like the clouds, it'll be all right". Not so. The trees look annoying because they are slightly out of focus, but you can see the sharp edge of the cloud. f6.3, 18mm. Shoulda stopped down the lens more.

In case there was any doubt, the kit lens can focus:

So I have no one to blame but myself...

Depth of field preview

The D40 has no DOF preview button. But, does a digital camera really need one? If you have time to stop down the aperture you have to to shoot an image, look at it in the LCD and decide - do I open up or close down the aperture more?

Thursday, August 20, 2009

D40 : Exposure : Sunsets


Color and saturation adjusted a bit

A sunset? No.

Actually, this is not a sunset. Its a city rise. The red glow is from Revere, or one of those other cities to the east. This shot was taken facing east.

In short: The D40 can do sunset exposures just fine (These were all shot with aperture priority). Some shots were with -1 EV compensation.

The autofocus will work as long as there is even a tiny bit of light and the subject has contrast. I used the tree to focus, and this was nice and contrasty against the sky.

D40 mini-accessories

I got the Tiffen 52mm UV (Haze) filter and the Matin heavy duty LCD protector from the same company I bought the camera from. The LCD protector is one step up from the films you could get for your PDA. This is a thin sheet of polycarbonate that sticks to your LCD. I misaligned it at first and had to pull it off, and it came off without leaving any residue. It sticks quite well but is very reflective. In bright light I have a feeling it might reduce the contrast of the LCD. I plan to leave the protector on until it gets very scuffed, and then replace it leaving the original LCD scratch free (hopefully).

The Tiffen filter is cheap, its not multi-coated, so you can expect a little glare when you have a bright light source. An example is shown below:

The reflection is due to the UV filter.

The only other thing I want to get is a 4GB card. My 1GB card will store about 270 Fine JPEGs, probably half the amount of RAW.

Focusing screens for the D40

Yes, you can get them for the D40.

There's a very pricey one from Katz Eye. It runs into $100 to $155. It has some good reviews.
There is a pretty cheap one you can get on ebay right now ($34 with shipping), and it might be the same you get from (This one costs $40).

A video tutorial for replacing the focusing screen:

But I would use rubber tipped tweezers, rather than straight metal ones. And don't film your self doing it... :)

The google search url.

An amateur's review of the D40: Long exposure time examples

I am pretty pleased with high exposure times on the D40. The last time I really played with long exposure was with my film camera.

What I am NOT pleased with is MY inability to focus properly - aargh! But in my defense there isn't a focusing screen (like split prism, or ground glass). I wonder if I can get one of those for the D40?

Click on the examples to get a 1200px downsampled versions.

If you ignore my amateur focus you will note the amazing lack of noise even on 30s long exposures. I like this sensor.

In the last two pictures below I let the D40 decide the exposure and used aperture priority. The D40, like the F65, knows exposure. (For normal photos, on the F65, I ended up deciding to let the camera decide it. Whenever I took over I did something wrong. Only for special shots - like the really long exposures - did I put everything on manual. I think the same is going to be true of the D40.)

Its amusing though, these last two shots have the same exposure, but they are lit differently. It could be that the street lamps vary in brightness over time.

In short: great sensor!

Tuesday, August 18, 2009

An amateur's review of the D40: High ISO example

Picture (click to get a downsampled 1280px image). This was taken at sunset with 800 ISO setting with kit lens at 18mm zoom, no VR, handheld.

The overall impression is good, but is marred by the combination of hand shake and higher ISO that gives the photo a softness that hurts the edginess of the main subject. The auto focus was perhaps also not the best.

The following full size segments of the original image illustrate this point:

Thursday, August 13, 2009

Nikon D40 at different ISO levels

ISO 200

ISO 400

ISO 800

ISO 1600

ISO 3200

First impressions of Nikon D40

Obtained this as refurbished from Adorama for $375 with 18-55mm DX AF-S lens. Adorama shipped it for free and it arrived overnight (NY to MA). Its declared as refurbished but the body is blemish free. I would say this is a new camera (unless some one soaked the interior in dust/water/acid etc). There is a 90 warranty from Nikon on both lens and body.

From the exif information on the pictures I see that the camera has about 2600 shutter actuations, and the serial number has two dots punched in at the start and end. According to this page those dots indicate Nikon has refurbished the camera.

I found a few dust specks on the viewfinder. (Aiming at the sky and looking around the field of view). I remember on the old Nikons, that my uncle had, you could pop-off the pentaprism and clean it all out, but I don't think you can do that here.

The camera with this lens weighs about as much as my Nikon F65 (or N65), which is to say, less than a kilo.

The battery needs to be charged, but it had enough juice to go through the initial set up. It prevented me from taking any pictures though. Battery is charging, owner is pretending to do data analysis.

Things to do when battery charges (other than go grocery shopping)
  1. Look for dead/hot pixels on sensor
  2. Take shots at different ISO levels to see noise
  3. Evaluate exposure modes
  4. Look for dead/hot pixels on display LCD
  5. Evaluate auto focus accuracy
Things to buy:
  1. Plastic protector for the LCD screen
  2. UV filter (as protection) for the DX lens
PS. Sorry Ben, but I decided I wanted to see if the price premium on DSLRs was worth it over point and shoots. If I find 6 MP too restricting I might upgrade in a few years...

Wednesday, August 12, 2009

Moveable tabs on Firefox

I know firefox is so unhip now, with that chrome or plating or whatever they call it browser. But really, being able to drag a tab from one window to another? Nice.

Differences between Nikon pro and consumer DSLRs

It wasn't always clear from reading up on websites and forums what exactly are the differences between consumer DSLRs and pro DSLRs. I'm making a short generic list here of what Nikon pro cameras may have that consumer one's don't (listed according to how important they were to me)
  1. Aperture stop down to preview depth of field
  2. Fast mechanical shutter (rather than a combination of slow mechanical shutter and sensor gating which can lead to streaking in extreme situations )
  3. Exposure bracketing (shoot three or more shots in quick succession with slightly different exposure levels)
  4. 1/3 EV steps for ISO rating (i.e. pro cameras can do ISOs inbetween 100 and 200)
  5. Motor for AF lens in body (If you have many old lenses this counts as a cost savings)
  6. Metal body that can withstand physical shocks better
  7. Dust removal on the sensor
  8. More auto focus points
  9. Top panel LCD for settings summary
Basically, a "pro camera" is designed to withstand being dropped and bumped (you're in the streets covering a protest - it could get rough), has quick auto focus with many points (you're trying to shoot a guy shooting a hoop - you got zero time to focus and click), has many many MP (you grabbed the best shot you could, and the president's face is on the top right corner of the photo - you gotta crop it to 1/8th the size to make a good composition, will it still look good?), has better sensor protection (you're in the savannah, you just changed from a macro to a tele to snap a lion pouncing on a gazelle - you don't want to be bothered with sand on your sensor).

Me? All I really want is a little more than 3MP (my current Canon A510) to make nicer larger prints, greater depth of field freedom, and a little less of that annoying softness and blue fringing that comes with cheaper optics.

Also, as a random tidbit: if you want to compare a DSLR lens (made for the DX format sensor) to a film camera lens for the Nikon's the multiplying factor is 1.5. So the 28-80 kit lens that came with my F65 will work as a 42-120 lens on a Nikon DSLR. As a corollary, the 18-55 kit lens with the D40 does the same work as the 28-80 for the film camera.

Sunday, August 2, 2009

CSS : dot (.) vs hash (#)

. -> class selector
# -> id selector

.bregma { width:10cm; } -> <div class="bregma">
#lambda {widh: 5cm;} -> <div id="lambda">

PS. The # symbol is called an octothorpe (Thanks to Vicki for catching the spelling mistake)

Friday, July 31, 2009

New version of matplotlib out : 3D plotting!

The hard working folks at matplotlib have put out a new release candidate with interesting new features.

Remember - its a release candidate. Not (yet) for production.

Get it from
Install using sudo easy_install ~/Downloads/matplotlib-0.99.0.rc1_r0-py2.5-macosx-10.5-i386.egg
On mac you will hear complaints about
Searching for matplotlib==0.99.0.rc1-r0
No local packages or download links found for matplotlib==0.99.0.rc1-r0
error: Could not find suitable distribution for Requirement.parse('matplotlib==0.99.0.rc1-r0')
but it will be installed fine.

Most interesting is the incorporation of 3D plotting.

from mpl_toolkits.mplot3d import axes3d
import pylab

x = y = pylab.arange(-1,1.0,.05)
X, Y = pylab.meshgrid(x,y)
Z = pylab.cos(pylab.pi*X) * pylab.sin(pylab.pi * Y)
fig = pylab.figure(); ax = axes3d.Axes3D(fig); cset = ax.contourf(X, Y, Z, 20);

Tuesday, July 21, 2009

Class and Instance Methods in Ruby

From here
class A
def self.class_method
puts 'class method'

def instance_method
puts 'instance method'

Thursday, July 16, 2009


Lets you select a folder in finder and opens a terminal cd'd to that folder.

Interestingly, this is a free but not opensource app. This seems to be the culture around Macs - apps written specifically for Mac are less likely to be opensourced.

Thanks to Jon Hendry for the link.

Wednesday, July 15, 2009

Installing mongrel on mac OS X with ruby 1.9.1

What. A. Pain.

UPDATE: I'm now using thin instead of mongrel.

When you go to do sudo gem install mongrel it will barf with some errors related to a file called http1.c Stop. Do not adjust the channel. There is nothing wrong with your computer. Somebody goofed in the source code. From instructions here, you have to make the following changes to the source. Remember to edit as root, don't run gem install mongrel again - you will overwrite your corrections.

This stuff was worked out by Alan and clarified by Ami Mahloof
1) cd /usr/local/lib/ruby/gems/1.9.1/gems/mongrel-1.1.5/ext/http11

2) sudo vi / mate on http11.c and:

replace line 77 with

for(ch = RSTRING_PTR(f), end = ch + RSTRING_LEN(f); ch < end; ch++) {

replace line 172 with
colon = strchr(RSTRING_PTR(temp), ':');

replace line 174 with

rb_hash_aset(req, global_server_name, rb_str_substr(temp, 0, colon - RSTRING_PTR(temp)));

replce line 176,177

rb_str_substr(temp, colon - RSTRING_PTR(temp)+1,

replace lines 298 299

dptr = RSTRING_PTR(data);
dlen = RSTRING_LEN(data);

3) save the file, close the vi / textmate

4) sudo ruby extconf.rb && sudo make && sudo make install

5) cd ../../lib/mongrel
(or the full path "/usr/local/lib/ruby/gems/1.9.1/gems/mongrel-1.1.5/lib/mongrel"

6) mate / sudo vi handlers.rb

7) fix the case statements in

On lines 208-212, change instances of ": false" to "then false"

8) sudo gem install gem_plugin (else mongrel may hang)
Full error message:
local:Rriki2 kghose$ sudo gem install mongrel
Building native extensions.  This could take a while...
ERROR:  Error installing mongrel:
ERROR: Failed to build gem native extension.

/usr/local/bin/ruby extconf.rb install mongrel
checking for main() in -lc... yes
creating Makefile

gcc -I. -I/usr/local/include/ruby-1.9.1/i386-darwin9.6.0 -I/usr/local/include/ruby-1.9.1/ruby/backward -I/usr/local/include/ruby-1.9.1 -I. -D_XOPEN_SOURCE -D_DARWIN_C_SOURCE   -fno-common -D_XOPEN_SOURCE=1 -O2 -g -Wall -Wno-parentheses  -fno-common -pipe -fno-common  -o http11.o -c http11.c
http11.c: In function ‘http_field’:
http11.c:77: error: ‘struct RString’ has no member named ‘ptr’
http11.c:77: error: ‘struct RString’ has no member named ‘len’
http11.c:77: warning: left-hand operand of comma expression has no effect
http11.c:77: warning: statement with no effect
http11.c: In function ‘header_done’:
http11.c:172: error: ‘struct RString’ has no member named ‘ptr’
http11.c:174: error: ‘struct RString’ has no member named ‘ptr’
http11.c:176: error: ‘struct RString’ has no member named ‘ptr’
http11.c:177: error: ‘struct RString’ has no member named ‘len’
http11.c: In function ‘HttpParser_execute’:
http11.c:298: error: ‘struct RString’ has no member named ‘ptr’
http11.c:299: error: ‘struct RString’ has no member named ‘len’
make: *** [http11.o] Error 1

Gem files will remain installed in /usr/local/lib/ruby/gems/1.9.1/gems/mongrel-1.1.5 for inspection.
Results logged to /usr/local/lib/ruby/gems/1.9.1/gems/mongrel-1.1.5/ext/http11/gem_make.out

Saturday, July 11, 2009

Avoid buffered printing into nohup.out

Invoke with

python -h

this prevents print statements from being buffered (e.g. when being dumped into a nohup.out file)

Wednesday, July 8, 2009

Include program listing in latex document

Use the listings package, in combination with the colors package

Use \usepackage[usenames]{color} to preload 68 color definitions (otherwise you only get about 5 boring colors).

Documentation for colors is here.

Sunday, July 5, 2009

Internet speed over HMS secure connection

Hmm... I guess they throttle downloads, but I have 5/20 service, how can I upload faster?

This is the speed test without secure conection

Wednesday, July 1, 2009

Matplotlib (numpy) : max index of multidimensional array

If x is multidimensional pylab array x.argmax() returns the flat index (converts the multidimensional array into a flat vector before finding the max/min). The pylab function unravel_index comes to our help

pylab.unravel_index(x.argmax(), x.shape)

From a hint here.

Monday, June 29, 2009

Parallel python: destroying job_servers

Remember to destroy job servers after they are done.

My nice animation generator crashed sometime over the weekend with the following death rattle:

Traceback (most recent call last):
File "", line 168, in
tscale = tscale, dpi = dpi)
File "", line 92, in process_file
job_server = pp.Server(ppservers=ppservers)
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/", line 336, in __init__
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/", line 494, in set_ncpus
range(ncpus - len(self.__workers))])
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/", line 141, in __init__
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/", line 148, in start
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/", line 594, in __init__
errread, errwrite)
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/", line 1009, in _execute_child
errpipe_read, errpipe_write = os.pipe()
OSError: [Errno 24] Too many open files

Saturday, June 27, 2009

Parallel python : clusters

  1. If the jobs are too short they may not get submitted to the remote machine and may all execute on local. pp submits jobs to remote machines only after running out of workers on local
  2. File handles are opened on the remote machines.
  3. You can't pass a file handle as a function argument
    TypeError: can't pickle file objects

Parallel python

Among the many libraries for parallel processing in python I chose what seemed to have one of the simpler interfaces - Parallel Python.

There is one trick that is not apparent from the docs: The function you call has to be 'self sufficient'. You basically have to write up the function you call as if it were a script. So the function has to have the imports etc within itself.

This is not a biggie, but it gets confusing until you figure it out.

Original code:

# module x -----------------
import a
import b

def f1(g):

def f2():

def f3():

# Calling script
import x


Parallelized code
# module px ------------------------------
def pf(g)

# New calling script
import pp
import px

ppservers = ()
job_server = pp.Server(ppservers=ppservers)

jobs = []
jobs.append(job_server.submit(, (2,))
jobs.append(job_server.submit(, (4,))
jobs.append(job_server.submit(, (6,))
jobs.append(job_server.submit(, (8,))

Thursday, June 25, 2009

Matplotlib interactive plotting pads figure size

With interactive backend (eg Tk)

f = pylab.figure(figsize = (7,10))
f.get_size_inches() -> array([ 7.175, 10.175])

The saved pdf is also of this size.

It becomes extreme with smaller figures:

f = pylab.figure(figsize = (1,1))
f.get_size_inches() -> array([ 3.275, 1.175])

This last I can see as a function of getting the tool bar to fit.

With Agg the saved figure is the size you ask for

Wednesday, June 24, 2009

Only on wikipedia:

Reading a paper about neural circuitry. Came across the word 'resonant circuitry'. Looked it up on wikipedia. Knew it had to do with band pass circuitry. Looked that up on wikipedia. Read in that entry that Thomas Pynchon claimed the character Kilroy was based on the circuit diagram for a band pass filter. You be the judge.

Any hoo, went to Pynchon's page on wikipedia and started to read. On the entry for his novel V, read about yoyodyne corp. Hey I know that from the GPL. Went to the (fictional) webpage for yoyodyne corp... Wait. Whaa!? What's all thsi? What's the time? Where am I? What am I doing here? Wasn't I just reading a paper on neural circuitry?

Tuesday, June 23, 2009

Sourceforge community choice awards

So I voted as follows:

KeePass Password Safe for Best Project (I would have liked matplotlib in the list)
eViacam for Best New Project
matplotlib for Best Project for Academia
Audacity for Best Project for Multimedia (mencoder, if it was on sf, would have won mine) for Best Project for Government
KeePass Password Safe for Most Likely to Change the Way You Do Everything
(I would have liked matplotlib in the list)

I would have liked to see Matplotlib in the categories '
Best Project' and 'Most Likely to Change the Way You Do Everything' too, but KeePass is a good choice.

I didn't vote in a category when I didn't know any of the choices. eViacam is a mistake. I tried to get rid of it but couldn't cancel that vote. I picked it at first even though I knew nothing about it because I thought it was a good idea for a project, but later decided it wasn't right to vote on a project I haven't used.

Sunday, June 21, 2009

Redirecting nohup output

nohup command 2>file1.txt 1>file2.tex &

nohup command 2>/dev/null 1>/dev/null &

pdb to set break points

import pdb; pdb.set_trace()
Where you want the code to stop. Once in debugging mode type 'r' to return from trace and be where you stopped the program. From a tip here.

Friday, June 19, 2009

Efficient pickling in python

Use the highest protocol you have available - which will usually be binary. So as example:
import cPickle
import pylab

z = pylab.random((10,10))
z ndarray 10x10: 100 elems, type `float64`, 800 bytes
len(cPickle.dumps(z)) -> 2371
len(cPickle.dumps(z, cPickle.HIGHEST_PROTOCOL)) -> 934

z = pylab.random((50,50))
z ndarray 50x50: 2500 elems, type `float64`, 20000 bytes
len(cPickle.dumps(z)) -> 55586
len(cPickle.dumps(z, cPickle.HIGHEST_PROTOCOL)) -> 20134
For some reason, I thought the default was to use the most efficient protocol, but it isn't.

Type: builtin_function_or_method
Base Class:
String Form:
Namespace: Interactive
dump(obj, file, protocol=0) -- Write an object in pickle format to the given file.

See the Pickler docstring for the meaning of optional argument proto.
Hence my slow processing and bloated data files...
Note that load figures out the protocol automatically...

Removing a non-empty directory using python - shutil

The os module has functions (os.rmdir, os.removedirs) that can remove empty directories but they will not mess with non-empty ones.

The shutil module has a function (rmtree) that will do this (From this thread in the python mailing list).

Tuesday, June 16, 2009

3D in matplotlib

One of the frustrating things about matplotlib is its lack of 3D plotting. If matplotlib were a mediocre library it wouldn' hurt. But it is a GREAT library and produces really good looking plots. So the lack of 3D plotting sometimes makes me want to go out and strangle somebody. There has however always been a furtive 3D plotting feature in matplotlib, and apparently, its being worked on some more. Yeehah! No more futzing with pyVtk.

Important links:
  1. Docs for 3D features
  2. Installing the svn code (that has all this goodness)

Searching in sourceforge mailing lists

Drop down the 'mailing lists' tab. There is a search option there.

Drawing circles using matplotlib

Use the pylab.Circle command

import pylab #Imports matplotlib and a host of other useful modules
cir1 = pylab.Circle((0,0), radius=0.75,  fc='y') #Creates a patch that looks like a circle (fc= face color)
cir2 = pylab.Circle((.5,.5), radius=0.25, alpha =.2, fc='b') #Repeat (alpha=.2 means make it very translucent)
ax = pylab.axes(aspect=1) #Creates empty axes (aspect=1 means scale things so that circles look like circles)
ax.add_patch(cir1) #Grab the current axes, add the patch to it
ax.add_patch(cir2) #Repeat

Modifying a firefox plugin

I dislike the indistinct icons in the cookie manager plugin CSlite.

In Firefox/extensions/{00084897-021a-4361-8423-083407a033e0}/ (the directory for this particular extension) we find chrome/cookiesafe.jar

From a hint in a thread here, we learn that the .jar file is just a .zip file, and we can extract that to find skin/classic/cookiesafe and a whole bunch of .gifs there.

We can then change those gifs and change the icons that are used in the application.

Friday, June 12, 2009

CS Lite

I was using the Permit Cookies extension for firefox, but it hasn't been updated for a few cycles of firefox updates. It works fine, but I started to look for something else. I found CS lite which has a more convenient interface (a popup menu from the status bar). The only gripe I have so far is that the icons in the status bar are not very clean.

Searching for a cheap graphics card that will play Crysis

Downloaded and started playing the Crysis demo. On my integrated graphics MoBo, naturally I had to set everything down to low, but even then it can get choppy when in the jungle. And the graphics look so crappy I think - hmm, maybe I'll just play Far Cry. Tom's hardware has a great page where you can compare the performance of various cards with various games at various settings when searching for a cheap graphics card for crysis.

Thursday, June 11, 2009

Mommy, why does my mac update reset my trackpad tap settings?

And make it all mysterious and make me think like my computer has hung, until I figure out that its just the track pad.... grrrrr....

And it reset my keyboard repeat rate too...

Wednesday, June 10, 2009

Changing axes tick label size

In matplotlib, the way to change axes tick label sizes is to do:
for label in ax.get_xticklabels() + ax.get_yticklabels():
From matplotlib-users

Wednesday, May 27, 2009

Tick label formatting in matplotlib

Use the Format** functions as illustrated in the major-minor demo. e.g.

major_formatter = pylab.FormatStrFormatter('%2.1f')

Wednesday, May 20, 2009

Installing Ruby on Mac OS X

tar -xvf ruby-1.9.1-p129.tar.gz
cd ruby-1.9.1-p129
./configure --enable-shared --enable-pthread CFLAGS=-D_XOPEN_SOURCE=1
make -j4
sudo make install

Installs in /usr/local/bin, make sure to point path there

Tuesday, May 19, 2009

HP battery recall

Finally, some useful information on Slashdot. I knew all those hours invested in reading inane comments riveting articles would pay off. HP is recalling batteries. The list is here

Those geneticists and their Excel

Mistaken Identifiers: Gene name errors can be introduced inadvertently when using Excel in bioinformatics

If you are too lazy to get to the punchline:

"MatchMiner [1] and GoMiner [2] are two bioinformatics program packages we
published recently in another Biomed Central Journal, Genome Biology. When we
were beta-testing those programs on microarray data, a frustrating problem
occurred repeatedly: Some gene names kept bouncing back as "unknown." A little
detective work revealed the reason: Use of one of the research community's most
valuable and extensively applied tools for manipulation of genomic data can
introduce erroneous names. A default date conversion feature in Excel (Microsoft
Corp., Redmond, WA) was altering gene names that it considered to look like
dates. For example, the tumor suppressor DEC1 [Deleted in Esophageal Cancer 1]
[3] was being converted to '1-DEC.' Figure 1 lists 30 gene names that suffer an
analogous fate."

Wednesday, May 6, 2009


A search on google for
pylab append to array
brings up a page full of Matlab hits. Why, I wonder?

Tuesday, May 5, 2009

Function definitions in scripts

My MATLAB origins betrayed me. I kept thinking - you can't put function definitions in a script in Python. Man, I gotta write a module and THEN a script that calls the module? But, in Python, you can include function defs in a 'script'. So the following works perfectly well:

def func1():
print '1'

def func2():
print '2'


Take that MATLAB.

Monday, May 4, 2009

Excel tips

  • window->freeze panes : keep cells to the left and top (inclusive) of selected cell frozen as you scroll

Friday, May 1, 2009


Todays xkcd made me laugh. A small pause in the downhill journey of that site.

Tuesday, April 21, 2009

GPL logo

is not very good. Its kind of crowded and ungainly.

Don't you think they should have come up with something more visually appealing? To me, even plain type looks better! Like this


The growing up of the GPL

I first ran into the GPL in some book. It could have been a coding book or a latex book. I didn't care about licenses at that age, but I remembered it because it contained the following extract:
You should also get your employer (if you work as a programmer) or your school, if any, to sign a "copyright disclaimer" for the program, if necessary. Here is a sample; alter the names:
Yoyodyne, Inc., hereby disclaims all copyright
interest in the program `Gnomovision'
(which makes passes at compilers) written
by James Hacker.

signature of Ty Coon, 1 April 1989
Ty Coon, President of Vice
Now, these guys must be fun to hang out with! That's what I thought. Recently, I looked up GPL3. Stallman's blurb on why we should switch was all peachy and militant. But what really disappointed me was the end of the license:
If the program does terminal interaction, make it output a short notice like this when it starts in an interactive mode:
    Copyright (C)   
This program comes with ABSOLUTELY NO WARRANTY; for details type `show w'.
This is free software, and you are welcome to redistribute it
under certain conditions; type `show c' for details.
So boring! These guys are grownup now. They're not any fun anymore...

Monday, April 20, 2009


Ahh Ruby... Suppose you have a string class that corresponds to a class name and you want to call a certain method on it, what do you do? Well you could do

eval class + '.method'

but that will be slow (as evals are slow) and apparently that looks ugly.

But the creator of Ruby must have run into that issue often. Because what you can do, is


This is an inflector that tries to find a declared constant with the name in the string. And in Ruby classes are constants...

Ruby regular expression evaluater


Saturday, April 11, 2009

Thursday, April 9, 2009

Running a module as a script

Contents of

def haha():
print 'haha'

def mama():
print 'mama'

if __name__ == "__main__":

Now invoke as a script




Saturday, April 4, 2009

Federal free file works

I think the IRS did a good job on the free file interface. The interface works on Firefox3.0 on a mac. I'm assuming it works on windows. There are a few minor glitches with cursor positioning on some text fields (when you need to go back and delete/edit stuff in a field) but its an allround good job. If all government stuff was like this it wouldn't hurt so much to pay the taxes.

Don't use 'type' as a field in a Rails database

I'm working on a new rails application. In one of the tables I have a field called 'type'. Everything was going fine, but whenever I set the 'type' field and saved the record it ignored the contents and set it to NULL. I couldn't figure it out.

Tangentially, I had set a bunch of fields to be 'string' and these are translated to be varchar(255) in the sqlite database and I wanted to test what happened if I went over the 255 character limit. So I typed in a string 260 characters long into the 'type' field using an external database app and saved the record. It saved just fine, and kept all the characters. Now I thought, let me load this in my rails app and see if it still sets it to NULL. I loaded it and got back this error message:

ActiveRecord::SubclassNotFound in ItemsController#show

The single-table inheritance mechanism failed to locate the subclass: 'abcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyz0'. This error is raised because the column 'type' is reserved for storing the class in case of inheritance. Please rename this column if you didn't intend it to be used for storing the inheritance class or overwrite Item.inheritance_column to use another column for that information.

Now they tell us !

Convention : good.
Informative error messages : good.
Not telling us what the convention is : bad

Bad Rails people, bad rails people.

To peek into the database I'm using the Sqlite manager plugin for firefox by Mrinal Kant. Its awesome.

Thursday, April 2, 2009

Free e-file for federal

Excellent! I always wondered why the IRS didn't allow you to e-file for free (unless you were below a certain income limit). I mean its free to paper file. It must cost the IRS a lot to go through badly handwritten paper forms. But this year I see that you can e-file for free whatever your income. Good.

Wednesday, April 1, 2009

Can't call one python script from another

Bookmarks in Eclipse!

I was jumping round my analysis code and getting tired of hunting to find the apropriate line I was working on. I thought wouldn't it be nice if there was a bookmark system. So I right clicked on the gutter in eclipse. And. Whaddya know? THERE'S A BOOKMARK SYSTEM. I love this IDE.

Friday, March 27, 2009

Psycho on Mac

svn co psyco-dist
sudo python install

Saturday, March 21, 2009

Maximum number of open files in Python

So here I was merrily writing a python module to extract neural data from the Cerebus system's giant .nevs file and split it into smaller files - one per neural unit. I had been extracting one channel at a time and all was well. So now I copy the code over to the lab machine and tell it to extract all the data and split it into 96*4 files.

Traceback (most recent call last):
File "", line 11, in
File "/Users/kghose/Experiments/Python/", line 270, in fragment
IOError: [Errno 24] Too many open files: '../Neural/DJ2009/20090320//Frag//electrode64unit02.bin'

Whaaa? Whats all this? Well, it turns out you can't have too many files open at the same time. And the resource module can tell you exactly how many files you can have open at a given time:

(256L, 9223372036854775807L)

Which means the current soft limit is 256 files and the hard limit is some number so large only astronomers and people who write stimulus bills can deal with it. And so for my application, I can change the limit by doing:

resource.setrlimit(resource.RLIMIT_NOFILE, (500,-1))

(-1 means set the hard limit to the maximum possible)

Why I love matplotlib + python

Note this graph. The upper trace just goes from 0 to 1.6ms. The lower trace was a series of such waveforms ocurring from 0 to 10 s. I zoomed in to see one wave. When you do that in Matlab the tick labels would take the form 8.168, 8.169, 8.170 which is legitimate, but gets hard to read when you zoom in a lot and you get lost in the digits. Matplotlib, on the other hand, produces the zoom plot pictured here. It shows you the offset on the right (8.168) and then just shows you the vernier values. Now THAT is what I call service. The matplotlib guys are really good. Its like they actually use their product for data analysis. They must be scientists or something :).

Percent sign in python format string

From here: just use a % sign as an escape character.


print '22%' works

print '%d' %(22) works

print '%d%' %(22) throws an error and the solution is

print '%d%%' %(22)

People say this is hacky. They 'fixed' it in Python 3.0 but I'm not upgrading soon. Imagine having to go through all your code because of this nonessential thing!

Sunday, March 15, 2009

Slow Matlab editor on Mac OS X

From macoshints:

If you've upgraded to Leopard and now find the IDE of the new Matlab release 2007b to be super-slow, create a file called java.opts in your /Applications/MATLAB_R2007b/ directory, and add the following line to it:
Then restart Matlab. This will switch Matlab back to using the Quartz drawing enginge (as it was in previous versions of OS X).

Saturday, March 14, 2009

Getting clips from DVD

Sometimes I find howlers on DVD that I want to store. On a Mac the sequence is:
  1. Open the DVD in VLC and find out which title and chapter the clip is in.
  2. Use MacTheRipper to extract that title and chapter to .vob format
  3. Open it up in VLC, note the seconds offset for the part of the clip we want and then use File->Streaming/Export wizard to export the clip. I use H.264 (3072 kb/s) for video and MPEG-4 audio and mp4 container.
Here's an example:

The what controls what again?


I thought the author of the Harry Potter books invented the word 'Muggle' but I ran into it in Steinbeck's 'Sweet Thursday'. Couldn't figure out what it meant though.