Skip to main content

Posts

Showing posts from 2009

Splinter Cell: Chaos Theory - disabling lights

Well, the right click (alt fire) on the suppressed pistol activates some magic device that can temporarily knock out a light - for about 10s or so. Why did I have to search on the internet and track down vague clues until I hit this page and found out for sure? Shouldn't there  be a manual somewhere? UPDATE: I gotta read the notes on the equipment screen more carefully...

UPDATE2: The manual is useful. Save trees/animal skins, don't print it out.

Max os x service start up : StartupItem

A good guide is available here

File:  /Library/StartupItems/RRiki/StartupParameters.plist:

{
 Description = "RRiki server";
 Provides    = ("RRiki");
 Uses        = ("Ruby");
 OrderPreference = "Last";
 Messages =
 {
  start = "Starting RRiki server";
  stop = "Stopping RRiki server";
  restart = "Restarting RRiki server";
 };
}

File: /Library/StartupItems/RRiki/RRiki

#!/bin/sh

##
# RRiki service startup script
##

. /etc/rc.common

StartService ()
{
    ConsoleMessage "Starting RRiki"
    cd /Users/kghose/Source/Rriki2/
    /usr/local/bin/thin start -eproduction -d
}

StopService ()
{
    ConsoleMessage "Stopping RRiki"
    cd /Users/kghose/Source/Rriki2/
    /usr/local/bin/thin stop 
}

RestartService ()
{
    ConsoleMessage "Restarting RRiki"
    cd /Users/kghose/Source/Rriki2/
    /usr/local/bin/thin restart -eproduction -d
}

RunService "$1"

To manipulate the service:
sudo Syste…

Calling an action when a selection box changes

in partial (view)

<%= select('history', 'sel', history_list, {:selected => selected}, {:class => 'historyselect'}) %>
<%= link_to_remote 'x',
      :url => {:controller => 'rriki',
               :action => 'clear_history'},
      :html => {:title => 'Clear history'},
      :method => :get %>
<%= observe_field 'history_sel',
    :url => {:controller => 'rriki',
             :action => 'select_item_in_history'},
    :with => 'history_sel' %>

changing the select will call the action select_item_in_history with params['history_sel'] set to the selected value.

Rails: Debugging form helpers generated with remote calls (AJAX)

When parts of a webpage are updated using AJAX calls (e.g remote functions) these updated parts do not show up when you ask Firefox for the webpage html (using CTRL+U). A hack to see the generated html from a suspect rails call is to simply print out the html on the webpage instead of executing it by doing:

<%=h ...rails command... %> Instead of

<%= ...rails command... %>

html_options for rails select tag

I had a select tag that I wanted to mess with a little to apply some styles etc. to. The select tag has an html_options hash and I was trying to use that. My initial attempt was

select('history', 'sel', history_list, html_options => {:class => 'historyselect'}) WRONG

This is not python, and even though the rails docs setout the syntax as:

select(object, method, choices, options = {}, html_options = {})

ruby can not accept keyword arguments. Only positional, so you have to do:

select('history', 'sel', history_list, {}, {:class => 'historyselect'})

The empty braces signify the empty 'options' argument, and the next set of braces signify the html_options hash

PAN = FOA

At NIH when you look at a grant they all have a Program Announcement number (PA). When you go to grants.gov they ask for the Funding Opportunity Announcement number (FOA). The first test of eligibility to apply for the grant is to realize that they both mean the same thing...

HP Pavillion Dv2700 intermittent fan

Problem: Laptop (HP Pavillion DV2700) has an intermittent fan. Fan goes on for a second and then stops, goes on again, stops. While this may be OK for the hardware (but who knows) it drives the user nuts.
Cause: Laptop bios upgraded to F.2E (latest)
My Fix: Downloaded and installed F.25 (older version) and this stopped the intermittent fan. I hope that this does not lead to overheating problems, I should monitor CPU temps.

50mm f/1.8 AF-D

I'm talking about this lens.
Its made in China, it costs about $130 now, bumped up from about $100 a year back. The focusing ring is dampened and easy to grip (Its a MF on the D40). The mount is metal, but not machined accurately - mine is a tight fit on the camera (and others have commented on this).
The image quality is excellent. This is a set of images of a deck of playing cards shot at the nearest range and at different apertures. The images are 100% crops from center (left) to edge (right). The DOF at f/1.8 is really shallow. The cards look washed out at f/1.8, but if you look at the red diamond and the edge of the middle card, they are sharp, indicating that the DOF is shallow enough (.1" or less) that the card goes out of focus by the time you go from center to edge. At f/2.5 the sharpness and contrast are remarkable:

The spelling is 'coArse'

So here I was, sitting at a class in MIT and the instructor puts up a slide and there is a legend that says 'course coding'. It is clear from the concept that the meaning is 'coarse coding', with 'coarse' being the opposite of fine. I keep my big mouth shut, because we are here for the science, and I don't want to side track people pointing out pipifax things like typos on a slide. A few slides later we come across coarse coding, and it is again spelled 'course'. I keep quiet. Again, and again repeatedly on different slides. Now I know its not a typo. The instructor thinks that's the right spelling. I keep quiet.

Some days later I come across the misspelling on some website. Then again on slashdot, and now I have to open my big mouth to relieve the pressure.

Its 'coarse'. 'COARSE'. Meaning...well, not fine. 'course' means a path, like a race course, or the course of a river.

Buyer beware

A month or so ago, my spouse got me something from Brookstone. It was a great idea, but I realized that the product did not work out for me. On the receipt there was a note saying that returns could be made up to 60 days. We went a month or so after purchase and were told that Brookstone's policy was to refund the money before 30 days were up and to give store credit after that.

I personally think this is misleading. That store policy should be prominently displayed on the receipt where it says 60 days to return. Otherwise its a shady move - probably false advertising. A little bit on the range of mail in rebates - the whole idea with the mail in rebate is that a certain fraction of people will forget to mail it in, so you get more people to buy (they think they are paying less for it) but your actual revenues are more (because the average price is more than you say it is with rebate).

Brookstone's policy is in the similar vein. A certain fraction of customers (like us) will see…

Spotting photographic equipment scammers

Go to nexttag or some other site and figure out the price range of the productGo to several different resellers and check the prices.Do a search with the reseller's name along with the phrase 'bait and switch'Find the product on amazon, adorama, ritzcamera and keh.com
If the
Price of the product is much below (say 30%) the well know retailers' pricesAccessories are correspondingly more costlyExtended warranties are correspondingly more costlywebsite sells the same product at a higher price and it looks like one is body only and the other is kit (but the website does not tell you)Then be wary. They could be
Trying to bait and switch you (sell you another product after saying out of stock)
Trying to sell you over priced accessories and warranties to make upSelling a used or refurb kit as newJust plain old scam you - take your money and not deliver the product.

Macro lens

A lens of a given focal length has a certain field of view. If you can focus the lens close enough, you can get large reproductions of objects. Such lenses are called macro lenses.

Maximum reproduction ratio

1:6.6 maximum reproduction ratio -> 10mm object will take up 1.515151515151 (sorry had to do that) mm on the sensor.

DX sensor on D40 23.7mm -> 3008 pixels so 1.51 mm -> 192 pixels -> about 1in for a 200dpi print.

So with the 50mm 1.8 AF-D lens if you shoot at the closest distance (about .5m) a 1cm object can be printed at 1in decently. Not that you would want to use this lens that way...

50mm f/0.7

Its billed as the world's fastest lens. A limited number were built for NASA (glamor portraits in outer space - astronauts photographed only by the light of their own instruments) and Stanley Kubrik snagged a few to make some obscure movie shot mostly by candle light (yep, he spent all his money on those fancy high speed lenses and couldn't afford real lighting).

Here is a picture of one. And here is a page by the guy who modified the lenses for Kubrik.

PS. You've probably figured out I was kidding about some of the stuff about astronaut portraits and Kubrik running out of money...

How the D40 handles a multi dimensional problem

There are three dimensions to exposure:
ApertureShutter speedSensitivityIn film days adjusting sensitivity on the fly was not practical, so you basically had a two dimensional plane once you loaded film.

The life of the camera computer was easy when you set the camera to A or S modes: all it had to do was adjust the free variable to maintain correct exposure. In M mode the camera merely informed you of exposure letting you roam freely in this 2D space.

The D40 can be setup exactly this way if you select fixed ISO. If you select auto ISO, however, the camera now is handed two free parameters (for A and S) and one free parameter (for M). How does the D40 deal with this?

For A mode the computer minimizes ISO and maximizes shutter speed. You can set a lower limit for Shutter speed and upper limit for ISO and the camera will drop shutter speed until it hits the lower limit and the start to bump ISO.

For S mode the computer minimizes ISO. It will maximize the aperture until it reaches the lens&#…

IR Remote for Nikon

I didn't buy this for my Film SLR because I cost too much. Currently Nikon's brand (ML-L3) costs $15 on Amazon/Adorama etc. and on Nikon Mall costs $20. However, if you look here (bestofferbuy) you can get an 'alternate version', called the YN ML-L3, for $4.80 and free shipping. I was a little wary at first and wondered if this was a scam, but I decided to try it out and ordered two (aha, see how this works, they price it less and that gets the sucker customer to buy more). I guessed it would take a long time to get here, but it took about a week and a half. I tried out the first one and it works fine. The unit comes with battery installed but with a plastic tab to keep it from discharging. These shady Chinese operators are moving upscale. (Actually it ships from Hong Kong. Don't know if that's any different anymore).

I have plans for the other one. An intervalometer costs an insane amount of money. Say like $140. What I want to do is order a 555 and some pots f…

What will 6MP get you?

So, how big a print can I make with a 6MP (3000x2000pixel) image, if I assume that the image is meant to be seen all at once?

From wikipedia (eye acuity) we find that a human with 20/20 vision can resolve 2 arcminute per line pair. This means two pixels separated by 2/60 = .033 degrees they can be seen separately

Then a 1000 pixels can subtend 1000/2 * .033 = 16.66 degrees, so a 6MP image can subtend 50x33 degrees.

At a distance of .3m (arm's length) this works out to .358m x .195m (14.3"x7.8")

At a distance of 1m this works out to 1.192m x .65m (47.5"x25.87")

I don't think this is too shabby.

D40 quick controls

Set Fn button to control ISOSet AE/AF lock to AE onlyIn A/S (Aperture/Shutter priority) modesWheel alone controls Aperture(Shutter)Wheel + function controls ISOWheel + (+/-) controls exposure compensation (I keep forgetting this)In M (Full manual) modes
Wheel controls shutterWheel + (+/-) controls apertureWheel + function controls ISOExposure is indicated on the bar graphIn Auto mode you only get to control ISO

Digital vs FIlm and Point and Shoot vs SLR

I'm sure Google crashes when you do this search. But here's what I think:
The only way to get photographs you attach to is to take a lot of them as often as possible, look at the results and link it with what you did.
Digital makes this process cheaper and faster. Digital gives you instant feedback. From the instant LCD display, to the home viewing on the big monitor to the prints at the dime store.More pictures, more fun.I think the guys who say 'all you need is a pinhole' are right. Really. It ain't the camera. I have photos I treasure from my Dad's Exakta RTL (with a busted exposure meter), from point and shoot Yashikas and Kyoceras, and from my Canon A510. From my D40 so far I have perhaps two photos I treasure for artistic reasons (out of may be 500). The camera and the lens just don't matter. Just take a lot of photos. And have fun.UPDATE: One thing though - the D40 has really good low light capability. I took several indoor shots with room lighting and…

The moon and six pence

Photography increases my admiration for the human visual system. I look up into the night sky and I see the full moon, I see the stars around it and the houses below. Then I put my camera on a tripod and try to capture the same scene. WOW! The moon is BRIGHT! I mean you could shoot it hand-held (1/30th even at f13). But wait, at that exposure the stars are gone, not to mention the house. I go down to 1" at f13. Now I get the stars, not really the house and the moon is one big halo with no internal structure.

Shutter mechanisms

I got fascinated by the camera shutter mechanisms and started to do some reading:
A great set of images of a shutter opening and closing. Read the tech details of how it was done.A very nice writeup with attractive hand drawn sketches of the principle of the focal plane shutter and some funny effects that occur with fast moving objects or flash use.I wanted to know more about the shutter mechanism.
Nikon has two pages with tiny drawings [1],[2]
A search on google images with the word "shutter mechanism" and filter by line art gives interesting resultsAn example of how shutter speed knobs interact with the actual shutter is given here.

Unhappy with shutterfly photo scans

In 2006 I had taken a bunch of film photos and had them processed by shutterfly. Shutterfly also scanned these photos and put them in my account so I was happy. Finally this year I decided I wanted the scans on my computer and I ordered a DVD of them (you can't download the hi-res versions from the website). I was unpleasantly surprised when I got the scans back. They are 270 ppi or so, amounting to 1536x1024 from 35mm film. I would have expected something more detailed from them such as 2000 ppi or so. But it was a bad surprise as some of the photos were landscapes and I was looking forward to enlargements. I guess I'll have to dig out the negatives from the boxes.

D40 : Burst mode test

Kingston SDGC 4GB class 6 (SD6/4GB) card. Auto focus (Prefocused by half shutter press before burst). Auto mode. Large/Fine Jpeg. Off white wall with a few objects on it.
23 pictures in 11 seconds. Buffer sat at around r4/r5 during burst. After burst the card activity light flickered for about 4s.

The D40 and the F65

My own first SLR is the Nikon F65 which I got in 2001 for about $400 (more than my D40!). It was billed as an entry level SLR for beginners. That's me, a beginner for about a decade now. I've used a digital compact (Canon A510) since 2005 and details of the F65 slowly faded from my mind as it stayed stashed in the closet. But after I got the D40 I began to remember things - didn't it have a DOF preview, didn't it have this? That? So I got it out of the closet and lookee here, the F65 (Nikon entry level film) has:
Depth of field previewExposure bracketingFive (5) AF areasPentaprism
Top LCD
It doesn't have two control wheels, but has the control wheel + modifier buttons.

Hmm... methinks the Nikon engineers are losing to the Nikon marketing people (C'mon DOF preview, exposure bracketing, 5 AF areas these don't add to cost - they probably COST money to take away, the only reason to take these small things away is to make people buy the more expensive one isn't …

Hmm...

I was reading an essay by a photographer who shoots 4x5 format for landscapes. Its very entertaining to read about how he focuses on a ground glass screen and then uses a loupe to check focus. But here's a sentence that I found entertaining because its technically wrong, but I think I know what he means:

"Fourth, the inverted image on the ground glass creates a more direct visual impression because the brain does not have to flip the image upside down. This last remark is based on the fact that our eyes act as lenses and thus project an upside-down image of the world to the brain, which then has to flip it right side up. Because the ground glass image is inverted it is projected to the brain right side up thereby nullifying the need for the brain to rectify the image."

Its hard to explain why this is wrong but basically the brain is used to how it gets the images and it is harder for the brain to process an upside down image. Consider the empirical results that we find it …

D40 : Manual lens

I have a F65 which came with a 28-80mm lens. I used this lens on the D40. It required manual focus, the red focus dot did not work as a focus indicator and I could not select a focus area. (I had gotten the impression that the focus circuitry would still be working so that I could use the focus indicator to judge focus). The dot works. It just goes off when you are way off focus, flashes when you are close and is steady when you are focused. Now you know why my moon shot was out of focus and I was complaining that there was no red dot :P

The exposure meter still gave readings, and they looked sensible. Here are two shots of the moon. One with the kit lens at 55m and the other with the manual lens at 80mm with a UV filter on it. The 80mm shot is blurry, either because of my poor focus, or because of a cloud infront of the moon. I'm going to try some more shots later at night.

Kit lens at 55mm


Film camera lens at 80mm
This is the appropriate 1.5x magnification you expect from the differ…

D40 on flikr

It was interesting that the D40 even beat out compact cameras from Nikon...
The link to the live page is here.

How fast a card should I get for my dSLR?

I couldn't find a scientific answer to this one. Funnily enough I couldn't even find online how fast the D40 writes out its data. Perhaps its in the manual and I should just look. And what's with all these different speed metrics for SD cards !?

Anyway, here's how I went about it. From yesterday's shoot I have 101 pictures totaling 204.6 MB (Large Fine Jpegs. I ain't no RAWer) = 2Mb/picture

The D40 can shoot bursts of 2.5 frames a second = 5Mb/s. If we are to keep up with that we should have a card that is capable of at least 5Mb/s write speed.

Thanks to wikipedia's page on SD cards, we learn that class 6 SDHC cards write at 6 MB/s and above and the 'x' notation e.g. 40x, 50x means 40x150kB/s.

So we will need a class 6 card doing 5MB/s or more or a 33x card.

Lets see how that works out for an amateur shooting like 10 pictures or so at a go.

D40 : Kit lens (18-55mm Nikkor Af-S G)

You need to watchout for depth of field at 55mm. I have to reprogram myself after 4 years of using a compact. I keep thinking f5.6 is 'Ok for things at inifinity, like that building over there'







But it ain't. You have to do tricks like 'hyper focus', i.e. focus at infinity (like a building behind this one) and then recompose, or stop down the lens more.

Lets actually work this out:
Circle of confusion (Zeiss formula)
c = d/1730
where d = sqrt(23.7^2 + 15.6^2) = 28.37mm
c = 0.0164mm

Hyper focal distance (Wikipedia)
H = f^2/Nc
For the shot I had
f = 55mm
N = 5.6

H = 32937.71 mm = 32 m

The building was probably 10-15m away, and I focused on that, reducing my DOF. I should have kept my aperture at around

N = f^2/Hc = 55^2/(15000*.0164) = f12 for a good shot.

Alternatively, I should have focused on the farthest part of the building, locked focus and recomposed.

Stop down the lens Kaushik, stop it down. This 'ere's a REAL camera :)

D40 : Spot metering and exposure lock

Metered on the sun reflection on the water and recomposed


Metered on the cobblestones and recomposed

The D40 : Summary

The D40 is a great camera. It has spot metering, exposure lock and focus lock. These are important things for me because I like to play with pictures taken against light, which are challenging even for film. It's evaluative metering is superb, which is for everything else. I have no problems with the controls, though some people say they are cumbersome. At most, you have to hold down a modifier button while spinning the thumb wheel. The exposure lock button is the only inconvenient thing - you have to jam your thumb next to your eye as you look through the viewfinder. I haven't made prints yet. Theoretically 6 MP will yield 10"x 6.6" prints at 300dpi and 20"x 13" at 150dpi. We'll see how that turns out. The D40 is as light as my film F65 and lighter than the fancier older film Nikons. It feels solid in the hand, more solid than the F65. The kit lens (18-55 Nikkor G) is great, its quiet and has great depth of field. Be aware that you need to stop down qu…

D40 : Depth of field

Having shot almost exclusively with a compact digital (Canon A510) for the last four years I forgot how much fun and challenging optics is.

The small sensor and the short focal length of the compact, much like older, rangefinder point and shoots (e.g. yashika mg-1) gave almost pinhole camera like behavior. Basically with these cameras infinity was about an arms length or so away and I would be more causal with the focus.

With the D40 its back to the F65 habits. Depth of field is back down (which is what I wanted). But I'm not used to it any more.

For this shot (click for full size crop) I thought "I'll just focus on them clouds there, the trees are at infinity, just like the clouds, it'll be all right". Not so. The trees look annoying because they are slightly out of focus, but you can see the sharp edge of the cloud. f6.3, 18mm. Shoulda stopped down the lens more.

In case there was any doubt, the kit lens can focus:

So I have no one to blame but myself...

Depth of field preview

The D40 has no DOF preview button. But, does a digital camera really need one? If you have time to stop down the aperture you have to to shoot an image, look at it in the LCD and decide - do I open up or close down the aperture more?

D40 : Exposure : Sunsets

Original


Color and saturation adjusted a bit


A sunset? No.

Actually, this is not a sunset. Its a city rise. The red glow is from Revere, or one of those other cities to the east. This shot was taken facing east.

In short: The D40 can do sunset exposures just fine (These were all shot with aperture priority). Some shots were with -1 EV compensation.

The autofocus will work as long as there is even a tiny bit of light and the subject has contrast. I used the tree to focus, and this was nice and contrasty against the sky.

D40 mini-accessories

I got the Tiffen 52mm UV (Haze) filter and the Matin heavy duty LCD protector from the same company I bought the camera from. The LCD protector is one step up from the films you could get for your PDA. This is a thin sheet of polycarbonate that sticks to your LCD. I misaligned it at first and had to pull it off, and it came off without leaving any residue. It sticks quite well but is very reflective. In bright light I have a feeling it might reduce the contrast of the LCD. I plan to leave the protector on until it gets very scuffed, and then replace it leaving the original LCD scratch free (hopefully).

The Tiffen filter is cheap, its not multi-coated, so you can expect a little glare when you have a bright light source. An example is shown below:

The reflection is due to the UV filter.

The only other thing I want to get is a 4GB card. My 1GB card will store about 270 Fine JPEGs, probably half the amount of RAW.

Focusing screens for the D40

Yes, you can get them for the D40.

There's a very pricey one from Katz Eye. It runs into $100 to $155. It has some good reviews.
There is a pretty cheap one you can get on ebay right now ($34 with shipping), and it might be the same you get from focusingscreen.com (This one costs $40).

A video tutorial for replacing the focusing screen:


But I would use rubber tipped tweezers, rather than straight metal ones. And don't film your self doing it... :)

The google search url.

An amateur's review of the D40: Long exposure time examples

I am pretty pleased with high exposure times on the D40. The last time I really played with long exposure was with my film camera.

What I am NOT pleased with is MY inability to focus properly - aargh! But in my defense there isn't a focusing screen (like split prism, or ground glass). I wonder if I can get one of those for the D40?

Click on the examples to get a 1200px downsampled versions.



If you ignore my amateur focus you will note the amazing lack of noise even on 30s long exposures. I like this sensor.

In the last two pictures below I let the D40 decide the exposure and used aperture priority. The D40, like the F65, knows exposure. (For normal photos, on the F65, I ended up deciding to let the camera decide it. Whenever I took over I did something wrong. Only for special shots - like the really long exposures - did I put everything on manual. I think the same is going to be true of the D40.)

Its amusing though, these last two shots have the same exposure, but they are lit differen…

An amateur's review of the D40: High ISO example

Picture (click to get a downsampled 1280px image). This was taken at sunset with 800 ISO setting with kit lens at 18mm zoom, no VR, handheld.



The overall impression is good, but is marred by the combination of hand shake and higher ISO that gives the photo a softness that hurts the edginess of the main subject. The auto focus was perhaps also not the best.

The following full size segments of the original image illustrate this point:









Nikon D40 at different ISO levels

ISO 200


ISO 400


ISO 800


ISO 1600


ISO 3200

First impressions of Nikon D40

Obtained this as refurbished from Adorama for $375 with 18-55mm DX AF-S lens. Adorama shipped it for free and it arrived overnight (NY to MA). Its declared as refurbished but the body is blemish free. I would say this is a new camera (unless some one soaked the interior in dust/water/acid etc). There is a 90 warranty from Nikon on both lens and body.

From the exif information on the pictures I see that the camera has about 2600 shutter actuations, and the serial number has two dots punched in at the start and end. According to this page those dots indicate Nikon has refurbished the camera.

I found a few dust specks on the viewfinder. (Aiming at the sky and looking around the field of view). I remember on the old Nikons, that my uncle had, you could pop-off the pentaprism and clean it all out, but I don't think you can do that here.

The camera with this lens weighs about as much as my Nikon F65 (or N65), which is to say, less than a kilo.

The battery needs to be charged, but it had eno…

Differences between Nikon pro and consumer DSLRs

It wasn't always clear from reading up on websites and forums what exactly are the differences between consumer DSLRs and pro DSLRs. I'm making a short generic list here of what Nikon pro cameras may have that consumer one's don't (listed according to how important they were to me)
Aperture stop down to preview depth of fieldFast mechanical shutter (rather than a combination of slow mechanical shutter and sensor gating which can lead to streaking in extreme situations )Exposure bracketing (shoot three or more shots in quick succession with slightly different exposure levels)1/3 EV steps for ISO rating (i.e. pro cameras can do ISOs inbetween 100 and 200)Motor for AF lens in body (If you have many old lenses this counts as a cost savings)Metal body that can withstand physical shocks better
Dust removal on the sensorMore auto focus points
Top panel LCD for settings summaryBasically, a "pro camera" is designed to withstand being dropped and bumped (you're in the…

CSS : dot (.) vs hash (#)

. -> class selector
# -> id selector

so
.bregma { width:10cm; } -> <div class="bregma"> #lambda {widh: 5cm;} -> <div id="lambda">

PS. The # symbol is called an octothorpe (Thanks to Vicki for catching the spelling mistake)

New version of matplotlib out : 3D plotting!

The hard working folks at matplotlib have put out a new release candidate with interesting new features.

Remember - its a release candidate. Not (yet) for production.

Get it from http://drop.io/xortel1#
Install using sudo easy_install ~/Downloads/matplotlib-0.99.0.rc1_r0-py2.5-macosx-10.5-i386.egg
On mac you will hear complaints about
Searching for matplotlib==0.99.0.rc1-r0
Reading http://pypi.python.org/simple/matplotlib/
Reading http://matplotlib.sourceforge.net
Reading https://sourceforge.net/project/showfiles.php?group_id=80706&package_id=278194
Reading https://sourceforge.net/project/showfiles.php?group_id=80706&package_id=82474
Reading http://sourceforge.net/project/showfiles.php?group_id=80706&package_id=82474
Reading http://sourceforge.net/project/showfiles.php?group_id=80706
No local packages or download links found for matplotlib==0.99.0.rc1-r0
error: Could not find suitable distribution for Requirement.parse('matplotlib==0.99.0.rc1-r0')but it will be installed fine.

M…

Installing mongrel on mac OS X with ruby 1.9.1

What. A. Pain.

UPDATE: I'm now using thin instead of mongrel.

When you go to do sudo gem install mongrel it will barf with some errors related to a file called http1.c Stop. Do not adjust the channel. There is nothing wrong with your computer. Somebody goofed in the source code. From instructions here, you have to make the following changes to the source. Remember to edit as root, don't run gem install mongrel again - you will overwrite your corrections.

This stuff was worked out by Alan and clarified by Ami Mahloof
1) cd /usr/local/lib/ruby/gems/1.9.1/gems/mongrel-1.1.5/ext/http11 2) sudo vi / mate on http11.c and: replace line 77 with for(ch = RSTRING_PTR(f), end = ch + RSTRING_LEN(f); ch < end; ch++) { replace line 172 with colon = strchr(RSTRING_PTR(temp), ':'); replace line 174 with rb_hash_aset(req, global_server_name, rb_str_substr(temp, 0, colon - RSTRING_PTR(temp))); replce line 176,177 rb_str_substr(temp, colon - RSTRING_PTR(temp)+1, RSTRING_LEN(tem…

Internet speed over HMS secure connection

Hmm... I guess they throttle downloads, but I have 5/20 service, how can I upload faster?

This is the speed test without secure conection

Parallel python: destroying job_servers

Remember to destroy job servers after they are done.

My nice animation generator crashed sometime over the weekend with the following death rattle:

Traceback (most recent call last):
File "animate_array_job.py", line 168, in
tscale = tscale, dpi = dpi)
File "animate_array_job.py", line 92, in process_file
job_server = pp.Server(ppservers=ppservers)
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/pp.py", line 336, in __init__
self.set_ncpus(ncpus)
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/pp.py", line 494, in set_ncpus
range(ncpus - len(self.__workers))])
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/pp.py", line 141, in __init__
self.start()
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/pp.py", line 148, in start
shell=True)
File "/Library/Framew…

Parallel python : clusters

If the jobs are too short they may not get submitted to the remote machine and may all execute on local. pp submits jobs to remote machines only after running out of workers on local
File handles are opened on the remote machines.
You can't pass a file handle as a function argument
TypeError: can't pickle file objects

Parallel python

Among the many libraries for parallel processing in python I chose what seemed to have one of the simpler interfaces - Parallel Python.

There is one trick that is not apparent from the docs: The function you call has to be 'self sufficient'. You basically have to write up the function you call as if it were a script. So the function has to have the imports etc within itself.

This is not a biggie, but it gets confusing until you figure it out.

Original code:

# module x -----------------
import a
import b

def f1(g):
f2
f3

def f2():

def f3():

# Calling script
import x

x.f1(2)
x.f1(4)
x.f1(6)
x.f1(8)


Parallelized code
# module px ------------------------------
def pf(g)
import x <------ NOTE THIS. ALL IMPORTS AND OTHER FUNS HAVE TO BE WITHIN THIS FUN
x.f(g)

# New calling script
import pp
import px

ppservers = ()
job_server = pp.Server(ppservers=ppservers)

jobs = []
jobs.append(job_server.submit(px.pf, (2,))
jobs.append(job_server.submit(px.pf, (4,))
jobs.append(job_server.submit(px.pf, (6,))
jobs.append(j…

Matplotlib interactive plotting pads figure size

With interactive backend (eg Tk)


f = pylab.figure(figsize = (7,10))
f.get_size_inches() -> array([ 7.175, 10.175])


The saved pdf is also of this size.

It becomes extreme with smaller figures:


f = pylab.figure(figsize = (1,1))
f.get_size_inches() -> array([ 3.275, 1.175])


This last I can see as a function of getting the tool bar to fit.

With Agg the saved figure is the size you ask for

Only on wikipedia:

Reading a paper about neural circuitry. Came across the word 'resonant circuitry'. Looked it up on wikipedia. Knew it had to do with band pass circuitry. Looked that up on wikipedia. Read in that entry that Thomas Pynchon claimed the character Kilroy was based on the circuit diagram for a band pass filter. You be the judge.



Any hoo, went to Pynchon's page on wikipedia and started to read. On the entry for his novel V, read about yoyodyne corp. Hey I know that from the GPL. Went to the (fictional) webpage for yoyodyne corp... Wait. Whaa!? What's all thsi? What's the time? Where am I? What am I doing here? Wasn't I just reading a paper on neural circuitry?

Sourceforge community choice awards

So I voted as follows:

KeePass Password Safe for Best Project (I would have liked matplotlib in the list)
eViacam for Best New Project
matplotlib for Best Project for Academia
Audacity for Best Project for Multimedia (mencoder, if it was on sf, would have won mine)
OpenOffice.org for Best Project for Government
KeePass Password Safe for Most Likely to Change the Way You Do Everything (I would have liked matplotlib in the list)

I would have liked to see Matplotlib in the categories 'Best Project' and 'Most Likely to Change the Way You Do Everything' too, but KeePass is a good choice.

I didn't vote in a category when I didn't know any of the choices. eViacam is a mistake. I tried to get rid of it but couldn't cancel that vote. I picked it at first even though I knew nothing about it because I thought it was a good idea for a project, but later decided it wasn't right to vote on a project I haven't used.

Efficient pickling in python

Use the highest protocol you have available - which will usually be binary. So as example:
import cPickle
import pylab

z = pylab.random((10,10))
z ndarray 10x10: 100 elems, type `float64`, 800 bytes
len(cPickle.dumps(z)) -> 2371
len(cPickle.dumps(z, cPickle.HIGHEST_PROTOCOL)) -> 934

z = pylab.random((50,50))
z ndarray 50x50: 2500 elems, type `float64`, 20000 bytes
len(cPickle.dumps(z)) -> 55586
len(cPickle.dumps(z, cPickle.HIGHEST_PROTOCOL)) -> 20134
For some reason, I thought the default was to use the most efficient protocol, but it isn't.

cPickle.dump?
Type: builtin_function_or_method
Base Class:
String Form:
Namespace: Interactive
Docstring:
dump(obj, file, protocol=0) -- Write an object in pickle format to the given file.

See the Pickler docstring for the meaning of optional argument proto.
Hence my slow processing and bloated data files...
Note that load figures out the protocol automatically...

3D in matplotlib

One of the frustrating things about matplotlib is its lack of 3D plotting. If matplotlib were a mediocre library it wouldn' hurt. But it is a GREAT library and produces really good looking plots. So the lack of 3D plotting sometimes makes me want to go out and strangle somebody. There has however always been a furtive 3D plotting feature in matplotlib, and apparently, its being worked on some more. Yeehah! No more futzing with pyVtk.

Important links:
Docs for 3D features
Installing the svn code (that has all this goodness)

Drawing circles using matplotlib

Use the pylab.Circle command

import pylab #Imports matplotlib and a host of other useful modules cir1 = pylab.Circle((0,0), radius=0.75, fc='y') #Creates a patch that looks like a circle (fc= face color) cir2 = pylab.Circle((.5,.5), radius=0.25, alpha =.2, fc='b') #Repeat (alpha=.2 means make it very translucent) ax = pylab.axes(aspect=1) #Creates empty axes (aspect=1 means scale things so that circles look like circles) ax.add_patch(cir1) #Grab the current axes, add the patch to it ax.add_patch(cir2) #Repeat pylab.show()

Modifying a firefox plugin

I dislike the indistinct icons in the cookie manager plugin CSlite.

In Firefox/extensions/{00084897-021a-4361-8423-083407a033e0}/ (the directory for this particular extension) we find chrome/cookiesafe.jar

From a hint in a thread here, we learn that the .jar file is just a .zip file, and we can extract that to find skin/classic/cookiesafe and a whole bunch of .gifs there.

We can then change those gifs and change the icons that are used in the application.

CS Lite

I was using the Permit Cookies extension for firefox, but it hasn't been updated for a few cycles of firefox updates. It works fine, but I started to look for something else. I found CS lite which has a more convenient interface (a popup menu from the status bar). The only gripe I have so far is that the icons in the status bar are not very clean.

Searching for a cheap graphics card that will play Crysis

Downloaded and started playing the Crysis demo. On my integrated graphics MoBo, naturally I had to set everything down to low, but even then it can get choppy when in the jungle. And the graphics look so crappy I think - hmm, maybe I'll just play Far Cry. Tom's hardware has a great page where you can compare the performance of various cards with various games at various settings when searching for a cheap graphics card for crysis.

Verizon FiOS 20/5 serivice over wireless

From other parts





Installing Ruby on Mac OS X

ftp ftp://ftp.ruby-lang.org/pub/ruby/1.9/ruby-1.9.1-p129.tar.gz
tar -xvf ruby-1.9.1-p129.tar.gz
cd ruby-1.9.1-p129
./configure --enable-shared --enable-pthread CFLAGS=-D_XOPEN_SOURCE=1
make -j4
sudo make install


Installs in /usr/local/bin, make sure to point path there

Those geneticists and their Excel

Mistaken Identifiers: Gene name errors can be introduced inadvertently when using Excel in bioinformatics

If you are too lazy to get to the punchline:

"MatchMiner [1] and GoMiner [2] are two bioinformatics program packages we
published recently in another Biomed Central Journal, Genome Biology. When we
were beta-testing those programs on microarray data, a frustrating problem
occurred repeatedly: Some gene names kept bouncing back as "unknown." A little
detective work revealed the reason: Use of one of the research community's most
valuable and extensively applied tools for manipulation of genomic data can
introduce erroneous names. A default date conversion feature in Excel (Microsoft
Corp., Redmond, WA) was altering gene names that it considered to look like
dates. For example, the tumor suppressor DEC1 [Deleted in Esophageal Cancer 1]
[3] was being converted to '1-DEC.' Figure 1 lists 30 gene names that suffer an
analogous fate."

Sourceforge nominations

My choices:
- Matplotlib

- Inkscape

Function definitions in scripts

My MATLAB origins betrayed me. I kept thinking - you can't put function definitions in a script in Python. Man, I gotta write a module and THEN a script that calls the module? But, in Python, you can include function defs in a 'script'. So the following works perfectly well:

def func1():
print '1'

def func2():
print '2'

func1()


Take that MATLAB.

GPL logo

is not very good. Its kind of crowded and ungainly.

Don't you think they should have come up with something more visually appealing? To me, even plain type looks better! Like this
GPLv3

The growing up of the GPL

I first ran into the GPL in some book. It could have been a coding book or a latex book. I didn't care about licenses at that age, but I remembered it because it contained the following extract:
You should also get your employer (if you work as a programmer) or your school, if any, to sign a "copyright disclaimer" for the program, if necessary. Here is a sample; alter the names: Yoyodyne, Inc., hereby disclaims all copyright
interest in the program `Gnomovision'
(which makes passes at compilers) written
by James Hacker.

signature of Ty Coon, 1 April 1989
Ty Coon, President of Vice
Now, these guys must be fun to hang out with! That's what I thought. Recently, I looked up GPL3. Stallman's blurb on why we should switch was all peachy and militant. But what really disappointed me was the end of the license:
If the program does terminal interaction, make it output a short notice like this when it starts in an interactive mode:
Copyright (C)
This program comes with A…

Constantize.

Ahh Ruby... Suppose you have a string class that corresponds to a class name and you want to call a certain method on it, what do you do? Well you could do

eval class + '.method'

but that will be slow (as evals are slow) and apparently that looks ugly.

But the creator of Ruby must have run into that issue often. Because what you can do, is

class.constantize.method

This is an inflector that tries to find a declared constant with the name in the string. And in Ruby classes are constants...

Federal free file works

I think the IRS did a good job on the free file interface. The interface works on Firefox3.0 on a mac. I'm assuming it works on windows. There are a few minor glitches with cursor positioning on some text fields (when you need to go back and delete/edit stuff in a field) but its an allround good job. If all government stuff was like this it wouldn't hurt so much to pay the taxes.

Don't use 'type' as a field in a Rails database

I'm working on a new rails application. In one of the tables I have a field called 'type'. Everything was going fine, but whenever I set the 'type' field and saved the record it ignored the contents and set it to NULL. I couldn't figure it out.

Tangentially, I had set a bunch of fields to be 'string' and these are translated to be varchar(255) in the sqlite database and I wanted to test what happened if I went over the 255 character limit. So I typed in a string 260 characters long into the 'type' field using an external database app and saved the record. It saved just fine, and kept all the characters. Now I thought, let me load this in my rails app and see if it still sets it to NULL. I loaded it and got back this error message:


ActiveRecord::SubclassNotFound in ItemsController#show

The single-table inheritance mechanism failed to locate the subclass: 'abcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklm…

Free e-file for federal

Excellent! I always wondered why the IRS didn't allow you to e-file for free (unless you were below a certain income limit). I mean its free to paper file. It must cost the IRS a lot to go through badly handwritten paper forms. But this year I see that you can e-file for free whatever your income. Good.

Bookmarks in Eclipse!

I was jumping round my analysis code and getting tired of hunting to find the apropriate line I was working on. I thought wouldn't it be nice if there was a bookmark system. So I right clicked on the gutter in eclipse. And. Whaddya know? THERE'S A BOOKMARK SYSTEM. I love this IDE.

Maximum number of open files in Python

So here I was merrily writing a python module to extract neural data from the Cerebus system's giant .nevs file and split it into smaller files - one per neural unit. I had been extracting one channel at a time and all was well. So now I copy the code over to the lab machine and tell it to extract all the data and split it into 96*4 files.

Traceback (most recent call last):
File "convert_nev.py", line 11, in
File "/Users/kghose/Experiments/Python/nev.py", line 270, in fragment
IOError: [Errno 24] Too many open files: '../Neural/DJ2009/20090320//Frag//electrode64unit02.bin'

Whaaa? Whats all this? Well, it turns out you can't have too many files open at the same time. And the resource module can tell you exactly how many files you can have open at a given time:

resource.getrlimit(resource.RLIMIT_NOFILE)
(256L, 9223372036854775807L)

Which means the current soft limit is 256 files and the hard limit is some number so large only astronomers and people who write…

Why I love matplotlib + python

Note this graph. The upper trace just goes from 0 to 1.6ms. The lower trace was a series of such waveforms ocurring from 0 to 10 s. I zoomed in to see one wave. When you do that in Matlab the tick labels would take the form 8.168, 8.169, 8.170 which is legitimate, but gets hard to read when you zoom in a lot and you get lost in the digits. Matplotlib, on the other hand, produces the zoom plot pictured here. It shows you the offset on the right (8.168) and then just shows you the vernier values. Now THAT is what I call service. The matplotlib guys are really good. Its like they actually use their product for data analysis. They must be scientists or something :).

Percent sign in python format string

From here: just use a % sign as an escape character.

so:

print '22%' works

print '%d' %(22) works

print '%d%' %(22) throws an error and the solution is

print '%d%%' %(22)

People say this is hacky. They 'fixed' it in Python 3.0 but I'm not upgrading soon. Imagine having to go through all your code because of this nonessential thing!

The time travelling Mac

Finder has a little bug:

Slow Matlab editor on Mac OS X

From macoshints:

If you've upgraded to Leopard and now find the IDE of the new Matlab release 2007b to be super-slow, create a file called java.opts in your /Applications/MATLAB_R2007b/ directory, and add the following line to it: -Dapple.awt.graphics.UseQuartz=true Then restart Matlab. This will switch Matlab back to using the Quartz drawing enginge (as it was in previous versions of OS X).

Getting clips from DVD

Sometimes I find howlers on DVD that I want to store. On a Mac the sequence is:
Open the DVD in VLC and find out which title and chapter the clip is in.Use MacTheRipper to extract that title and chapter to .vob format
Open it up in VLC, note the seconds offset for the part of the clip we want and then use File->Streaming/Export wizard to export the clip. I use H.264 (3072 kb/s) for video and MPEG-4 audio and mp4 container.Here's an example:


The what controls what again?

Muggles

I thought the author of the Harry Potter books invented the word 'Muggle' but I ran into it in Steinbeck's 'Sweet Thursday'. Couldn't figure out what it meant though.