Skip to main content

Parallel python

Among the many libraries for parallel processing in python I chose what seemed to have one of the simpler interfaces - Parallel Python.

There is one trick that is not apparent from the docs: The function you call has to be 'self sufficient'. You basically have to write up the function you call as if it were a script. So the function has to have the imports etc within itself.

This is not a biggie, but it gets confusing until you figure it out.

Original code:

# module x -----------------
import a
import b

def f1(g):
f2
f3

def f2():

def f3():

# Calling script
import x

x.f1(2)
x.f1(4)
x.f1(6)
x.f1(8)



Parallelized code
# module px ------------------------------
def pf(g)
import x <------ NOTE THIS. ALL IMPORTS AND OTHER FUNS HAVE TO BE WITHIN THIS FUN
x.f(g)

# New calling script
import pp
import px

ppservers = ()
job_server = pp.Server(ppservers=ppservers)

jobs = []
jobs.append(job_server.submit(px.pf, (2,))
jobs.append(job_server.submit(px.pf, (4,))
jobs.append(job_server.submit(px.pf, (6,))
jobs.append(job_server.submit(px.pf, (8,))

Comments

Popular posts from this blog

Python: Multiprocessing: passing multiple arguments to a function

Write a wrapper function to unpack the arguments before calling the real function. Lambda won't work, for some strange un-Pythonic reason.


import multiprocessing as mp def myfun(a,b): print a + b def mf_wrap(args): return myfun(*args) p = mp.Pool(4) fl = [(a,b) for a in range(3) for b in range(2)] #mf_wrap = lambda args: myfun(*args) -> this sucker, though more pythonic and compact, won't work p.map(mf_wrap, fl)

Flowing text in inkscape (Poster making)

You can flow text into arbitrary shapes in inkscape. (From a hint here).

You simply create a text box, type your text into it, create a frame with some drawing tool, select both the text box and the frame (click and shift) and then go to text->flow into frame.

UPDATE:

The omnipresent anonymous asked:
Trying to enter sentence so that text forms the number three...any ideas?
The solution:
Type '3' using the text toolConvert to path using object->pathSize as necessaryRemove fillUngroupType in actual text in new text boxSelect the text and the '3' pathFlow the text

Drawing circles using matplotlib

Use the pylab.Circle command

import pylab #Imports matplotlib and a host of other useful modules cir1 = pylab.Circle((0,0), radius=0.75, fc='y') #Creates a patch that looks like a circle (fc= face color) cir2 = pylab.Circle((.5,.5), radius=0.25, alpha =.2, fc='b') #Repeat (alpha=.2 means make it very translucent) ax = pylab.axes(aspect=1) #Creates empty axes (aspect=1 means scale things so that circles look like circles) ax.add_patch(cir1) #Grab the current axes, add the patch to it ax.add_patch(cir2) #Repeat pylab.show()