Scavenger - Now With History Based Profiling

I have just added yet another scheduler to Scavenger (yes, there is three of them now). This scheduler is the "smartest" of the three as it uses history based profiling in order to be able to choose the right surrogate for the job. History based profiling in a cyber foraging system is nothing new - the Chroma system used that some 6-7 years ago. What is new is the way that the profiling information is used within the Scavenger system. In Scavenger the profiling information can be used not only to tell you something about how the task will perform on certain surrogates (i.e., the ones where you have earlier performed the task), it is also capable of making an informed estimate of the running time on an unknown surrogate... more on that later ;-)

Building the Python Imaging Library (PIL) for iPhone/iPod Touch

As my last post hints at I have started experimenting with building PIL for my iPod Touch. Doing so is non-trivial and I have therefore decided to write a small tutorial here, so that others may be spared of wasting their time on this project ;-)

What I did was to compile the thing directly on the iPod, and to do that I went through the following steps:

1. Jailbreak: I used QuickPWN 2.2.5 under Windows (it did not work on my Mac for some reason) and that was quite easy.
2. Install SSH on the iPod using Cydia (there is an entry in the main menu showing you how to do that).
3. SSH in to your iPod as the root user with password "alpine".
4. Install Python, the gcc compiler, the code signing tool, and make by issuing this command: apt-get install python iphone-gcc ldid make (alternatively you could install it using Cydia it you like).
5. Download the SDK headers, you may find them along with a description of how to use them here.
6. [Optional] Download the libjpeg source if you want jpeg support. Untar the source bundle, enter the directory containing the source and issue the following commands:

   export C_INCLUDE_PATH=/var/include/
   ./configure --prefix=/usr/
   make
   make install
   make install-lib

7. Download the PIL source and unpack it.
8. Enter the source directory and issue the following commands:

   python setup.py build
   python setup.py install
   ldid -S /usr/lib/python2.5/site-packages/PIL/*.dylib

   
   

After going through all of these steps I ended up with a working PIL library on my iPod. Oh, and thanks to Renaud Blanch for pointing out to me that I needed to sign the dynamic libraries for them to work.

Scavenger on iPhone/iPod Touch

The first thing I did after implementing the new Presence lib was of course to install Scavenger on my iPod Touch. And I am happy to report that it works! :-)

The Python version installable on a jailbroken iPhone/iPod is version 2.5.1, and since Scavenger is developed using Python 2.6 in a "backwards compatible" way everything worked like a charm.

The next step is getting the Python Imaging Library working on the iPod...

New Presence Consumer Lib

I have just created a new Presence library - one that does not depend on the Presence daemon. This lib may be used by Python applications that only act as consumers of Presence services, i.e., application that do not advertise services through Presence but merely uses services announced by other peers.

So why did I make this new consumer-only lib? Because it enables devices that are not capable of running the Presence daemon to use Presence anyway (as a client/consumer right now). This means for example, that I can discover and use Presence services from my jailbroken iPod.

The new lib can be found in /trunk/libs/python_consumer_lib.

More about the __cmp__ method in Python

I just stumbled upon this little something about overloading the __cmp__ method in a Python class. If you are not careful, you will end up having a class whose instances can not be compared to None (as I always do to see if some optional parameter has been passed to a method or function). Lets take a small example:

class Foo(object):

def __init__(self, x):

    self._x = x

def __cmp__(self, other):

    if type(other) == Foo:

        return cmp(self._x, other._x)

    else:

        raise TypeError("Foo cannot be compared to %s"%str(type(other))) f1 = Foo(1)

f2 = Foo(2)

print 'f1 is smaller than f2:', f1 < f2 # True

print 'f1 is larger than f2:', f1 > f2  # False

print 'f1 is equal to f2:', f1 == f2    # False

print 'f1 is None:', f1 == None         # Crash!

This code crashes in the final statement, because the comparison to None calls the __cmp__ method, and it has no way of comparing to None. This could of course be fixed by adding "if other == None: return -1" but the right way to fix this would be by overloading another method called __eq__ that is used to test for object equality.

class Foo(object):

def __init__(self, x):

    self._x = x

def __eq__(self, other):

    if type(other) == Foo:

        return cmp(self, other) == 0

    else:

        return False

def __cmp__(self, other):

    if type(other) == Foo:

        return cmp(self._x, other._x)

     else:

         raise TypeError("Foo cannot be compared to %s"%str(type(other)))



f1 = Foo(1)

f2 = Foo(2)

print 'f1 is smaller than f2:', f1 < f2 # True print 'f1 is larger than f2:', f1 > f2  # False

print 'f1 is equal to f2:', f1 == f2    # False

print 'f1 is None:', f1 == None         # False

Now the comparison to None (or in fact to any object that is not of type Foo) is caught by the __eq__ method, and comparisons to other type Foo objects are correctly forwarded to the __cmp__ method.

Emacs ido-mode (minor mode)

I have just spent some time play around with the Emacs "InteractivelyDoThings"-mode; ido-mode. Ido-mode replaces your regular "find-file" and "switch-buffer" mini-buffer with a much more flexible version; a version where you are allowed to do fussy matching, tab-complete your way to opening files on remote servers, and much much more...

Check out this screencast by Stuart Halloway if you have the time, or just dive into this EmacsWiki-page for a short explanation of what ido-mode does for you.

The "with"-statement in Python

In Python 2.5 and onwards a new "with" statement has been introduced. I recently read a really great article by Fredrik Lundh about the inner workings of this statement, so please read it if you are interested in all the dreary details. The syntax of the with statement is shown below:

with expression [as name]
    code block

What it boils down to is this: The with statement expects its expression to evaluate to an object that has two methods: __enter__ and __exit__. These will then be called before and after executing the code within the statement. To quote the article "Now, when the “with” statement is executed, Python evaluates the expression, calls the __enter__ method on the resulting value (which is called a “context guard”), and assigns whatever __enter__ returns to the variable given by as. Python will then execute the code body, and no matter what happens in that code, call the guard object’s __exit__ method. "

This means, that you can use the with statement for other things than merely locking (which is probably the mostly know use). Consider the code below:

with open('foo.txt', 'r') as infile:
    print infile.read()

This would open the file, assign it the name infile in the statement code block, read and print the file, and finally close the file again. This is all because a file objects __enter__ method returns the opened file, and its __exit__ method closes the file. Pretty neat, huh? :-)