PyMW - Master Worker Computing in Python

PyMW (Master Worker Computing in Python) PyMW is a Python module for parallel master-worker computing in a variety of environments, including BOINC. With the PyMW module, users can write a single program that scales from multi-core machines to global computing platforms.

Download

PyMW can be downloaded from here.

Installation

A BOINC project has to be prepared to accept applications written in Python. BOINC comes with an install script for PyMW. The setup script - pymw_setup.py - can be found in the bin directory of the BOINC project. An absolute path to the PyMW working directory has to be provided with the -p or --pymw switch to the setup script. The assimilator program for PyMW will copy the results to this directory. Example to run the setup script:

cd /home/kadam/projects/sandbox/bin
pymw_setup -p /home/kadam/pymw/examples/tasks

The PyMW setup script executes the followings, respectively:

Insert pymw_assimilator in the daemons section of config.xml
Insert pymw application in project.xml
Create pymw directory in the app directory of the project
Create client application executables for Linux platform
Create client application executables for Windows platform
Call xadd
Call update_versions

Platforms

PyMW is available and has been tested on the following platforms:

Linux: i686-pc-linux-gnu
Windows: windows_intelx86
OS X: i686-apple-darwin

By default PyMW assumes that the python interpreter is installed on the client computers. The Python interpreter is more likely to be installed on linux clients. On Windows PyMW assumes that Python is installed and the path environment variable contains the path to the interpreter. If python is installed on a windows box, but it's not in the path, than PyMW will look in the default installation directory, which is C:\Python25 with the current version of Python.

If python is not installed on the client computer, PyMW jobs will fail. The license of Python makes it possible to deploy the interpreter as part of the client application, however the current version of the interpreter is about 35 MB, which is not very network friendly. A later version of PyMW will come up with a solution that transfers the interpreter compressed over the network to reduce network load, but make PyMW widely available.

Usage

Once successfully executed, the project is ready to handle PyMW jobs. The setup script can be called multiple times. However, if the pymw client application is already registered with the project, the setup script will only alter the working directory of the pymw_assimilator in the config.xml file.

To use PyMW with the BOINC interface PyMW has to be configured with the -i or --interface switch. Also the home directory of the BOINC project needs to be passed to PyMW with the -p or --project_home switch. Example to run PyMW with the BOINC interface:

cd /home/kadam/pymw/examples
python monte_pi.py -i boinc -p /home/kadam/projects/sandbox/

For further information on using PyMW, please refer to the documentation of PyMW.

Home