distributed self-stabilization made easy
libcircle is an API for distributing embarrassingly parallel workloads using self-stabilization. Details on the algorithms used may be found at http://dl.acm.org/citation.cfm?id=2389114.
The basic program flow when using libcircle is the following:
#include <libcircle.h>
/* An example of a create callback defined by your program */
void my_create_some_work(CIRCLE_handle *handle)
{
/*
* This is where you should generate work that needs to be processed.
* For example, if your goal is to lstat files on a large filesystem,
* this is where you would readdir() and and enqueue directory names.
*
* This should be a small amount of work. For example, only enqueue the
* filenames from a single directory.
*/
while((data_to_process = readdir(...)) != NULL)
{
handle->enqueue(data_to_process);
}
}
/* An example of a process callback defined by your program. */
void my_process_some_work(CIRCLE_handle *handle)
{
/*
* This is where work should be processed. For example, this is where you
* should lstat one of the files which was placed on the queue by your
* create_some_work callback. Again, you should try to keep this short and
* block as little as possible.
*/
handle->dequeue(&my_data);
...
finished_work = lstat(my_data, ...);
...
store_in_database(finished_work);
}
/*
* Initialize state required by libcircle. Arguments should be those passed in
* by the launching process. argc is a pointer to the number of arguments,
* argv is the argument vector. The return value is the MPI rank of the current
* process.
*/
int rank = CIRCLE_init(&argc, argv);
/*
* Processing and creating work is done through callbacks. Here's how we tell
* libcircle about our function which creates the initial work. For MPI nerds,
* this is your rank 0 process.
*/
CIRCLE_cb_create(&my_create_some_work);
/*
* After you give libcircle a way to create work, you need to tell it how that
* work should be processed.
*/
CIRCLE_cb_process(&my_process_some_work);
/*
* Now that everything is setup, lets execute everything.
*/
CIRCLE_begin();
/*
* Finally, give libcircle a chance to clean up after itself.
*/
CIRCLE_finalize();