Lisp in Parallel : lparallel

lfarm is a distributed version of lparallel which replaces worker threads with remote processes. For example lfarm:pmap will subdivide the input sequence(s), send the parts to remote machines for mapping, and then combine the results. Likewise lfarm:future wraps remote task execution in the metaphor of promises. Most of the lparallel kernel API is retained with minor variations. Support for remote closures is available on some popular platforms (SBCL, CCL, LispWorks, Allegro).

• plet now exploits type declarations
• defpun*, defpun/type*, and psort* are now deprecated — instead use the unstarred versions and pass :use-caller t to make-kernel
• parallel compilation is now safe

• exported types: kernel, channel, ptree
• added ptree-computed-p — query the computed state of a ptree node
• make-kernel now aborts cleanly when a worker fails to initialize, e.g. when make-thread fails or when a :context function aborts
• check-kernel now returns a kernel instance
• added a front-end lock to some ptree functions — removes the requirement that some calls be exclusive
• improved performance of functions defined by defpun

• added readers kernel-name and kernel-context
• added restart kill-errors to workers — removes debugger popups
• attempting to submit a task to an ended kernel now signals an error
• suicidal calls to kill-tasks inside workers are now permitted

Below is another example of Concurrent Hello World. The macros receive and run are just 9 lines each, given below the fold.

(defpackage :example (:use :cl :node))
(in-package :example)

(defun hello (hello-queue world-queue)
  (receive hello-queue
    (:say-hello (n)
      (cond ((plusp n)
             (format t "Hello ")
             (send world-queue :say-world n))
            (t
             (send world-queue :quit)
             (return))))))

(defun world (world-queue hello-queue)
  (receive world-queue
    (:say-world (n)
      (format t "World!~%")
      (send hello-queue :say-hello (- n 1)))
    (:quit ()
      (return))))

(defun main (n)
  (let ((hello-queue (make-queue))
        (world-queue (make-queue)))
    (run
      (make-node 'hello hello-queue world-queue)
      (make-node 'world world-queue hello-queue)
      (send hello-queue :say-hello n))))

Continue reading →

The major version bump is for some incompatible changes, though they are unlikely to cause trouble.

• keyword arguments to psort besides :key have been replaced with a single :granularity argument; the old arguments are now ignored
• removed deprecated aliases from 1.2.0 and 1.3.0 (you may not be aware of them since they haven’t been listed in the documentation)
• A function defined with defpun is now optimized for N worker threads where N is the number of cores. The old behavior is available with defpun*, which defines a function that is optimized for N-1 workers (and has less overhead).
• added psort* — like psort but targets N-1 workers
• improved performance of psort
• task categories are now compared with eql; same for ptree node ids

The benchmarks page has been updated to reflect recent optimizations, which includes optimizations in SBCL itself.