Lisp in Parallel : lparallel

• added broadcast-task — executes a given task inside each worker
• added kernel-worker-index — determine if the current thread is a
  worker thread, and if so, obtain its assigned index

• added the ability to bind multiple values in plet, e.g.
  (plet ((x 1) ((y z) (values 2 3))) (+ x y z)) ;=> 6
• added slet — serial/non-parallel let with the same syntax as plet for binding multiple values
• various optimizations

• added a :timeout option to try-pop-queue and try-receive-result; this requires the latest version of bordeaux-threads (lparallel will still run with older versions of bordeaux-threads as long as the :timeout option is not used)
• optimizations to defpun
• optimizations from smaller generated code size

lfarm is a distributed version of lparallel which replaces worker threads with remote processes. For example lfarm:pmap will subdivide the input sequence(s), send the parts to remote machines for mapping, and then combine the results. Likewise lfarm:future wraps remote task execution in the metaphor of promises. Most of the lparallel kernel API is retained with minor variations.

• plet now exploits type declarations
• defpun*, defpun/type*, and psort* are now deprecated — instead use the unstarred versions and pass :use-caller t to make-kernel
• parallel compilation is now safe

• exported types: kernel, channel, ptree
• added ptree-computed-p — query the computed state of a ptree node
• make-kernel now aborts cleanly when a worker fails to initialize, e.g. when make-thread fails or when a :context function aborts
• check-kernel now returns a kernel instance
• added a front-end lock to some ptree functions — removes the requirement that some calls be exclusive
• improved performance of functions defined by defpun

• added readers kernel-name and kernel-context
• added restart kill-errors to workers — removes debugger popups
• attempting to submit a task to an ended kernel now signals an error
• suicidal calls to kill-tasks inside workers are now permitted

Below is another example of Concurrent Hello World. The macros receive and run are just 9 lines each, given below the fold.

(defpackage :example (:use :cl :node))
(in-package :example)

(defun hello (hello-queue world-queue)
  (receive hello-queue
    (:say-hello (n)
      (cond ((plusp n)
             (format t "Hello ")
             (send world-queue :say-world n))
            (t
             (send world-queue :quit)
             (return))))))

(defun world (world-queue hello-queue)
  (receive world-queue
    (:say-world (n)
      (format t "World!~%")
      (send hello-queue :say-hello (- n 1)))
    (:quit ()
      (return))))

(defun main (n)
  (let ((hello-queue (make-queue))
        (world-queue (make-queue)))
    (run
      (make-node 'hello hello-queue world-queue)
      (make-node 'world world-queue hello-queue)
      (send hello-queue :say-hello n))))

Continue reading →

The major version bump is for some incompatible changes, though they are unlikely to cause trouble.

• keyword arguments to psort besides :key have been replaced with a single :granularity argument; the old arguments are now ignored
• removed deprecated aliases from 1.2.0 and 1.3.0 (you may not be aware of them since they haven’t been listed in the documentation)
• A function defined with defpun is now optimized for N worker threads where N is the number of cores. The old behavior is available with defpun*, which defines a function that is optimized for N-1 workers (and has less overhead).
• added psort* — like psort but targets N-1 workers
• improved performance of psort
• task categories are now compared with eql; same for ptree node ids

The benchmarks page has been updated to reflect recent optimizations, which includes optimizations in SBCL itself.