Mapping

It should be no surprise that arrays are faster than lists for parallel mapping. The open-coded versions of pmap and pmap-into, which are triggered when a single array is mapped to an array, are particularly fast in SBCL when the array types are declared or inferred. For the extreme case of a trivial inline function applied to a large array, the speed increase can be 20X or more relative to the non-open-coded counterparts.

Condition handling under the hood

To the user, a task is a function designator together with arguments to the function. However the internal representation of a task is like a generalization of a closure. A closure is a function which captures the lexical variables referenced inside it. Implementation-wise, a task is a closure which captures the task handlers present when the task is created. A closure bundles a lexical environment; a task additionally bundles a dynamic environment. This is the basic theory behind parallelized condition handling in lparallel.

Communicating via conditions

Because task handlers are called immediately when a condition is signaled inside a task, condition handling offers a way to communicate between tasks and the thread which created them. Here is a task which transfers data by signaling:

(defpackage :example (:use :cl :lparallel :lparallel.queue))
(in-package :example)

(define-condition more-data ()
  ((item :reader item :initarg :item)))

(let ((channel (make-channel))
      (data (make-queue)))
  (task-handler-bind ((more-data (lambda (c)
                                   (push-queue (item c) data))))
    (submit-task channel (lambda ()
                           (signal 'more-data :item 99))))
  (receive-result channel)
  (pop-queue data))

; => 99

receive-result has been placed outside of task-handler-bind to emphasize that handlers are bundled at the point of submit-task. (It doesn’t matter where receive-result is called.)

Multiple kernels

It may be advantageous to have a kernel devoted to particular kinds of tasks. For example one could have specialized channels and futures dedicated to IO.

(defvar *io-kernel* (make-kernel 16))

(defun make-io-channel ()
  (let ((*kernel* *io-kernel*))
    (make-channel)))

(defmacro io-future (&body body)
  `(let ((*kernel* *io-kernel*))
     (future ,@body)))

Since a channel remembers its associated kernel, submit-task and receive-result do not depend upon the value of *kernel*. In the promise API, only future and speculate need *kernel*.