It should be no surprise that arrays are faster than lists for parallel mapping. The open-coded versions of
pmap-into, which are triggered when a single array is mapped to an array, are particularly fast in SBCL when the array types are declared or inferred. For the extreme case of a trivial inline function applied to a large array, the speed increase can be 20X or more relative to the non-open-coded counterparts.
Condition handling under the hood
To the user, a task is a function designator together with arguments to the function. However the internal representation of a task is like a generalization of a closure. A closure is a function which captures the lexical variables referenced inside it. Implementation-wise, a task is a closure which captures the task handlers present when the task is created. A closure bundles a lexical environment; a task additionally bundles a dynamic environment. This is the basic theory behind parallelized condition handling in lparallel.
Communicating via conditions
Because task handlers are called immediately when a condition is signaled inside a task, condition handling offers a way to communicate between tasks and the thread which created them. Here is a task which transfers data by signaling:
(defpackage :example (:use :cl :lparallel :lparallel.queue)) (in-package :example) (define-condition more-data () ((item :reader item :initarg :item))) (let ((channel (make-channel)) (data (make-queue))) (task-handler-bind ((more-data (lambda (c) (push-queue (item c) data)))) (submit-task channel (lambda () (signal 'more-data :item 99)))) (receive-result channel) (pop-queue data)) ; => 99
receive-result has been placed outside of
task-handler-bind to emphasize that handlers are bundled at the point of
submit-task. (It doesn’t matter where
receive-result is called.)
It may be advantageous to have a kernel devoted to particular kinds of tasks. For example one could have specialized channels and futures dedicated to IO.
(defvar *io-kernel* (make-kernel 16)) (defun make-io-channel () (let ((*kernel* *io-kernel*)) (make-channel))) (defmacro io-future (&body body) `(let ((*kernel* *io-kernel*)) (future ,@body)))
Since a channel remembers its associated kernel,
receive-result do not depend upon the value of
*kernel*. In the promise API, only