- added readers
- added restart
kill-errorsto workers — removes debugger popups
- attempting to submit a task to an ended kernel now signals an error
- suicidal calls to
kill-tasksinside workers are now permitted
Below is another example of Concurrent Hello World. The macros
run are just 9 lines each, given below the fold.
(defpackage :example (:use :cl :node)) (in-package :example) (defun hello (hello-queue world-queue) (receive hello-queue (:say-hello (n) (cond ((plusp n) (format t "Hello ") (send world-queue :say-world n)) (t (send world-queue :quit) (return)))))) (defun world (world-queue hello-queue) (receive world-queue (:say-world (n) (format t "World!~%") (send hello-queue :say-hello (- n 1))) (:quit () (return)))) (defun main (n) (let ((hello-queue (make-queue)) (world-queue (make-queue))) (run (make-node 'hello hello-queue world-queue) (make-node 'world world-queue hello-queue) (send hello-queue :say-hello n))))
The major version bump is for some incompatible changes, though they are unlikely to cause trouble.
- keyword arguments to
:keyhave been replaced with a single
:granularityargument; the old arguments are now ignored
- removed deprecated aliases from 1.2.0 and 1.3.0 (you may not be aware of them since they haven’t been listed in the documentation)
- A function defined with
defpunis now optimized for N worker threads where N is the number of cores. The old behavior is available with
defpun*, which defines a function that is optimized for N-1 workers (and has less overhead).
psortbut targets N-1 workers
- improved performance of
- task categories are now compared with
eql; same for ptree node ids
The benchmarks page has been updated to reflect recent optimizations, which includes optimizations in SBCL itself.
- The latest CLISP and ECL appear close to passing all tests, and may indeed pass on some platforms.
- I am not aware of any outstanding CL implementation bugs affecting lparallel besides those reported in CLISP and ECL. I am also not aware of any bugs in lparallel itself.
- If you happen to be using bignums on 32-bit x86 CCL 1.8 or earlier, you should get latest from the CCL repository and build a new image.
- lparallel does not seem complex enough to warrant a mailing list, yet some things may not be entirely simple either. Feel free to ask questions or offer feedback on this thread, or send me email.
- I plan to remove some old deprecated aliases in version 2.0 (they are not shown in the documentation here). Now is the time to suggest incompatible changes, before the 2.0 bump.
- I have been hesitant to add a nickname to the lparallel package. The separation of the lparallel API into a handful of packages was meant to encourage people to use a subset of the API, e.g.
(:use :lparallel.cognate). However some people always write package-qualified symbols, and for them an
llnickname would be convenient. I am not exactly against this, but it does entail a bit of peril in the form of increased likelihood of conflict.
- I have noticed this pattern being used:
(let ((*kernel* (make-kernel ...))) ...). This is not recommended for three reasons. First, it makes the kernel object inaccessible to other (non-worker) threads, preventing the use of
kill-tasksin the REPL for example. Second,
end-kernelis likely to be forgotten, resulting in a kernel that is not garbage collected. Third, even if we properly abstract this pattern by writing a
with-temp-kernelmacro that calls
end-kernel, such a macro lends itself to suboptimal code because multiple uses of it would defeat the benefits of a thread pool. These issues are avoided by calling
(setf *kernel* ...)or by binding to an existing kernel, for example
(let ((*kernel* *io-kernel*)) ...).
with-temp-kernelmacro may still be convenient in non-critical cases such as testing, yet I hesitate to include it in the lparallel API for the reasons mentioned above.
- optimized cognate functions and macros when they are called inside worker threads; e.g.
(future (pmap ...))no longer blocks a worker
clear-ptree-errors— for resuming after an error
clear-ptree— for recomputing from scratch
- improved task handling for ptrees
defpunno longer transforms
When an evaluation fails or is interrupted, it may be convenient to automatically kill tasks created during the evaluation. One use for this might be for debugging a set of long-running tasks. Here is a solution using alexandria’s
(defpackage :example (:use :cl :lparallel :alexandria)) (in-package :example) (defun call-with-kill-on-abort (fn task-category) (let ((*task-category* task-category)) (unwind-protect-case () (funcall fn) (:abort (kill-tasks task-category))))) (defmacro with-kill-on-abort ((&key (task-category '*task-category*)) &body body) `(call-with-kill-on-abort (lambda () ,@body) ,task-category)) (defun foo () (with-kill-on-abort (:task-category 'foo-stuff) (pmap nil #'sleep '(9999 9999))))
Example run in SLIME:
CL-USER> (example::foo) ; ... then hit C-c-c WARNING: lparallel: Replacing lost or dead worker. WARNING: lparallel: Replacing lost or dead worker. ; Evaluation aborted on NIL.
As always, worker threads are regenerated after being killed.