tech.v3.parallel.queue-iter
Read an iterator from in a separate thread into a queue returning a new iterator. This allows a traversal of potentially blocking information to happen in a separate thread with a fixed queue size.
default-executor-service
(default-executor-service)Default executor service that is created via 'newCachedThreadPool with a custom thread factory that creates daemon threads. This is an executor service that is suitable for blocking operations as it creates new threads as needed.
iter-fn
(iter-fn item)Create a non threadsafe clojure fn that will iterate the iterator. Once iterator is exhausted further invocations of the function will return nil.
queue-fn
(queue-fn src-fn & [options])Given a clojure fn, create a new thread that will read that fn and place the results into a queue of a fixed size. Returns new fn. Iteration stops when the src-fn returns nil.
Options:
:queue-depth- Queue depth. Defaults to 16.:log-level- When set a message is logged when the iteration is finished.:executor-service- Which executor service to use to run the thread. Defaults to a default one created via default-executor-service.:close-fn- Function to call to close upstream iteration.
queue-iter
(queue-iter iter & [options])Given an object with a conversion to iterator, create a new thread that will read that iterator and place the results into a queue of a fixed size. Returns new iterator.
Options:
:queue-depth- Queue depth. Defaults to 16.:log-level- When set a message is logged when the iteration is finished.:executor-service- Which executor service to use to run the thread. Defaults to a default one created via default-executor-service.