Releases: wlandau/crew
Releases · wlandau/crew
CRAN patch
simplified Shiny/promises docs
crew 0.9.4
- Do not use extended tasks in Shiny vignette.
- Add a new
retry_tasks
argument with defaultTRUE
(#170). - Avoid Base64 encoding functions from
nanonext
Delegate promises
minor patch
crew 0.9.2
- Use
.args
rather than...
inmirai::mirai()
to make sure arguments continue to be passed as local variables inmirai
>= 0.13.1.9012. - Add new controller methods
autoscale()
,descale()
, andstarted()
to facilitate different kinds of Shiny apps. - Deprecate the
scale
andthrottle
methods ofcontroller$promise()
.promise()
now always callsautoscale()
to make sure one and only one auto-scaling loop is running asynchronously. Auto-scaling thus continues even after the promise resolves. - Add a second example vignette that simulates coin flips.
- Add a new
error
argument tocollect()
(#166).
Shiny-related improvements
crew 0.9.1
- Rewrite the async Shiny vignette with
crew
promises and Shiny extended tasks (#157, @jcheng5). - Clarify the intent of
controller$promise(mode = "one")
in the vignette on promises (@jcheng5). - Implement an
error
argument inpop()
which may help with integration withExtendedTask
(@jcheng5). - Handle task errors in the Shiny vignette (@jcheng5).
Promises and backlogged task management for {targets}
crew 0.9.0
- Require
nanonext
>= 0.12.0 andmirai
>= 0.12.0. - Return to always re-launching backlogged inactive workers (#79, shikokuchuo/mirai#95).
- Implement
push_backlog()
andpop_backlog()
to manage cases when it is not desirable to push to saturated controllers (ropensci/targets#1220). - Invisibly return the
mirai
object of a task pushed to the controller. This allows users to interact with the task directly, e.g. to create a promise object withpromises::as.promise()
(#146, @jcheng5). - Add a new
walk()
method for controllers and controller groups to submit tasks in batch and return control immediately without waiting for any task to complete (#148, @jcheng5). - Revive the
collect()
method for popping multiple tasks at once (#148, @jcheng5). - Add controller group methods
nonempty()
,resolved()
,unresolved()
, andunpopped()
to help with #148. - Make the
mirai
dispatcher error message extremely verbose.
Vital safety for workers using SIGTERM
- Configure workers to send themselves a termination signal if the connection to the dispatcher is broken (#141, @psychelzh). Huge thanks to @shikokuchuo for the support through shikokuchuo/mirai#87, shikokuchuo/mirai#88, and shikokuchuo/nanonext#25! The signal itself is platform-dependent and determined by the new function
crew_terminate_signal()
. - Implement
crew_monitor_local()
to help users monitor and terminate local R processes created bycrew
andmirai
. - Implement new utility function
crew_terminate_process()
to terminate a process manually without resorting toSIGKILL
on Windows. - Throw a warning from
controller$map()
if at least one task threw one.warnings = FALSE
suppresses this behavior. - Set
output = TRUE
indaemon()
sostdout
andstderr
streams print. - Add new arguments
local_log_directory
andlocal_log_join
to write to local log files.
Event-driven wait()
crew 0.7.0
- Migrate from
asyncdial
toautoexit
. - Use
Sys.info()[["user"]]
to get the user increw_clean()
(#135, @luwidmer). - Use condition variables to wait for tasks more efficiently (#108).
- Because of #108,
controller$map()
can no longer be used if there are tasks waiting to be popped. - Use a
cli
progress bar inmap()
. - Encapsulate non-function
R6
members in theprivate
list and add active bindings where interfaces are necessary (#137). Ad hoc tests that absolutely need to modify private objects can useobject$.__enclos_env__$private
trick, but these tests should be skipped on CRAN in case there is a change toR6
that breaks this. - Drop
.signal
frommirai::mirai()
since allmirai
tasks signal as ofmirai
version 0.11.2.9025. - Implement
crew_throttle()
, a decoupled mechanism for throttling that can be applied to scaling. - Bring back the
throttle
argument, powered bycrew_throttle()
. - Retry
mirai::status()
again indaemons_info()
and make it configurable usingseconds_interval
andseconds_timeout
in both the client and the launcher (#128).
Local async for plugins, simplification, and performance
crew 0.6.0
- Migrate checks to enforce features in version 0.5.0 for reverse dependencies.
- Drop check of backlogged workers, c.f. shikokuchuo/mirai#63 (comment) (#79, #124, @shikokuchuo).
- Deprecate
seconds_exit
becauseexitlinger
inmirai
is now obsolete (#125, @shikokuchuo). - Use
mirai::nextget("cv")
to count unresolved tasks instead of looping through all the task objects (#131). - Remove throttling and
collect()
in auto-scaling. Simplifies much of the code. Made possible by the efficiency gains in #131. - Simplify
wait()
. seconds_interval
inmap()
no longer defaults tocontroller$client$seconds_interval
.launcher$terminate_workers()
terminates one or more workers, andlauncher$terminate
terminates the whole launcher.- Add infrastructure to let custom launcher plugins launch and terminate workers asynchronously (#133). Launchers can set a positive number in the
processes
field to set the number of localmirai
daemons for asynchronous requests to launch and terminate the serious workers. Then,launch_worker()
andterminate_worker()
can optionally make use oflauncher$async$eval()
to send these asynchronous calls.
Robustness, random numbers, and TLS configuration
crew 0.5.0
- Suppress interactive browser on Windows which launched on each worker previously (@psychelzh).
- Migrate to the new host/daemon nomenclature in
mirai
0.9.1 (#96). - Suppress
status()
retries (@shikokuchuo, #100). - Implement
launch_max
to error out if workers repeatedly launch without completing any tasks (#101, @shikokuchuo, @multimeric). - Detect discovered workers more robustly in
launcher$done()
. - Add a new
algorithm
argument to thepush()
,shove()
, andmap()
methods of controllers and controller groups, as well as arguments/fields increw_eval()
and its return value (#113, @shikokuchuo). - As a default for pseudo-random number generation, leverage widely-spaced L'Ecuyer streams as supported by
mirai::nextstream()
(#115, @shikokuchuo). - Move README documentation to vignettes.
- Add a new
crew_tls()
function for TLS configuration (#120). - Deprecate the
tls_enable
andtls_config
arguments ofcrew_client()
etc. in favor of atls
argument which acceptscrew_tls()
objects (#120).