Skip to content

Daemon cleanup callback #586

Closed
Closed

Description

@king-of-poppk

I am looking for a way to run some "cleanup logic" after a mirai has completed. Obviously this cleanup logic has to run on the same daemon that executed the mirai. Is there such a functionality? Is there the ability to run a mirai on a specific daemon? How can one identify daemons?

With everywhere it is possible to run the cleanup logic on all daemons, but even with .min == 1, all existing daemons are waited on which introduces unwanted delays.

I can also obviously collocate the cleanup logic with the rest of the logic in a single mirai, but the reason I am trying to split this up is because that cleanup logic takes some time and I want to be able to do something early with the result while the cleanup runs in the background.

I have also tried collocating but setting up the cleanup logic via later::later so as to not block on it, but then it seems I still need a way to trigger later::run_now() on that specific daemon and then this is the same problem as above where you replace "cleanup logic" with "later::run_now() callback", since later::run_now() is blocking...

A different way to solve the problem would be to figure out why later::later() callbacks do not run automatically in the background once the daemon has returned. But I don't know how this can be done.

Activity

  1. shikokuchuo commented on Apr 28, 2026

    @shikokuchuo
    Member

    You'd need to call later::run_now() for your later callbacks to run as the daemon is never idle at the top level. Calling that before every run would work if that's an option, but obviously that adds time to the start of your call.

    Another possibility to consider if cleanup is heavy is to just have single task daemons, e.g. daemons(n, maxtasks = 1).

    Other than that I don't think you can get around co-locating the cleanup code.

  2. king-of-poppk commented on Apr 28, 2026

    @king-of-poppk
    Author

    daemons(n, maxtasks = 1) would lead to another question which is: "How do I reduce each worker's startup time". I think I have a lot of work to do there, in particular renv is taking a lot of time to do its startup check. With callr I could easily ignore .Rprofile and assign .libPaths() to copy the host's .libPaths() but with mirai it seems there is no such flexibility.

  3. shikokuchuo commented on Apr 28, 2026

    @shikokuchuo
    Member
  4. king-of-poppk commented on Apr 28, 2026

    @king-of-poppk
    Author

    Is it possible to configure this per .compute?

  5. king-of-poppk commented on Apr 28, 2026

    @king-of-poppk
    Author

    And actually, now I realize maxtasks=1 may not cut it, because a lot of the initialization cost is not renv, but booting up heavy callr sessions inside the daemon. But I'll try it out anyway. I just think that the daemon running later::run_now() when idle would suffice to solve my problem, so is there an intrinsic limitation to doing that due to mirai's architecture?

  6. shikokuchuo commented on Apr 28, 2026

    @shikokuchuo
    Member

    A daemon is never idle in the same way as we might be idle at a REPL, it's always waiting for the next task.

  7. locked and limited conversation to collaborators on Apr 28, 2026
  8. converted this issue into a discussion #593 on Apr 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions