How to schedule existing Fibers

[sled]

Hello there,

I'm currently working with an API client library that returns fibers for API request (https://github.com/microsoftgraph/msgraph-sdk-ruby). Now I'm wondering how I'd schedule those fibers using Async.

Here's a simplified synchronous example to fetch users and its calendar events:

# Need to get the users synchronously first
users_fiber = ApiClient.users.all
users = users_fiber.resume

users.each do |user| # this loop could run concurrently
  events_fiber = user.calendar_events.all
  events = events_fiber.resume
  # do something with events
end

How would I run this asynchronously with Async? Here's what I came up with, but it doesn't feel right:

# Need to get the users synchronously first
users_fiber = ApiClient.users.all
users = users_fiber.resume

scheduler = Async::Scheduler.new
Fiber.set_scheduler(scheduler)

users.each do |user| 
  Fiber.schedule do
    events_fiber = user.calendar_events.all
    events = events_fiber.resume
     # do something with events
  end
end

scheduler.run
Fiber.set_scheduler(nil)

I'm not sure whether Fiber.schedule { other_fiber.resume } is correct and whether it's a good idea to manage the scheduler this way i.e. calling Fiber.set_scheduler(...) and calling scheduler.run manually.

[ioquatix]

When running in an event loop, all fibers will run concurrently, even those created with Fiber.new. However, you won't get any of the other features of Async when using such fibers (e.g. Async::Task#wait and #stop). Therefore, you shouldn't need to change your code at all. If you resume a fiber, and it performs a blocking operation, and there is an active scheduler, it will use the event loop for concurrent execution.

[sled]

@ioquatix thanks for your reply! The API client prepares the request and yields before executing it, so resuming the fiber will run a blocking HTTP request and return the parsed response.

I'm still struggling a bit getting used to Async because there are multiple ways to achieve the same thing.

I was wondering whether using an Async { } block is more sensible, because then I don't have to manage the state of Fiber.scheduler and error handling if one of the fibers fail might be easier. However should I use a barrier or just wait on all the tasks? E.g.:

barrier = Async::Barrier.new

Async do
  users.each do |user|
    barrier.async do
      events_fiber = Demo.client.users.by_user_id(user.id).calendar_view.get(request_configuration)
      events = events_fiber.resume
      # Do something with events
    end
  end
  barrier.wait
end

versus

Async do
  users.map do |user|
    Async do
      events_fiber = Demo.client.users.by_user_id(user.id).calendar_view.get(request_configuration)
      events = events_fiber.resume
      # Do something with events
    end
  end.map(&:wait)
end

Because this will be a Map-Reduce kind of operation, I think using Async is better than Sync correct?