serveIsolates function

Future<ServerIsolates> serveIsolates(
  1. RouterFactory factory,
  2. InternetAddress address,
  3. int port, {
  4. required int isolates,
  5. void onIsolateError(
    1. Object? error,
    2. Object? stackTrace
    )?,
})

Serves factory from isolates isolates, all sharing one port.

Dart runs one isolate on one thread, so a single server uses one core, and isolates do not share memory. Using the rest of the machine means running the server several times over rather than adding threads to it.

serve on its own uses one core. On a four-core machine that leaves three idle under any load, and nothing reports it — the throughput ceiling looks like the application's rather than the process's.

This is what uvicorn --workers and gunicorn -w do for Python, for the same reason: the runtime cannot spread one server across cores, so it is run several times behind a shared socket. A threaded runtime needs none of it, so there is no equivalent to copy; this is here for the isolate model, not as a convenience on top of serve.

The operating system load-balances accepted connections across sockets bound with shared: true, which is what lets several isolates answer one port.

Router buildApp() => Router()..route('/', get(home));

void main() async {
  final cluster = await serveIsolates(
    buildApp,
    InternetAddress.anyIPv4,
    8080,
    isolates: Platform.numberOfProcessors,
  );
}

State does not cross isolates. Anything shared, a cache or a counter, has to live outside the process; each isolate gets its own copy of whatever factory builds.

Implementation

Future<ServerIsolates> serveIsolates(
  RouterFactory factory,
  InternetAddress address,
  int port, {
  required int isolates,
  void Function(Object? error, Object? stackTrace)? onIsolateError,
}) async {
  if (isolates < 1) {
    throw ArgumentError.value(isolates, 'isolates', 'must be at least one');
  }

  final local = await serve(factory(), address, port, shared: true);

  final spawned = <Isolate>[];
  final handles = <SendPort>[];
  final workers = <_Worker>[];

  for (var i = 1; i < isolates; i++) {
    final worker = _Worker(i);
    final isolate = await Isolate.spawn(
      _serveInIsolate,
      _IsolateSeed(factory, address, local.port, worker.ready.sendPort),
      debugName: 'dust_server isolate $i',
      onExit: worker.exits.sendPort,
      onError: worker.exits.sendPort,
    );
    worker.isolate = isolate;

    final SendPort handle;
    try {
      handle = await worker.started.future;
    } on Object {
      // Nothing the caller knows about is serving yet, so leaving the local
      // server bound and the earlier isolates alive would leak a port and a
      // heap each.
      worker.dispose();
      isolate.kill(priority: Isolate.immediate);
      for (final earlier in workers) {
        earlier.dispose();
        earlier.isolate?.kill(priority: Isolate.immediate);
      }
      await local.close(drain: Duration.zero);
      rethrow;
    }

    handles.add(handle);
    worker.ready.close();
    spawned.add(isolate);
    workers.add(worker);
  }

  handles.add(_localHandle(local));
  final cluster = ServerIsolates._(spawned, handles, address, local.port)
    .._workers.addAll(workers);

  // An isolate that dies afterwards is otherwise invisible: the port stays
  // bound by the survivors, so traffic keeps flowing at reduced capacity with
  // nothing to say so. This cannot restart it — a replacement cannot rebind
  // the socket — but it can stop the loss being silent.
  for (var i = 0; i < workers.length; i++) {
    final index = i;
    workers[i].onDeath = (message) {
      cluster._dead.add(index);
      if (message is List && message.length == 2) {
        onIsolateError?.call(message.first, message.last);
      }
    };
  }

  return cluster;
}