FetchableDevEnvironment API

[sheremet-va]

There is a growing need to standardise the API around environment communication. Even the default SSR environment currently doesn't expose any methods to run the code.

We have decided to implement RunnableDevEnvironment for the default SSR environment that exposes the runner and import directly to decrease the boilerplate (no need to run createModuleRunner manually):

const ssrEnvironment = server.environments.ssr
const entry = await ssrEnvironment.import('/entry-point.js')
await entry.handle(request)

const module = ssrEnvironment.runner.evaluatedModules.getModuleByUrl('/entry-point.js')

module.exports === entry

Taking this idea further, we think it would be good for the ecosystem to have a standard way to run the code via the Fetch API. This is already how most third-party tools operate, and we have seen more and more requests to support this out of the box. This is where the new FetchableDevEnvironment comes in.

FetchableDevEnvironment is a DevEnvironment that has a dispatchFetch method:

class FetchableDevEnvironment extends DevEnvironment {
  dispatchFetch(request: Request): Promise<Response>
}

FetchableDevEnvironment class is not exposed (only its type). The class is considered an implementation detail and instead, we will expose a createFetchableDevEnvironment function to create the environment like we already do with createServer. You can call it inside the environments.dev.createEnvironment config option:

import { createServer, createFetchableDevEnvironment } from 'vite'

const server = await createServer({
  environments: {
    edge: {
      createEnvironment(name, config) {
        return createFetchableDevEnvironment(name, config, {
          requestHandler(req: Request): Promise<Response> {
            // ... process request and return a response
          },
        })
      },
    },
  },
})

const response: Response = await server.environments.edge.dispatchFetch(
  new Request(convertToUrl('/entry-point.js')),
)

dispatchFetch is not called anywhere by Vite itself. The expectation is that frameworks can use this to run the code in the adapter's environment:

import { isFetchableDevEnvironment, isRunnableDevEnvironment } from 'vite'

function handleRequest(environment: DevEnvironment, request: Request) {
  if (isFetchableDevEnvironment(environment)) {
    return await environment.dispatchFetch(request)
  }
  else if (isRunnableDevEnvironment(environment)) {
    const entry = await environment.import('/entry-point.js')
    return await entry.default.fetch(request)
  }
  // ignore browser - fail on others
  else if (environment.consumer !== 'client') {
    throw new Error(`the environment "${environment.name}" doesn't support "dispatchEvent" or "import" API`)
  }
}

We are also thinking about providing utility functions to convert the http.IncommingMessage to Request and http.ServerResponse to Response. We might also need a function to combine Response + http.ServerResponse.

import { toFetchRequest, combineResponses } from 'vite' // name suggesions are welcome

server.middlewares.use(async (req, resp) => {
  const request = toFetchRequest(req)
  const response = await handleRequest(server.environments[condition], request)
  combineResponses(resp, response)
})

Fetchable Module Runner API

To make things easier on the module runner side, we also want to introduce a createFetchableModuleRunner function to create a runner that uses global fetch to fetch the module information. If global fetch is not available, the createFetchableModuleRunner will throw an error.

import { createFetchableModuleRunner } from 'vite/module-runner'

const runner = createFetchableModuleRunner({
  serverUrl: 'http://localhost:3000',
  environmentName: 'edge',
  hmr: createHmrConnection(), // custom function, no changes here
})

export default {
  async fetch(request) {
    const entry = await runner.import('/entry-point.js')
    return entry.handle(request)
  },
}

To achieve this, Vite implements a middleware that intercepts all requests to /@vite/import/{env}/{url} (the naming is based on already established /@vite/client and /@vite/env) and responds with the result of environment.fetchModule back to the runner:

// very simple example of an internal implementation, error handling omitted

server.middlewares.use(async (req, resp) => {
  if (req.url.startsWith('/@vite/import/') {
    const { environmentName, moduleUrl, importer } = parseUrl(req.url)
    const moduleInfo = await server.environments[environmentName].fetchModule(moduleUrl, importer)
    resp.status(200).set({ 'Content-Type': 'application/json' }).end(JSON.stringify(moduleInfo))
  }
})

Default SSR environment as FetchableDevEnvironment

To reduce boilerplate even further, we could also expose the dispatchFetch method on the default SSR environment. At the minimum, this would require:

• entryPoint option in the SSR section of the config
• standardised interface for the entry point (export default { fetch })

The internal implementation might look something like this:

const environment = new FetchableEnvironment(name, config, {
  async handleRequest(request) {
    const entry = await environment.import(environment.config.entryPoint)
    const response = await entry.default.fetch(request)
    validateResponse(response)
    return response
  },
})

await server.environments.ssr.dispatchFetch(
  new Request(convertToUrl('/entry-point.js'))
)

Currently, we don't have plans to do this, but any feedback regarding this is welcome.

Previous discussions:

• Environment API
• Runtime API PR
• Runtime API Feedback discussion
• Initial Docs PR
• Initial Implementation PR

[Rich-Harris]

This looks great! Especially the createFetchableModuleRunner stuff.

Is the hmr option required? If we can rely on fetch being available, then couldn't that be used to notify the client of updates (e.g. via a stream) and the server of errors (e.g. via a POST request)? I'm sure it's more complicated than that, just trying to understand.

The purist in me wants to avoid making environments.ssr 'special' by making it a RunnableDevEnvironment. It feels like being stricter about the separation between environment and runner would make the API easier to learn in the long run (otherwise the relationship is somewhat less clear), and avoid the footgun described in the docs:

One of the problems that was exposed with vite.ssrLoadModule is over-reliance on the server state inside the processed modules

I appreciate that such a restriction would make migrations from Vite 5 more awkward though, and possibly prevent other valid use cases.

[patak-dev]

The purist in me wants to avoid making environments.ssr 'special' by making it a RunnableDevEnvironment

I thought the same before, but once we have the concept of RunnableDevEnvironment, we can avoid the ssr environment being special by making every environment be a RunnableDevEnvironment by default. Before, if you wanted a new environment named server, you were forced to configure it with a dev: { createEnvironment } and create a ModuleRunner. Now just adding environments: { server: {} } would give us a ready-to-use environment. The simplicity of ssrLoadModule() helped quite a bit. I'm leaning towards having the default be as easy to use to end up helping more than it hurts.

[patak-dev]

@sheremet-va maybe we don't expose environment.import() and force users to be more explicit with environment.runner.import()?

[sheremet-va]

@sheremet-va maybe we don't expose environment.import() and force users to be more explicit with environment.runner.import()?

We can discuss this in a runnable PR 😄

But I would also prefer having a single function.

[sheremet-va]

If we can rely on fetch being available, then couldn't that be used to notify the client of updates (e.g. via a stream) and the server of errors (e.g. via a POST request)? I'm sure it's more complicated than that, just trying to understand.

I did not consider that at all, to be honest. We can always experiment with custom HMR implementations. I am just not sure if every runtime supports streams and such 🤔 Either way, I would love to provide a default HMR connection that just works.

[jacob-ebey]

How do I dispatch a http://localhost:5173/some/sub/route?a=a&b=b to /entry-point.js? convertToUrl('/entry-point.js', "/some/sub/route?a=a&b=b")?

[jamesopstad]

There's a lot to read here but it seems most of the discussion so far has focused on two aspects of the proposal.

1. The dispatchFetch method in the FetchableDevEnvironment interface and what its parameters should be:

interface FetchableDevEnvironment {
  dispatchFetch(request: Request): Promise<Response>
}

2. A convention for entry modules and whether this should be prescribed:

export default {
  async fetch(request: Request) {
    // Returns `Response`
  }
}

Both of these were originally proposed by @jacob-ebey in the Discord thread that started discussion about introducing a standard request based API.

We support the dispatchFetch signature that was originally proposed ((request: Request) => Promise<Response>) rather than the newly proposed signature ((entry: string, request: Request) => Promise<Response>). The newer proposal does not support all of our use cases at Cloudflare. I shared the reasoning in this comment - #18191 (reply in thread) - but can elaborate more if it's not clear. Aside from this, in my view the arguments for including the entry here haven't yet made a compelling case for why it's necessary.

Regarding a standardised module format, I don't have strong views about this and think there are legitimate arguments on both sides. Jacob originally argued (again on Discord) that the FetchableDevEnvironment proposal was dependant on it but this doesn't necessarily have to be the case. We will be using the default export convention in the example above because it aligns with the entry format for Cloudflare Workers. I believe that part of the motivation for considering this more generally is that frameworks are increasingly standardising on using the Web Fetch API in their own request handlers. It would be great to hear from a broader range of framework and runtime authors on this topic.

[benmccann]

We have used the Web Fetch API for quite awhile. There have been a few drawbacks we've faced such as many platforms not supporting early hints, web sockets, etc. but overall have been happy with there being an emerging standard

[jacob-ebey]

@jamesopstad I'd appreciate an expansion on what breaks and why for your use-cases as I just don't see it.

[jamesopstad]

@jacob-ebey, I think if you try to get the fixture in your repo working then it will make more sense. I'll try and explain in more general terms though so that it makes sense to others.

The core of what we're discussing is standardising on web requests/responses as a method for interacting with environments. One thing I really like about this is that it can make the code that frameworks write for their development server much closer to the code that is used for production. In production, the request comes in and there is some sort of wrapper that directs it to the relevant build output code. In development, the request comes in and there is some sort of wrapper that directs it to the relevant development code imported by the module runner. In the ideal case, the same entry file can be used for development and for generating the build output.

The proposed dispatchFetch method is a way to send a request from Vite to the environment in development. It doesn't necessarily have to be the only way though. Platform features might require that environments also hold references to each other that allow them to communicate directly. This is the case for Service Bindings in Cloudflare Workers. If worker A contains a handler like the following then the WORKER_B service binding means that Miniflare internally calls the fetch handler in the worker B environment:

export default {
  async fetch(request, env, ctx) {
    const fetchResponse = await env.WORKER_B.fetch(request);
    const fetchResult = await fetchResponse.json();

    return Response.json({
      name: 'Worker A',
      worker_b_fetch_result: fetchResult,
    });
  },
}

The entry path must already be defined in the worker B environment or it wouldn't know what code should be called to handle the request.

I hope this explains why I feel that the (request: Request) => Promise<Response> signature is a better fit for the dispatchFetch method: a) it more closely mirrors production b) the entry path must already be initialised for other use cases such as service bindings etc.

[jacob-ebey]

I'll get bindings working if they aren't already. Should be simple in the setup I already have.

[benmccann]

fetch is sort of a confusing name to me. I guess the idea is that you can call it with the fetch API? But it's weird to name an API with the same name as the API that calls it. Doubly so when the request signatures differ. And this API is not actually fetching anything. Not to mention that you can call web APIs with wget/curl or any number of APIs besides fetch. Perhaps something like handleRequest might be a clearer name for the method than fetch

[sheremet-va]

The naming in the proposal is indeed based on the Fetch API because we rely on Request and Response from that API, so it makes sense to me personally.

[j4w8n]

I don't yet completely understand all of this, but I'm liking it. I'm currently building a plugin, which is a directory-based API router that should support multiple runtimes; and, since this is a fresh project, I'm using the v6 beta.

The proposed helper functions, to convert requests and responses, would be great. As of now, I've stolen SvelteKit's versions of these, via their getRequest and setResponse utilities (ref).

As for using a FetchableDevEnvironment for ssr by default: Seems like a decent idea, but I didn't understand why the example implementation shows using a config option to acquire the entry point file, but then the public dispatchFetch call still has to declare it. I'm probably just missing some context - I'm new:

const environment = new FetchableEnvironment(name, config, {
  async handleRequest(request) {
    const entry = await environment.import(environment.config.entryPoint)
    const response = await entry.default.fetch(request)
    validateResponse(response)
    return response
  },
})

await server.environments.ssr.dispatchFetch(
  new Request(convertToUrl('/entry-point.js'))
)

[sheremet-va]

convertToUrl is just an example. You would normally get the request from a server middleware.

[magne4000]

To add my 2 cents to the discussion, I think that this proposal would benefit from more standard middlewares.
Having Vite middlewares complying with Connect syntax, while also having other parts of Vite using the Web Fetch API feels a bit messy.

What I would like to be able to do:

server.middlewares.use(async (request: Request) => {
  // We can directly reuse `request` in `dispatchFetch`
  return new Response(...);
});

I created universal-middleware for this kind of purpose, where we could have support for both new and old syntax.
This lib allows one to write middlewares using this universal syntax, while also providing wrappers to directly target popular servers (like @universal-middleware/express, that would convert the middleware above to an express one).

The new syntax would internally be wrapped using @universal-middleware/express to avoid breaking changes. But it could transition to fully drop Connect in the future IMO.