HttpApiClient: Redacted request values appear in plain text in HttpClientError #9838

HttpApiClient drops Redacted from request values before it builds the HttpClientRequest. When the request then fails, the HttpClientError holds the request with every secret in plain text, in the query, the headers and the body. Anything that renders the error shows them: logs, Cause pretty-printing, toJSON.

import { Effect, Layer, Redacted, Schema } from "effect"
import { HttpClient, HttpClientError } from "effect/http"
import { HttpApi, HttpApiClient, HttpApiEndpoint, HttpApiGroup } from "effect/http-api"

const Api = HttpApi.make("api").add(
  HttpApiGroup.make("g").add(
    HttpApiEndpoint.post("join", "/join", {
      query: { token: Schema.RedactedFromValue(Schema.String) },
      headers: { "x-token": Schema.RedactedFromValue(Schema.String) },
      payload: Schema.Struct({ JoinToken: Schema.RedactedFromValue(Schema.String) })
    })
  )
)

// a client whose every request fails
const failing = Layer.succeed(
  HttpClient.HttpClient,
  HttpClient.make((request) =>
    Effect.fail(new HttpClientError.HttpClientError({ reason: new HttpClientError.TransportError({ request }) }))
  )
)

const program = Effect.gen(function*() {
  const client = yield* HttpApiClient.make(Api, { baseUrl: "http://x" })
  const secret = Redacted.make("s3cret")
  const error = yield* Effect.flip(
    client.g.join({ query: { token: secret }, headers: { "x-token": secret }, payload: { JoinToken: secret } })
  )
  console.log(JSON.stringify(error.toJSON()))
})

Effect.runPromise(Effect.provide(program, failing))

The output, trimmed:

"urlParams":{"_id":"UrlParams","params":{"token":"s3cret"}}
"headers":{"content-type":"application/json",...,"x-token":"s3cret",...}
"body":{"_id":"effect/HttpBody","_tag":"Uint8Array","body":"{\"JoinToken\":\"s3cret\"}",...}

The error's message is safe, because it only contains the method and the URL with redacted query values. The problem is the request the error holds.

I found this while building a Docker Engine client. Swarm join tokens, unlock keys and secret data are Redacted in its endpoint schemas, and a failed request put them into the error. That client now strips the request from errors itself.

Cause

The client encodes the query, the headers and the payload with their schemas (HttpApiClient.ts L483–L498). Encoding a Redacted codec produces its plain value, and the client puts the encoded values into the request (L535–L561). After that, core's redaction no longer recognises the secrets:

• UrlParams redacts only values that are still Redacted, or whose names match UrlParams.CurrentRedactedNames. Passing a Redacted straight to HttpClientRequest.setUrlParam stays redacted. The same value sent through HttpApiClient doesn't.
• Headers holds only strings, and redacts by name: authorization, cookie, set-cookie and x-api-key by default.
• HttpBody.Uint8Array decodes JSON and text bodies for inspection (HttpBody.ts L278), so the encoded JSON is shown as it is.

The same applies to a payload sent as URL parameters (GET endpoints) and to form bodies.

Proposal

A value that is Redacted when the caller passes it in should stay redacted in the request the client builds:

• Query: wrap the encoded value of a Redacted field in Redacted again. UrlParams already sends a Redacted value unwrapped and keeps it redacted in inspection.
• Headers: let Headers hold Redacted values the way UrlParams does, or mark those header names as redacted on that request.
• Body: this is the hard part, because the body is serialised bytes. The body could carry a separate inspection-only rendering, with the Redacted fields replaced by <redacted>, and its toJSON would show that instead of the bytes.

Whether to do this in HttpApiClient alone, or to give HttpClientRequest a general way to mark values as redacted, is a maintainer decision. I'm happy to work on a PR.

Out of scope

• Secrets that aren't typed Redacted, such as an environment variable in a JSON body. Those stay the caller's job.