S3

The transport that is not a mount: serve a driver as an S3-compatible HTTP endpoint.

mountx/s3 speaks enough of the S3 REST API — path-style, SigV4-signed — for rclone, the AWS CLI, an AWS SDK, or a presigned URL in a browser to talk to any FsDriver as if it were a bucket. There is no kernel involved and nothing here produces a mountpoint, which is why it is a separate subpath rather than another choice for mountx/auto: auto's whole contract is "hand back a mounted directory", and a gateway never has one to hand back.

import { createS3Server } from "mountx/s3";
import { createMemoryDriver } from "mountx/drivers/memory";

await using server = await createS3Server(createMemoryDriver()).listen();
server.url; // http://127.0.0.1:<port>

// aws --endpoint-url $server.url s3 ls s3://mountx

#Quick start

#One driver, or several

// One driver, one bucket (default name "mountx"):
const one = createS3Server(driver, { bucket: "photos" });

// Several drivers, several buckets:
const many = createS3Server({ buckets: { photos: photosDriver, notes: notesDriver } });

The two shapes are told apart by the driver side: a value with stat, readdir and open is a driver whatever else it carries; only then is a buckets property looked at. createS3Server() returns immediately — nothing is bound until listen() — but it throws right away for a bucket name that cannot be a URL path segment, or for a host it will not bind.

#rclone

The two flags below are not stylistic — this gateway implements ListObjectsV2 only and has no CreateBucket, and rclone needs to be told both things or its first request fails:

# rclone.conf, or the equivalent RCLONE_CONFIG_MX_* environment variables
[mx]
type = s3
provider = Other
access_key_id = AKIAIOSFODNN7EXAMPLE
secret_access_key = wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
endpoint = http://127.0.0.1:PORT
region = us-east-1
force_path_style = true
list_version = 2
no_check_bucket = true

rclone sync ./photos mx:mountx/photos
rclone lsjson -R mx:mountx

• list_version=2 — rclone picks the list API from the provider: V2 for AWS, V1 for everything else, including Other. This gateway only implements ListObjectsV2, so an unconfigured remote gets 501 NotImplemented: ListObjects (V1) is not implemented by this gateway on its first listing.
• no_check_bucket=true — rclone's S3 backend sends CreateBucket before an upload unless a listing has already marked the bucket as existing; this gateway's buckets are the drivers it was constructed with; there is nothing to create, so it answers NotImplemented and rclone reports failed to prepare upload.
• force_path_style=true — this gateway speaks path-style URLs only (/bucket/key), never virtual-hosted (bucket.host/key).

An empty directory needs both --create-empty-src-dirs (rclone's local backend otherwise drops it before the sync sees it) and --s3-directory-markers (its S3 backend otherwise has no key to represent one) — rclone sync --create-empty-src-dirs --s3-directory-markers ./tree mx:mountx/tree round-trips one, both directions.

#Presigned URLs

mountx/s3 exports the same SigV4 codec the session verifies against, so a presigned URL needs no server:

import { presignRequest, formatAmzDate, uriEncode } from "mountx/s3";

const path = "/mountx/notes/hello.txt";
const signed = presignRequest({
  method: "GET",
  path,
  headers: [{ name: "host", value: "127.0.0.1:PORT" }],
  credentials: { accessKeyId, secretAccessKey },
  region: "us-east-1",
  timestamp: formatAmzDate(Date.now()),
  expiresIn: 300, // seconds
});
const query = signed.query.map((q) => `${uriEncode(q.name)}=${uriEncode(q.value)}`).join("&");
const url = `http://127.0.0.1:PORT${path}?${query}`;

curl "$url" # no Authorization header anywhere — the signature lives in the query

#Who may connect

SigV4 is only authentication when there is a secret to verify against, so the rule is split on whether credentials was passed:

• No credentials — every signature is parsed but never checked, so the bind is loopback-only: a non-loopback host — including the unspecified 0.0.0.0/::, which bind every interface — is refused outright, before a socket opens, with a named S3BindError.
• With { accessKeyId, secretAccessKey } — every request is verified with strict SigV4 (both the header form and the presigned-query form), clock skew bounded to ±15 minutes, and any host is allowed.

The loopback check is literal: localhost, ::1, and any 127.x.y.z (plus the ::ffff: IPv4-mapped spelling a dual-stack listener uses). Nothing resolves a name, so a hostname that happens to point at 127.0.0.1 is still refused — the check is a fact about the address handed in, not about where it currently resolves.

#Semantics

#ETags are derived, not MD5

An object's ETag is the first 32 hex characters of sha256("dev:ino:size:mtimeMs"), suffixed -1. The -1 is the shape a multipart ETag has, which is deliberate: it is the signal to a client that this is not a content hash. rclone reads it that way — rclone check announces it is using MD5, finds none it can verify, and falls back to comparing size and modification time instead, reporting the files as matching and the hashes as "could not be checked" rather than treating the mismatch as a difference. --size-only gets the same result with no hash step logged at all. The ETag is otherwise stable across repeated GETs of the same object.

#mtime, not much else

x-amz-meta-mtime (epoch seconds — a bare integer when whole, three decimals when not) is rclone's own metadata convention, and the only x-amz-meta-* header this gateway keeps. On PUT (and on CompleteMultipartUpload, from the value given to CreateMultipartUpload) it maps to driver.utimes() when the driver declares times; on GET/HEAD it is emitted from stat.mtime. Every other x-amz-meta-* header is dropped. Resolution stops at the millisecond — the header has no room for more — so a file with sub-millisecond mtime precision re-transfers on the next rclone sync unless it is run with --modify-window 1ms.

Content-Type is not stored at all: every GET/HEAD answers application/octet-stream.

#Directories are prefixes

There is no directory object except for an empty one. PUT key/ (empty body) is a recursive mkdir; DELETE key/ is an rmdir (ENOTEMPTY → 409 BucketNotEmpty). A directory with children needs no marker — its children are the listing — but an empty one would otherwise vanish from every listing, so it lists as a zero-byte object with key key/. That is exactly what lets sync recreate it, and exactly what rclone's --s3-directory-markers flag asks for.

#The NotImplemented boundary

The capabilities philosophy applies to the wire as much as to a driver: an operation this gateway does not implement answers a well-formed S3 NotImplemented (501) error document, never a fall-through to a neighbouring one — GET /bucket/key?acl is refused, and must never quietly answer the object's bytes because the sub-resource went unrecognized. CreateBucket/DeleteBucket answer the same way; bucket existence here is "was a driver given to createS3Server()", not something a request can create. ?versioning is refused too, which is what makes rclone purge log Failed to read versioning status, assuming unversioned and then carry on correctly.

Supported: ListBuckets, HeadBucket, ListObjectsV2, GetObject, HeadObject, PutObject, DeleteObject, DeleteObjects, CopyObject, and the five multipart operations below. Conditionals (If-Match/If-None-Match/If-Modified-Since/If-Unmodified-Since, 304/412) and a single Range (416 when unsatisfiable) are implemented on GET/HEAD.

#Multipart

CreateMultipartUpload, UploadPart, CompleteMultipartUpload, AbortMultipartUpload and ListParts stage their parts through the driver, under a reserved bucket-root prefix, .mountx-multipart/<uploadId>/part-<N> — invisible to every other operation (a direct GET/HEAD/PUT of a staging key answers as though it did not exist, and it is skipped in every listing). CompleteMultipartUpload streams the staged parts together in the order the client listed them, because parts arrive out of order and out of size, so the offset of part N is unknowable until every part before it exists. Aborting an upload, and closing the server, both sweep the staging area; an interrupted upload leaves no debris behind once either happens.

#The write-in-place caveat

A PUT (and multipart completion) writes the object in place: there is no temporary file and no rename, because the driver interface has no atomic-create primitive to build one on. What is guaranteed is the first byte — the destination is not opened, so an existing object is not truncated and a new one is not created, until the first payload byte has arrived (and, for a signed aws-chunked body, been verified). An upload rejected at or before its first byte therefore leaves the bucket exactly as it was; one that fails after writing has begun leaves whatever had been written by then — a partial object where a whole one used to be, not the old object restored.

#Serving without mounting, on purpose

There is nothing platform-specific here: createS3Server is node:http, and Node's HTTP server runs wherever Node runs. Nothing in this transport has been exercised on Windows, but nothing in it depends on anything Linux- or macOS-specific either — it is an HTTP server serving an already-portable driver interface.

#createS3Server(source, options?)

function createS3Server(
  source: FsDriver | { buckets: Record<string, FsDriver> },
  options?: S3ServerOptions,
): S3Server;

#S3ServerOptions

Extends S3SessionOptions, so everything the session takes is settable here too.

#S3Server

interface S3Server extends AsyncDisposable {
  readonly session: S3Session;
  readonly host: string;
  readonly port: number;
  readonly url: string; // e.g. "http://127.0.0.1:54321"; IPv6 bracketed
  readonly buckets: string[];
  readonly connections: number;
  listen(): Promise<S3Server>; // idempotent; resolves once bound
  close(): Promise<void>; // stop accepting, drain, drop, sweep multipart staging — idempotent
}

close() drains in a deliberate order: stop accepting, let in-flight responses finish (bounded by drainTimeout), drop what's left, and only then sweep the multipart staging area — sweeping first could race a part still being written by a request that was allowed to finish.

#S3BindError / isS3BindError() / isLoopbackHost()

class S3BindError extends Error {
  readonly code: "ERR_S3_BIND";
  readonly host: string;
}
function isS3BindError(error: unknown): error is S3BindError;
function isLoopbackHost(host: string): boolean;

The one error type createS3Server() throws for an address, named so it can be caught rather than pattern-matched on a message. isLoopbackHost() is the literal check described above, exported on its own because it is worth asking without constructing a server first.

#S3Session

new S3Session(buckets: Record<string, FsDriver>, options?: S3SessionOptions)

One HTTP request in, one S3 reply out, with no socket anywhere — which is what makes the whole protocol testable with no listener and no client, the same posture FuseSession and NfsSession keep.

session.buckets; // Map<string, Loopback>
session.bucketNames; // string[], in listing order
session.stats; // { requests, replies, errors, operations: Map<string, number>, assertions }
await session.handleRequest(head, body?); // → S3StreamResponse; never rejects
await session.close(); // idempotent; sweeps multipart staging

handleRequest's boundary is streaming, unlike NfsSession.handleCall(bytes): the body in and the body out may each be an AsyncIterable<Uint8Array>, because buffering a multi-gigabyte PUT to parse it — or a multi-gigabyte GET to answer it — is not something a gateway may do.

#S3SessionOptions

#The layers below

Everything except server.ts touches no socket and no listener, which is what makes the protocol testable by a signing JS client built out of the same codecs (test/s3/client.ts, the conformance column's transport):

All of it is re-exported from mountx/s3 by name, except the generic XML layer xml.ts is itself built on (XmlNode, xmlDocument, parseXml, and the rest) — primitives rather than anything a consumer of an S3 gateway composes with, the same treatment mountx/nfs gives its own sub-struct helpers.

#Not available

• CreateBucket/DeleteBucket, bucket ACLs and policies, object ACLs, versioning, and every list-type=1 request. All refused with a well-formed NotImplemented rather than a fake answer — see the boundary above.
• Symlinks, hardlinks, permissions and access time. An S3 object has none of these, so there is nothing for the gateway to carry even where the driver underneath has one.
• Windows, not because anything here is platform-specific, but because it has not been run there.

#Next

• FUSE and NFSv3 — the two transports that produce a mountpoint.
• mountx/auto reference — the chooser that picks among the three mount transports, and why S3 is not an option there.