S3

Serve a driver through an S3-compatible HTTP endpoint. This transport does not create a mount.

mountx/s3 implements the required parts of the path-style S3 REST API with Signature Version 4 (SigV4). It lets rclone, the AWS CLI, an AWS SDK, or a presigned URL in a browser access an FsDriver as a bucket.

This transport does not use a kernel filesystem client or create a mount point. Therefore, it has a separate subpath and is not a choice for mountx/auto. Auto must return a mounted directory, but a gateway does not create one.

import { createS3Server } from "mountx/s3";
import { createMemoryDriver } from "mountx/drivers/memory";

await using server = await createS3Server(createMemoryDriver()).listen();
server.url; // http://127.0.0.1:<port>

// aws --endpoint-url $server.url s3 ls s3://mountx

#Quick start

#One driver, or several

// One driver, one bucket (default name "mountx"):
const one = createS3Server(driver, { bucket: "photos" });

// Several drivers, several buckets:
const many = createS3Server({ buckets: { photos: photosDriver, notes: notesDriver } });

The function identifies the two input shapes from the driver side. A value with stat, readdir, and open is a driver, even if it also has a buckets property. Otherwise, the function checks for buckets.

createS3Server() returns before it binds a socket. Binding starts only when you call listen(). The function immediately rejects a bucket name that cannot be a URL path segment. It also rejects a host that it cannot bind.

#rclone

Set the following options before you use rclone. The gateway implements only ListObjectsV2, does not implement CreateBucket, and requires path-style URLs. Without these settings, the first request can fail:

# rclone.conf, or the equivalent RCLONE_CONFIG_MX_* environment variables
[mx]
type = s3
provider = Other
access_key_id = AKIAIOSFODNN7EXAMPLE
secret_access_key = wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
endpoint = http://127.0.0.1:PORT
region = us-east-1
force_path_style = true
list_version = 2
no_check_bucket = true

rclone sync ./photos mx:mountx/photos
rclone lsjson -R mx:mountx

• list_version=2: rclone selects the list API from the provider: V2 for AWS, V1 for everything else, including Other. This gateway only implements ListObjectsV2, so an unconfigured remote receives 501 NotImplemented: ListObjects (V1) is not implemented by this gateway on its first listing.
• no_check_bucket=true: without this flag, rclone sends CreateBucket before an upload unless a previous listing found the bucket. This gateway creates buckets from the drivers passed to its constructor. A request cannot create another bucket. The gateway returns NotImplemented, and rclone reports failed to prepare upload.
• force_path_style=true: this gateway speaks path-style URLs only (/bucket/key), never virtual-hosted (bucket.host/key).

To copy an empty directory, use both --create-empty-src-dirs and --s3-directory-markers. Without the first flag, the local backend drops the directory before sync. Without the second, the S3 backend has no key to represent it. The command rclone sync --create-empty-src-dirs --s3-directory-markers ./tree mx:mountx/tree preserves an empty directory in both directions.

#Presigned URLs

mountx/s3 exports the same SigV4 codec the session verifies against, so a presigned URL needs no server:

import { presignRequest, formatAmzDate, uriEncode } from "mountx/s3";

const path = "/mountx/notes/hello.txt";
const signed = presignRequest({
  method: "GET",
  path,
  headers: [{ name: "host", value: "127.0.0.1:PORT" }],
  credentials: { accessKeyId, secretAccessKey },
  region: "us-east-1",
  timestamp: formatAmzDate(Date.now()),
  expiresIn: 300, // seconds
});
const query = signed.query.map((q) => `${uriEncode(q.name)}=${uriEncode(q.value)}`).join("&");
const url = `http://127.0.0.1:PORT${path}?${query}`;

curl "$url" # no Authorization header anywhere — the signature lives in the query

#Who may connect

SigV4 authenticates a request only when the server has a verification secret. Connection rules therefore depend on whether you pass credentials:

• No credentials: the server parses but does not verify signatures. It can bind only to loopback. Before opening a socket, it rejects a non-loopback host with S3BindError. This includes 0.0.0.0 and ::, which bind every interface.
• With { accessKeyId, secretAccessKey }: strict SigV4 verifies every request. Verification supports header and presigned-query forms and allows ±15 minutes of clock skew. Any host is permitted.

The loopback check compares literal addresses: localhost, ::1, and any 127.x.y.z (plus the ::ffff: IPv4-mapped spelling a dual-stack listener uses).

The check does not resolve hostnames. Therefore, it refuses a hostname even when the name currently points to 127.0.0.1. It checks the supplied address, not its current destination.

#Semantics

#ETags are derived, not MD5

The entity tag (ETag) of an object is the first 32 hex characters of sha256("dev:ino:size:mtimeMs"), suffixed -1.

The -1 suffix gives the ETag the shape of a multipart ETag. This signals to a client that the value is not a content hash. rclone interprets the value this way. rclone check first reports that it is using MD5. It finds no hash that it can verify, so it compares size and modification time instead. Matching files pass, and rclone reports that hashes could not be checked. --size-only produces the same result with no hash step logged at all.

The ETag is otherwise stable across repeated GETs of the same object.

#mtime, not much else

x-amz-meta-mtime uses epoch seconds. It contains an integer for a whole second and three decimals otherwise. It is rclone's own metadata convention, and the only x-amz-meta-* header this gateway keeps.

On PUT, the value maps to driver.utimes() when the driver declares times. CompleteMultipartUpload uses the value supplied to CreateMultipartUpload. On GET and HEAD, the response uses stat.mtime.

Every other x-amz-meta-* header is dropped.

The header supports only millisecond resolution. If a file has a more precise modification time, the next rclone sync transfers it again. Use --modify-window 1ms to prevent that transfer.

Content-Type is not stored. Every GET and HEAD request returns application/octet-stream.

#Directories are prefixes

Only an empty directory has its own object.

PUT key/ (empty body) is a recursive mkdir; DELETE key/ is an rmdir (ENOTEMPTY → 409 BucketNotEmpty).

A directory with children needs no marker because its children make it appear in listings. An empty directory has no children, so the gateway represents it as a zero-byte object with the key key/.

This marker lets sync recreate the empty directory. The rclone --s3-directory-markers flag enables this behavior.

#The NotImplemented boundary

The gateway reports unsupported protocol operations explicitly.

An operation that it does not implement returns a valid S3 NotImplemented (501) error document.

An unsupported operation does not fall through to a related operation. For example, the gateway refuses GET /bucket/key?acl. It does not return object bytes when it fails to recognize the acl subresource.

CreateBucket and DeleteBucket return the same response. A bucket exists only when you pass its driver to createS3Server(). Requests cannot create buckets.

The gateway also refuses ?versioning. As a result, rclone purge logs Failed to read versioning status, assuming unversioned and then continues correctly.

The gateway supports ListBuckets, HeadBucket, ListObjectsV2, GetObject, HeadObject, PutObject, DeleteObject, DeleteObjects, and CopyObject. It also supports the five multipart operations below.

GET and HEAD support If-Match, If-None-Match, If-Modified-Since, and If-Unmodified-Since. They return 304 or 412 when a condition requires that response. They also support one byte range and return 416 when it cannot be satisfied. PUT supports three of the four conditional headers, as described below.

#Conditional writes

Before it reads the body, PutObject evaluates If-None-Match, If-Match, and If-Unmodified-Since. These headers support conditional creation and compare-and-swap (CAS) updates:

• If-None-Match: * creates a regular file object only if the key is absent. The gateway opens the file with O_CREAT|O_EXCL. The driver performs this operation atomically. If another writer creates the key first, the conditional request returns 412 PreconditionFailed.
• If-Match: "<etag>" replaces an object only if it still has that ETag. The gateway serializes the comparison and write with other conditional PUT requests for the same key. This provides CAS behavior within one S3Session. An unconditional PUT takes no lock and can still overwrite the object.
• If-Match for an absent key returns 404 NoSuchKey. S3 uses this response instead of the RFC 9110 412 response.
• If-Unmodified-Since writes only when the object has not changed after the specified time. The gateway ignores this header when the key is absent because there is no modification time to compare.
• If-Modified-Since is not evaluated on a PUT. RFC 9110 §13.2.2 limits it to GET and HEAD, and a PUT cannot return 304.

A directory marker uses mkdir, which has no exclusive-create form. The session serializes conditional requests for that key, but the O_CREAT|O_EXCL guarantee applies only to regular file objects.

CreateMultipartUpload and CompleteMultipartUpload do not support conditional writes. You cannot use them for a compare-and-swap update.

#Multipart

The five multipart operations stage parts through the driver. They are CreateMultipartUpload, UploadPart, CompleteMultipartUpload, AbortMultipartUpload, and ListParts. Parts use the reserved bucket-root prefix .mountx-multipart/<uploadId>/part-<N>.

Other operations cannot see staging keys. Direct GET, HEAD, and PUT requests treat a staging key as absent. Listings also omit it.

CompleteMultipartUpload streams staged parts in the order listed by the client. Parts can arrive out of order and with different sizes. Therefore, the offset of part N is unknown until all preceding parts exist.

Both upload abort and server close remove the staging area. After either operation finishes, an interrupted upload leaves no staged parts.

#The write-in-place caveat

A PUT and multipart completion write the object in place.

There is no temporary file and no rename, because the driver interface has no atomic-create primitive to build one on.

The gateway guarantees behavior before the first byte. It does not open the destination until the first payload byte arrives. Therefore, it does not truncate an existing object or create a new one before that point. For a signed aws-chunked body, it also verifies the byte first.

An upload rejected at or before its first byte does not change the bucket. If an upload fails after writing starts, the written prefix remains. A partial object replaces the previous complete object; the gateway does not restore the old value.

#Serving without mounting, on purpose

This transport has no platform-specific code. createS3Server uses node:http, which runs wherever Node runs.

This transport has not been tested on Windows. However, it does not depend on Linux or macOS behavior. It is an HTTP server for the portable driver interface.

#createS3Server(source, options?)

function createS3Server(
  source: FsDriver | { buckets: Record<string, FsDriver> },
  options?: S3ServerOptions,
): S3Server;

#S3ServerOptions

S3ServerOptions extends S3SessionOptions. You can set every session option here.

#S3Server

interface S3Server extends AsyncDisposable {
  readonly session: S3Session;
  readonly host: string;
  readonly port: number;
  readonly url: string; // e.g. "http://127.0.0.1:54321"; IPv6 bracketed
  readonly buckets: string[];
  readonly connections: number;
  listen(): Promise<S3Server>; // idempotent; resolves once bound
  close(): Promise<void>; // stop accepting, drain, drop, sweep multipart staging — idempotent
}

close() drains in a fixed order. It stops accepting requests. It lets in-flight responses finish, up to drainTimeout. It drops the remaining connections and then removes the multipart staging area. Removing the staging area first could conflict with a part that an allowed request is still writing.

#S3BindError / isS3BindError() / isLoopbackHost()

class S3BindError extends Error {
  readonly code: "ERR_S3_BIND";
  readonly host: string;
}
function isS3BindError(error: unknown): error is S3BindError;
function isLoopbackHost(host: string): boolean;

S3BindError is the address error from createS3Server(). You can catch this type instead of matching error-message text. isLoopbackHost() performs the literal address check described above. You can call it without creating a server.

#S3Session

new S3Session(buckets: Record<string, FsDriver>, options?: S3SessionOptions)

S3Session converts one HTTP request to one S3 reply without a socket. Therefore, tests need no listener or external client. FuseSession and NfsSession use the same separation.

session.buckets; // Map<string, Loopback>
session.bucketNames; // string[], in listing order
session.stats; // { requests, replies, errors, operations: Map<string, number>, assertions }
await session.handleRequest(head, body?); // → S3StreamResponse; never rejects
await session.close(); // idempotent; sweeps multipart staging

Unlike NfsSession.handleCall(bytes), handleRequest has a streaming boundary. The request and response bodies can each be an AsyncIterable<Uint8Array>. The gateway must not buffer a multi-gigabyte PUT for parsing or a multi-gigabyte GET for response.

#S3SessionOptions

#The layers below

Only server.ts uses a socket or listener. Therefore, the signing JavaScript client in test/s3/client.ts can test the protocol with the same codecs:

mountx/s3 re-exports all these APIs by name. It does not export the generic XML primitives used by xml.ts, such as XmlNode, xmlDocument, and parseXml. S3 gateway consumers do not combine these primitives directly. mountx/nfs applies the same rule to its substructure helpers.

#Not available

• CreateBucket/DeleteBucket, bucket ACLs and policies, object ACLs, versioning, and every list-type=1 request. The gateway refuses all these operations with a well-formed NotImplemented response. See the boundary.
• Symlinks, hardlinks, permissions and access time. An S3 object has none of these, so there is nothing for the gateway to carry even where the driver underneath has one.
• Windows. The transport has no known platform-specific requirement, but it has not been tested on Windows.

#Next

• celld: use this gateway as the shared bucket for a celld fleet.
• FUSE, 9P, and NFS: the three transports that produce a mount point.
• mountx/auto reference: the chooser that selects among the three mount transports, and why S3 is not an option there.