csfs

Client Side File System

GitHub · csfs.emdzej.pl

About

One read API for data that lives somewhere a browser can reach: a static HTTP host, a directory the user picked, the origin private file system — or inside a zip archive in any of those.

import { httpFileSystem } from "@emdzej/csfs-http";
import { withArchives } from "@emdzej/csfs-zip";

const fs = withArchives(httpFileSystem("https://example.test/data"));

await fs.read("/pr/index.dat");           // Uint8Array
await fs.read("/media.zip#/photo.png");   // out of the archive, one Range request
const file = await fs.file("/huge.bin");
await file?.slice(0, 1024).bytes();       // 1 KB fetched, not the file

Nothing is downloaded that was not asked for, and the same code runs against every backend.

Status

Version 0.3.0, on npm as @emdzej/csfs-*. Tested in Node 22 and 24, and in Chromium, Firefox and WebKit.

0.3.0 came out of an audit of the whole tree and its first end-to-end tests in real browsers — see the news entry and the changelog, which opens with what to check when upgrading from 0.2.0.

pnpm add @emdzej/csfs-http @emdzej/csfs-zip
npx @emdzej/csfs-cli manifest ./data

Why the interface looks like this

A file is modelled on Blob — size, slice, arrayBuffer, bytes, stream, text. Reading part of a file is the operation that makes remote data usable at all: an archive is read from its end, a sorted index is binary-searched, a video is seeked. An interface whose only read is "give me the whole thing" forces a download per lookup. Because Blob and File already have this shape, the local backends need no adapter.

Absence is not an error. file() and directory() return null, so testing whether something exists needs no try/catch. Errors are kept for things that really went wrong: a refused permission, or a host whose answer disagrees with its manifest.

A read can be cancelled. bytes, text, arrayBuffer and read take a { signal }. Work shared between readers — one download of a file several slices want — stops only when the last of them gives up.

Writing is a separate interface. Two of the backends cannot write; a combined interface would make every consumer check capabilities it never uses.

Archives

Two ways in, for two different situations.

// `#` addressing, when the caller knows the archive is there
await fs.read("/pack.zip#/inside.txt");
await fs.read("/outer.zip#/inner.zip#/deep.txt");   // nesting works

// an archive answering for a directory that does not exist
const fs = withTransparentArchives(httpFileSystem(base), [
  { archive: "/drawings.zip", serves: "/drawings", entry: "basename" },
]);
await fs.read("/drawings/1132/1132C000.png");

That second case is not exotic. A parts catalogue ships 38,488 drawings as one flat drawings.zip and as a tree bucketed by name, and every reference in the data uses the tree's shape. A real file always wins, so a tree that was extracted keeps working, and a half-extracted one falls back file by file.

An entry is inflated on its first read, not when it is looked up. An entry stored without compression is never copied at all: its bytes are a range of the archive, so a slice of it is a Range request — which makes an archive stored inside another one about as cheap to read as one on its own.

The manifest

HTTP cannot list a directory. A static host will serve any file you name and tell you nothing about what is there, so a tree served over HTTP carries a description of itself.

csfs manifest ./data --label "my tree" --archive "/drawings.zip:/drawings:basename"

Measured on a real tree: 43,915 entries describing 15.30 GB come to 4.02 MB of JSON, 0.42 MB gzipped — 9.4:1 at a host's default level, paid once when the tree is opened.

Extracted, the same bytes need 228,515 entries and the manifest grows to 14.72 MB. Declaring an archive is what makes the difference, because paths are what a manifest costs, not bytes.

Packages

Package	What it is
@emdzej/csfs-core	The contract, paths, BlobFile/RangeFile, walk
@emdzej/csfs-zip	Archives as a file system, and # addressing
@emdzej/csfs-http	Static HTTP, manifest-driven, Range reads
@emdzej/csfs-fsa	A picked directory (File System Access)
@emdzej/csfs-opfs	The origin private file system
@emdzej/csfs-node	node:fs, for tooling and tests
@emdzej/csfs-manifest	The manifest format and its builder
@emdzej/csfs-cli	Build manifests and inspect a tree from a terminal

What each backend costs

• HTTP — one Range request per read, streamed as it arrives. Needs a manifest. A host that ignores Range still works: its 200 is the whole file, which csfs slices itself rather than mistaking it for the slice. A response that disagrees with the manifest about a file's length is an error, not a short read.
• A picked directory — each path segment is a round trip. Permission does not survive a reload, so a remembered directory needs one click.
• OPFS — no prompt ever, which is the reason to import into it. But it is evictable unless persist() is granted, and shared across the origin, so use a namespace. It also differs by engine: WebKit's folds case the way its disk does, Firefox asks the user before persist() answers, and Safari's private windows have none at all.

Zip handling is @zip.js/zip.js, not ours — zip64, data descriptors, cp437 names, and archives whose local headers lie while the central directory holds the truth.

Used by: eperx, dialogysx, masax