Skip to main content
Deno 2 is finally here 🎉️
Learn more
Module

x/earthstar/src/entries/npm.ts

Storage for private, distributed, offline-first applications.
earthstar-project/earthstar
Go to Latest
import * as earthstar from "https://deno.land/x/earthstar@v10.2.0/src/entries/npm.ts";

Earthstar is a small and resilient distributed storage protocol designed with a strong focus on simplicity and versatility, with the social realities of peer-to-peer computing kept in mind.

This is a reference implementation written in Typescript. You can use it to add Earthstar functionality to applications running on servers, browsers, the command line, or anywhere else JavaScript can be run.

Example usage

import { Replica, ReplicaDriverMemory, Crypto, Peer } from "earthstar";

const shareKeypair = await Crypto.generateShareKeypair("gardening");

const replica = new Replica({
	 driver: ReplicaDriverMemory(shareKeypair.shareAddress),
	 shareSecret: shareKeypair.secret,
});

const authorKeypair = await Crypto.generateAuthorKeypair("suzy");

await replica.set(authorKeypair, {
	 path: "/my-note",
	 text: "Saw seven magpies today",
});

const allDocs = await replica.getAllDocs();

const peer = new Peer();

peer.addReplica(replica);

peer.sync("https://my.server")

This module also exposes server APIs for for building always-online peers. The below example reads some on-disk JSON to initiate some share replicas, and stores their data using the filesystem.

import {
ExtensionKnownShares,
ExtensionSyncWeb,
Server,
} from "https://deno.land/x/earthstar/mod.ts";

const server = new Server([
new ExtensionKnownShares({
	 knownSharesPath: "./known_shares.json",
	 onCreateReplica: (shareAddress) => {
		 return new Earthstar.Replica({
			 driver: new ReplicaDriverFs(shareAddress, "./share_data"),
		 });
	 },
}),
new ExtensionSyncWebsocket(),
]);

Classes

An attachment driver which persists attachments using the local filesystem. Works with Node.

An attachment driver which persists attachments in memory. Works everywhere.

A server won't accept writes

A discovery service for finding peers on the local network, to be used with Peer.discover.

A replica driver which perists to LocalStorage, which stores a maximum of five megabytes per domain. If you're storing multiple shares, this limit will be divided among all their replicas. Works in browsers and Deno.

An in-memory replica driver. Its contents will be lost when it is closed. Works everywhere.

A strorage driver which persists to SQLite. Works in Deno and browsers.

Generic top-level error class that other Earthstar errors inherit from.

A server extension for populating a server with known shares. Use this to specify which shares you'd like your server to sync with others.

A server extension for exposing the contents of shares, so that you can request documents by their path and have them served over HTTP. Can be used to create wikis, websites, image galleries, and more.

A server extension which reads settings from a specified share, e.g. which shares to host on the server. Settings are modified by other peers syncing with this server.

An extension which enables synchronisation over the web via HTTP.

A replica holding a share's data, used to read, write, and synchronise data to.

A server URL is bad or the network is down

A syncing partner to be used with local instances of IPeer. Works everywhere.

A syncing partner created from an inbound HTTP connection (i.e. a web client).

A syncing partner to be used with servers reachable via the internet. Works everywhere.

Holds many shares' replicas and manages their synchronisation with other peers. Recommended as the point of contact between your application and Earthstar shares.

A replica holding a share's data, used to read, write, and synchronise data to.

A cached, synchronous interface to a replica, useful for reactive abstractions. Always returns results from its cache, and proxies the query to the backing replica in case of a cache miss.

An IReplica or IReplicaDriver was used after close() was called on it.

A replica driver which persists data to the filesystem.

A replica driver which stores data in memory. All data is lost when the replica is closed.

A replica driver which persists data to IndexedDB.

An IReplica or IReplicaDriver was used after close() was called on it.

An extensible server able to synchronise with other peers.

The core server logic. Combine this with a HTTP framework to create a fully-fledged server.

Get and set values from a common pool of settings for Earthstar clients, such as an author, shares, share secrets, and servers.

Syncs the data of a Peer's replicas with that of another peer.

Validation failed on a document, share address, author address, etc.

Variables

Lowercase alphabetical characters.

Uppercase alphabetical characters.

All characters allowed in an identity's public address.

All characters allowed in an identity's pub key.

All characters allowed in an identity's short name.

All characters allowed in base32.

Higher-level crypto functions. Mostly used for generating new author and share keypairs.

A version of the ICryptoDriver interface backed by Chloride. Faster than noble. Works in Node and the browser (with some polyfilling on your part).

A version of the ICryptoDriver interface backed by noble/ed25519. The slowest crypto driver available, but works everywhere.

A verison of the ICrptoDriver interface backed by native Node crypto functions. Requires a recent version of Node, perhaps 12+? Does not work in the browser.

All digits.

Validator for the 'es.4' format. Checks if documents are spec-compliant before ingesting, and signs them according to spec.

Format for 'es.5' documents. Supports attachments and share keypairs.

Helper for querying Earthstar docs using a glob-style query string.

A helper used by globToQueryAndRegex -- see that function for details.

All characters permitted in a document's path.

All special characters permitted in a document's path.

All characters allowed in a share's address.

All characters allowed in a share's key.

All characters allowed in a share's name.

Functions

Put a short name and pub key together into an identity address.

Put a share name and encoded pub key together into a share address.

Encode uint8array bytes to base32 string

Decode base32 string to a uint8array of bytes. Throw a ValidationError if the string is bad.

Check that an identity address is valid.

Check that a share address is valid.

example usage: myArrayOfArrays.sort(arrayCompare)

Creates an invitation URL. Validates the share address, that servers are valid URLs, and the secret against the share address if given.

Convert a keypair (author / share) back into a raw KeypairBytes for use in crypto operations.

Return whether a document is expired or not

Combine a shortname with a raw KeypairBytes to make an AuthorKeypair

Combine a name with a raw KeypairBytes to make an ShareKeypair

Returns a valid share address generated using a given name.

Given an array of format names, and an array of IFormat, returns an array of IFormat restricted to those with matching names.

Returns an object with format names as keys, and corresponding IFormat as values.

Returns the default formats if no formats are given.

Returns the default format if no formats are given.

Check if any value is a subclass of EarthstarError (return true) or not (return false)

Check that a string contains only printable ASCII

Check if any value is a subclass of EarthstarError (return false) or not (return true)

Check that a string only contains character from a string of allowed characters.

Parse an author address into its parts.

Parse a share or author address into its parts

Parses an invitation URL. Validates the share address, secret (if given), and any server URLs.

Parse a share address into its parts.

This is a low-level helper for the template matching code; probably don't use it directly.

Find documents whose path matches the glob string. See documentation for globToQueryAndRegex for details on glob strings.

Given a template string like "/posts/{postId}.json", query the replica for docs with matching paths.

Set the crypto driver used for all cryptographic operations.

Returns a promise which is fulfilled after a given number of milliseconds.

Interfaces

A keypair used by individual entities to sign documents.

The core properties all documents must implement, regardless of format.

Contains data written and signed by an identity.

Contains data written and signed by an identity.

A partial es.4 doc that is about to get written. The rest of the properties will be computed automatically.

A partial es.5 doc that is about to get written. The rest of the properties will be computed automatically.

Higher-level crypto functions. Not used directly for the most part, but useful for generating new keypairs.

A crypto driver provides low-level access to an implementation providing ed25519 cryptography, e.g. Chloride, noble/ed25519, Node crypto.

Formats are each responsible for one document format such as "es.4". They are used for signing and validating documents, as well as generating new documents from a given input.

Holds many shares' replicas and manages their synchronisation with other peers. Recommended as the point of contact between your application and Earthstar shares.

An attachment driver provides low-level access to a replica's attachments. ReplicaAttachmentDrivers are not meant to be used directly by users; let the Replica talk to it for you.

A document driver provides low-level access to a replica's documents. ReplicaDocDrivers are not meant to be used directly by users; let the Replica talk to it for you.

A replica driver provides low-level access to a replica's documents and attachments. ReplicaDrivers are not meant to be used directly by users; let the Replica talk to it for you.

Implement this interface to create an Earthstar server extension.

Provides a syncer with the means to connect the peer being synced with (the partner).

Options for configuring a new replica.

  • validators: Validators for the kinds of documents this replica will replicate, e.g. FormatValidatorEs4.
  • driver: A driver the replica will use to read and persist documents.

Describes a query for fetching documents from a replica.

Filters a query by document attributes.

Options to initialise a Syncer with.

Type Aliases

An identity's public address.

The human-identifiable portion of an identity's public address, e.g. suzy.

An attachment associated with a document.

A short string with a timestamp and hash of the document's path and author.

A document with it's attachment merged onto a new attachment property.

Verifies a given type is of type IFormat

Extracts a IFormat's config type

Extracts a IFormat's document type, e.g. DocEs5

Extracts a IFormat's input type, used to generate a new document.

Extracts a IFormat's name, e.g. es.5

Verifies a given type is an array of IFormat

Represents fetching all historical versions of a document by authors, or just the latest versions.

  • IngestEventSuccess — a new doc was written
  • IngestEventFailure — refused an invalid doc
  • IngestEventNothingHappened — ingested an obsolete or duplicate doc

The path of a document, e.g. /images/teapot.png.

An event representing when a QuerySource has processed all existing documents.

  • ExpireEvent - An ephemeral document has expired
  • IngestEvent — the result of a replica ingesting a document
  • DocAlreadyExists — processing an old doc as you catch up

A mode representing what kind of docs are desired from a query stream.

  • existing - Only pre-existing documents.
  • new - Only documents written after the stream is initiated
  • everything - Both pre-existing and incoming documents.
  • DocAlreadyExists — processing an old doc as you catch up
  • IngestEvent — the result of a replica ingesting a document
  • ExpireEvent - An ephemeral document has expired
  • AttachmentIngestEvent - A new attachment has been ingested
  • AttachmentPruneEvent - An attachment without a corresponding document has been pruned.
  • ReplicaEventWillClose — the replica is about to close
  • ReplicaEventDidClose — the replica has closed

A share's public address.

A keypair used to write to a specific share

The human-identifiable portion of a share's address, e.g. gardening.

An event with an Earthstar document and corresponding ID.

A type of message one SyncAgent can send to another.

An event sent when a SyncAgent doesn't want anything anymore, though it'll still serve HAVE requests.

A special event for the implementation of a PlumTree. Asks the recipient to begin lazily messaging us.

An event to be passed on to a RangeMessenger

The current status of a SyncAgent.

Signals that a SyncAgent wants a document/documents from another SyncAgent

An 'appetite' which determines when the syncer will stop syncing.

  • once - The syncer will only attempt to reconcile existing docs and then stop.
  • continuous - Indefinite syncing, including existing docs and new ones as they are ingested into the replica.

An event for disclosing which shares a Peer has without actually revealing them. Another peer can use the salt to hash their own shares' addresses and see if they match.

An event a Syncer can send or receive.

A map of sync statuses by the share address they're associated with.

A SyncAgentEvent addressed to a specific share address.

A UNIX timestamp in microseconds.