earthstar API tour
Table of contents
- Introduction
- Peer
- Replica
- Replica drivers
- Validators
- Identity
- Query
- Doc
- Crypto
- setGlobalCryptoDriver
- Syncers
- QueryFollower
- ReplicaCache
Introduction
earthstar
is a reference implementation of Earthstar written in Typescript. You can use it to add Earthstar functionality to programs running on servers, browsers, or the command line.
It can be imported via URL into a browser:
<script type="module"> import * as Earthstar from "https://cdn.earthstar-project.org/js/earthstar.bundle.v9.3.2.js"; </script>
Or Deno:
import * as Earthstar from "https://deno.land/x/earthstar@v9.3.2/mod.ts";
or installed with NPM:
npm install earthstar@9.3.2
And import what you need for your specific platform:
import { Replica } from "earthstar";
import { ReplicaDriverIndexedDB } from "earthstar/browser";
import { ReplicaDriverSqlite } from "earthstar/node";
To integrate Earthstar into your program, your application will probably need a single instance of Earthstar's Peer
class, which holds the replicas for many shares and is able to synchronise them with other peers.
Nearly all of Earthstar's classes can be configured to suit your specific use-case, For example, a Replica
can be configured to persist data to IndexedDB if you're building a browser app, or Sqlite if you're building a server or command-line tool.
Peer
A Peer
is responsible for holding many Replica
s, and offers many ways to access and manage them. A Peer
can hold one Replica
for each share.
There is also a Peer.sync
method which can be passed 'syncable' things such as another Peer
or a Websocket or HTTP URL to a replica server. The Peer
will automatically create a new syncer
const myPeer = new Peer();
// Add a replica
myPeer.addReplica(myReplica);
// Sync with a local instance of a `Peer`
myPeer.sync(myOtherPeer);
// Sync with a replica server using Websockets
myPeer.sync("wss://our.club/earthstar");
// ... Some time later...
myPeer.stopSyncing();
When two peers synchronise with one another, they perform a 'salted handshake' to determine if they have any shares in common without actually revealing what those shares are. Whichever shares they have in common are synced.
Technical docs forPeer
Replica
A local replica of a share's documents. Provides operations for querying and setting documents.
import { ReplicaAsync, ValidatorEs4, ReplicaDriverMemory } from "earthstar";
// Construct a replica
const myReplica = new Replica(
// Needs a share address,
`+myshare.a8puorlpz1`,
// A document validator,
ValidatorEs4,
// And a driver to teach it how to read and write documents
new ReplicaDriverMemory(`+myshare.a8puorlpz1`)
);
// Query for documents
const txtResults = myReplica.query({
filter: {
pathEndsWith: ".txt"
}
});
// Set a document
await myReplica.set(myKeypair, {
format: "es.4",
path: "notes/ink.txt",
content: "Don't forget to get some new ink."
});
Technical docs for Replica
Replica drivers
Earthstar can be used on in browsers, on servers, the command line, or anywhere JavaScript can be run. There's no single way to persist data in all these environments, which is why we made Replica
able to use different ReplicaDrivers
suited to the environment its being used in.
Universal drivers:
ReplicaDriverMemory
(no persistence)
Web platform drivers:
ReplicaDriverLocalReplica
ReplicaDriverIndexedDB
Deno drivers:
ReplicaDriverSqlite
Node drivers:
ReplicaDriverSqlite
It's possible to create your own replica drivers which conform to IReplicaDriver
. Technical docs for IReplicaDriver
Validators
Earthstar shares are able to store documents of different formats. But most of the time, you'll want to use the built-in one from this library: es.4
. As such, Earthstar currenly only offers one format validator: FormatValidatorEs4
.
Every time a new document is ingested by a ReplicaAsync
it is checked by a validator. In the case of FormatValidatorEs4
, it checks whether a document's signature matches its author, space address, and content.
It's possible to write your own validator which adheres to IFormatValidator
. Technical docs for IFormatValidator
Identity
An identity is a keypair containing a public address and a secret. This keypair is used to sign documents whenever they're written. The public address can be freely distributed, but the secret should be, well, secret.
You can generate identities using Crypto.
Note that documents are not encrypted (yet), only signed. This means we can verify that the document has not been tampered with on its journey to you, making it possible for syncing to hop across several peers without worrying about them messing with your data. They can still read your data though -- all docs in a space are readable by everyone in the space.
Technically speaking, these are ed25519 keypairs encoded in base32.
Technical docs forAuthorKeypair
Query
Used to describe a document query. Used with ReplicaAsync.query
and QueryFollower
.
Query
Doc
Represents some cryptographically-signed user data. The contents of a Doc
can only be changed via ReplicaAsync
. A document cannot be moved from one share to another.
Doc
Crypto
Crypto
is used internally for signing and verifying documents using ed25519. In most cases, you will want to use it to generate new identity keypairs:
const keypair = Crypto.generateAuthorKeypair("suzy");
console.log(keypair);
/*
{
"address":"@suzy.bv3cclqhvnbn4fwjur7u4nimm4cznbzigqyihlfwi6acck5b4q54q",
"secret":"bxrishindifrczhm5riqd4mozx4o5htksqthd64yph4rlfs6ej53n"
}
*/
Technical docs for Crypto
setGlobalCryptoDriver
Because ed25519 is not part of any JS standard library (or WebCrypto API), there are several third-party implementations, some of which are only available for certain runtimes.
To balance compatibility and performance, Earthstar uses noble/ed25519 by default. This works in browsers, Deno, and Node.
Earthstar exports a setGlobalCrypto
function to change this:
import { setGlobalCryptoDriver, CryptoDriverChloride } from "earthstar";
setGlobalCryptoDriver(CryptoDriverChloride);
Technical docs for setGlobalCryptoDriver
Syncers
Most of the time you will want to use the Peer.sync
convenience method to synchronise with other peers. But if you are building something special — for example a server, or something with Web Workers and BroadcastChannel
— Syncer
provides a more flexible API which you can use in combination with classes conforming to earthstar-streaming-rpc
's ITransport
type.
import * as Rpc from "https://deno.land/x/earthstar_streaming_rpc/mod.ts";
const myPeer = new Peer();
// This syncer will listen for other syncers using the `my_channel` BroadcastChannel.
const broadcastChannelSyncer = new Syncer(
myPeer,
(methods) => new Rpc.TransportBroadcastChannel({
methods,
deviceId: myPeer.peerId,
channel: 'my_channel'
})
);
// Don't forget to close once done!
broadcastChannelSyncer.close();
Technical docs for Syncer
QueryFollower
Listens to events from a Replica
, and if it matches the given query, sends an event to a SuperBus
.
Here's a basic usage:
const myMarkdownFollower = new QueryFollower(myReplica, {
filter: {
pathEndsWith: ".md",
},
});
// Print every newly created / updated Markdown doc to the console
myJpgFollower.bus.on((event: LiveQueryEvent) => {
if (event.kind === "success") {
console.log("A markdown doc was updated or created!");
}
});
// Start the follower.
myJpgFollower.hatch();
A QueryFollower
must always be hatched first. Hatching makes it catch up with all existing documents which match the query, after which it starts listening for new events.
When you're done with a QueryFollower
, you should close it to stop callbacks from running:
myJpgFollower.close();
Technical docs for IQueryFollower
ReplicaCache
ReplicaAsync
offers a strictly asynchronous interface to a share's contents. Sometimes a synchronous API would make life far easier, and ReplicaCache
does its best to provide that.
ReplicaCache
offers a synchronous API for fetching documents by storing queried documents in an internal cache.
When you first query a document, there is no result. But ReplicaCache can be subscribed to for updates, at which point you return to the cache and get an instant result:
import { ReplicaCache } from "earthstar";
const cache = new ReplicaCache(myReplica);
const myQuery = { filter: { pathStartsWith: "/about/" } };
let aboutDocs = cache.queryDocs(myQuery); // This will be an empty list.
cache.onCacheUpdated(() => {
// Trigger a re-querying of the cache!
aboutDocs = cache.queryDocs(myQuery); // Now there should be something in there.
});
ReplicaCache
has an optional TTL (time-to-live) option which can be provided to its constructor:
const cache = new ReplicaCache(myReplica, 2000); // ms, 1000 by default
When you fetch an expired result from the cache, ReplicaCache will then return the latest result from its backing replica.
Technical docs forReplicaCache