Earthstar has a new specification powered by Willow.
Earthstar v11 is now in beta.
Navigate

Earthstar

How it works

Earthstar helps you do two things:

Store and retrieve data on your own device (even when you're offline)
Synchronize data with other people (for collaboration and backup)

This all works without a central server, but you can run server(s) to make it easier for devices to connect with each other.

These things combined can be used to make rich collaborative applications, like chatrooms, games, or message boards — or web services, like websites, wikis or image galleries.

Share

A share is like a shared folder. They're typically used by a small group of people who trust each other: a group of friends; a family, or any community tied together by a common interest.

Each share has an address like +gardening.bhyux4opeug2ieqcy36exrf4qymc56adwll4zeazm42oamxtr7heq. The first part helps you see which share you're interacting with, and the second scrambly part is used for cryptographic operations.

Every share address also has an accompanying secret. If you know the secret, you can write data to that share.

A share is just an address. It's a name for a self-contained universe of data peers are interested in.

A share's data is stored on its users' devices using replicas.

Replicas

A replica is a concrete copy of the data in a share, stored on a user's device.

Replicas are stores of data which you can put wherever you want: your computer, a USB key, a friend's internet-connected Raspberry Pi. They can use different storage technologies to persist their data, such as SQLite or IndexedDB.

When two replicas are configured to use the same share address, they can synchronise their data with each other, and this can be used to create collaborative applications like chatrooms, forums, or games.

Because replicas aren't connected to each other all the time, it's not guaranteed that all replicas have the same data on them at any given moment. Earthstar can only guarantee that replicas' data will be eventually consistent.

Each replica holds the data for one share.

Data stored in a replica are persisted as separate Documents.

Documents

Documents describe and contain your share's data.

Every document has a path like /my-story.txt. Paths can be used to group different bits of data together and make them more discoverable, e.g. /posts/2021/bird-thoughts.md.

Our community has used this to create a few conventions for storing documents for different purposes e.g. profile information, micro-blogging, long-form discussions. These emerging standards allow multiple apps to use the same data.

Documents can be temporary: they can optionally state the time at which they should be auto-deleted. After this time, replicas will delete that document completely. Users can also delete documents on demand by overwriting them with empty documents.

Documents can have attachments. These attachments can be anything: music, video, programs. Documents and attachments come separately, so it's possible to hold a document without its attachment (and you may even want this, e.g. in the case of large files).

Documents can either belong to a single user, or be writable by everyone with the share address and secret. In that case many people can write to the same document, and your replica will store the last version from each author. This means you can always roll back or resolve conflicts if something went wrong.

When a public name is put into a document's path, prefixed with a tilde, then only the user with that identity can write documents to that path. Otherwise anyone can add a document to that path.

Every document has a signature to help Earthstar verify its authorship. So if a document claims it was written by @suzy, you can be confident it was really written by @suzy and not altered by anyone else. These signatures are created using keypairs.

Keypairs

A person's identity in Earthstar is represented by a keypair, which is a bit like a username and password.

It's made up of an address and a secret. The address is made up of a 4-letter "shortname" you choose and something called a public key, which can look like this: @suzy.bzfkbawuj2cdxsxilvso4a2xczmcx6char4pqd2tjhdmcoy5i4ebq.

The address is publicly shareable information, and the secret should be kept, well, secret. A keypair's address or secret can never be changed, so keep it safe!

When a public name is put into a document's path, prefixed with a tilde (e.g. /about/~@suzy.bzfkbawuj2cdxsxilvso4a2xczmcx6char4pqd2tjhdmcoy5i4ebq/displayName.txt), then only the user with that identity can write to that document. Otherwise anyone can edit the document.

But what about the ways we express our identity, like names, pronouns, or avatars?

This kind of information is stored as documents within the commons, e.g. in your "about me" document. This has two big benefits: your identity can freely fluctuate between the shares you use; and you can express your identity with as little or as much detail as you want!

Syncing and Earthstar servers

Earthstar peers are able to sync with each other through the internet, but how do they find each other to begin with?

We've designed Earthstar so that it doesn't disclose which shares peers hold unless the the other peer knows about those shares itself, using something called a zero knowledge proof.

This way, malicious peers cannot scan a discovery service to find out which IP addresses are interested in what data.

It also means that Earthstar shares are undiscoverable. Users must disclose share addresses to each other manually.

In many p2p systems there's an emphasis on peers connecting directly to one another. This adds a few complications, such as establishing this direct connection to begin with, and needing to be online at the same time (which is especially salient to Earthstar's serving of small groups).

For this reason we encourage the use of Earthstar servers: lightweight, redundant, always-online peers reachable via a public URL. These servers are used to stash documents for syncing while peers are online.

Because they're servers, they can do things like serve responses to browsers too. That can be very handy.

We are striving to make running an Earthstar server as simple as possible. See how easy it is in our 'Run a server' tutorial.

Every share has its own separate network of Peers; data does not spread from one share to another. But a Peer can hold replicas of multiple shares at a time.

Applications

Applications are just programs which write and read through a replica's documents and re-present them as a friendly user interface. For example, imagine a program which reads a folder full of images and displays them as a gallery.

Earthstar has no opinion on how applications are created, and provides a reference Javascript implementation that you can use in your vanilla JS, jQuery, React, Svelte, Vue, Angular, or whatever app.

Earthstar apps can typically run in browsers, or can be native apps. They just need to be able to talk to the other peers to sync data.

Important reminder about secrets

A share's address grants discovery and replication access to a replica. The share's secret confers write access, so be careful who you share it with. It's not currently possible to remove people from a share; instead you can make a new share and migrate everybody else there.

Author keypairs also have an address and a secret. The address is safe to share (it's how people can identify you) but don't share your secret with anyone — treat it like a password.

Example:

Finally, note that documents are not encrypted. They are signed, which means we can prove they were not tampered with during their multi-hop journey to your computer.

Apps will be able to encrypt the content of documents and Earthstar will provide helper functions for this. Document metadata (author, path) is needed for syncing and will not be encrypted.

Blocking & harassment

We care deeply about making safe and healthy online spaces. That said, blocking is difficult due to the distributed nature of Earthstar and we're still figuring it out.

As a first step, replica servers tend to be the bottlenecks in the network so they might be a natural place to apply blocklists. Apps should also provide users with the ability to hide other users.

Get started