Navigate

Earthstar

How it works

Earthstar helps you do two things:

Store and retrieve data on your own device (even when you're offline)
Synchronize data with other people (for collaboration and backup)

This all works without a central server, but you can run server(s) to make it easier for devices to connect with each other.

These things combined can be used to make rich collaborative applications, like chatrooms, games, or message boards — or web services, like websites, wikis or image galleries!

Share

Previously known as: workspace, space

A share is like a shared folder. They're typically used by a small group people who trust each other: a group of long-time friends; a family, or a community tied together by a common interest.

Each share has an address like +bookclub.a18bu7axtn. The first part helps you see which share you're interacting with, and the second, scrambly part makes it hard for others to guess.

It's important that it's hard to guess, because anyone who knows the address can read and write data to that share. So it should be kept out of public view and shared only with people you trust.

A share is just an address. It's a reference to a self-contained universe of data peers are interested in.

A share's data is stored on its users' devices using replicas.

Replicas

Previously known as: storages, pockets

A replica is a concrete copy of the data in a share.

Replicas are stores of data which you can put wherever you want: your computer, a USB key, a friend's internet-connected Raspberry Pi. They can use different storage technologies to persist their data, such as SQLite or IndexedDB.

When two replicas are configured to use the same share address, they can synchronise their data with each other, and this can be used to create collaborative applications like chatrooms, forums, or games.

Each replica holds the data for one share.

Data stored in a replica are persisted as separate Documents.

Documents

Documents describe and contain your share's data.

Every document has a path like /my-story.txt. Paths can be used to group different bits of data together and make them more discoverable, e.g. /posts/2021/bird-thoughts.md.

Our community has used this to create a few conventions for storing documents for different purposes e.g. profile information, micro-blogging, long-form discussions. These emerging standards allow multiple apps to use the same data.

Documents can be temporary: they can optionally state the time at which they should be auto-deleted. After this time, replicas will delete that document completely. Users can also delete documents on demand by overwriting them with empty documents.

Documents can either belong to a single user, or be writable by everyone with the share address. In that case many people can write to the same document, and your pocket will store the last version from each author. This means you can always roll back or resolve conflicts if something went wrong.

Every document has a signature to help Earthstar verify its authorship. So if a document claims it was written by @suzy, you can be confident it was really written by @suzy and not altered by anyone else. These signatures are created using identities.

Identities

A person's identity in Earthstar is represented by a keypair, which is a bit like a username and password.

It's made up of an address and a secret. The address is made up of a 4-letter "shortname" you choose and something called a public key, which can look like this: @suzy.bzfkbawuj2cdxsxilvso4a2xczmcx6char4pqd2tjhdmcoy5i4ebq.

The address is publicly shareable information, and the secret should be kept, well, secret. A keypair's address or secret can never be changed, so keep it safe!

When a public name is put into a document's path, prefixed with a tilde (e.g. /about/~@suzy.bzfkbawuj2cdxsxilvso4a2xczmcx6char4pqd2tjhdmcoy5i4ebq/displayName.txt), then only the user with that identity can write to that document. Otherwise anyone can edit the document.

But what about the ways we express our identity, like names, pronouns, or avatars?

This kind of information is stored as documents within the commons, e.g. in your "about me" document. This has two big benefits: your identity can freely fluctuate between the shares you use; and you can express your identity with as little or as much detail as you want!

Syncing and replica servers

Earthstar peers are able to sync with each other, but how do they find each other to begin with?

We've designed Earthstar so that it cannot disclose which shares peers hold unless the the other peer knows about those shares itself.

This way, malicious peers cannot scan a discovery service to find out which IP addresses are interested in what.

It also means that Earthstar shares are undiscoverable. Users must disclose share addresses to each other manually.

In many p2p systems there's an emphasis on peers connecting directly to one another. This adds a few complications, such as establishing this direct connection to begin with, and needing to be online at the same time (which is especially salient to Earthstar's serving of small groups).

For this reason we encourage the use of Replica servers, small redundant always-online peers reachable via URL. These servers are used to stash documents for syncing while peers are online.

We are striving to make running a replica server as simple as possible. Find out how to start your own.

Every share has its own separate network of Peers; data does not spread from one share to another. But a Peer can hold multiple share replicas at a time.

Applications

Applications are just programs which write and read through a replica's documents and re-present it as a friendly user interface. For example, imagine a program which reads a folder full of images and displays them as a gallery.

Earthstar has no opinion on how applications are created, and provides a Javascript API that you can use in your vanilla JS, jQuery, React, Svelte, Vue, Angular, or whatever app.

Earthstar apps can typically run in browsers, or can be native apps. They just need to be able to talk to the other peers to sync data.

Important reminder about secrets

A share address gives you full read and write permission to the documents stored there. Only share it with people when you want to invite them to the share. It's not currently possible to remove people from a share; instead you can make a new share and migrate everybody else there instead.

Your Identity keypair has two parts, the address and secret. The address is safe to share (it's how people can identify you) but don't share your secret with anyone -- treat it like a password.

Example:

Finally, note that documents are not encrypted. They are signed, which means we can prove they were not tampered with during their multi-hop journey to your computer. Anyone with the commons address can read all the documents.

Apps will be able to encrypt the content of documents and Earthstar will provide helper functions for this. Document metadata (author, path) is needed for syncing and will not be encrypted.

Blocking & harassment

We care deeply about making safe and healthy online spaces. That said, blocking is difficult due to the distributed nature of Earthstar and we're still figuring it out.

As a first step, replica servers tend to be the bottlenecks in the network so they might be a natural place to apply blocklists. Apps should also provide users with the ability to hide other users.

We are also interested in distributed/delegated blocking and reputation ideas such as TrustNet.

Get started