Remote Sync/Sharing

[zef (Zef Hemel)]

There’s a risk I try to solve too many problems with one “silver bullet solution” (ha!), but I have three immediate problems to solve, and one potentially cool one for the future:

1. The publish case as mentioned. I have content locally in my space and want to publish to somewhere.
2. Library (and plug) updates. If I put in some mirror/remote frontmatter on a library page, I can use that to auto update it from an external source. In most scenarios this is pull only (give me the new version of the “Git” library, for instance). But if I’m a developer a library (like the Git one) I’d like to be able to reverse that direction to a push.
3. Future: collaboration: I have a page locally and want to collaborate on it with another SilverBullet user without having to give each other access to each other’s spaces. We could use an intermediate server somewhere that we can use as a sync point. The most basic version would do this in a manual sync style: I make a change, sync, then my collaborator does the same to pull in my change. Obviously this is not real-time collaboration (yet) but that could be a future iteration (some may remember we had exactly this feature in SB at some point in the past).

In my currently working version (locally) I use the term “mirror”, but based on @inos-github 's terminology suggestion of using “share” I may change naming again.

Here’s how it works:

I’m on a page that I’d like to mirror. I hit Cmd-p, and a dialog appears:

I select the Gist option (I already have a github token configured). Then, it creates new Github gist and updates my frontmatter:

I make a change, hit Cmd-p again and it pushes that new version to the existing gist.

If I select the Github repo one, it will ask me for a repo, branch and file name and do something similar.

For instance, to publish a new library to my libraries repo I have this frontmatter:

---
mirror.uri: "https://github.com/zefhemel/silverbullet-libraries/blob/main/test.md"
mirror.hash: f4fd734c
mirror.mode: push
---

Note: the hashes are used to check if local changes have been made since the last push.

At the same time I’m working on Library management - #7 by zef what I intend to do next is that whenever you install a library from a repostory, it will populate the mirror.* frontmatter for those downloaded libraries in pull mode. Upgrading these libraries would then be as simple as a “mirror” operation (conceptually hitting Cmd-p on those files) to pull in the latest version.

Upgrading all libraries would be implemented by querying all pages with a mirror key in pull mode and updating them all.

@PowerUser

This almost seems like an event system, where event handlers (remotes) are called with the updated articles each time they’re modified. I might suggest this be built around an existing event system if possible, since there will probably be other uses for an event system in the future.

What I’ve built now is a service registry on top of the event bus that was already there. When you run the “Mirror” command and no mirror is configured, it will run service discovery on a mirror:onboard selector. Many services may match this selector (currently Github and Github gists).

Similarly, when a mirror key is configured, the readURI and writeURI services are called on the configured URI (which may have many implementations like a Gist, Github, Ghost one — all extensible).

@PowerUser

• What level of the stack are these going to run at? Is it the go/server level or the deno/plug level or the browser level? Some might be better than other for writing extensions, especially if API keys might be needed. I don’t know if I would want to store these in my notes repo as plaintext.

Nothing happens on the server, really. 90% of the server code is CRUD operations on files (list, read, write, delete), plush an endpoint to run shell commands and proxy http request (to avoid CORS issues). So the answer is: this is operated fully from the browser.

As for handling credentials: I don’t have a great solution for this. My current practice is that I have a page named SECRET(.md) that has a space-lua block with all keys. This file is in my .gitignore so it doesn’t end up in my repo (which I use for versioning and backup). There are probably fancy encryption things we can do with locking and unlocking things, but haven’t gotten to think about that much.