Who's using Online mode?

Since we have 0 tracking in SilverBullet I have no statistics on this, so let me ask this way, then follow up with context on why I’m asking.

SilverBullet has two client modes (Client Modes): Online is the default, it does not store any significant amount of data in the client (browser) and does not work offline, then there’s Sync button (which you get once you hit the Sync button) which syncs all content locally and enables offline mode.

Which one are you using?

  • I’m using Online mode only
  • I’m using Sync mode only
  • I’m using a mix of both
0 voters

Especially if you select using a mix or using online only, please leave a reply with the reason why.

Ok, then the promised context to this question. As you know I’m always on the hunt to remove features and simplify SilverBullet to keep the project manageable in terms of scope. Those who have been long-time SilverBullet users may remember that the first version had a two modes (similar to what we have now), then a second iteration which completely removed the “Online” mode: that is, it always synced all your content locally and operated fully in-client without much use of the server (beyond syncing content). Then, later, the pendulum swung back and I added an Online mode again. We’re getting closer to SilverBullet 3 year anniversary, so perhaps it’s time to swing the other way again :wink:

I’m now investigating what the impact would be to remove Online mode, so let me lay what we would lose and what we would gain.

What we would lose

The “in-and-out” edit use case

Beside being an interesting technical challenge to solve, one reason to (re) introduce Online mode was to better support the use case of “Oh, let me just quickly login to my notes on this foreign device to make a quick edit.”

With Sync mode only, this could be quite painful, because basically any new client first needs to perform a full sync to become fully operable. If you have thousands of pages, or lots of attachments, this could take a lot of time. There’s some optimizations to consider here, like not syncing attachment, or lazy syncing them. However page content always would need to be synced to make sure the indexes (and with that query results) are going to be complete.

The question is: do people actually do/need this? In practice, I never do. I access my SB on my phone, on my Chromebook/tablet and on my desktop. All in Sync mode. The initial sync takes some time, but after that I’m all good.

Support for huge spaces

Related to the previous point: if you have a massive space, think many tens of thousands or more pages, sync mode may not scale. I have 1400 pages in my space and everything works perfectly fine for me, but I’ve heard people who try to use spaces with tens or hundreds of thousands of pages, syncing all that content to every client may not be practical.

The solution? People would have to trim down their space, or split them into multiple.

Server-side indexing

The second advantage that I can think of (although it’s not heavily used) is that if indexing of your space becomes a heavy-weight operation (like for instance, when you try to do things like generating embeddings for retrieval with LLMs, like @justyns is doing) you’d have to redo that embedding generation for every single client, whereas when you’d use (only) Online mode, that indexing happens once on the server and that’s it. Note that currently this point is moot if you use some clients in Sync mode, those clients would also need to perform this expensive indexing on their own.

Custom HTTP endpoints (niche)

The third thing we’d lose, although I also don’t think this is heavily used, is server-side custom endpoints, see: Space Script

What we would gain

No more client modes

The existence of two client modes is a common source of confusion and it’s hard to explain the trade-offs between the modes. It also adds UX overhead in that there’s this ever-present Sync button up top.

By removing Online mode, this would be simpler: you login to SB, your content syncs and everything is available, also offline. No need to think about it.

I suspect that a lot of people don’t use Sync mode because they don’t know it’s there or understand what it is, or simply forget to hit that button. The result is that they will actually get worse performance, because Sync mode is actually significantly faster in day-to-day operation (because there’s far fewer round-trips to the server). The effect would be that those users would just get a faster experience, because they’d be forced into Sync mode.

Technical simplicity

This may not affect you all, but it does those who work on SilverBullet, its plugs, and those who use more advanced features like Space Script. It’s often not super obvious where code will run: on the server? On the client? Will it work the same? Why is this syscall available here, but not there? By removing Online mode, the answer would be simpler: everything runs in the browser, you always have access to all syscalls. Done.

It also reduces QA effort. I spend a fair amount of time testing changes ensuring they work both in Online and Sync mode, and although it’s pretty rare there are inconsistencies, I still need to do that testing every time.

There is a fair amount of infrastructure and code that enables the flexibility of having logic be able to run both in the browser and on the server. As a technical feat I’m actually quite proud of how well this works, and doesn’t require all that much maintenance, but every line of code is another source of bugs.

If we eliminate Online mode, the server component would become super simple: basically a dumb data store, all it will do is serve up and store files, all the interesting stuff happens in the client.

This also opens the door to perhaps rewriting the server to run on node.js again. I’m fairly happy with Deno, but for many people it’s a bit of a barrier to contribute because it’s a bit of a niche JS runtime.

What would become easier

There are a few areas that I haven’t really pushed into, because they would be quite hard to tackle. By eliminating Online mode these doors would re-open:

  1. End-to-end encryption. With this feature, you would no longer have to assume a trustworthy server. The server would just store encrypted binary blobs. Even if your server gets hacked, nobody would be able to decrypt your content without knowing the key. This would always be an optional mode, because it would limit certain features.
  2. Multi-tenant and SaaS hosting options. Right now you can only host a single space on a single SilverBullet instance. I spent some time in the past trying to make SB multi-space and multi-tenant capable, but would run into issues of security, especially if people would start to host plugs and space scripts outside their direct control. However, since removing Online mode would make the server more of a “dumb block storage” it would actually be way easier to scale. Even offering SilverBullet as a hosted SaaS would be on the table.

Opinions?

3 Likes

For those using Online mode (all the time or in hybrid): what’s your reason for using it?

I was using a mix of both. After reading your explanation, it seems I gain nothing from using online mode. So I changed my answer to sync only and will stop using online mode.

I was using online mode mostly because it seemed to be faster to receive change from other client. However, recently I realised that the change were there already and my query of recent changes simply needed a refresh.

I can see issues with people that always clear their browser on close, as they will need a full sync every time. It should be possible to whitelist SB, so not a real problem… It could lead to confusion from user unaware of the syncing behaviour.

I use online mode only, but I don’t think I fully understood sync mode. After reading your explanation, it sounds like sync mode would be beneficial for me. I only ever use SB on my phone or laptop, and I have a small space, so syncing would be a non-issue. Unless my note-taking explodes at some point, which I doubt.

Maybe I don’t understand it well, but I use online mode with my phone and at least two other workstations (often all editing the same page concurrently) and in the past I have been burnt by inconsistency issues and conflicts causing all sorts of hassles. Having one authoritative source that then is backed by a git repo is my preference. How that is implemented, I’m less attached to, but I don’t want to have to resolve conflicts manually. It is very un-ergonomic and can lead to data loss.

1 Like

The main reason I don’t use sync, is because it happened a couple times that all my files were duplicated in my space, and it was a pain to fix it all. Since then I use exclusively online mode.

And it happens sometimes that I want to connect to my space from a random computer, so that’d be annoying to have to load the entire data on that machine.

I have a hard time seing the added value of the sync mode to be honnest, keeping only online mode would be my favourite option.

2 Likes

I use a mix of online and sync mode mostly so I can test the LLM Plug without having to explicitly switch back and forth.

In the past I did have some issues with sync mode though where I’d get a lot of conflicts. When that was happening, having online mode to fallback to was nice. I’m pretty sure sync has been stable for me for a while now though, or at least I haven’t noticed any recent issues.

On my phone - I like sync/offline mode because I don’t need to connect to my vpn/internet just to look something up in my notes or make new notes.

On pretty much all other devices, it doesn’t matter too much if I use sync or not. I could think of use cases for having an online-mode and also custom api / http endpoints, but I don’t currently use them.

Maybe it’d be possible to have a full sync and a lite-sync option? The lite sync could just sync index.json to get metadata/etc, and lazy-load everything else. IMO that’d work for public facing read-only instances or quick online editing.

Online mode is great for work/untrusted devices where I don’t want personal notes sync’d.

All my personal devices run in sync mode.

In saying that, I could certainly live without online mode.

I don’t totally trust syncing I pretty much just always use online only mode and connect to it via VPN from my phone. My phones always on VPN so isn’t a big deal.

I think I tried the syncing and just had some issues with it working properly I had issues in past with things having merge conflicts.

1 Like

I only use Online mode, and it works well for me.

I run Silverbullet on my own server, so my ideal use is:

  • When I load a file, it loads immediately from my server
  • When I save a file, it saves immediately to my server

I want my server to be the ‘source of truth’ for every file, and I back it up accordingly. I’d worry if Online mode went away - like others, I’ve had problems with sync solutions in the past (not Silverbullet-related).

1 Like

I also would prefer to always use sync, and sometimes switch away because of faults with sync mode. In addition to conflict files, I also have the occasional “dammit, something is out of sync, please resync everything” moment which is especially painful on the phone.

I’m using Online mode nearly always, in part probably laziness (because it is the default) but also because i did get some conflicted copies of files early on. I think this was self-inflicted, using sync mode on two devices. Staying in online mode “seems like a good idea” but I will use sync mode more often as a result of this discussion.

It seems from other comments i’m not the only one confused by these two modes, or at least by the terminology. I know this has been discussed before, but to me a more intuitive term for sync mode would be offline mode.

(I’m running SB on a local server using tailscale. )

I use online mode pretty much exclusively. I regularly access my space from two personal computers, a phone, a tablet, and a work computer (sometimes simultaneously). I also (less frequently) need to log in from other unexpected devices. Online mode seems to work really well for that device-hopping use case. I don’t have to worry about one device syncing and overwriting what I’m doing on another device.

I’ve been burned in the past by the sync options provided by other apps, resulting in data loss or duplication. I’m just way more comfortable knowing that the document I see is the document saved on the server.

2 Likes

Running SilverBullet though PWA in Online mode on 3 computers, and on a PWA on my Phone in Sync mode

The reasoning for Sync mode on the phone is that there are times where i have no network and still need my notes, as well as since SilverBulllet is running on my home server and my ISP at times gives me a new external IP (have not solved this on my side yet though i understand there are workarounds) this breaks the connection to my home server and Sync mode allows me to still access my notes until i update my DNS

If i had to choose is go for all Sync mode for sure

Thank you all for your input, I appreciate it!

Let me summarize some of the feedback and respond to it:

I’m using Online mode, but for no specific reason
This is what I suspected, and almost a reason to switch it off, because I think that in many scenarios (but not all, which I’ll get back to) using Sync mode should be the better choice.

With one little caveat that it should work reliably, which brings me to the next one.

I’m using Online mode because I don’t trust Sync
You either hit issues with Sync in SilverBullet specifically at some point, or with sync in general in life.

I recognize this concern. However, I don’t think it’s a good reason to not double down on it. In fact, in the time that SilverBullet only had sync mode, a lot of issues and bugs were ironed out. Things like “this needs a full sync cycle before I see my changes on other devices” were things specifically addressed, for instance, by immediately syncing the currently open file. Some people layed out scenarios where having the same page open on two devices would either result in conflicts or in not reliably/quickly syncing those changes. That’s a bug, this should work reliably and at some point in time worked fine. If it doesn’t anymore, that’s something to fix.

This type of issue is more likely to get fixed and improved if everybody suffers from it. I, too, sometimes get in a weird conflict generating cycle, which is solved after I reload and I’m like… ok, whatever. However, if I know this is the reality of everybody I’d prioritize finally looking into it.

I think Sync mode slowly regressed a bit in quality over the last months because I could tell myself “well people are not all using this, and a refresh fixes it”, or “this gets better after the first sync cycle” and I don’t use it for development (only briefly for testing). By putting such issues more in your (and my) face, I’m more motivated to actually fix them properly. And there wouldn’t be any more way to talk myself out of it.

Which brings me to the next one.

I only use Online mode and don’t really need Sync
Some people have said they only use Online mode and they’re ok with that, even to the degree they’d be ok with removing Sync mode.

On this one I can be quite clear, even from a pure self-centered perspective (no offense to you all, but I am still my #1 target demographic myself :wink: ) : this is never going to happen (famous last words). It’s an absolute key feature for me that I have access to my notes without an Internet (or LAN) connection, especially on my phone. For connectivity, but also performance reasons. I use the Tailscale setup myself when I access my notes from my phone outside my house and the latency there is pretty huge and unpredictable. However, since all my content is local it doesn’t affect me at all. Everything is always pretty snappy.

So until omnipresent network connection is going to be a solved thing (any day now, I’m sure) Sync mode is here to stay.

I use a mix, because I have many devices or use somebody else’s computer to get access
This is the biggest potential blocker to ditch Online mode, and in a sense the “origin story” of having this mode at all.

There’s two aspects to this:

  1. “Time to access”: How much time you’d need to go from login to accessing the content you care about: do you need to wait a full sync cycle to do anything?
  2. Security: what traces do you leave behind when you login on a “foreign” device?

The security concern is one to be addressed also for Sync users. It’s pretty non-obvious how to wipe a client clean right now other than doing this (somehow) at the browser level. This isn’t good. I think it would make a lot of sense to have an explicit “Logout” type button or command that does exactly this: remove all locally stored content and burn your authentication tokens.

This leaves the issue of forgetting to logout, even though you should have. Doing a remote wipe is not realistic. The only real solution for this would be to implement on-client encryption, which would be something to explore. This is also doable I think.

Then on the need to wait for a full sync cycle you’d need to get access to your content. This is where I’d like to do a bit of out-loud thinking:

Imagine this to be the new reality: when you first login to your space on a new device, you immediately get access to the index page (or whichever page you were linked to): this page will be retrieved directly from the server. In the background a full sync kicks off, but until that finishes you can already do a lot of things:

  1. You can navigate between pages by clicking links. Whatever content isn’t already synced (including attachments) would be fetched from the server on-the-fly.
  2. The page picker will be populated with all pages in your space (not based on indexed data, as it is right now) but based on the plain list of .md files in your space (perhaps with some heuristic filters on what we suspect may be template files etc.)
  3. The compromise: All (inline and widget) templates and queries would be disabled.

Then, when the initial sync finishes, all regular functionality would flip on.

Two observations about this approach:

  1. For sync users, this would actually be a better experience than they get right now. Right now, you can do very little without a full sync having completed. You can see a page and click links, but e.g. the page picker will only show synced content (and may even be empty for a long time). Queries will pretend to render, but actually produce empty results. What you need to do is wait for the sync to finish and effectively reload the page to get in the desired state. It’s one of the “this is fine, it’s just the first run” type of experience things I have come to accept, but would actually address now.
  2. For Online users, this would of course be a worse experience, but perhaps one we can live with.

For those who want to login from a “foreign” device, and don’t want to sync any content locally, perhaps there can be a “I’m on a public computer” style checkbox that many web apps have, which would block any sync to happen, but that would mean that also things like queries and templates would not render in this mode. Not ideal, but perhaps an acceptable trade-off.

1 Like

Just to add to this, I use online only specifically to avoid syncing everything when I’m operating in things like VMs or accessing stuff while I happen to be in a clean chrome instance for dev work. Having it full sync all the time would be a major pain.

I also access my notes on my work PC, so I’d prefer if they didn’t get the joy of seeing everything automatically because it all downloads.

Finally, would this at all effect the idea of read-only access somehow in the future? Because that would surely always be online-only etc. Not something I need or want, just something that might need to be considered.

Edit: I’ve also had cases where sync mode on my phone missed pages that could be found in online mode. No idea why, but I trust online more

1 Like

I am using online mode as I mostly using SB on my lan and very rare cases login remotely. If I am offline and I need my notes and even I had it in online mode I anyway can’t use them reasonably any way.

I never use sync mode, mainly because I’ve never not had an internet connection for the last… well years and years. And if that does happen I still have a physical notebook with me.

Multi tenancy would be nice, but I could also just run an extra docker container. I’d prefer if SB stayed a really good note taking/writing app, and let things like encryption, multiple instances, etc to other systems. I want an always-online markdown editor that stores my files as files.

4 Likes

An other user case I had recently:
I had to have multiple people (~20) to write collaboratively documentation. And I know that SB is single user, but that worked like a charm and was super effective. I can only imagine that would be impossible if there was only the sync mode available.

ps. I know I could use things like headgedoc for that, but in this case it involved multiple tens of .md files, that was so much more effective doing it through SB, and all was directly synced to git :love_you_gesture:

1 Like