Reliability concerns

Can you identify when this happens? After upgrading SilverBullet. Is a reindex triggered? Could this be happening on a client that is “laggy” in terms of sync (hasn’t been updated for a while)? As you probably know, space styles are only really loaded on boot so if the styles aren’t (indexed) there at that point, they won’t be loaded until a System: Reload.

Anecdotally I think I’ve seen such issues at various points. Does it still happen recently? Please, if it does, have a look at JS logs and share them. I think there are issues here, but any information to debug would be useful.

Depending on what release channel you are, I did some things in my attempts to split the index phase in 2 that introduced new message queues that I later stopped using. If you have a client that was running such an “intermediate” version at some point and then upgraded to a newer build that removed this support, you may have ended up with a client state that still had items eternally queued. I’ve encountered this myself. It should only affect edge users, though. You can fix it indeed with a client reset or Client: Clean which also cleans queues.

If this still happens, please report it, and any JS logs would be helpful. If the number you get stuck on is 67 (six-seven!), this a special case that I’d also like to know about.

I use SilverBullet on mobile and find it acceptable, but not amazing. The gestures are indeed not super reliable, but seem to work for me (iOS). I haven’t put much focus on this myself, but it would be good to have github issues for any problems you have (and your device setup).

Generally, doing good mobile UX in web browsers remains hard. Some things you’d want to do are effectively not possible…

Yes. I resisted the concept of Live Preview for a long time (early versions of SB didn’t have it), because of the jumping around of content. However, I do not have better ideas on how to fix this behavior. It’s kind of inherent in the model I think.

Let’s try to keep this topic focused on reliability (which at least technically is my profession, SRE) as opposed to overall UX and “comfort” (which is its own can of worms and I think very much deserves a separate discussion thread).

I use CF Zero Trust on top. I’m starting to wonder whether what happens is that CF Zero Trust occasionally boots me out, and silverbullet takes that to mean “oh we should log him out of this too”. In CF Zero Trust I have it configured for “Instant Auth” (there’s only one authentication mechanism, so I don’t even notice the re-log-in), so I’m not entirely sure. Certainly some - but not all - of the reliability issues I describe are related to CF Zero Trust (…and I’ve had other reliability issues with tailscale, [example]). I try to be a helpful complainy-user :sweat_smile:, e.g. in Cloudflare Zero Trust session revocation handling · Issue #1091 · silverbulletmd/silverbullet · GitHub

Not that I’ve noticed, but the concern is always there.

Syncing attempts happen every 5 seconds or so. However, if any of the components in the chain (cloudflare, network, silverbullet backend) decide not to cooperate, this aggression doesn’t help much. Also, bugs in the client can, in theory, make syncing attempts not happen - and we folks trying to improve reliability are living on nightly, so bugs are to be expected.

The UX I’d expect here is, on a big screen, a small footer like “Last sync completed 4s ago (2025-01-26 10:39:02 UTC)”, and on a little screen something like a colored dot in the corner indicating something like “green is <5s, red is >60s, yellow in the middle”.

There’s already a little bit of a clue in the title - top-saved-color vs top-unsaved-color - but I believe this is for syncing to primary (local), not secondary (server). Also, it has different semantics - it won’t tell you if it’s done pulling changes from the server after a sleep.

Well, I’ve seen “client needs reloading” on silverbullet.md, so I’m guessing that runs the edge build… and for good reason! You want to see that your reliability fixes work, and so do I. Could we perhaps have a tracking issue for why this is necessary, when it’s expected to stop, and a link to said issue in the message? Also, can we consider not showing this message as a modal popup (needs several confirmations each time), and just go ahead and reload, notifying the user that it happened later? It’s not like the user is getting an opportunity to cleanly finish their work before the reload.

I am 80% sure that I’ve seen more than one reindex after upgrading silverbullet. That is, I’ll upgrade, have an expected reindex, everything will be working fine for hours/days, and then it’ll decide to reindex again (without another upgrade; on the same device). I’m thinking “reindex was only mostly complete, not all complete, and there’s a big difference”.

I use Chrome exclusively; this always happens on “full reindexes”, but also after “long” periods of inactivity. For instance, I just opened silverbullet on chrome-based PWA on my Android phone now (no upgrades in weeks), and instead of the contents of my main page, I get this:

Lua error: Error evaluating “string.split(space.readPage(name,”\n")[1]": Not found (Origin: [[Library/Std/APIs/Template@912]])

Because PWA doesn’t have a reload button (and the drag-from-top-to-reload thing doesn’t work there), I ended up opening this in “regular chrome” on the device, which flashed the error message for a moment, but then corrected itself. This also fixes it for the PWA. I think I have to do this about once a day.

I get it, and it absolutely makes sense for the nature of this project. My complaining walks a fine line between “Zef won’t see my problem and even if I send a fix, something else might clobber it” and “we introduce so much friction to the development process that Zef gives up on this and it becomes abandonware”.

I mean more of “they change all the time”, or more accurately “it’s not 100% clear that Zef isn’t planning to change them anymore”.


Thank you for taking our points seriously! I’m very encouraged that this seems somewhere near top-of-mind for you thanks to this thread :sweat_smile:

The Client: Clean suggestion is an interesting one (…is it the same as Client: Wipe?). I think some “reliability discoverability” would perhaps help here - maybe have silverbullet check itself for some of the known issues you suggest a Client: Clean for, and have the software display this suggestion itself. Not sure if that makes sense.

Phone clients being notoriously hard to debug, that’s maybe also something we can do something about. Perhaps something nutty like having the javascript errors be always reported to an endpoint on the backend, so we don’t have to set up USB debugging and then “catch silverbullet in the act”? A quick conversation with a friendly neighborhood LLM shows a gross hack:

window.onerror = function(message, source, lineno, colno, error) {
  // Send this to your own API endpoint
  fetch('/api/logs', {
    method: 'POST',
    body: JSON.stringify({ message, lineno, user: currentUser.id })
  });
};

One other point of note here, with software engineering management hat on, is how to wrangle this laundry list of “reliability paper-cuts”. If this were my corporate job, I’d say “make github issues for all of these, label them or use a milestone or whatever github finds appropriate, deduplicate these against them, and have an overall reliability track or issue update”, but that’d all be handled by some kind of project manager, and I’m highly aware that this is a one-man show. I’m curious to know where you’re thinking of taking this, and fully understand if the answer is “no, I’ll need to leave this mostly scratch-my-own-itch”.

I think separate github issues is definitely the way to go, also because it enables discovery for other users, and other people chiming in and commenting if they see similar issues or not. That type of feedback helps me prioritize as well. If it’s you then maybe you’re just doing something weird, but if there’s a bunch of others who hit the same issue then I rather look at it sooner.

Fair enough. Reliability improvements · Issue #1798 · silverbulletmd/silverbullet · GitHub should be a step in the right direction.

I realize I’m a bit late to the party :slight_smile: but I can confirm most of the worries reported by @lutzky while only running stable versions.

The “not quite knowing” if my changes are saved is the biggest point for me, especially on mobile. Perhaps just an animated icon in the top bar would be enough, just like most editors online do.

While on Desktop I find SB to be great, I use it a lot on mobile too, mostly to jot down quick ideas and notes, and that experience is less than ideal.

I use another local-first, sync-to-the-server-later PWA: Actual Budget. It’s absolutely phenomenal and I’ve never, ever had issues when using it on 3 devices (same devices I use SB on, same docker server, same Cloudflare tunnel without additionl auth). I understand that syncing transactions is way easier than whole freetext documents, but maybe we could take some inspiration from other apps.

It appears they have a custom-built CRDT! actual/packages/crdt

What I just pushed to edge is a change where the “dirty” indicator (page title color) only changes back to it’s “your safe” state after a single file sync has completed. So unless your client is entirely in its offline state (yellow bar), this dirty indicator should now be a more accurately reflect whether your changes made it to the server or not.

Can you tell me more about this one, what code is being executed there? There may be a boot race condition happening here.

A while ago already I added the SB_LOG_PUSH environment variable you can set to 1 (or anything else). When enabled, clients should send (most) of their logs to the server.

Sure; for files in Inbox/, I always show them on my “front page”.

inbox = {}

function inbox.notes()
  local res = template.each(query[[
    from index.tag "page"
    where string.startsWith(name, "Inbox/")
  ]], template.new[==[
**[[${name}|${string.sub(name,7)}]]** - ${string.split(space.readPage(name),"\n")[1]}
]==])
  if some(res) then
    return [==[# Inbox notes
]==] .. string.trimEnd(res)
  else
    return ""
  end
end

@zef the last release is another example of lack of stability of the platform (breaking changes, deprecating process missing ..) that I mentioned
Really, I'm very discouraged to maintain community plugins because the maintenance cost is too high. Moreover, many time, users redevelops features that are yet provided by another plugin. Currently, it's easy to install and update plugs but to find something...

Like I said before, It's not easy to develop and debug space Lua and moreover if after a new release, you have to debug them. It's a nightmare.
For me, the priority is: stability.

After all, SB is great product but I think it must to reach next level: maturity.

Thanks

To make this actionable you’ll have to be more specific about what broke. I put great effort into making things (especially syscalls) backwards compatible. It’s possible this was less the case on the pure Lua side because @mjf has been basically fixing our version to be more consistent with actual Lua which can mean regressions if you relied on previous bugs, basically. I think this work is more or less done now, so I hope for fewer or no such issues moving forward.