You may have seen hints here and there that I’ve recently spent a lot of time on the Lua side of things and I’m extremely happy with what this will unlock and elegance it can bring to the system. As I probably have mentioned elsewhere, if I would have seen this 3 years ago (I started SB about 3 years ago) I would have designed things very differently.
If you haven’t already, watch this video for a glimpse. A lot of this is already somewhat out of date, but you’ll get the gist:
Up to now I developed the Lua stuff in “parallel universe” mode, there are now “Lua ways” to do many things that previously required more specific functionality.
For instance:
- Templates can be implemented using the
${lua expression}
syntax (that you can use anywhere) and a few Lua APIs. No need for a specialtemplate
fenced code block syntax + language - Queries can be implemented with a slight overloading of Lua syntax and become much more elegant and general purpose this way: Space Lua/Lua Integrated Query so no more need for a special
query
fenced code block syntax + language. - Most (though not all) safe things you could do with Space Script you can also in Lua, but often slightly nicer (for instance because in Space Lua we abstract away from all async/await JS stuff).
- Live Template Widgets can now also be implemented in Lua by listening and responding to specific events (not publicly used yet, but believe me: it’s more elegant), so they don’t need a special
hooks
template mechanism anymore either. - Live Embeds are unnecessary, because they can simply become Lua expressions, like
${embed.youtube "url"}
. No need for specific syntax here either. - Transclusions also become unnecessary because, again, you can just do this with a Lua call (not implemented yet, but would be trivial), e.g.
${embed.page[[other page]]}
without the need for custom syntax (although we may want to keep this one because of Obsidian compatibility). - Space Config? Not really necessary, we can have a Lua API for that.
The whole Objects story basically doesn’t change. All of this still makes perfect sense in a Lua world. You just query them using the Lua Integrated Query.
So a lot that needed to be custom built before, can now (often trivially) be replaced with Space Lua versions.
This is very exciting (to me).
However, there is a but…
The but
Parallel universes are expensive, especially if you’re a team of… well, me, with limited time and brain capacity. SilverBullet is a hobby. I have a “real” job and family as well. While I initially though I’d be fine with a slow and phased migration story from the old to the new, I’m now more concerned about the cost of doing this. I have already encountered a lot of parts of the code base where it’s very hard to keep things working in parallel. There’s a lot legacy, ugly code in there that I’d be eager to get rid of, but cannot because it would break existing functionality.
I also see bug reports and people running into issues where I think: well, that won’t be a problem in the Lua world, should I even bother to fix that bug? Actually, do I even have energy to fix bugs that I know are in parts of the system I don’t really want to keep supporting long term?
I really spend a lot of my (limited) mental energy right now thinking “how do I do this, without breaking that other feature?”
What I would love to do right now. And would have done absolutely, if I were SB’s only user, would be to rip the band aid and remove all the “legacy” features I just mentioned (as some other questionable features that I’m not sure anybody uses) in on fell swoop.
Gone.
Oh, that would be so nice.
However, the (happy) reality is that I do have active SilverBullet users and that’s great. I’m very happy you’re all here, but you do make my life a bit harder
Still I think something needs to happen, because ultimately there’s a slight risk of me burning out on the complexity of maintaining this system of ever increasing size and complexity, and that may not be good either.
So, as I’m writing this, I want to propose something and see what you all say.
SilverBullet 2.0
Versioning is weird, because there never was a SB 1.0. But let’s consider all the 0.* versions SilverBullet 1.0 for simplicity.
What I am considering is doing a last 0.x release, and then go in hardcore “rip everything we no longer need” out mode. A lot, if not all, features I mentioned at the top would disappear, plus perhaps some more.
I don’t want to break everybody’s happy space (hah) right now, so what everybody can (and perhaps should) do is pin their version to that last 0.x release. It will be there, you can use that until the end of time. If people get upset about me removing their favorite feature in this version, or just performing this rug pull, I’d be happy to support whoever is interested to maintain a branch for people who don’t want to migrate to 2.0. There can be a “SilverBullet classic”. There always could be of course, this is an open source project, people can always fork.
Then, the brave ones (I hope all of you) can do the migration of their space to what ultimately will become a 2.0 release, whenever they’re ready.
How to migrate
What would be involved in such a migration for you?
- If you’re a casual SilverBullet user, e.g. using just the standard stuff like pages, basic markup, page links, stuff like that: nothing will really happen. The fundamentals don’t change.
- If you’re using templates or queries: those blocks will simply not be rendered anymore and just appear as fenced code blocks. You’d have to manually translate your queries to Lua queries. This shouldn’t be hard (the syntax is 90% the same), and I’m happy to help in this process. You’d also have to manually translate templates in the same way (this may be slightly more involved, but ultimately more powerful as you learn the Lua syntax).
- If you’re using space script, all of that will need to be converted to Lua. I think I’d also remove space script (although this one I may be talked out of). The scripts would simply stop being activated. Since space script allows for really random JavaScript to be run, there may be cases where you cannot translate this to Lua. However, I’d be very interested in those cases to see if we can somehow support with Lua after all.
- Many (not included, third-party) plugs would likely break or need adjustment.
For a little while, some features may simply disappear. However, as you know I’m open to feedback. If you miss something, I’m happy to rethinking and add it back (but perhaps in a slightly tweaked more Lua-focused way).
If SilverBullet were a company with a full development team backing it, I wouldn’t propose this approach. But it’s not. It’s primarily me, and I have to balance where I spend my energy and keep my motivation high. This is why I’m proposing this more radical strategy.
Let me know what you think.