So over the last few weeks I’ve been tinkering with PICO-8, which is a nice game development environment to play with. Recommended if you’re looking into introducing your kids to programming (as I am). As a programming language it uses Lua, which is a programming language I was somewhat familiar with, but not deeply.
I dug into Lua a little more, and I like it quite a bit. First of all, because it’s a pretty simple, yet powerful language. The syntax is quite limited, so it’s easy to learn, and probably not all that hard to implement.
And then I started to think… could Lua be a solution to a design problem I’ve been thinking about for a few months now…
Here’s the issues I see that Lua may solve:
Security
Space script is a pretty cool feature, but also unsafe. The way it works is that it effectively just evals whatever JavaScript you throw at it on both the client and server. If you write this code yourself, this should be all good (unless you really do scary things), but the moment you start to copy & paste random scripts without understanding what they do, we get into scary territory. It’s pretty hard to sandbox running random JavaScript code in a browser environment (ironically). Some other tools do this by Wasm compiling a JavaScript interpreter and then using that, which is pretty heavy weight.
There are some ways of running/using Lua in the browser:
- You can WASM compile the official Lua interpreter and run it in the browser, for instance: GitHub - ceifa/wasmoon: A real lua 5.4 VM with JS bindings made with webassembly
- There is a JavaScript interpreter available: GitHub - fengari-lua/fengari: 🌙 φεγγάρι - The Lua VM written in JS ES6 for Node and the browser
- There is a Lua grammar available for the Lezer parser (which SB uses for all its editor parsing) that seems pretty complete: GitHub - R167/lezer-lua: Lezer-based parser for Lua this doesn’t run Lua, but it parses it. And… building an interpreter on top of this seems feasible if you know what you’re doing (famous last words).
So with some options on the table, it seems feasible to offer space-lua as an alternative to JavaScript and make it safe to run.
Accessibility
While JavaScript is a very popular language, it is also a complex one with lots of weird “features”. I’ve noticed that a fair number of SB users are writing space script, but don’t always really know the language. They mostly use ChatGPT to help them.
Lua can be a solution here too, because it’s an easier language to learn with fewer weird edge cases.
Unification of template language, expression language and space script
But here we get to the the thing that gets me even more excited. It is possible that we can replace SB’s expression language, and potentially tweak the template language to use Lua expressions instead, and to use it for queries as well. This would mean that SB would no longer have a fully custom expression language, but that those expressions would just support arbitrary Lua expressions, which would make them even more powerful.
Already, syntactically it’s quite surprisingly close and I’m thinking if this can be done in such a way that it is largely or entirely backwards compatible.
The Plan
There’s a big caveat with the Lua implementations that exist today, and that is that to allow Lua code to call asynchronous APIs (e.g. any syscall), you’d have to write Lua code with callbacks, which would become quite ugly. I already have a solution to work around this for SB’s expression language (where you can mix synchronous and async calls transparently), and I think I can expand that to a full Lua implementation.
This would mean that I’d end up writing a custom Lua interpreter for SilverBullet. I think this is feasible, and can be done in reasonable time. I happen to (literally) have a PhD in compiler construction and programming language design, so this is fact not the first time I’d do something like this. But you know… famous last words.
What this would mean for SilverBullet
I wouldn’t remove space script as it is today, perhaps ever, but we’d add a second option, let’s call it space-lua. This option would be enabled on all SilverBullet instances, because it would be perfectly secure. This would also mean that many built-in functions can be reimplemented in Space Lua and distributed via Libraries, which is something I’m not 100% comfortable doing right now with space script.
Space Lua would be the recommended new way of extending SB with custom commands etc. basically the use cases of Space Script today. Just safer and with a simpler language.
It would mean that SB’s Expression Language would become Lua Expressions. I still have to see if I’d tweak Lua expressions slightly to also support e.g. the != operator (which we use today) in addition to Lua’s version /= so that we can maintain backwards compatibility, but beyond that it should largely make the expression language more powerful than it is today.
I still have to see what this would look like, but it’s also conceivable that we’d expand/replace the template language with a Lua version. Conceptually:
{{for page in all(query("page"))}}
* {{page.name}}
{{end}}
Or something along those lines, or even expand the Lua syntax with SB queries natively.
Anyway, I’m thinking out loud here, but I’m really warming up to this plan.
Let me know what you think.
.