Migrating SilverBullet Plugs from Deno to Node

Plugs and Libraries
1

This guide describes how to migrate a SilverBullet plug from the old Deno-based build system to the new Node/npm-based one.

Note: you can do this at your leasure, your old Deno builds should continue to work. However, newer versions of the APIs will no longer be published to JSR, just as an npm package.

What changes

Before (Deno) After (Node)
deno.jsonc with tasks and import map package.json with scripts and dependencies
deno.lock package-lock.json
deno run -A <url>/plug-compile.js npx plug-compile
jsr:@silverbulletmd/[email protected] @silverbulletmd/silverbullet from npm

What stays the same: Your .plug.yaml manifest, your .ts source files, and the @silverbulletmd/silverbullet/... import paths all remain unchanged.

Steps

1. Delete Deno files

rm deno.jsonc deno.lock

2. Create package.json

{
  "name": "silverbullet-yourplug",
  "version": "0.1.0",
  "private": true,
  "scripts": {
    "build": "npx plug-compile yourplug.plug.yaml"
  },
  "dependencies": {
    "@silverbulletmd/silverbullet": ">=2.5.3"
  }
}

Replace yourplug with your plug name matching your .plug.yaml file.

3. Create .gitignore

node_modules/
*.map

4. Install and build

npm install
npm run build

This produces yourplug.plug.js (and a .map sourcemap file) in the current directory. Use --dist <path> in the build script if you want the output elsewhere.

See the silverbullet-mermaid repository (to-node branch) for a complete example of this migration.

2

Going to be honest here, I don't like this, still have to fight the *** **** esbuild is. I'm still forced into some specific built system and I still have to hack my way around it. Bundle size also increased by +10% .... The built code actually feels worse than deno now ...

3

Bundle size increased? That’s very odd…

I’m open to more radical ideas around plug development. What I tried to do here is the minimal possible change to not produce much work for people.

4

Ok, taking back what I said on bundle size, that is actually in the "frontend", I really have no clue why tho.
I guess I'm really limit testing the build system and build systems drive me up the wall, but I would have wished that the plugin API was just available as a library and I could do something like the following

Silversearch.initPlugin({
     name: "silversearch",
     functions: {
         "search": {
             func: () => console.log("Searching!"),
             command: {
                 name: "Silversearch: Search",
                 key: "Ctrl-s"
             }
         }
    }
})

Then I could just use the build system of my choice, or none at all, meaning I would have full control for debugging purposes, could use vite features to e.g. import files as raw strings, etc. (One would also get typechecking on the manifest.)

5

Bit off-topic, and of course I might be wrong, but as a developer who has worked with both Node and Deno, moving out from Deno to Node does sound a bit stupid, sorry.

Deno is supposed to be more secure, it supports npm packages too, the DX is better, so... why...? :exploding_head:

I was looking into a web-hosted PKB. SilverBullet seems like a good fit, but I cannot avoid being judgmental about this move... Unless I'm missing something here (idk, users prefer Node because reasons), that does not sound like a sound system design decision to me.

6

I can't speak for Silverbullet+, but the backend of the "default" SB is written in go and the frontend runs in the browser. So neither deno nor node are used at runtime anymore, it's really only about the build step, which involves using mostly, if not only, npm packages and it's just generally easier to use node than forcing deno into being node. The DX is just 10x better.

7

So the move away from Deno has a long history. A bit of context, for those who are interested:

The very first prototypes of SilverBullet were built on node.js as backend, and built the frontend with node.js-tooling as well. Then I had this idea of allowing for a sandboxed plugin mechanism where plugins could transparently either be run in the browser (in an iframe or webworker) or on the server. I managed to get this to work on the server somewhat with node, but it was clear that node was not really designed with much server-side sandboxing support.

Deno was about to hit 1.0-ish around this time, and I liked it a lot. Node "done right" from the same original author as node, and their CTO (Bert) that I actually knew from a previous company, all built on Rust! Some sort of limited permission support for workers. Great!

And for many years it was a reasonable choice. However, it's also been a source of intermediate frustration. The Deno team moves fast and sometimes breaks things, and over the years I've had to report various regressions. They were quickly fixed, but still a distraction. They've changed tactics quite a lot on package management (HTTP imports, import maps, JSR, varying levels of supporting npm) that resulted in unwanted churn trying to just stay things up to date with the latest best practices. Then there were a bunch of cool features like Deno KV that started out very promising, but were not very deeply developed. Sometimes seemingly abandoned.

Since Deno is a VC-backed company, from a long-term thinking perspective, with my "CTO hat" on it's always been a bit of a gamble to build on such technology if the business model has not proven yet. A few years later, I think it's still not been proven, and in fact the recent lay-off at Deno doesn't instill much confidence that they've hit that part out of the park yet. So for me, the long-term future of Deno is unclear.

At the same time, a few things happend:

Node.js has started to catch up on support of various APIs (probably under pressure from Deno and Bun), like fetch and others, and there may be typescript (ish) support there at some point.

The other development is that as of SilverBullet v2, the need for a back-end in SilverBullet is almost symbolic. I rewrote the remaining backend from Deno to Go in a few days, and now have another implementation in Rust that lives in SilverBullet+. There's not much there.

As @MrMugame said, in terms of maturity of toolchains, there's a lot of difference. Deno comes with a bunch of nice things out of the box (like fmt, lint etc.) but the Node ecosystem is much richer. Given that I only really have the client (frontend) to care about now, SilverBullet gets almost 0 value out anything specific that Deno offers. It is however a barrier to entry in terms of onboarding new people, because a lot more people know node/npm than know Deno.

That's why a few months ago I decided I'd slowly step away from the Deno dependency and a release or two back that finally happened. Building plugs this way is part of that.