Pub Plug CLI vs Browser output is different

I’m trying to incorporate the silverbullet-pub plugin into my CI pipeline to publish all pages with a tag of public. The output of running the command in the browser vs a shell command is different. It seems the cli command isn’t picking up the tag for the page but the browser one is.

This is the command I’m running in the container:
deno run -A --unstable-kb --unstable-worker-options /silverbullet.js plug:run /space pub.publishAll

As an example, this is effectively the output on the cli but its missing the public tag

All pages [
  {
    name: "My Public Page",
    ....,
    tag: "page",
    tags: [],
    itags: ["page"]
  }
]

This is the frontmatter block:

---
tags: public
---

pub uses your existing .silverbullet.db index to do its work, if this file doesn’t exist it should index the space first and then do the publish step. Did you by any chance check in .silverbullet.db* files, or have stale ones in your checkout? Try cleaning them out, then run the CLI version again.

I’m actually testing the process on my computer first. I’m running sb in a container and to test out the publish cli command I just get a shell into the same container and manually run it there. So they SHOULD be using the same db unless my cli command isn’t correct.

The output of both running plug:run and the SB server should list the location of the database file used at boot. Are they the same?

If so, this may an issue related to how you run it. The database used is SQLite which isn’t set up for having multiple processes use its file at the same time I think. So ideally the server isn’t running at the same time as the CLI. How do you intend to run it on a CI context? Personally I run it straight with Deno there.

I shut down my compose stack and ran the following:

Docker run with a shell into the container

docker run -it --rm --entrypoint bash -v ./space:/space --user 1000 zefhemel/silverbullet

Launching silverbullet inside the container:

deno run -A --unstable-kb --unstable-worker-options /silverbullet.js plug:run /space pub.publishAll

The output shows that it’s using the correct database, however it’s only publishing the index page.

space path /space
Function to run: pub.publishAll with arguments []
Using KV as a database backend (/space/.silverbullet.db).
Activated plug federation
Activated plug markdown
Activated plug editor
Activated plug index
Activated plug query
Activated plug sync
Activated plug plug-manager
Activated plug search
Activated plug template
Activated plug emoji
Activated plug tasks
Activated plug share
Activated plug mermaid
Activated plug git
Activated plug pub
Booting up worker for index
Loaded 1 functions and 0 commands from space-script
Current space index version 4
Space index is up to date
Loading builtins attributes into index
Booting up worker for pub
Publishing to A { prefix: "_public/" }
All pages [
...
]
Cleaning up destination directory
Booting up worker for federation
Publishing []
Done writing published paegs
Writing index page Public
Writing Public
Booting up worker for markdown
Indexing _public/index as page
Booting up worker for search
Booting up worker for tasks
Publishing index.json
Stopping sandbox for federation
Stopping sandbox for markdown
Stopping sandbox for editor
Stopping sandbox for index
Stopping sandbox for query
Stopping sandbox for sync
Stopping sandbox for plug-manager
Stopping sandbox for search
Stopping sandbox for template
Stopping sandbox for emoji
Stopping sandbox for tasks
Stopping sandbox for share
Stopping sandbox for mermaid
Stopping sandbox for git
Stopping sandbox for pub

Ok, could you try deleting the hidden .silverbullet.db file and then run the CLI command again? This should result in a full index first. Let’s see if that changes anything.

That did get the cli command to output the correct files!

Ok. Can’t really explain that, but in a CI setup this will work then. Just make sure you don’t commit the .silverbullet.db file.

I’ve now updated the README etc, but I realized the better approach is to have the build process use the in-memory data store by setting the SB_DB_BACKEND=memory environment variable.

I added this a CI section now: https://pub.silverbullet.md

1 Like