Indexing incrementally and for performance

Hello, I have some questions about indexing. I have a large graph (~2k files, ~60k blocks). Indexing on mobile takes hours, during which I have to keep the browser window open and not have the phone shut off. Desktop indexing takes many minutes. A couple of questions:

  • If I edit a .md file directly (outside SilverBullet), will indexing update incrementally, or does it trigger a broader reindex? How about if I add, remove, or move a .md file?
  • Is there anything I can do to improve indexing performance? My notes are entirely bullet‑based with no paragraphs, and I plan to keep them that way since my PKM is outline‑centric.

In light of the ongoing indexing rewrite, how will these assumptions hold? I plan to run SilverBullet locally on both my desktop and my phone, pushing updates directly to .md files from desktop to phone. The phone instance will be read‑only (at least for now). Hence the questions above.

Hi @zef, would love to get your thoughts on this. Happy to clarify if the question is not clear.

Hours? That’s… a lot. And this is on 2.4?

It is incremental, whenever any file change is detected (locally triggered or synced from remote) it will index just those files. It does happen in every client individually, though.

We’d have to identify where the indexing time is spent. In 2.4 I already optimized a lot and disabled some things, but plugs like Silversearch likely slow things down if you use that.

Heh, yes. I’m running SB (the binary) on the phone as well. This is on 2.3. Let me try it on 2.4 and get back.

Very helpful to know it’s file-level incremental. Makes my idea for running SB locally on the phone viable. Also means the huge initial indexing time matters less.

I do use Silversearch. If there’s an easy way to currently profile the indexing time, I’d be interested to try it out. If not, no big deal.

Thanks much for the responses!