Migrating from Obsidian-Tasks

I’d been using Obsidian for almost a year and I thought it was great but it has its issues, particularly with portions of it not being open source. I was excited to learn about SilverBullet. What Zef has done largely on his own is incredible, so huge thanks for his amazing work are in order.

For me though there’s still some missing features, particularly around task management. I intend to try to fix that with the following Space Script:

const rrule = import("https://esm.sh/[email protected]")

const checkbox = /^\[([^\]]+)]\s+/
const hashTags = /(^|\s)#[^ !@#$%^&*(),.?":{}|<>]+/g
const taskId = /[a-zA-Z0-9-_]+/.source

const priorities = {
  "🔺": 0, // highest
  "⏫": 1, // high
  "🔼": 2, // medium
  "🔽": 4, // low
  "⏬": 5, // lowest
}
const matchPriority = Object.keys(priorities).join("|")

// Match anywhere in the string
const matchers = {
  priority: new RegExp(`\\s+(${matchPriority})`, "u"),
  start: /\s+🛫\s*(\d{4}-\d{2}-\d{2})/u,
  created: /\s+➕\s*(\d{4}-\d{2}-\d{2})/u,
  scheduled: /\s+(?:⏳|⌛)\s*(\d{4}-\d{2}-\d{2})/u,
  deadline: /\s+(?:📅|📆|🗓)\s*(\d{4}-\d{2}-\d{2})/u,
  completed: /\s+✅\s*(\d{4}-\d{2}-\d{2})/u,
  cancelled: /\s+❌\s*(\d{4}-\d{2}-\d{2})/u,
  recurrence: /\s+🔁\s*([a-zA-Z0-9, !]+)/iu,
  depends: new RegExp(`\\s+⛔️\\s*(${taskId}( *, *${taskId} *)*)`, "iu"),
  id: new RegExp(`\\s+🆔\s*(${taskId})`, "iu"),
  tags: new RegExp(hashTags.source, "iu"),
}

// Only match at the end for iterative extraction
const extractors = Object.entries(matchers).map(([prop, regex]) => [
  prop,
  new RegExp(regex.source + "$", regex.flags),
])

const parsePriority = (emoji) => priorities[emoji] ?? 3

const differenceInCalendarDays = (left, right) => {
  left = Temporal.PlainDate.from(left)
  right = Temporal.PlainDate.from(right)
  return left.until(right, { largestUnit: "days" }).days
}

const dueCoefficient = 12.0
const scheduledCoefficient = 5.0
const startedCoefficient = -3.0
const priorityCoefficient = 6.0
const parseUrgency = (task) => {
  const now = Temporal.Now.plainDateISO().toString()
  const { deadline, scheduled, start, priority } = task
  let urgency = 0.0

  if (deadline) {
    const overdue = differenceInCalendarDays(deadline, now)
    const dueMult = overdue >= 7
      ? 1.0
      : overdue >= -14.0
      ? ((overdue + 14.0) * 0.8) / 21.0 + 0.2
      : 0.2
    urgency += dueCoefficient * dueMult
  }

  if (scheduled && differenceInCalendarDays(now, scheduled) >= 0)
    urgency += scheduledCoefficient * 1

  if (start && differenceInCalendarDays(now, start) > 0)
    urgency += startedCoefficient * 1

  switch (priority) {
    case 0: urgency += priorityCoefficient * 1.5; break
    case 1: urgency += priorityCoefficient * 1.0; break
    case 2: urgency += priorityCoefficient * 0.65; break
    case 3: urgency += priorityCoefficient * 0.325; break
    case 5: urgency -= priorityCoefficient * 0.3; break
  }

  return Math.round(urgency * 100) / 100
}

silverbullet.registerFunction({ name: "parseUrgency" }, parseUrgency)

// Continue matching and trimming attributes from the end of the
// task until none are left, or we hit a max limit in which case
// there is probably something wrong. If a duplicate property is found
// it is discarded. This has the effect of last property wins.
const extractAttributes = (name) => {
  const attrs = { tags: [] }
  const extractTag = ([prop, regex]) => {
    const match = regex.exec(name)
    if (!match) return false

    const value = match[1]
    const previous = attrs[prop]
    if (Array.isArray(previous)) previous.push(value)
    else attrs[prop] = value

    name = name.slice(0, match.index)
    return true
  }

  let max = 20
  while (extractors.some(extractTag) && max) max--
  if (!max) console.warn(`Attribute limit exceeded for:`, name)

  const { priority } = attrs
  if (priority) attrs.name = `${name.trimEnd()} ${priority}`
  else attrs.name = name
  
  // attrs.happens = happens
  attrs.priority = parsePriority(priority)
  // attrs.urgency = parseUrgency(attrs)

  return attrs
}

silverbullet.registerAttributeExtractor({ tags: ["task"] }, (text) => {
  let { tags, ...attrs } = extractAttributes(
    text.trimEnd().replace(checkbox, "")
  )

  // Re-append the tags to the name. Currently, SilverBullet extracts
  // tags ahead of this function running so it will never have items
  // but it is kept here for feature parity with Obsidian Tasks.
  if (tags.length) attrs.name += ` ${tags.join(" ")}`

  // If the task is cancelled, override the done state
  if (text.startsWith("[-]")) attrs.done = true

  return attrs
})

const appendDate = async (from, emoji) => {
  const now = Temporal.Now.plainDateISO().toString()
  await syscall("editor.dispatch", {
    changes: { from, insert: ` ${emoji} ${now}` },
  })
}

const finalDates = new RegExp(
  `${matchers.completed.source}|${matchers.cancelled.source}`,
  "gu"
)

const stateChanges = {
  // Completed
  "x": ({ to }) => appendDate(to, "✅"),
  // Incomplete, remove finalization dates from the task
  " ": async ({ from, text, to }) => {
    const insert = text.replaceAll(finalDates, "")
    if (insert === text) return 

    // The raw text includes the checkbox area which has the old task
    // state and we don't want that.
    const [current] = checkbox.exec(text) || [""]
    const offset = current.length
    await syscall("editor.dispatch", {
      changes: {
        from: from + offset, 
        to, 
        insert: insert.slice(offset),
      },
    })
  },
  // Cancelled
  "-": ({ to }) => appendDate(to, "❌")
}

// Add or remove finalization date when changing a task
silverbullet.registerEventListener(
  { name: "task:stateChange" },
  ({ data }) => stateChanges[data.newState]?.(data)
)

const toDateString = (date) => date
  .toTemporalInstant()
  .toZonedDateTimeISO(Temporal.Now.timeZoneId())
  .toPlainDate()

// Setup a new recurring task when one is completed.
silverbullet.registerEventListener(
  { name: "task:stateChange" },
  async ({ data }) => {
    if (data.newState !== "x") return
    const { from, text } = data
    const match = matchers.recurrence.exec(text)
    if (!match) return // non-recurring

    const [,deadline] = matchers.deadline.exec(text) || []
    if (!deadline) {
      return syscall(
        "editor.flashNotification",
        "No deadline found for recurrence"
      )
    }

    const { RRule } = await rrule
    const start = new Date(`${deadline}T00:00:00`)
    const rule = new RRule({
      ...RRule.parseText(match[1].trimEnd()),
      dtstart: start,
    })
    const next = rule.after(start)
    const nextLine = text.replace(
      matchers.deadline,
      ` 📅 ${toDateString(next)}`
    )

    await syscall("editor.insertAtPos", `${nextLine}\n- `, from)
  }
)

Features implemented and roadmap:

  • Extract all Obsidian-Tasks style emoji attributes. The RegExps are copied nearly verbatim from Obsidian-Tasks. Priorities are numerically identical. :white_check_mark: 2024-03-04
  • Completed/Cancelled dates are appended AND deleted when checked/unchecked. :white_check_mark: 2024-03-04
  • Recurring tasks are copied to a new line when completed. :white_check_mark: 2024-03-05
  • Compute urgency property (source) :white_check_mark: 2024-03-08
    Currently, SB extracts deadline ahead of us so we are unable to fully compute it during indexing. It is registered as a custom function parseUrgency(task) which can be used in templates and queries.
  • Compute happens property (the earliest of start date, scheduled date, and due date)
  • Add heading property set to the value of the nearest heading.
7 Likes

I updated the script, which now creates new tasks when a task recurrence emoji :repeat: is found. The recurrence string uses natural language and is parsed using rrule. For instance:

  • Clean Dishwasher :repeat: every month :date: 2024-03-01

When you complete the task, a new one is added using the deadline date as a reference point. The next one will be added above the one you just finished with a new deadline corresponding to the recurrence rule, resulting in the following:

  • Clean Dishwasher :repeat: every month :date: 2024-04-01
  • Clean Dishwasher :repeat: every month :date: 2024-03-01 :white_check_mark: 2024-03-05

I fixed a bug that came about as a result of a small refactor. Recurring tasks should work now!

This is pretty incredible, and I’m both surprised and delighted you can import external libraries in space script. I never even tried this. Very nice work, this is very cool.

On a more “philosophical” level this brings up the question: is this how we want people to (ab)use space script? Isn’t this pushing past the reasonable limit of how much JavaScript code you are going to want to write (and edit) on a SilverBullet page, and is that a good idea? I mean SB is not a proper IDE. You don’t get very good error messages I’m sure. Is there a threshold where it would be better to develop something like this as a plug rather than a space script?

This is an honest question and I don’t have a real answer. What has your experience been developing this way?

Btw, this seems to be implementing this so that’s cool:

1 Like

Personally, I really like the idea of being able to import from external sources. Being a self-hosted application for personal knowledge management, I think it should be up to each individual whether they want to use this script. From a security standpoint, there’s some valid concerns, but no more or less than installing a plug. With that in mind, it could certainly be locked down with a well made Content Security Policy (CSP) which, if you were to add anything, it should be convenient management of this CSP.

As for improving the Developer eXperience, I did encounter a bit of friction around importing the script. You may have noticed it’s not an ESM import, but rather an async runtime import. This was the only way I found to get it to work, and I couldn’t even await it at the top-level. I think if one were to improve the platform, it would be to both allow ESM imports, as well as top-level await.

To be clear, this is how plugs work too. While a lot of functionality is distributed with SB itself, there’s third-party optional plugs too, e.g. Plugs/TreeView and Plugs/AI

Very cool, thanks for sharing! Much more thorough than my attempt. I’ll be trying it out, recurring tasks are something I wanted to look into soon too.

1 Like

I’ve used your awesome script and I was able to reach the start, created, scheduled, deadline, completed, recurrence property.

For the priority despite my effort the only priority I get is the number 3, the other are non considered.
Even in the table view of the tasks the icon is not stripped.

Thanks for using it! I did encounter bug where the priority isn’t properly extracted. I’ll post a fix a little later.

Huh, for some reason I’m no longer able to edit the post. @zef, is there an edit limit that I’ve hit?

In any case, here’s the fix. Change the line that looks like this:

To this (remove ?: from the regex):

Also note that properties will only be extracted from the end of the string, so you’ll need to move the priority emoji to the end of the line.

1 Like

Not sure, maybe Discourse has some fancy logic around this. I just made your original post a wiki, not sure if that helps. Can you edit it again?

1 Like

A couple small improvements were made:

  • Priority matching emoji have been centralized. For those who wish to change the emoji used for determining priority, you just have to change it in the one spot towards the top.
  • The priority emoji is put back in to the name attribute so it will show up when displaying the task. This gives more clear insight in to task order for the upcoming urgency property.
  • Cancelled tasks (those with [-]) are considered done.
2 Likes

@Pietzaahh This is a great script.
We have so many options that I don’t know which one to use these days :sweat_smile:

Anyhow, I checked back with my colleague “Gemini” and it gave me back this script with priorities working.
It also refactored a bit more the code.

Hope you can use it to continue implementing your ideas:

```space-script
const rrule = import("https://esm.sh/[email protected]")

const checkbox = /^\[([^\]]+)]\s+/
const hashTags = /(^|\s)#[^ !@#$%^&*(),.?":{}|<>]+/g
const taskId = /[a-zA-Z0-9-_]+/.source

// Priority emojis and their mappings
const priorityEmojis = {
   "❗": 1, // highest
   "⏫": 2, // high
   "🔼": 3, // medium
   "🔽": 4, // low
   "⏬": 5, // lowest
}

const matchPriority = Object.keys(priorityEmojis).join("|")

// Task-related emojis for date attributes  
const taskEmojis = {
    "🛫": "start",    
    "➕": "created",
    "⏳": "scheduled", 
    "📅": "deadline",
    "✅": "completed",
    "❌": "cancelled"
};

// Matchers including priority and additional task emojis
const matchers = {
  priority: new RegExp(`\\s+(${matchPriority})`, "u"),
  ...Object.fromEntries(Object.entries(taskEmojis).map(
     ([emoji, attribute]) => [attribute, new RegExp(`\\s*${emoji}\\s*(\\d{4}-\\d{2}-\\d{2})`, "u")]
  )),
  recurrence: /\s+🔁\s*([a-zA-Z0-9, !]+)/iu,
  depends: new RegExp(`\\s+⛔️\\s*(${taskId}( *, *${taskId} *)*)`, "iu"),
  id: new RegExp(`\\s+🆔\s*(${taskId})`, "iu"),
  tags: new RegExp(hashTags.source, "iu"),
}

const extractors = Object.entries(matchers).map(([prop, regex]) => [
  prop,
  new RegExp(regex.source + "$", regex.flags),
])

// Helper function to parse priorities from strings (fallback)
const parsePriority = (emoji) => emoji ? priorityEmojis[emoji] || 3 : 3; // Default to medium

// Extract attributes including priority and dates
const extractAttributes = (task) => {
  let attrs = { tags: [] }

  task = task.replace(checkbox, "").trim(); // Trim checkbox first

  for ([prop, regex] of extractors) {
    const match = regex.exec(task)
    if (!match) continue;

    const value = attrs[prop];
    if (Array.isArray(value)) value.push(match[1])
    else attrs[prop] = match[1] 

    task = task.slice(0, match.index).trimEnd();
  }

  // Prioritize emoji-based priorities
  attrs.priority = priorityEmojis[task.match(new RegExp(matchPriority))] || parsePriority(attrs.priority); 

  attrs.name = task.trim();
  return attrs;
}

silverbullet.registerAttributeExtractor({ tags: ["task"] }, (text) => {
  let { tags, ...attrs } = extractAttributes(text);

  // Tags stay at the end for feature parity
  if (tags.length) attrs.name += ` ${tags.join(" ")}`

  // If the task is cancelled, override the done state
  if (text.startsWith("[-]")) attrs.done = true

  return attrs
})

const appendDate = async (from, emoji) => {
    const now = Temporal.Now.plainDateISO().toString();
    await syscall("editor.dispatch", {
        changes: { from, insert: ` ${emoji} ${now}` },
    });
}

const finalDates = new RegExp(
  `${matchers.completed.source}|${matchers.cancelled.source}`,
  "gu"
)

const stateChanges = {
  // Completed
  "x": ({ to }) => appendDate(to, "✅"),
  // Incomplete, remove finalization dates from the task
  " ": async ({ from, text, to }) => {
    const insert = text.replaceAll(finalDates, "")
    if (insert === text) return 

    // The raw text includes the checkbox area which has the old task
    // state and we don't want that.
    const [current] = checkbox.exec(text) || [""]
    const offset = current.length
    await syscall("editor.dispatch", {
      changes: {
        from: from + offset, 
        to, 
        insert: insert.slice(offset),
      },
    })
  },
  // Cancelled
  "-": ({ to }) => appendDate(to, "❌")
}

// Add or remove finalization date when changing a task
silverbullet.registerEventListener(
  { name: "task:stateChange" },
  ({ data }) => stateChanges[data.newState]?.(data)
)

const toDateString = (date) => date
  .toTemporalInstant()
  .toZonedDateTimeISO(Temporal.Now.timeZoneId())
  .toPlainDate()

// Setup a new recurring task when one is completed.
silverbullet.registerEventListener(
  { name: "task:stateChange" },
  async ({ data }) => {
    if (data.newState !== "x") return
    const { from, text } = data
    const match = matchers.recurrence.exec(text)
    if (!match) return // non-recurring

    const [,deadline] = matchers.deadline.exec(text) || []
    if (!deadline) {
      return syscall(
        "editor.flashNotification",
        "No deadline found for recurrence"
      )
    }

    const { RRule } = await rrule
    const start = new Date(`${deadline}T00:00:00`)
    const rule = new RRule({
      ...RRule.parseText(match[1].trimEnd()),
      dtstart: start,
    })
    const next = rule.after(start)
    const nextLine = text.replace(
      matchers.deadline,
      ` 📅 ${toDateString(next)}`
    )

    await syscall("editor.insertAtPos", `${nextLine}\n- `, from)
  }
)
```

One thing that would be great to improve is to update “cancelled” tasks without the need to click on the - [-] part, just when the - is inserted, at change task state.
I tried to fix that but it didn’t work out.

I tried to add the following, in case it matters:

// Cancelled
"-": async ({ to }) => {  // Changed to async
    await appendDate(to, "❌"); // Directly append the date
}

New version with Urgency parsing. Note that currently SB extracts the deadline attribute before this script is able to parse it, meaning a critical property of urgency is missing at the time of extraction. Instead, I made the parsing available as function call: parseUrgency(task)

Is it possible to add a “Category” property?

I usually categorize with an icon to differentiate the reminder outfit.

My task are of this type:

  • :date: 2024-06-06 :moneybag: Home Insurance
  • :date: 2024-07-15 :classical_building: Visit air show
  • :date: 2024-09-10 :toolbox: Work Travel
  • :classical_building: Visit air museum

where the :toolbox: :moneybag: :classical_building: are my category work, money , museum, etc?.

Usually the category is the first “character” of the string.

Thanks.

Marco

That’s a neat way to categorize tasks! I’ll admit I probably won’t implement this myself as my goal with this post is to mimic Obsidian-Tasks, but you can fairly easily augment the task objects yourself using the SilverBullet API alongside mine.

I can get you started here:

const contexts = {
  "🧰": "work",
  "💰": "money",
  "🏛": "museum",
}
const emoji = Object.keys(contexts).join("|")
const matcher = new RegExp(`\\s+(${emoji})`, "u")

silverbullet.registerAttributeExtractor({ tags: ["task"] }, (text) => {
  const match = matcher.exec(task)
  return match
    ? { context: contexts[match[1]] }
    : undefined
})

Feel free to start new thread and we can discuss it further!

1 Like

Thanks for the scripts which I am trying to get up and running in my workflow. One thing I noticed recently, if you edit the tasks on their actual page (where they originate) it is properly repeating and adding a new task with the new due date set. But if I click the task completed from somewhere else using a live query, it completes the task but the new task with the repeating is not created. So some event is not being triggered when completing from a live query as opposed to completing from the real page. Any ideas?

Yes, this is a known issue: Event `task:stateChange` isn't fired for "external" tasks · Issue #788 · silverbulletmd/silverbullet · GitHub

1 Like