Hello! syncabook is awesome, and indeed Storyteller does take "the opposite" approach when it comes to forced alignment.
Others have linked to the docs, where I go into detail about the syncing algorithm, but at a high level:
Storyteller uses Whisper to transcribe the audio to text (this is the most computationally expensive part of the process)
Then we use a Levenshtein-distance-based fuzzy search algorithm to find each chapter in the text (this is attempting to account for the difference between audio and text versions, as you said!)
Then for each chapter, we find the start and end timestamp of each sentence, again using a fuzzy search across the transcription.
In general, Storyteller does a pretty good job; it treats the ebook as the source of truth, which means that at the moment it sometimes misses introductory and ending pieces of the audiobook, though it's on the roadmap to have some support for explicitly triggering those when that happens.
Others have linked to the docs, where I go into detail about the syncing algorithm, but at a high level:
Storyteller uses Whisper to transcribe the audio to text (this is the most computationally expensive part of the process)
Then we use a Levenshtein-distance-based fuzzy search algorithm to find each chapter in the text (this is attempting to account for the difference between audio and text versions, as you said!)
Then for each chapter, we find the start and end timestamp of each sentence, again using a fuzzy search across the transcription.
In general, Storyteller does a pretty good job; it treats the ebook as the source of truth, which means that at the moment it sometimes misses introductory and ending pieces of the audiobook, though it's on the roadmap to have some support for explicitly triggering those when that happens.