FWIW I think most of the "tokenization problems" List of actual tokenizarion lim... | Hacker News

Hacker News new | past | comments | ask | show | jobs | submit

login

TZubiri 76 days ago | parent | context | favorite | on: Something weird is happening with LLMs and chess

FWIW I think most of the "tokenization problems"

List of actual tokenizarion limitations 1- strawberry 2- rhyming and metrics 3- whitespace (as displayed in the article)

Consider applying for YC's Spring batch! Applications are open till Feb 11.
Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact