Jekyll2022-06-10T11:24:38-05:00https://ndingwall.github.io/blog/feed.xmlNick DingwallPersonal blog. Opinions and mistakes my own.Tokenization for language modeling: Byte Pair Encoding vs Unigram Language Modeling2020-07-09T00:00:00-05:002020-07-09T00:00:00-05:00https://ndingwall.github.io/blog/tokenization