IEEE

Electrical and Computer Engineering Researches (ICECER), International Conference on

This paper proposes and evaluates a novel training methodology for a Byte-Pair Encoding (BPE) tokenizer that utilizes a comprehensive morphological dictionary instead ...