The tokenize () method is a wrapper for the StringTokenizer class. The StringTokenizer constructor takes two String arguments: one for the input and one for the delimiter.

In this article, you'll take advantage of the commonly used StringTokenizer class to perform better tokenization of complicated and information-rich strings.

It doesn't tokenize, it just splits; there's a difference -- if your tokens have embedded separators (consider, a space-delimited sequence of tokens, where tokens can have backslash-escaped ...