/sys/doc/ Documentation archive

include "sys.m";
sys := load Sys Sys->PATH;

tokenize: fn(s, delim: string): (int, list of string);

Words are delimited by the maximal sequences of any character from the delim string. Tokenize skips delimiter characters at the beginning and end of s, so each element in the returned list has non-zero length.

If s is nil or contains no words, tokenize returns a count of zero and a nil list.

Delim may be nil or the empty string, specifying no delimiter characters. The resulting word list will be nil (if s is nil or the empty string) or a single-item list with a copy of s.