Token Count Function in SSIS expression

The TOKENCOUNT function returns back the number of times a Token delimiter appears in a string value.

Syntax:

TOKENCOUNT(character_expression, delimiter_string)

Character expression: A string that contains tokens separated by delimiters.

Delimiter string: A string that contains delimiter characters. For example, ";," contains three delimiter characters semi-colon, a blank space, and a comma.

Let’s see an example.

Expression: ‘Bagesh Kumar Singh Greater Noida UP’

The TOKENCOUNT function returns 6 because the string contains three tokens: "Bagesh", "Kumar", “Singh”, “Greater”, “Noida, UP".

  

Keep in mind Delimiter string can’t be empty. If we keep it null it will throw an error.

  

TOKENCOUNT returns 0 (zero) if the character_expression is null.

TOKENCOUNT works only with the DT_WSTR data type. A character_expression argument that is a string literal or a data column with the DT_STR data type is implicitly cast to the DT_WSTR data type before TOKEN performs its operation. Other data types must be explicitly cast to the DT_WSTR data type.