MTL Toolbox: Difference between revisions

MTL Toolbox (view source)

8 bytes added , 1 May

m

46,077

edits

@@ Line 71: / Line 71: @@
 ** Example: {{TE|^thaau}}
-Tokenizer: the default tokenizer ("simple") is used. It only does case folding of [[ASCII]] characters, so [[Ø]] is not folded to lower case.
+Tokenizer: the default tokenizer ("simple") is used. Because it only does case folding of [[ASCII]] characters, [[Ø]] and ø do not match each other.
 == See also ==