Laboratoire ICAR » Plateforme TXM

• the word space separators field default value '[\p{Z}|\p{C}]+' is incorrect -> fix it to '[\p{Z}\p{C}]+'
  • MD: OK fixed in 0.7.7
• the end of sentence characters field default value '[\p{Ps}|\p{Pe}|\p{Pi}|\p{Pf}|\p{Po}|\p{S}]' is incorrect -> fix it to '[\p{Ps}\p{Pe}\p{Pi}\p{Pf}\p{Po}\p{S}]'
  • MD: OK fixed in 0.7.7
• when any tokenizer parameter field is edited, the import module is always run with the "MISSING TOKENIZER KEY: punct_strong" message, which means that orthographic sentences are never analyzed?
  • MD: OK fixed in 0.7.7, the "punct_strong" was not well updated causing this message display
• what is the purpose of the 'OK' button? -> suggestion: remove the 'OK' button
  • MD: OK removed the "OK" button in 0.7.7

see ticket #1347 for more tokenizer import fixes