Bug #1872

RCP: 0.7.8beta: slow annotation creation when many word annotations are created

Added by Matthieu Decorde over 4 years ago. Updated 11 months ago.

Status:Feedback Start date:09/19/2016
Priority:Normal Due date:
Assignee:- % Done:

80%
Category:Annotation Spent time: -
Target version:TXM 0.8.1

Description

When many annotations are created the current code creates as many SQL transactions as annotations to create.

Note : Windows TXM seems slower as annotations are created.

Solution

  1. Create annotation creation methods without transaction and update concordance editor to use only one transaction.
  2. flush & clear the entity manager each new 1000 created entity
  3. if not enough, commit the transaction each new 10000 created entity

Annotation creation time differs from annotation update time :
Create F10 C20: 122s
Create F1 C20: 14s
Create F10 C10: 87s
Create F1 C10: 15s
Update F1,F10 C10,20: 2,5s -> flushing is enough to speed up the process

Validation test

  • Create a concordance with [] query
  • annotate more than 50 000 lines in the concordance
  • check if the duration is OK

History

#1 Updated by Matthieu Decorde about 4 years ago

  • Description updated (diff)
  • % Done changed from 0 to 60

FR
L'annotation est plus rapide depuis qu'il n'y a qu'une seule session. Il reste toujours le problème de la vérification du chevauchement qui se fait pour chaque annotation et pour toutes les structures CQP.

les voies d'amélioration sont :
  • de ne pas vérifier les chevauchements
  • de limiter la vérification qu'à certaines structures CQP
  • de vérifier le chevauchement de toutes les annotations d'une seule fois
  • de changer de techno ?

#2 Updated by Matthieu Decorde over 3 years ago

  • Target version changed from TXM 0.7.8 to TXM 0.8.0a (split/restructuration)

#3 Updated by Sebastien Jacquot over 2 years ago

  • Target version changed from TXM 0.8.0a (split/restructuration) to TXM 0.8.0

#4 Updated by Matthieu Decorde almost 2 years ago

  • Target version changed from TXM 0.8.0 to TXM 0.8.2

#5 Updated by Sebastien Jacquot 12 months ago

  • Subject changed from RCP: 0.7.8beta: slow annotation creation when multiple annotation are created to RCP: 0.7.8beta: slow annotation creation when many annotations are created
  • Description updated (diff)
  • Status changed from New to Feedback
  • Target version changed from TXM 0.8.2 to TXM 0.8.1
  • % Done changed from 60 to 80

#6 Updated by Matthieu Decorde 12 months ago

  • Subject changed from RCP: 0.7.8beta: slow annotation creation when many annotations are created to RCP: 0.7.8beta: slow annotation creation when many word annotations are created
  • Description updated (diff)
  • % Done changed from 80 to 50

#7 Updated by Matthieu Decorde 11 months ago

  • Description updated (diff)
  • % Done changed from 50 to 80

#8 Updated by Matthieu Decorde 11 months ago

  • Description updated (diff)

#9 Updated by Matthieu Decorde 11 months ago

  • Description updated (diff)

Also available in: Atom PDF