Task #2678

Improve partition creation process duration (improve query for part creation)

Added by Sebastien Jacquot about 4 years ago. Updated over 2 years ago.

Status:New Start date:03/29/2016
Priority:Normal Due date:
Assignee:- % Done:

0%

Category:Diagnostic and optimization Spent time: -
Target version:TXM 0.8.4

Description

Queries of type: [_.sp_n="97"] expand to sp
are very heavy.

There are some optimization in /org.txm.searchengine.cqp.core/src/org/txm/searchengine/cqp/corpus/query/SubcorpusCQLQuery.java to avoid them when the part is of "text" level:

if ("text".equals(structure.getName())) { // optimisation only for the text structure
            queryString = "<"+property.getFullName()+"=\""+CQLQuery.addBackSlash(value)+"\">[] expand to "+structure.getName();
        } else {
            queryString = "[_."+property.getFullName()+"=\""+CQLQuery.addBackSlash(value)+"\"] expand to "+structure.getName();
        }

to generate a query of type: <text_booktitle="Waterloo">[] expand to text
Is there a way to extend this optimization?
eg. [_.sp_n="97"] expand to sp
is equivalent to:
<sp_n="97">[] expand to sp

NOTE: after discussion with MD, the 2 queries are not always equivalent. The optimized query "<sp_n="97">[] expand to sp" may not work on a subcorpus. But in the context of Part creation they may always be equivalent?

Solution 1

  • manage all structures levels
  • be able to know if the structure is sometimes out of the current corpus, if not use the <structure> query

Solution 2

  1. execute the query: <sp_n="97">[] expand to sp
  2. if no matches are found then execute the query: [_.sp_n="97"] expand to sp

History

#1 Updated by Sebastien Jacquot about 4 years ago

  • Description updated (diff)

#2 Updated by Sebastien Jacquot about 4 years ago

  • Description updated (diff)

#3 Updated by Serge Heiden about 4 years ago

  • Subject changed from Improve partition creation duration process (improve query of part creation) to Improve partition creation process duration (improve query for part creation)
  • Description updated (diff)

#4 Updated by Serge Heiden almost 4 years ago

  • Description updated (diff)

#5 Updated by Matthieu Decorde over 2 years ago

  • Target version changed from TXM 0.8.2 to TXM 0.8.4

Also available in: Atom PDF