Feature #2167
Caches on Java side some R data values
Status: | New | Start date: | 04/25/2017 | |
---|---|---|---|---|
Priority: | Normal | Due date: | ||
Assignee: | - | % Done: | 0% |
|
Category: | Stats / R | Spent time: | - | |
Target version: | TXM 0.8.4 |
Description
Discuss about caches on Java side some R data values or not.
At this time, some results do that and some others don't.
Eg. At this moment it seems there is a bug with CAH and large corpus, R consumes up to 2 Go od RAM and never ends. Maybe cache can help.
Eg. Lexical Table, add cache to getFreqs() method to not query R each time frequencies are needed.
(Discuss about this because we may want the Java object reflects the R object. At this moment it is not uniform for all TXM results)
Original :
@Override public List<Integer> getFreqs() { ArrayList<Integer> freqs = new ArrayList<Integer>(); ArrayList<double[]> cols = new ArrayList<double[]>(); int Nrows = this.getNRows(); int Ncols = this.getNColumns(); for (int i = 0; i < Ncols; i++) try { cols.add(this.getCol(i).asDoubleArray()); } catch (RException e) { // TODO Auto-generated catch block org.txm.utils.logger.Log.printStackTrace(e); } catch (RWorkspaceException e) { // TODO Auto-generated catch block org.txm.utils.logger.Log.printStackTrace(e); } catch (StatException e) { // TODO Auto-generated catch block org.txm.utils.logger.Log.printStackTrace(e); } int sum = 0; for (int i = 0; i < Nrows; i++) { sum = 0; for (int j = 0; j < Ncols; j++) sum += (int) cols.get(j)[i]; freqs.add(sum); } return freqs; }
Proposal :
@Override public List<Integer> getFreqs() { // caching if(this.frequencies == null) { this.frequencies = new ArrayList<Integer>(); ArrayList<double[]> cols = new ArrayList<double[]>(); int Nrows = this.getNRows(); int Ncols = this.getNColumns(); for (int i = 0; i < Ncols; i++) try { cols.add(this.getCol(i).asDoubleArray()); } catch (RException e) { // TODO Auto-generated catch block org.txm.utils.logger.Log.printStackTrace(e); } catch (RWorkspaceException e) { // TODO Auto-generated catch block org.txm.utils.logger.Log.printStackTrace(e); } catch (StatException e) { // TODO Auto-generated catch block org.txm.utils.logger.Log.printStackTrace(e); } int sum = 0; for (int i = 0; i < Nrows; i++) { sum = 0; for (int j = 0; j < Ncols; j++) sum += (int) cols.get(j)[i]; this.frequencies.add(sum); } } return this.frequencies; }
History
#1 Updated by Sebastien Jacquot about 6 years ago
- Description updated (diff)
#2 Updated by Sebastien Jacquot about 6 years ago
- Description updated (diff)
#3 Updated by Sebastien Jacquot over 4 years ago
- Target version changed from TXM 0.8.0a (split/restructuration) to TXM 0.8.2
#4 Updated by Sebastien Jacquot over 3 years ago
- Subject changed from Lexical Table, add cache to getFreqs() method to not query R each time frequencies are needed to Caches on Java side some R data values
- Description updated (diff)
- Category changed from Development to Stats / R
#5 Updated by Sebastien Jacquot over 3 years ago
- Description updated (diff)
#6 Updated by Matthieu Decorde about 2 years ago
- Target version changed from TXM 0.8.2 to TXM 0.8.4