semantic-repository-0.5.2: deploymet update

hi,
as some of you know, semantic repository version 0.5.2 has some problem with IOException (too many files open)
which was stopping repository to perform further search. thats what was reason to get empty search result.
after searching a while, we found the problem (which was also mentioned in lucene FAQ document).

by default bash shell allow limited files to open, since we had many index files, lucene has to open them up during performing search.
so when it exceeds the limit of 1025 files (which is default on our production environment). our application threw the following exception -
Caused by: java.io.FileNotFoundException: /var/indexes/ads-index/_gm.tvd (Too many open files)
at java.io.RandomAccessFile.open(Native Method)
at java.io.RandomAccessFile.(RandomAccessFile.java:212)
at org.apache.lucene.store.FSDirectory$FSIndexInput$Descriptor.(FSDirectory.java:506)
at org.apache.lucene.store.FSDirectory$FSIndexInput.(FSDirectory.java:536)
at org.apache.lucene.store.FSDirectory.openInput(FSDirectory.java:445)
at org.apache.lucene.index.TermVectorsReader.(TermVectorsReader.java:70)

after increasing this limit to 10,000 we didn’t find such problem exists. we had to apply “ulimit -n 10000″ command on bash shell to make it works.

TODO: possible bug, probable our system is opening up several index searcher.

about semantic repositroy –

semantic repository service is intended for indexing content from different sources and maintain multi indexes for different types of content and perform different types of search. yet another solr type indexing service on top of lucene but it will gradually support content versioning and more semantic search result.

Advertisement