Setting up Document Store

Owned by Sreekanth TVN (Unlicensed)
Last updated: Apr 08, 2013 by Nora Roggeveen-Sams (Unlicensed)Version comment

1.System Requirements

  • Tomcat 6.0.26 or higher
  • JDK 1.6.0.23 or higher
  • RAM - 3GB(minimum available)
  • Processor - 2 GHz or higher
  • Hard disk - The DocStore WAR file is about 50MB in size.
  • The actual repository size depends on the volume of the documents to be added.
  • The account/user under which Tomcat runs should have rwx access to the DocStore repository folder.

2.Getting the Source Code from SVN

  • Check out the ole-trunk directory from the following SVN Repository:

              ole-trunk

      The following projects will be checked out.

  • ole
  • ole-cfg-dbs
  • ole-common
  • ole-dependencies
  • ole-docstore -- Refer to it as DOCSTORE_PROJ
  • ole-rice2

3.Building the Code

  • Open command prompt and go to the folder: %DOCSTORE_PROJ%

             Run “mvn clean install”.

            oledocstore.war file will be created in %DOCSTORE_PROJ%\ ole-docstore-webapp\target folder.

4.Deployment

4.1  Setting up properties

Go to %TOMCAT_HOME%\bin directory.

Create/open setenv.bat file under %TOMCAT_HOME%\bin directory

Define the following variables in setenv.bat file

Set JAVA_OPTS as below.

set JAVA_OPTS=%JAVA_OPTS% -Xms512m -Xmx1g -XX:MaxPermSize=256m
set JAVA_OPTS=%JAVA_OPTS% -Dapp.environment=local
set JAVA_OPTS=%JAVA_OPTS% -DOLE_DOCSTORE_HOME="\opt\docstore"
set JAVA_OPTS=%JAVA_OPTS% -DOLE_DISCOVERY_HOME="\opt\docstore\discovery"

Save the file setenv.bat                                        

4.2 Deploying the War file to Tomcat

  •    Copy oledocstore.war file from %DOCSTORE_PROJ%\ ole-docstore-webapp\target to %TOMCAT_HOME%\webapps folder

5.Running Docstore and Discovery

Start the Tomcat Server.

Open a browser.

Navigate to the following URLs.

The following directory structure is created for DocumentStore and Discovery related configuration after the application is started.

DocumentStore

  • \opt\docstore\properties 

Discovery

  • \opt\docstore\discovery\properties
  • \opt\docstore\discovery\solr-config

If the application is not accessible, the following properties may need to be edited and Tomcat restarted.

  •       Open documentstore.properties file under \opt\docstore\properties and modify the port in the following property to match the Tomcat port. (Default is 8080)

                     local.docSearchURL=http://localhost:8080/oledocstore/

  •       Open ole-discovery.properties file under \opt\docstore\discovery\properties and modify the port in the following property to match the Tomcat port. (Default is 8080)

                    docSearchURL=http://localhost:8080/oledocstore/

6.String Ingest

  Refer to User Guide for details

Open a browser.

        Go to URL http://localhost:8080/oledocstore/ and click on “Ingest String Content” tab.

        Copy the sample xml provided in the screen.

        Click Submit button.

The response xml in the browser gives the result of the stream ingest process.

The data can also be verified using the URLs given in the previous section.

7.Bulk Ingest

  Refer to User Guide for details

Copy the following sample data files to the folder specified by the property  local.batch.upload.dir (only this property) in the file \opt\docstore\propertiesdocumentstore.properties. This property should be changed to a directory suitable to the local user's system. (e.g. /ole-test-data/bulkingest). Create this folder if it does not exist.

Please note: There may be issues with file/folder permissions with this directory. Make sure this directory is fully accessible by all users.

  • DOCSTORE_PROJ \ole-docstore-engine\src\test\resources\org\kuali\ole\bulkIngest\files\bulkIngest-Work-Bib-Marc-20.xml    --- This is a sample test data file
  • DOCSTORE_PROJ \ole-docstore-engine\src\test\resources\org\kuali\ole\bulkIngest\files\bulkIngest-Work-Inst-OleML-20.xml   --- This is a sample test data file

Copy the first file (Bib data) to the destination folder specified by the property local.batch.upload.dir in the documentstore.properties file.

After the first file is ingested, copy the second file (Instance data).

 Open a browser.

             Go to URL http://localhost:8080/oledocstore/admin.jsp

             Click on ‘BulkIngest’ button. Need to click only once.

 To verify that the data is stored in DocumentStore, go to URL

            http://localhost:8080/oledocstore

           Click ‘Refresh Summary’ button in the Summary tab.

           Note for the count for each DocType.

To verify that the data is indexed in DocStore, go to URL

         http://localhost:8080/oledocstore/discovery

         Click ‘Refresh’ button in the Summary tab.

         Note for the count for each DocType.

8.LOG Files

Open log4j.properties file under \opt\docstore\properties

Now check the following properties to find the location of log files:

  •  log4j.appender.rolling.File=logs/ole-docstore.log
  •  log4j.appender.errorAppender.File=logs/ole-docstore-error.log

Open corresponding files specified by above properties. If the path is relative , look for it in %TOMCAT_HOME%\bin directory

9.DocStore within OLE(Optional)

Copy the oledocstore.war files into the Tomcat running OLE.

Before building OLE make sure the following properties are correctly set in local-build.properties file in %OLE_HOME% directory.

Copy the file ‘olebibeditor.properties’ provided along with this document in to your %USER_HOME% directory.

Modify the following property in the above file as per your local environment.

 Open a browser.

      Go to URL

       http://localhost:8080/ole-local/portal.do?channelTitle=DocumentStoreSearch&channelUrl=discovery.do

       Click on ‘Search’ button.

Open documentstore.properties file under \opt\docstore\properties and modify the port in the following property to match the Tomcat port. (Default is 8080)

local.docSearchURL=http://localhost:8080/oledocstore/

Operated as a Community Resource by the Open Library Foundation