Page Comparison

Description

Export files Create user-configurable profiles to specify options governing import of bibliographic, instance, and instance item data from OLE for transmission to a variety of destinations, including the public discovery layer, external utilities, or other databases. Exported files may be into OLE. Profiles will be applied to imports of any size, ranging from several records to copies of the entire bibliographic and instance portions of the database. Exports will be governed by profiles that support selection and data manipulation according user-defined criteria.

Exports may be ad hoc jobs performed immediately, may be scheduled to run at a particular date and time, or may be setup to run at regular intervals (e.g., nightly export to apply updates to the discovery layer). Output will be in a standard format, such as MARC or MARCXML.

Support of the public discovery layer is a critical export function. OLE does not provide a public discovery layer, and as a result, bibliographic and instance data must be exported from OLE for use by the discovery layer. Existing discovery layer systems expect MARC21 bibliographic records with a subset of associated instance data in local data fields. Institutions have their own particular requirements for type of export (full database, incremental updates) and scheduling to maintain a variety of discovery layer products. OLE needs to be able to export a full copy of the database and to export incremental updates to bib or instance data as required by the library. Updates include additions, changes, and deletions, as well as changes in staff-only status. Data that formerly were staff-only should be added to the exporta single record to thousands of records. Different profiles need to be created to deal with a variety of situations. Once created, a profile typically will be used many times over, and may be applied to whichever type of import it is appropriate – single bib, batch file import, etc.

Import profiles will harmonize with and enhance existing single bib import User Preferences and Staff Upload (the latter for creating requisitions).

User Stories

Priority Rank Order

Design Technical

Service Design and Implementation

Screen Flow Diagram

Layout use cases based on the screen flows indicating navigation between different screens and pop-ups. 

Security

• Roles – list of roles to be defined
• Permissions – list of permissions to be defined
• User Profiles – user profiles that may be applied roles/permissions (also for test/mock purposes) 

Rules

• Context & Agendas
• Terms & Prepositions
• Sample XML file

Component Design

• OJB.xml
• Class Diagram

• State Chart Diagram
• Psuedo Code

1. User creates a batch profile using the options provided in batch profile screen
2. User schedules a batch job for a given data and time
3. BatchProcessJobManager will perform the job scheduling by calling scheduleJob(ScheduleJobBo bo) method. The schedule information (storeds as cron expression) and the job information is stored in db
4. The Quartz scheduler which is configured in KRMSLocalSpringBean.xml calls the BatchProcessManager.execute(int processId) method.
5. BatchProcessManager will load the profile(BatchProcessProfile) created by user based on the process id supplied and call the factory class to create BatchProcess instance.
6. The BatchProcessFactory creates the appropriate BatchProcess instance based on the processType in the profile
7. The processBatch(BatchProcessProfile profile) is called to perform the batch process as per the profile information provided by the user.
   
   
   1. BatchProcessExportData
      This batchprocess is involved in performing export of bib / intance information based on the profile provided by the user. It has the following methods:

      1. batchExport(BatchProcessProfile profile) 

      This method is passed the BatchProcessProfile and writes its output to the filesystem. It is the method used for scheduled exports. It should function as follows

1. A call is made to the getBatchJobInfo(int processId) to get the current job info which executed the processBatch()
2. updateBatchProgress(BatchProcessJobBo jobBo) is called to updated the job running status to started and start time.
3. A call is made to the batchExport(BatchProcessProfile profile) method passing the profile information.
4. The needed values are pulled out of the profile information.
5. exportFilterCriteria - What exportFilterCriteria to use. The exportFormat is read from the exportFilterCriteria
   1. exportType - Full or Incremental
   2. exportId (processId)- Used for saving the incremental dateTime.
   3. exportFileTo - The directory and file name format.
   4. writeReportTo - The directory and report file name.
   5. chunkSize - Max number of records per file.
   6. staffOnly- If staffOnly records should be included.
   7. updateLeveInfo - bibOnly, instance.
6. If the export is incremental 
   1. Get the last export dateTime for the exportId
   2. save the current dateTime exportId pair.
7. call getSolrDocList(BatchProcessProfile profile) to get the solr doc list
8. A call to updateBatchProgress(BatchProcessJobBo jobBo) is called to updated the total number of records to be processed.
9. if there is more than one chunk split the solr results based on the number of chunks data
10. make a call to ExportService.getExportDataBySolr using the exportFormatProfile and one set of solr results which were split as per the chunk size.
11. If isMarc21 is true convert the returned MarcXML to Marc21.
12. Write the returned XML or Marc21 from the export service to files using the exportTo variable.
13. call the updateBatchProgress(BatchProcessJobBo jobBo) passing the jobBo which contains the current job information updated with records processed, % processed and time spent.
14. If there is more chunks make the next call to the ExportService.
15. Create the Report in the location provided in the profile.
16. updateBatchProgress(BatchProcessJobBo jobBo) is called to updated the job running status to completed and end time, total records processed, % processed and time spent.

Service Contracts

Service Implementation

Class Diagram:

 UI Screen

ER Diagram: Batch Process Profile

Batch Process

Data Model

Entity : BATCH DEFINITION