OLE Testing Data Persistence Options

There are a few options for getting testing data into the base dataset. KFS has used a few different methods over its development cycle. I'll try to list out some options. They are not mutually exclusive.

Part of this will involve updating the master datasource. More on that is described in OLE Master Data Source Management Process. But, I will refer to that master datasource here as OLEDBA. (Happens to be the name of the Oracle database schema.)

Option 1 - Update OLEDBA Directly

At any time, you can request that data be added to the master data source. Any database changes made to that schema will be exported into version control and then imported into the test environments when their databases are reloaded.

This option requires that you know in advance what tables need to be updated and how to load the data properly for the system functionality.

Option 2 - Stage Changes in a Test Environment

For more complex data updates (such as entering documents which have many affected tables), you can dedicate a (or bring up a special) test environment for the data entry. Then, bring down the instance and extract the updated data using a database tool. You then replay that data into the OLEDBA schema.

The trick with this option is identifying all the changes you want to extract. This is a place that the impex tool could help. Take an export of the database after the data entry into a new directory. Then, compare that to the baseline export from subversion. The data files which have been changed will tell you which tables were updated. This will probably still require some manual filtering out of the "noise" tables, but will definitely show you every table which was modified.

Option 3 - Special Batch Jobs to Load Data

This is a trick used by a couple modules within the KFS application. They have special batch steps which generate data in the test environments at some point after server startup. At the end of the generation process, they set a system parameter which lets the job know that it has run. This prevents it from running in the same instance in the future.

The purchasing module has a job which builds a large number of purchasing documents for the purpose of having something in there for users to test against. In their case, it is building them with hard-coded data. But, there is nothing saying that such a job could not be reading something else and creating the documents. (Such as a CSV or XML file.) This data could either be checked into the project itself so that it is compiled on the server. Or, the job could load it from somewhere else. (directly from SVN, in a new subdirectory in the ole-cfg-dbs project.

As an example, the configuration below is in spring-test-env-beans.xml, a special spring file which is only included in testing environments. It defines a new job with a trigger that runs it 5 minutes after server startup.

    <bean id="purapModuleConfiguration" parent="purapModuleConfiguration-parentBean">
   		<property name="jobNames">
			<list merge="true">
				<value>purapMassRequisitionJob</value>
			</list>
		</property>
		<property name="triggerNames">
			<list merge="true">
				<value>purapMassRequisitionJobTrigger</value>
			</list>
		</property>    
    </bean>
	
    <bean id="purapMassRequisitionStep" class="org.kuali.kfs.module.purap.batch.PurapMassRequisitionStep" parent="step">
	    <property name="documentService" ref="documentService" />
	    <property name="requisitionService" ref="requisitionService" />
	    <property name="purapService" ref="purapService" />
	    <property name="boService" ref="businessObjectService" />
	    <property name="psService" ref="persistenceStructureService" />
    </bean>
    
	<bean id="purapMassRequisitionJob" parent="scheduledJobDescriptor">
		<property name="steps">
			<list>
				<ref bean="purapMassRequisitionStep" />
			</list>
		</property>
	</bean>
	
	<bean id="purapMassRequisitionJobTrigger" parent="simpleTrigger">
		<property name="jobName" value="purapMassRequisitionJob" />
        <property name="startDelay" value="300000" />
        <property name="repeatCount" value="0" />
	</bean>