Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 7 Next »

OLE Change Management TBD

This document describes the mechanics of the database version control process. How the changes get into the master schema must still be decided by the OLE development team.

Process Overview

To manage and track database changes, the Kuali OLE project utilizes Subversion to track the master state of the database at any given point in time. The ole-cfg-dbs project in the Subversion repository is considered the definitive source and is stored as a combination of XML (for database structure) and flat files (for data.)

To make changes to this master SVN project easier, the project also manages a master database schema (OLEDBA). This schema is exported nightly to the ole-cfg-dbs/development path in SVN. This allows changes to be made to the master schema using normal database tools rather than requiring the "DBA" to edit the files in SVN. Any changes made to the schema during the day will be exported by the DB-Export-OLEDBA job in Hudson every night. For this reason, never update the files in the ole-cfg-dbs/development path directly, as they will be overwritten by the state of the OLEDBA schema the next night.

Then, using this SVN project, any of the development or testing databases can be refreshed from that stored definition. In fact, on a daily basis, the OLECI (for unit testing), OLELOCAL (for developers without a local database), and OLEDEV (for the http://dev.ole.kuali.org:8080/ole-dev/ instance) are deleted and restored from this project.

The additional complication to this is that there are actually three components to the full refresh process as will be discussed below.

Rice Data Overview

Since OLE is one of a number of Kuali Rice-based projects, an installation of it also includes certain Rice tables and their associated data. However, storing this data in our master schema leads to maintenance issues when the master structure changes and potential data collisions with other Kuali projects. As such, we rely on the "master" rice datasource directly.

The database impex tool (https://test.kuali.org/svn/kul-cfg-dbs/branches/clover-integration) allows for two database definitions to be overlaid upon one another. So, for a simple install of OLE, where the Rice server is integrated into the application, the impex tool first imports the Rice tables and data into the OLE database and then adds the OLE tables and data.

This, however, does not then include any of the OLE-specific data which needs to be stored in the Rice tables. This includes items such as:

  • Workflow Documents (KEW)
  • System Parameters (KNS)
  • Test Users (KIM)
  • Application-Specific Roles (KIM)
  • Permissions (KIM)
  • Responsibilities (KIM)

To import those, we have two additional steps. (This will hopefully be a single step in the future, when Rice provides for XML import/export for all of these objects.)

  1. Run Liquibase scripts to add the KNS and KIM data.
  2. Run the workflow XML Ingester tool (part of Rice) to add the workflow definitions.

The definitions for these items are stored in the following locations:

  • ole/work/db/rice-data (KIM/KNS)
  • ole/work/workflow (KEW)

After all "normal" import processes are done, these scripts/files are executed/imported as a final step to load the OLE data into the database.

ole-cfg-dbs Project Structure

Directory

Description

db-impex

The database import/export tool used to process the contents of the directory below. Loaded automatically from the Kuali SVN repository.

development

The OLE database description - see below.

rice

The matching Rice database description - loaded from the rice project.

Within the ole-cfg-dbs/development directory, there are four major items:

File/Directory

Description

schema.xml

This file represents the structure for the entire database schema. This contains all tables, views, sequences, indexes, and constraints.

data

Each file in this directory contains the data for a single table. These files are maintained in a pipe (|) delimited flat file format.

graphs

The "programs" used by Clover ETL to perform the import. Their main purpose is to map between the files and the database, translating any placeholders for pipes in the data. Graphs only exist for tables which contain data. For performance reasons, tables without any data have their graphs purged during the export process.

formats

The structure used for both the data file and the import into the database. Referenced by the graphs during execution.

Also, the ole-cfg-dbs project directory contains an svn:externals property which pulls in the correct version of Rice for OLE. (See the subversion documentation for more information.) This causes SVN to pull the needed definitions directly from the Rice project.

Rice Data Locations and Structure

The Rice data needed by OLE is stored within the main ole project in the locations below:

Directory

Description

ole/work/db/rice-data

Rice KNS and KIM data. (Parameters, Roles, Permissions, etc...)

ole/work/workflow

OLE workflow document definitions.

The non-workflow Rice data scripts use Liquibase for inserting the data into the Rice tables. Most of them (for practical reasons), simply load a CSV file with the OLE data into the needed tables. Review the Liquibase XML files in the ole/work/db/rice-data for the details. They are split up into two sets, demo and bootstrap. Under normal operation, both are executed. Usually, the demo scripts add data not in the bootstrap scripts but in at least one case (system parameters), it overlays the entire set of OLE parameters.

OLEDBA Master Schema

All changes to the base OLE database structure and baseline demonstration data must be done within this schema. The schema is located at:

JDBC URL:

jdbc:oracle:thin:@oracle.ole.kuali.org:1521:XE

Username:

OLEDBA

Changes made there will be exported into the ole-cfg-dbs SVN project nightly and used to refresh the OLELOCAL, OLEDEV, and OLECI databases. (as well as any developer satellite instances the next time they run the process)

Document TODO Notes

  • DBA schema
  • satellite databases
  • database refresh
  • Rice data
    • master schema from Rice project
    • OLE Data Overlay
      • Liquibase
      • Workflow
  • bootstrap vs. demo
  • No labels