| Note | ||
|---|---|---|
| ||
This document describes the mechanics of the database version control process. How the changes get into the master schema must still be decided by the OLE development team. |
| Table of Contents |
|---|
Process Overview
To manage and track database changes, the Kuali OLE project utilizes Subversion to track the master state of the database at any given point in time. The ole-cfg-dbs project in the Subversion repository is considered the definitive source and is stored as a combination of XML (for database structure) and flat files (for data.)
...
| Wiki Markup |
|---|
{note:title=OLE Change Management TBD} This document describes the mechanics of the database version control process. *How* the changes get into the master schema must still be decided by the OLE development team. {note} {toc} h2. Process Overview To manage and track database changes, the Kuali OLE project utilizes Subversion to track the master state of the database at any given point in time. The {{ole-cfg-dbs}} project in the Subversion repository is considered the definitive source and is stored as a combination of XML (for database structure) and flat files (for data.) To make changes to this master SVN project easier, the project also manages a master database schema (OLEDBA). This schema is exported nightly to the {{ole-cfg-dbs/development}} path in SVN. This allows changes to be made to the master schema using normal database tools rather than requiring the "DBA" to edit the files in SVN. Any changes made to the schema during the day will be exported by the [DB-Export-OLEDBA|https://ci.ole.kuali.org/job/DB-Export-OLEDBA/] job in Hudson every night. For this reason, never update the files in the {{ole-cfg-dbs/development}} path directly, as they will be overwritten by the state of the OLEDBA schema the next night. |
...
Then, using this SVN project, any of the development or testing databases can be refreshed from that stored definition. In fact, on a daily basis, the OLECI (for unit testing), OLELOCAL (for developers without a local database), and OLEDEV (for the [http://dev.ole.kuali.org/ole-dev/] instance) are deleted and restored from this project. |
...
The additional complication to this is that there are actually three components to the full refresh process as will be discussed below |
...
OLE Rice Data Maintenance
. h2. [OLE Rice Data Maintenance] h2. {{ole-cfg-dbs}} Project Structure |
...
Directory
...
Description
|| Directory || Description || | {{db-impex |
...
}} | The database import/export tool used to process the contents of the directory below. Loaded automatically from the Kuali SVN repository. |
...
development
...
The OLE database description - see below.
...
rice
...
The matching Rice database description - loaded from the rice project.
...
| | {{development}} | The OLE database description - see below. | | {{rice}} | The matching Rice database description - loaded from the rice project. | Within the {{ole-cfg-dbs/development}} directory, there are four major items: |
...
|| File/Directory |
...
Description
|| Description || | *schema.xml |
...
* | This file represents the structure for the entire database schema. This contains all tables, views, sequences, indexes, and constraints. |
...
data
...
Each file in this directory contains the data for a single table. These files are maintained in a pipe (|) delimited flat file format.
...
graphs
...
The "programs" used by Clover ETL to perform the import. Their main purpose is to map between the files and the database, translating any placeholders for pipes in the data. Graphs only exist for tables which contain data. For performance reasons, tables without any data have their graphs purged during the export process.
...
formats
...
The structure used for both the data file and the import into the database. Referenced by the graphs during execution.
Also, the ole-cfg-dbs project directory contains an svn:externals property which pulls in the correct version of Rice for OLE. (See the subversion documentation for more information.) This causes SVN to pull the needed definitions directly from the Rice project.
OLEDBA Master Schema
All changes to the base OLE database structure and baseline demonstration data must be done within this schema. The schema is located at:
...
JDBC URL:
| | *data* | Each file in this directory contains the data for a single table. These files are maintained in a pipe (\|) delimited flat file format. | | *graphs* | The "programs" used by Clover ETL to perform the import. Their main purpose is to map between the files and the database, translating any placeholders for pipes in the data. Graphs only exist for tables which contain data. For performance reasons, tables without any data have their graphs purged during the export process. | | *formats* | The structure used for both the data file and the import into the database. Referenced by the graphs during execution. | Also, the {{ole-cfg-dbs}} project directory contains an {{svn:externals}} property which pulls in the correct version of Rice for OLE. (See the [subversion documentation|http://svnbook.red-bean.com/en/1.5/svn.advanced.externals.html] for more information.) This causes SVN to pull the needed definitions directly from the Rice project. h2. {{OLEDBA}} Master Schema All changes to the base OLE database structure and baseline demonstration data must be done within this schema. The schema is located at: || JDBC URL: | jdbc:oracle:thin:@oracle.ole.kuali.org:1521:XE |
...
| || Username: |
...
OLEDBA
...
| OLEDBA | Changes made there will be exported into the {{ole-cfg-dbs}} SVN project nightly and used to refresh the OLELOCAL, OLEDEV, and OLECI databases. (as well as any developer satellite instances the next time they run the process) |
...
h2. Demo vs. Bootstrap Datasets |
...
All of the Kuali projects have the concept of a {{development}} dataset (hence the {{ole-cfg-dbs/development}} path), a {{demo}} dataset, and a {{bootstrap}} dataset. In most cases, the {{development}} and {{demo}} datasets are the same. The main difference between these is the removal of Kuali environment-specific URLs or email addresses. |
...
The {{bootstrap}} dataset is pared back even further, with almost all the non-essential data elements removed, leaving the database empty and ready for import of institutional data. |
...
Starting versions of these scripts (from the KFS project) can be found here: |
...
* {{ole/work/db/ole-demo.sql |
...
}} * {{ole/work/db/ole-bootstrap.sql |
...
}} These scripts would be run during the creation of the distribution and then their results re-exported into the {{ole-cfg-dbs}} SVN project for inclusion in the release. These scripts do *not* affect the Rice data. For that, the appropriate data files under {{ole/work/db/rice-data}} must be modified. |