PERFORMANCE BENCHMARK FOR INGEST
INGEST BIB DATA
Legacy data ingest:
Ingest 6 million bib records, processing time = 60hrs
On
2012-04-18 05:16:51,738 on DEV with batch size = 1000 (with about 1.5 million records already ingested)
Bulk Ingest Process for 10000 bib records:
Ingesting Time: 0:3:41.562(H:M:S.ms)
Indexing Time: 0:1:10.201(H:M:S.ms)
Total Process Time: 0:4:51.854(H:M:S.ms)
Incremental data ingest:
On 2012-04-18 05:16:51,738 on DEV with batch size = 1000 (with about 1.5 million records already ingested)
Bulk Ingest Process for 10,000 bib records:
Ingesting Time: 0:3:41.562(H:M:S.ms)
Indexing Time: 0:1:10.201(H:M:S.ms)
Total Process Time: 0:4:51.854(H:M:S.ms)
INGEST INSTANCE DATA
Legacy data ingest:
Ingest 10 million instance records, processing time may take 42 days!
Detailed time breakdown for ingesting 1000 records:
Time taken for ingest 1000 instance records :
Ingesting Time: 3.33 minutes
Time for Linking to Bib records: 2.51minutes
More detailed please read the xls file.
PERFORMANCE MINIMUM REQUIREMENT
Ingest about 20 million legacy data (including bib, instance..) need to finish in one week!