PERFORMANCE BENCHMARK FOR INGEST
INGEST BIB DATA
Legacy data ingest:
Ingest 6 million bib records, processing time = 60hrs
Incremental data ingest:
On 2012-04-18 05:16:51,738 on DEV with batch size = 1000 (with about 1.5 million records already ingested)
...
Total Process Time: 0:4:51.854(H:M:S.ms)
INGEST INSTANCE DATA
Legacy data ingest:
Ingest 10 million instance records, processing time may take 42 days!
Detailed time breakdown for ingesting 1000 records:
Time taken for ingest 1000 instance records :
...
More detailed please read the xls file.
PERFORMANCE MINIMUM REQUIREMENT
Ingest about 20 million legacy data (including bib, instance..) need to finish in one week!
PERFORMANCE ISSUES
...
ARCHITECTURE REVIEW
COMMENTS ON DOCSTORE ARCHITECTURE
John Pillans thought we actually don't need the JackRabbit