Migration from Ingest v1 to Ingest v2

Schedule

📦 dcb-service release to accompany with configuration notes

💥 boost server resources

Window	Round	Activity	Environment	Who	Status
Fri 22 Nov	Re-ingest Dry Run	Configure environment for ingest v2 📦	Staging	EBSCO FSE & K-Int
Fri 22 - Sun 24 Nov	Re-ingest Dry Run	Re-ingest standalones	Staging	K-Int	⏳
Sun 24 Nov	Re-ingest Dry Run	Prepare FOLIO libraries for harvest	Staging	EBSCO FSE
Mon 25 - Thu 28 Nov	Re-ingest Dry Run	Reingest FOLIO libraries 💥	Staging	K-Int	⏳
Thu 28 Nov - Wed 4 Dec	Re-ingest Dry Run	Verify ingest v2 and clustering changes	Staging	K-Int & EBSCO
Thu 5 Dec	Re-ingest	Configure environment for ingest v2 📦	Production	EBSCO
Thu 5 Dec	Re-ingest	Prepare UMSYS & NWMSU for harvest	Production	EBSCO FSE & K-Int
Fri 6 - Mon 9 Dec	Re-ingest	Re-ingest standalones and UMSYS & NWMSU 💥	Production	K-Int
Mon 9 - Sun 15 Dec	Re-ingest	Prepare and reingest ECS libraries 💥	Production	EBSCO & K-Int

Staging

Cohort 1 - Standalones

Enable
- ALTOONA
- CALS
- MISSOURI_RIVER
- RIVERSHARE
- SCCL
- SEMO
- SLOUC (St Louis County Test)
- STLOUIS_UNIVERSITY
Disable
- BETTENDORF
- CHRISTIAN_COUNTY
- COOLCAT
- DAVENPORT
- SPRINGFIELD_GREENE_COUNTY
- WASHU

Cohort 2 - FOLIO

Disable until harvest preparation complete, then enable and restart dcb-service

CALVARY_UNIVERSITY (Calvary Staging)
UMSYS (UMSYS - TEST)
TRUMAN (Truman State)

FSE Action (Staging)

22 Nov
1. Set environment flag - [UPD] - Sobha - Done
2. Deploy dcb-service released today to staging and trigger restart -[UPD] Sobha - Done
24-25 Nov Maintenance Window
1. Prepare Cohort 2 libraries for ingest
  1. FOLIO - analyse and reindex - US1334504: Prep libraries for harvest - on 11/24
  2. DCB - scale up DCB database size 4x - US1334505: Prep for DCB Harvest - MOBIUS-Staging - on 11/24
2. Set ingest flag to enabled in dcb for Cohort 2 libraries - US1334507: Prep libraries for harvest - on 11/24
3. Restart dcb-service to trigger ingest - US1334507: Prep libraries for harvest - on 11/24
4. Monitor ingest rate on dcb database and run vacuum as needed during ingest
  1. count on table ?? - Carole/Vijay/Tim for details. - US1334507: Prep libraries for harvest - on 11/24

Production Cohorts

05 Dec 2024 Cohort 1 - Standalones K-INT

ALTOONA
CALS
MISSOURI_RIVER
RIVERSHARE
SCCL
SEMO
SLOUC
STLOUIS_UNIVERSITY

05 Dec 2024 Cohort 2 - FOLIO EBSCO

Disable until harvest preparation complete, then enable and restart dcb-service

UMSYS
NORTHWEST_MISSOURI

09 Dec 2024 Cohort 3 - FOLIO EBSCO

Disable until harvest preparation complete, then enable and restart dcb-service

TBC

10 Dec 2024 Cohort 4 - FOLIO EBSCO

Disable until harvest preparation complete, then enable and restart dcb-service

TBC

11 Dec 2024 Cohort 5 - FOLIO EBSCO

Disable until harvest preparation complete, then enable and restart dcb-service

TBC

12 Dec 2024 Cohort 6 - FOLIO EBSCO

Disable until harvest preparation complete, then enable and restart dcb-service

TBC

FSE Action (Production)

Thu 5 Dec (maintenance window) Prepare environment
1. Set environment flag - [UPD] - Sobha - Done
2. Deploy dcb-service released today to production and trigger restart -[UPD] Sobha - Done
Thu 5 Dec (by end of maintenance window) Cohort 2
1. Prepare Cohort 2 libraries for ingest (UMSYS & NWMSU)
  1. FOLIO - analyse and reindex
  2. DCB - scale up DCB database size 4x [Prep for DCB Harvest - MOBIUS-Production]
2. Set ingest flag to enabled in dcb for Cohort 2 libraries
3. Restart dcb-service to trigger ingest
4. Monitor ingest rate on dcb database and run vacuum as needed during ingest
Mon 9 Dec (by end of maintenance window) Cohort 3
1. Prepare Cohort 3 libraries for ingest (remaining ECS libraries)
  1. FOLIO - analyse and reindex
  2. DCB - scale up DCB database size 4x [Prep for DCB Harvest - MOBIUS-Production]
2. Set ingest flag to enabled in dcb for Cohort 3 libraries
3. Restart dcb-service to trigger ingest
4. Monitor ingest rate on dcb database and run vacuum as needed during ingest
Tue 10 Dec (by end of maintenance window) Cohort 4
1. Prepare Cohort 4 libraries for ingest (remaining ECS libraries)
  1. FOLIO - analyse and reindex
  2. DCB - scale up DCB database size 4x [Prep for DCB Harvest - MOBIUS-Production]
2. Set ingest flag to enabled in dcb for Cohort 3 libraries
3. Restart dcb-service to trigger ingest
4. Monitor ingest rate on dcb database and run vacuum as needed during ingest
Wed 11 Dec (by end of maintenance window) Cohort 5
1. Prepare Cohort 5 libraries for ingest (remaining ECS libraries)
  1. FOLIO - analyse and reindex
  2. DCB - scale up DCB database size 4x [Prep for DCB Harvest - MOBIUS-Production]
2. Set ingest flag to enabled in dcb for Cohort 3 libraries
3. Restart dcb-service to trigger ingest
4. Monitor ingest rate on dcb database and run vacuum as needed during ingest
Thu 12 Dec (by end of maintenance window) Cohort 6
1. Prepare Cohort 6 libraries for ingest (remaining ECS libraries)
  1. FOLIO - analyse and reindex
  2. DCB - scale up DCB database size 4x [Prep for DCB Harvest - MOBIUS-Production]
2. Set ingest flag to enabled in dcb for Cohort 3 libraries
3. Restart dcb-service to trigger ingest
4. Monitor ingest rate on dcb database and run vacuum as needed during ingest

References

https://openlibraryfoundation.atlassian.net/issues/?jql=parent%20%3D%20%22DCB-1435%22%20OR%20issuekey%20%3D%20%22DCB-1435%22%20or%20issueLink%20%3D%20%22DCB-1435%22%20and%20issuetype%20!%3D%22Epic%22%20%20ORDER%20BY%20created%20DESC