/
Migration from Ingest v1 to Ingest v2
  • Current and Stable

    • Migration from Ingest v1 to Ingest v2

    Owned by Jag Goraya
    Last updated: Dec 17, 2024
    5 min read

    Schedule

    dcb-service release to accompany with configuration notes

    boost server resources

    Window

    Round

    Activity

    Environment

    Who

    Status

    Window

    Round

    Activity

    Environment

    Who

    Status

    Fri 22 Nov

    Re-ingest Dry Run

    Configure environment for ingest v2

    Staging

    EBSCO FSE & K-Int

    Fri 22 - Sun 24 Nov

    Re-ingest Dry Run

    Re-ingest standalones

    Staging

    K-Int

    Sun 24 Nov

    Re-ingest Dry Run

    Prepare FOLIO libraries for harvest

    Staging

    EBSCO FSE

    Slack

    Mon 25 - Thu 28 Nov

    Re-ingest Dry Run

    Reingest FOLIO libraries

    Staging

    K-Int

    Thu 28 Nov - Wed 4 Dec

    Re-ingest Dry Run

    Verify ingest v2 and clustering changes

    Staging

    K-Int & EBSCO

    Slack

    Thu 5 Dec

    Re-ingest

    Configure environment for ingest v2

    Production

    EBSCO

    Thu 5 Dec

    Re-ingest

    Prepare UMSYS & NWMSU for harvest

    Production

    EBSCO FSE & K-Int

    Thu 5 - Mon 9 Dec

    Re-ingest

    Re-ingest standalones and UMSYS & NWMSU

    Production

    K-Int

    Mon 9 - Sun 15 Dec

    Re-ingest

    Prepare and reingest ECS libraries

    Production

    EBSCO & K-Int

    Staging

    Cohorts

    Cohort 1 - Standalones

    • Enable

      • ALTOONA

      • CALS

      • MISSOURI_RIVER

      • RIVERSHARE

      • SCCL

      • SEMO

      • SLOUC (St Louis County Test)

      • STLOUIS_UNIVERSITY

    • Disable

      • BETTENDORF

      • CHRISTIAN_COUNTY

      • COOLCAT

      • DAVENPORT

      • SPRINGFIELD_GREENE_COUNTY

      • WASHU

    Cohort 2 - FOLIO

    Disable for ingest until harvest preparation complete, then enable and restart dcb-service

    • CALVARY_UNIVERSITY (Calvary Staging)

    • UMSYS (UMSYS - TEST)

    • TRUMAN (Truman State)

    FSE Action (Staging)

    1. 22 Nov

      1. Set environment flag - [UPD] - Sobha - Done

      2. Deploy dcb-service released today to staging and trigger restart -[UPD] Sobha - Done

    2. 24-25 Nov Maintenance Window

      1. Prepare Cohort 2 libraries for ingest

        1. FOLIO - analyse and reindex - US1334504: Prep libraries for harvest - on 11/24

        2. DCB - scale up DCB database size 4x - US1334505: Prep for DCB Harvest - MOBIUS-Staging - on 11/24

      2. Set ingest flag to enabled in dcb for Cohort 2 libraries - US1334507: Prep libraries for harvest - on 11/24

      3. Restart dcb-service to trigger ingest - US1334507: Prep libraries for harvest - on 11/24

      4. Monitor ingest rate on dcb database and run vacuum as needed during ingest

        1. count on table ?? - Carole/Vijay/Tim for details. - US1334507: Prep libraries for harvest - on 11/24

    Production

    Production Cohorts

    Dec 5, 2024 Cohort 1 - Standalones K-Int

    DCB-Service Restarted 2024-12-05 09:19 UTC

    ALTOONA
    CALS
    MISSOURI_RIVER
    RIVERSHARE
    SCCL
    SEMO
    SLOUC
    STLOUIS_UNIVERSITY

    Dec 5, 2024 Cohort 2 - FOLIO EBSCO FSE

    Disable for ingest until harvest preparation complete, then enable and restart dcb-service

    DCB-Service Restarted 2024-12-05 10:33 UTC

    UMSYS
    NORTHWEST_MISSOURI

    Dec 9, 2024 Cohort 3 - FOLIO EBSCO FSE

    Disable for ingest until harvest preparation complete, then enable and restart dcb-service

    DCB-Service Restarted 2024-12-09-11 09:29 UTC

    Calvary University (29515)
    Conception Abbey and Seminary College (120454)
    Harris-Stowe State University (89591)
    Kansas City University (15928)
    Missouri History Museum (78932)
    Missouri State Library (52021)
    Three Rivers College (41122)
    Truman State University (598276)

     

    Dec 10, 2024 Cohort 4 - FOLIO EBSCO FSE

    Disable for ingest until harvest preparation complete, then enable and restart dcb-service

    DCB-Service Restarted 2024-12-10 9:51 UTC

    A.T. Still University
    Benedictine College
    Cottey College
    Crowder College
    Culver-Stockton College
    Evangel University
    Fontbonne University
    Goldfarb School of Nursing at Barnes-Jewish College
    Kansas City Art Institute
    Kenrick-Glennon Theological Seminary

    Dec 11, 2024 Cohort 5 - FOLIO EBSCO FSE

    DCB-Service Restarted 2024-12-10 9:51 UTC

    Disable for ingest until harvest preparation complete, then enable and restart dcb-service

    Central Methodist University
    Concordia Seminary
    Covenant Theological Seminary
    Kansas City Kansas Community College
    Maryville University
    Missouri Southern State University
    Missouri Valley College
    Missouri Western State University
    North Central Missouri College
    Saint Louis Art Museum
    St. Louis Community College
    University of Health Sciences and Pharmacy in St. Louis
    Webster University
    William Woods University
    Nazarene Theological Seminary
    Lindenwood University
    Midwestern Baptist Theological Seminary
    Mineral Area College
    Missouri Baptist University
    Missouri Botanical Garden
    Moberly Area Community College

     

    Dec 12, 2024 Cohort 6 - FOLIO EBSCO FSE

    DCB-Service Restarted 2024-12-11 9:37 UTC

    Disable for ingest until harvest preparation complete, then enable and restart dcb-service

    Ozark Christian College
    Ozarks Technical Community College
    Park University
    Saint Paul School of Theology
    Southwest Baptist University
    St. Charles Community College
    State Fair Community College
    State Technical College of Missouri
    Westminster College
    William Jewell College
    Avila University
    Columbia College
    Drury University
    East Central College
    Hannibal-LaGrange University
    Jefferson College
    Lincoln University
    Logan University
    Metropolitan Community College
    Rockhurst University
    Southwestern Baptist Theological Seminary
    Stephens College

     

    FSE Action (Production)

    1. Thu 5 Dec (maintenance window) Prepare environment - UPD - Sobha - US1336167: Prep environment - on 12/5 during standard maintenance window into 12/5

    2. Thu 5 Dec (by end of maintenance window) Cohort 2 - UPD - Sobha - US1335955: Prep libraries for harvest - on 12/5 during standard maintenance window into 12/5

    3. Mon 9 Dec (by end of maintenance window) Cohort 3 - UPD - Sobha - US1336168: Prep libraries for harvest - on 12/9 during standard maintenance window into 12/9

    4. Tue 10 Dec (by end of maintenance window) Cohort 4 - UPD - Sobha - US1336169: Prep libraries for harvest - on 12/10 during standard maintenance window into 12/10

      1. Prepare Cohort 4 libraries for ingest

        1. FOLIO - analyse and reindex

        2. DCB - scale up DCB database size 4x [Prep for DCB Harvest - MOBIUS-Production]

      2. Set ingest flag to enabled in dcb for Cohort 4 libraries

      3. Restart dcb-service to trigger ingest

      4. Monitor ingest rate on dcb database and run vacuum as needed during ingest

    5. Wed 11 Dec (by end of maintenance window) Cohort 5 - US1336170: Prep libraries for harvest - on 12/11 during standard maintenance window into 12/11

    6. Thu 12 Dec (by end of maintenance window) Cohort 6 - US1336171: Prep libraries for harvest - on 12/12 during standard maintenance window into 12/12

     

    References

    Operated as a Community Resource by the Open Library Foundation