Import and export pickup locations through DCB Admin

Description

In Scope

  • File import parameters consistent with mappings import

    • File type: CSV, TSV

    • File size: 1MB max

  • Import Workflow

    • Minimum role to action import: LIBRARY_ADMIN

    • Import via Library > Location tab

    • Import validation responses as per mappings

    • Data validation as per

    • Add data change log support

    • Add to and sort Location grid by hidden column “Last imported”

  • Export workflow (see comment)

    • Grid export should produce an import-ready backup file

  • Documentation

    • Add new import user guide for pickup locations, similar to

 

References

 

Out of Scope

  • Abstracting import modal to support multiple use cases (both mappings and locations)

Attachments

7

Activity

Show:

Casey Henderson February 10, 2025 at 2:40 PM
Edited

I’m unable to replicate dplg6 not being imported, so I’ve added some logging to output the ignored items to the console. If you could have another try at this and tell me what the output is, that’d be really helpful in tracking it down.

Regarding the success toast issue, the success time was increased, but the minimum time (affecting both) was not. I have rectified this now, and the changes are live.

The TSV issue is probably because of the multiple TSV/CSV exports of the same file, although we should look at using the Excel integration we now get with Premium (especially if we’re expecting users to be using Excel). This should ensure we don’t get warnings like that in future.

Gill Osguthorpe February 10, 2025 at 12:29 PM

QA: 10/2: Thanks .

I exported the locations for Davenport as TSV then imported it without changing the file. One of the records was not imported - Loc code=dplg6. Is this due to PICKUP being uppercase?

The time the success toast is shown on screen is still 3 seconds - I think the error message time was increased but not the success message time.

I exported as TSV, imported the file, then exported as CSV and opened the resulting CSV. Excel showed the below message about removing leading zeros. Is this because they were inserted as part of the TSV import?

Casey Henderson February 7, 2025 at 1:59 PM
Edited

QA checklist - these changes are now live and should hopefully cover everything from your feedback

Gill Osguthorpe February 4, 2025 at 4:31 PM

QA 4/2/24 have completed QA . Thanks for including the checklist, it was really helpful. I worked against it and didn’t log the successes, just the issues.

1. Please change the button text from “Import from file” to “Import”.

2. I imported a test file with the Lon and Lat columns switched round to trigger the “expected headers” error and got this message, which is displayed when importing mappings.

  • The column headers in davenport locations_export.csv do not match the expected headers. The first three columns must be named “context”, “domain” and “lowerBound”. Please correct the column headers and retry.

3. After generating the above error I tested the max file size and incorrect format validation. the correct errors were displayed but the toast from the previous error was also displayed. See video.

Comment from Casey: Will be fixed as part as

4. The general error message also needs changing from:

  • “An error occurred uploading mappings. Please try again.” to

  • “An error occurred whilst importing locations. Please try again.”

5. The three column layout applied to the Category and Host LMS key-value pairs disappeared for some reason - it was present in the above video.

6. I made one of the agency codes invalid and imported. The generic error message was displayed. Instead can the error read:

  • One or more agency codes is invalid. Please correct and retry.

7. When the file contains invalid long/lat values the erroneous locations are not imported and a toast is displayed with a message like the below. The toast doesn’t remain on screen long enough to read. I originally missed the message. Can the length of time be increased from 3 to 6 seconds?

 

8. Irrespective of LMS type, a location must not be accepted if one already exists for the given Host LMS with the same localId.

  • I was able to import locations with the same Polaris ID of 999 and duplicated FOLIO IDs as well.

9. User cannot import a location with the same code as one that already exists.

  • This works in that entries with duplicate codes are ignored but there was no message to say that the records were not imported.

Casey Henderson February 3, 2025 at 10:00 AM

QA branch is available here https://libraries-dcb-hub-admin-scaffold-uat-git-dcb-1777-knowint.vercel.app/

There is also a comment containing all of my manual testing here.

Casey Henderson January 31, 2025 at 3:39 PM

DCB Service merged

There remains an issue where human readable usernames aren’t always recorded for import. I have investigated this behaviour today but have been unable to find a fix - it seems it may be related to the behaviour of the Micronaut UpdatedBy annotation in this specific context, which automatically takes the “username” attribute (which in our Keycloak set up isn’t actually the username, but rather the UUID) instead of the preferred_username. Unfortunately the solution we used to solve this issue for our GraphQL operations doesn’t seem to work in the context here, so we might have to spin this out as a separate issue.

Casey Henderson January 30, 2025 at 5:18 PM

30/01/25

Remaining (DCB Service)

  • Fix for issue where human readable usernames aren’t always recorded for import.

Remaining (DCB Admin)

  • Export of valid backup (and testing of this)

  • UX fixes

  • General testing before QA.

Jag Goraya January 29, 2025 at 1:51 PM

Thought so, thanks

Casey Henderson January 29, 2025 at 1:26 PM

Yes, the column headers must be in the same order or the validation of headers will fail.

Jag Goraya January 29, 2025 at 12:50 PM

Note that the file used is this one, so these tests are only applicable for files in this format, as required by the ticket.

Quick one - do the columns need to exist in the same order as that UGA reference sheet, or can they be in any order as long as the header field is the same? I realise I’ve got LOCTYPE a couple of columns earlier in most of the other config submissions, so thought I’d check before I embark on fixing up the files for the rest of the consortium.

Jag Goraya January 29, 2025 at 12:43 PM

What’s the expected behaviour in cases where locations already exist? Should they be deleted and replaced with the locations from import?

Yea, replace is what I was expecting, which I understand would be consistent with mappings import behaviour.

Done

Details

Assignee

Reporter

Target Version

Fix versions

Sprint

Priority

Created December 23, 2024 at 4:04 PM
Updated February 26, 2025 at 9:40 AM
Resolved February 10, 2025 at 3:57 PM