Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Info
titleJira #

https://jira.kuali.org/browse/OLE-1144 (OLE Search Executive- see also linked tasks and sub-tasks)

Overview of OLE Search:

As OLE grows to include more document types
Section
Column
width40%
Info
titleJira #

https://jira.kuali.org/browse/OLE-1144 (OLE Search Executive- see also linked tasks and sub-tasks)

Section
Column
width40%

Overview of OLE Search:

As OLE grows to include more document types (patrons, bibliographic formats, licenses, etc.), OLE will create integrated Search capabilities, where the search functions will provide inter-operability instead of some of the localized wildcards, limits on bibliographic fields, or other search-specific criteria below. Library users will be included in future testing and development of OLE Search, building upon our emerging search types for

  • Document Store Search – full Bibliographic searching 
    • Indexes, Facets, Sort, Boolean
  • Acquisitions Search 
  • Order Holding Queue
  • Receiving Queue
  • Fund/ Balance Inquiries
  • Load Reports

KFS inherited transactional Searches (no Bibliographic fields, only transactional- vendor, dates, etc.)

  • Payment Request
  • Purchase Orders
  • Requisition
  • Receiving

Other search functions available in OLE

  • Lookups
  • Lookup icon 
  • Custom Document Searches
  • Doc Search button
  • Saving Custom (Session) Searches 
Column
width40%
Panel
borderColor#A40000
bgColor#F8F8F8
titleBGColor#E8E8E8
titleContents
borderStyledashed
Table of Contents
minLevel1
outlinefalse

...

1.1 Searchable fields for all document categories, types and formats

1.1.1 Searchable fields for Bibliographic
Author No No No No No No Genre (marc genre/dc type) No No No No Edition No No No No No No No Barcode No No No No No No No Location No No No No No No Source No No No No No No No Record Type No No No No No

Field Name

Work-Bib-MARC

Work-Bib-DublinQ

Work-Bib-DublinUnQ

Work-Instance-OLEML

Work-Holdings-OLEML

Work-Item-OLEML

Work-License-ONIXPL

Work-License-PDF

Title

Yes

Yes

Yes

No

No

No

No

No

DublinUnQ

Title

Yes

Yes

Yes

No Author No

Yes

No Yes No

Yes

No

Subject

Yes

Yes

Yes

No

No

No

No

No

Description

Yes

Yes

Yes

No

No

No

No

No

Date of Publication

Yes

Yes

Yes

No

No

No

No

No

Format

Yes

Yes

Yes

No

No

No

No

No

Language

Yes

Yes

Yes No

SubjectYes

Yes

Yes

No

Publisher

Yes

Yes

Yes No

Publication DateYes

Yes

Yes

No

ISSN/ISBN/other (last for dc identifierfor dc identifier)

Yes

Yes

Yes

Genre (marc genre/dc type)

Yes

Yes

Yes

No Edition No

Yes

No Yes No

Yes

No

Bib IdentifierYesYesYes
Holdings IdentifierYesYesYes
Local IdYes

No

YesYes
Doc CategoryYesYesYes
Doc FormatYesYesYes
Doc TypeYesYesYes
StatusYesYesYes
Status Updated OnYesYesYes
Staff Only FlagYesYesYes
Created ByYesYesYes
Updated ByYesYesYes
Date EnteredYesYesYes
Date UpdatedYesYesYes

No

No

No

Encoding Level

No

No

No

No

Yes

No

No

1.1.2 Searchable fields for Holdings
No No No No No No Acquisition Method No No No No No No No No No No No No Volume Number No No No No Contract No No No No No No No

Field Name

Work-Holdings-OLEML

Work-Item-OLEML

Bib IdentifierYes

Yes

Holdings IdentifierYesYes
Item IdentifierYesYes
Receipt StatusYes

No

Copy NumberYesYes
Call NumberYesYes
Call Number TypeYesYes
Item PartYes

No

Call Number PrefixYesYes
Classification PartYes

Policy Type

No

No

No

Yes
Shelving Scheme CodeYesYes
Shelving Scheme ValueYesYes
No Shelving Order No Yes

Copies Reported Yes No

Location Level No Yes

No Yes No

Location Level NameYes

No Yes No

Access Status No Yes

Item Type

No No

Access Location

No Yes

No No

Statistical Search Code ValueYes No Yes No
Publisher Location Status Yes

No No

Imprint No Yes

No No

Platform

Yes

No No

Public Note Shelving Scheme Yes

No

Holding Note No YesNo No
Local Id No Yes

Yes No

Doc Category No Yes

Shelving Order Yes No

Doc Format No Yes

No Yes No

Doc Type No Yes

Yes No

Status No Yes

Address Ye No

Status Updated On No Yes No Yes No
Staff Only Flag No YesYes No
Claims Return FlagNo

Copy Number

Yes
Claims Return Flag Create DateNoYes
Claims Return NoteNoYes
Current BorrowerNoYes
Proxy BorrowerNoYes
Due Date TimeNoYes
Barcode ARSLNoYes
Volume NumberNoYes
EnumerationNoYes
ChronologyNoYes
Item Barcode

Licensee

No

No

No

No

Yes
Item StatusNoYes

No

Licensor

Item Type Full ValueNo

No

Yes
Item Type Code ValueNoYes
No Created ByYes No Yes
Status Updated By No Yes No Yes No
Date Entered No Yes No Yes No
Date UpdatedYes

No

Method

No

No

No

No

No

No Yes
1.1.4 Searchable fields for License

Field Name

Work-License-ONIXPL

Work-License-PDF

Contract Number

Yes

No

Type Licensee

No Yes

No No

Licensor

No Yes

No No

Status

Yes

No

Name Method

No Yes

No No

Type

No Yes

No No

Name

No

Yes

File Name

No

No

No

No

No

No

No

Yes

Date Uploaded

No

No

No

No

No

No

No

Yes

Owner No

No

No

No

No

No

No

Yes

Notes

No

No

No

No

No

No

No

Yes

1.2 Facet fields for all document categories, types and formats

Facet Field

Work-Bib-MARC

Work-Bib-DublinQ

Work-Bib-DublinUnQ

Work-Instance-OLEML

Work-Holdings-OLEML

Work-Item-OLEML

Work-License-ONIXPL

Work-License-PDF

Subject

Yes

Yes

Yes

No

No

No

No

No

Author

Yes

Yes

Yes

No

No

No

No

No

Format

Yes

Yes

Yes

No

No

No

No

No

Language

Yes

Yes

Yes

No

No

No

No

No

Publication Date

Yes

Yes

Yes

No

No

No

No

No

Genre

Yes

Yes

Yes

No

No

No

No

No

1.3 Field definitions for Work-Bib-MARC documents

Field

Data fields for search (MV- indicates multi-valued)

Data fields for short display

Data fields for detailed display

Data fields for Facet

ISSN

022 - a,z (MV)

first value

all values

same as search field

ISBN

020 - a,z (MV)

first value

all values

same as search field

Author/Creator

For each 100, 110: every subf except $6 (gives us 2 values for every tag). Also every subf except $t and $6 for: 111, 700, 710, 711, 800, 810, 811, 400, 410, 411) 
Ref: http://www.loc.gov/marc/bibliographic/
100 - Main Entry - Personal Name (NR)
110 - Main Entry - Corporate Name (NR)
111 - Main Entry - Meeting Name (NR)
700 - Added Entry - Personal Name (R)
710 - Added Entry - Corporate Name (R)
711 - Added Entry - Meeting Name (R)
711 800 - Series Added Entry -Meeting Personal Name (R)
800 810 - Series Added Entry -Personal  Corporate Name (R)
811 - Series Added Entry -Meeting Name (R)
400 - Series Statement/Added Entry - Personal Name(R)
410 - Series Statement/Added Entry - Corporate Name(R)
411 - Series Statement/Added Entry - Meeting Name(R)

first non-empty value of
100$a or 110$a.
Show blank if both are
missing.

All non-empty
indexed values

All non-empty
indexed values

Title

245 - all subf exc.

All non-empty
indexed values

All non-empty
indexed values

Title

245 - all subf exc. c and 6. Also, 130, 240, 246, 247, 440, 490, 730, 740, 773, 774, 780, 785, 830, 840) (MV)

245$a and 245$b

all values

 

Place of Publication

260 - a (MV)

first value

all values

same as search field

Description

505 - a (MV)
KG/LR: UPenn just included the MARC 505 in its Description index (which is distinct from its Format/Description index). Include just 505 $a.  The SMEs may want additional 5xx fields in the Description index, but 505 should be fine for November.

first value

all values

same as search field

Subject

600, 610, 611, 630, c and 6. Also, 130, 240, 246, 247, 440, 490, 730, 740, 773, 774, 780, 785, 830, 840) (MV)

245$a and 245$b

all values

 

Place of Publication

260 - a (MV)

first value

all values

same as search field

Description

505 - a (MV)
KG/LR: UPenn just included the MARC 505 in its Description index (which is distinct from its Format/Description index). Include just 505 $a.  The SMEs may want additional 5xx fields in the Description index, but 505 should be fine for November.

first value

all values

same as search field

Subject

600, 610, 611, 630, 650, 651, 653, 69X: every subf exc. $6, $2, $=, $? across these tags

600 - Subject Added Entry - Personal Name (R)
610 - Subject Added Entry - Corporate Name (R)
611 - Subject Added Entry - Meeting Name (R)
630 - Subject Added Entry - Uniform Title (R)
650 - Subject - Added Entry -  Topical Term (R)
651 - Subject Added Entry - Geographic  Name (R)
Ref : http://www.loc.gov/marc/bibliographic/bd6xx.html

first non-empty value of 600, 610, 611, 650, 651, 653, 69X : every subf exc . $6, $2, $= across these tags

No hyphens for X00, X10, and X11 fields 600, 610, 611, 700, 710, 711, etc), but hyphens for other fields.    
first non-empty value of 600$a, $d etc 610$a etc first non-empty value of 600$a,$d etc 610$a etc. , 610, 611, etc), but hyphens for other fields.    

Punctuation:
Follow punctuation and order given in
subfields:
600 - Subject Added Entry - Personal
Name (R)
610 - Subject Added Entry - Corporate Name (R)
611 - Subject Added Entry - Meeting
Name (R) , 610,611
(space between subfields) subfield subfield

Sample data:
600(subject - AuthorName) :
Kangxi,|cEmperor of China,|d1654-1722.
Facet/display:
kangxi,Emperor of China, 1654-1722.

Use double hyphen for subdivisions:
630 - Subject Added Entry - Uniform Title (R)
follow punctuation in fields, mainly using Periods between subfield value, except $v, $x, $y, $z
are "subdivisions" and use double - hyphen paired
with another subfield
Use double - hyphen for all subfields:
650 - Subject - Added Entry -  Topical Term (R)
651 - Subject Added Entry - Geographic  Name (R)
(double - hyphens between subfields) --
subfield -- subfield (period on end).
Sample data:
600(subject - AuthorName) :
Kangxi,|cEmperor of China,|d1654-1722.
Facet/display:
kangxi,Emperor of China, 1654-1722.
subfield

Sample data:
630(Standard title subject) :
Bible.|pN.T.|pLuke|xCommentaries.
Facet/display:
Bible.N.T.Luke -- Commentaries.

Use double - hyphen for all subfields:
 650, 651
(double - hyphens between subfields) --
subfield -- subfield (period on end).

Sample data:
650(subject-general):
Gardens|xSocial aspects|zChina|zBeijing|xHistory|y18th century.
Facet/display:
Gardens-Social aspectsChinaBeijingHistory-18th century.

All non-empty indexed
values.

Ordering of  multiple values:
Ordering of subject
headings in display
of individual bib
record should be in order of xml.

All non-empty  indexed values.

Hyphening pattern is same as display fields.

650(subject-general):
Gardens|xSocial aspects|zChina|
zBeijing|xHistory|
y18th century.
Facet/display:
Gardens--Socialaspects
-ChinaBeijing-
History--18th century.

Additional facets to be indexed:
1. Gardens - -Social aspects
2. Gardens - -Social aspects - -China
3. Gardens - -Social aspects - -China - -Beijing
4. Gardens - -Social aspects - -China - -Beijing - -History

Date of Publication

<marc:controlfield tag="008">[Date 1 in the 7-10 positions LR: Can also include 260 $c. (260-c is same as the value in control field. Use this if control field does not have pub date value.) (MV)does not have pub date value.) (MV).


Some records with 260 $c are having data other than date and data with no proper date.So for now 260 $c is ignored for publication date.

first value

all values

same as search field.

Keep decades, but add centuries and
"current year"
<controlfield tag="008">010108q10001999dcu b f000 0 eng d</controlfield>
The above example 1000-1999 means it should
appear with every facet 1000s,1100s, etc and every decade in 1000-1999.
Eliminate "odd" facets- don't want "s", "0000s",
or any years beyond the current year(manual removal 2020s and 9990s.
Only want single decades and single centuries.
<controlfield tag="008">001214k08000999xx zzz a n lat d</controlfield>.

Eg:
Two records with publication dates as 1905 and 1913.
The facets will be computed as:
20th century (2)-- 1900 to 1999.
1900s (1)-- 1900 to 1909.
1910s (1) – 1910 to 1919.

Edition

250 - a,b (MV)

first value

all values

same as search field

Form/Genre

655 - a, v (MV)

first value

all values

same as search field

Language

<marc:controlfield tag="008">[language code in the 35-37 positions]</marc:controlfield> LR: Add 546 $a (MV)
Language Codes (iso-639-3)

all values

all values

same as search field

Format

856 - q
245 - h  

LR: Format is very tricky b/c many MARC fields/subfields can be used to determine format.  

I think we could also consider adding the following:

Leader 06/07 – these are Type of Record and Bibliographic Material – each is a single letter characters and when combined, they seem to map to the UPenn bib_format field.  (See the worksheet now named Format – marc)
007 Physical Description Fixed Field-General Information – the character positions indicate physical format information.  See [http://www.loc.gov/marc/bibliographic/bd007.html
] 655 $a, $v (Genre/Form)
300 $e, $3 (Extent: Accompanying Material, Materials specified)
337 $a (Media Type)
338 $a (Carrier Type)
340 $a, $e, $m, $3 (Physical Medium: Material base, support, book format, materials specified)
Possibly other 3XX fields/subfields.

In the end, we may want to just touch base with some SMEs (Gwyneth? Bob? Stuart) to determine specifically what we could/should include)
(MV)

first value

all values

same as search field

...

1.4 Format field definitions for Work-Bib-MARC documents

Label

Marc Fields

Comments

Manuscript

Has any holdings  with "manuscripts" in location_name (gets only this value)

LR: MARC XML does not have location_name so this is irrelevant to the IU data that  OLE has for November.  Manuscript could be determined by the Leader 06/07.  06 values a, f, t equal manuscripts on their own.  07 values c and d seem to imply mauscript/archival collections/series.  We should check with the SMEs on this one.

Microformat

Has 245 $h containing "micro" OR has any holdings  with "micro" in location_name OR call_number starts "micro" (gets only this value)

LR: the 245 $h "micro" will work for the IU OLE MARCXML we have, but the reamaing text is specific to UPenn.

Archive

Has any holdings  with "archive" in location_name (gets only this value)

LR: This is specific to UPenn.  We may need to talk to IU about if they include Archive descriptions in their MARC records and how they designate them as such.

Thesis/Dissertation

bib_format is 'tm' AND has a 502 field

LR UPenn's bib_format seems to be a combination of the data values found in the 06/07 Leader fields.  For example, t in the 06 is Manuscript and m in the 07 is Monograph/Item and together they equal a Thesis/Dissertation.

Conference/Event

Has a 111 or 711 field [LR: Include 611 or 811]

 

Book

bib_format is 'aa', 'am' or 'ac' or 'tm'; exclude $h [micro*] and $k [kit]

LR: the 2 characters are from the Leader 06/07 the inclusions are 245 subfields

Sound recording

bib_format is 'im' or 'jm' or 'jc' or 'jd' or 'js'

LR: the 2 characters are from the Leader 06/07

Musical score

bib_format is cm, dm, ca, cb, cd or cs

LR: the 2 characters are from the Leader 06/07

Map/Atlas

bib_format is 'e*' or 'fm'

LR: the 2 characters are from the Leader 06/07

Video

bib_format is 'gm' AND 007/0 = v

LR: the 2 characters are from the Leader 06/07

Projected graphic

bib_format is 'gm' AND 007/0 = g

LR: 007 is a controlled field that indicates the format/physical description at general level and then associated subfields are more specific.

Journal/Periodical

bib_format is 'as' or 'gs'

LR: 007 is a controlled field that indicates the format/physical description at general level and then associated subfields are more specific.

Image

bib_format is 'km'

LR: the 2 characters are from the Leader 06/07

Datafile

bib_format is 'mm'

LR: the 2 characters are from the Leader 06/07

Newspaper

bib_format is 'as' AND (008/21 = 'n' OR 008/22 = 'e' )

LR: the 2 characters are from the Leader 06/07.  The 008 controlled field in those 2 positions provides the "form")

3D object

bib_format is 'r*'

LR: the single character maps to the 06 position in the leader.

Database/Website

bib_format is '*i'

LR: the single character maps to the 06 position in the leader.

Government document

bib_format is NOT c*, d*, i*, j* AND ( (008/28 = f, i, o and 260$b not 'press') )

LR: the single character maps to the 06 position in the leader. 008 is a fixed length controlled field and 260 $b is a type of publication.

Other

any bib_format not caught above

LR: Presumably relates to other 06/07 Leader data values not represented.

...

Field

DC-UnQ fields for Search

DC-Q fields for Search

Data fields for short display

Data fields for detailed display

Data fields for Facet

Author

<dc:creator> 

<dcvalue element="contributor" qualifier="author">

first value

All non-empty
indexed values

All non-empty
indexed values

Description

<dc:description> (MV)
Per Bob P.: Show only <dc:description>.

Per Bob P.:  Do not show Abstract description.
[show blank]

first value

all values

same as search field

Language

<dc:language>  (MV)
Language Codes (iso-639-3)

<dcvalue element="language" qualifier="iso">en_US</dcvalue>
Language Codes (iso-639-1-cc)

first value

all values

same as search field

Subject

<dc:subject> (R)

<dcvalue element="subject" qualifier="none">

first value

all values

same as search field

Title

<dc:title>

<dcvalue element="title" qualifier="none">

first value

all values

same as search field

Type

<dc:type> (MV)

<dcvalue element="type" qualifier="none">

first value

all values

same as search field

Date of Publication

<dc:date>

<dcvalue element="date" qualifier="issued">

first value

all values

same as search field

Format

<dc:format> (MV)

<dcvalue element="type" (This is covered in a separate field. So do not include it in Format)
<dcvalue element="format" qualifier="mimetype">

??? (LR: In looking back at the MARC to Qualified DC mapping it is not entirely clear, but it should be both the  format and type elements.

first value

all values

same as search field

Publisher

<dc:publisher> (MV)

<dcvalue element="publisher"

??? (KG/LR: publisher.  It doesn't appear in the crosswalk, but that could be that the UMD dataset did not include that tag)

first value

all values

same as search field

ISBN/ISSN/other

<dc:identifier>(ISSN)0198-9669</dc:identifier>  (MV)
 <dc:identifier>(ISBN)0306710382</dc:identifier> (MV)

<dcvalue element="identifier" qualifier="isbn">0-918006-48-1</dcvalue>

first value

all values

same as search field

...

Output: "To", "be,", "or", "what?"

2.3.2 Synonym Filtering

It is the process of synonym mapping. Each token is looked up in the list of synonyms and if a match is found, then the synonym is emitted in place of the token. The position value of the new tokens are set such they all occur at the same position as the original token

...

Synonyms are specified in a text file named 'synonyms‘synonyms.txt'txt’

The following are currently defined in this file.

...

The file named stopwords.txt specifies such words. Currently they are:

No Format

an and are as at

be but by

for

if in into is it

no not

of on or

s such

t that the their then there these they this to

was will with

...

  • All of these search selection searches for the words in the order they are entered.
  • As a phrase search selection does not search for the words in the order they appear.
  • Special characters such as '&' and ':' are searchable only if part of a longer phrase and the entire phrase is wrapped in quotation marks.  (If the quotation marks are left off, no hits will be returned.)
  • Wildcards (*, ?, #, !, []) do not function. 

...

DocType

links

Bibliographic

Instance

Instance

Bib,Holdings,Item

Holdings

Bib,Instance,Item

Item

Bib,Instance,Holdings

3.1 Short display

    Each record in the search results shows a subset of the record's fields.

3.2 Detailed display

   A link is (not yet) provided for each record in the search results.

   When this link is clicked, a popup, or a new tab of the browser is opened, with all the fields of the record.

3.3 Highlighting

In each search result, words matching the key words entered by the user are highlighted.

3.4 Facets

Facets are grouping of search results that help to analyze the search results and further filter or narrow down the search results.

These are helpful when the user cannot guess what exact keywords to search for.

Facet fields:

Author,Subject,Format,Language,Publication Date,Genre.

Top 5 facet values in the decreasing order of occurrences are shown for each facet field.

The remaining facets are seen by clicking the "more" link in a popup window.

The facet values in the popup are shown in alphabetical order.

Click on one or more facet values to filter search results and view only those records containing these facet values.

3.5 Sorting

By default, the search results are sorted by Title (A-Z)

The sorting options are:

    Title (A-Z) : sorts the results in the  ascending order (starts with 'a') of the title field value. This the default sorting order.
    Title (Z-A) :sorts the results in the descending  order (starts with 'z') of the title field value.
    Author (A-Z) :sorts the results in the ascending order (starts with 'a') of the author field value.
    Author (Z-A) : sorts the results in the descending order (starts with 'a') of the author field value.
    Pub date(new-old) : sorts the results in the descending order of the publication date field value.
    Pub date(old-new) : sorts the results in the ascending order of the publication date field value.  
    Relevance : it is default sort criteria provided by solr.

Records with empty or null values will appear at the top of the search results.

As per JIRA OLE_2194,

2nd indicator in MARC is non-filing character. need to use rules for these in applying sort/display standards. Ex. 245 1 3 $aAn April Shower- 2nd indicator is "3". Ignore first 3 characters, ie "An(space)" in applying sort rule

Test Template for sort

3.5.1 NISO Standard for Sort

...

NISO Search Results on docstore Search- in progress: https://jira.kuali.org/browse/OLE-2194  

4. Configurability of searchable fields

As seen above, there are different fields in the documents of different category/type/formats, that are to be indexed, searched for and displayed. How a field value should be extracted from a source document is also important.

This information about different document categories, types and formats and their corresponding fields is used in DocStore as well as OLE. It is preferable to store this information in a well encapsulated, reusable and maintainable manner.

This is done in an external xml configuration file (DocumentConfig.xml) which is loaded into a corresponding POJO name DocumentConfig. The configuration file is loaded at the time of startup of application and retained in memory until the application is shutdown. So any changes to this file will become effective only after restarting the application. There are limitations on what can be configured in this file. Changes to this file will need the existing documents to be re-indexed.

DocumentConfig.xml

It is available in %ole.docstore.home%/properties folder. (e.g. /opt/docstore/properties)

Field name convention:

Names of fields used for indexing/searching are suffixed with "_search".

Names of fields used for display are suffixed with "_display".

Names of fields used for facets are suffixed with "_facet".

Names of fields used for sorting are suffixed with "_sort".

Field Info/Attributes

Field Attribute

Purpose

Example

Id

Unique identifier of a field with a given [category, type, format]

id="ISBN_search

Name

Name of the field suitable for display

name="ISBN"

Type

Indicates the type of value of the field (informative purpose only)

type="text"

Field Definition

No Format

 A Field inside configurations file can be defined with attribute id is the name of the field inside solr by which it can be represented or indexed,

 name is the one to be displayable name of the filed and type is its entity type.

       <field id="ISBN_search" name="ISBN" type="text">

          <mapping type="custom">

               <include>020-a;z</include>

               <exclude/>                       

          </mapping>

     </field>

Mapping info can be defined for each field which specifies how the value(s) for the field should be extracted from the input file for the corresponding document. Mapping can be specified as XPATH value or a custom value.

Mapping Info/Attributes

Mapping Attribute

Purpose

Example

Type

Indicates how the mapping info is to be interpreted

type="custom"

Include

Values to be included

<include>020-a;z</include>

Exclude

Values to be excluded

<exclude/>

Mapping Entry:

...

,Holdings

3.1 Short display

    Each record in the search results shows a subset of the record's fields.

3.2 Detailed display

   A link is (not yet) provided for each record in the search results.

   When this link is clicked, a popup, or a new tab of the browser is opened, with all the fields of the record.

3.3 Highlighting

In each search result, words matching the key words entered by the user are highlighted.

3.4 Facets

Facets are grouping of search results that help to analyze the search results and further filter or narrow down the search results.

These are helpful when the user cannot guess what exact keywords to search for.

Facet fields:

Author,Subject,Format,Language,Publication Date,Genre.

Top 5 facet values in the decreasing order of occurrences are shown for each facet field.

The remaining facets are seen by clicking the “more” link in a popup window.

The facet values in the popup are shown in alphabetical order.

Click on one or more facet values to filter search results and view only those records containing these facet values.

3.5 Sorting

By default, the search results are sorted by Title (A-Z)

The sorting options are:

    Title (A-Z) : sorts the results in the  ascending order (starts with ‘a’) of the title field value. This the default sorting order.
    Title (Z-A) :sorts the results in the descending  order (starts with ‘z’) of the title field value.
    Author (A-Z) :sorts the results in the ascending order (starts with ‘a’) of the author field value.
    Author (Z-A) : sorts the results in the descending order (starts with ‘a’) of the author field value.
    Pub date(new-old) : sorts the results in the descending order of the publication date field value.
    Pub date(old-new) : sorts the results in the ascending order of the publication date field value.  
    Relevance : it is default sort criteria provided by solr.

Records with empty or null values will appear at the top of the search results.

As per JIRA OLE_2194,

2nd indicator in MARC is non-filing character. need to use rules for these in applying sort/display standards. Ex. 245 1 3 $aAn April Shower- 2nd indicator is "3". Ignore first 3 characters, ie "An(space)" in applying sort rule

Test Template for sort

3.5.1 NISO Standard for Sort

Anchor
TxSearch
TxSearch

The sort rules as recommended in the NISO standard are to be implemented. Please refer to NISO Standard for Sort for details.

NISO Search Results on docstore Search- in progress: https://jira.kuali.org/browse/OLE-2194  

Most of the rules for sorting are done in Solr by specifying filters like LowerCaseFilterFactory in schema.xml

4. Configurability of searchable fields

As seen above, there are different fields in the documents of different category/type/formats, that are to be indexed, searched for and displayed. How a field value should be extracted from a source document is also important.

This information about different document categories, types and formats and their corresponding fields is used in DocStore as well as OLE. It is preferable to store this information in a well encapsulated, reusable and maintainable manner.

This is done in an external xml configuration file (DocumentConfig.xml) which is loaded into a corresponding POJO name DocumentConfig. The configuration file is loaded at the time of startup of application and retained in memory until the application is shutdown. So any changes to this file will become effective only after restarting the application. There are limitations on what can be configured in this file. Changes to this file will need the existing documents to be re-indexed.

4.1 DocumentConfig Info

DocumentConfig.xml

It is available in %ole.docstore.home%/properties folder. (e.g. /opt/docstore/properties)

4.2 Field Definition

Field name convention:

Names of fields used for indexing/searching are suffixed with “_search”.

Names of fields used for display are suffixed with “_display”.

Names of fields used for facets are suffixed with “_facet”.

Names of fields used for sorting are suffixed with “_sort”.

Field Info/Attributes

Field Attribute

Purpose

Example

Id

Unique identifier of a field with a given [category, type, format]

id="ISBN_search"

Name

Name of the field suitable for display

name="ISBN"

Type

Indicates the type of value of the field (informative purpose only)

type="text"

Field Definition Example:

No Format
       <field id="ISBN_search" name="ISBN" type="text">
          <mapping type="custom">
              <include>020-a;z</include>
              <exclude/>                       
          </mapping>
       </field>

4.3 Mapping Definition

Mapping info can be defined for each field which specifies how the value(s) for the field should be extracted from the input file for the corresponding document. Mapping can be specified as XPATH value or a custom value.

Mapping Info/Attributes

Mapping Attribute

Purpose

Example

Type

Indicates how the mapping info is to be interpreted

type="custom"

Include

Values to be included

<include>020-a;z</include>

Exclude

Values to be excluded

<exclude/>

xpath Mapping Example:

No Format
       <field id="ContractNumber_search" name="Contract Number" type="text">

          <mapping type="xpath">

              <include>/publicationsLicenseExpression/licenseDetail/licenseIdentifier/IDValue/value</include>                       

          </mapping>
        </field>

Above is the example for the field of mapping type xpath.

4.4 Modifying configuration info

...

  1. Open the DocumentConfig.xml file.
  2. Add/modify/delete one or more fields of any [document category/type/format].
  3. Save the file.
  4. Reload the DocStore application. (Restart the Tomcat server.)
  5. Re-index the data related to the document category/type/format modified.

...

A field definition can be commented or deleted.

Test Template for Document Config

Transactional Search

OLE coding to-date for Acquisitions functions have utilized KNS Lookups, DocSearch (Detailed Search, Superuser Search), and named or session-based searches......

...