Libraries and metadata in a sea of information

Alan Manifold

As libraries, metadata, books and the world evolve, it seems quite possible that libraries of the future may have a closer relationship with metadata than with books. People have access to more information now than any library has ever, or could ever, put on the shelf. What people need are effective ways to find and identify relevant information. Helping them do so has been the mission of libraries since their beginning and will most likely continue to be their mission well into the future.

 Young woman browsing the shelves, c1950

Young woman browsing the shelves, c1950. State Library of Victoria digital collection.
Public Domain.

Evolution of metadata along with evolution of collection

As we look back at the history of metadata for libraries we can see two significant factors at work: the increasing size and scope of collections, and technological advancement. These two forces have been at work in shaping the requirements for metadata and the methods by which metadata is delivered. A quick look at this twin evolution will prove instructive for both understanding the past and anticipating the future.

As libraries grew beyond an individual's ability to know or remember all of their content, librarians created ledgers of acquisitions, or 'book catalogues', ordered by date of acquisition. Book catalogues were helpful for inventory, but were a terrible tool for discovery.

Two ideas had to emerge before finding things became possible in larger libraries. The first was classification, or cataloguing, which encompassed principles for organising the items on the shelf, and opened the door for subject searching and other discovery. The second was the card catalogue. With metadata becoming a critical aspect of the discovery process, there had to be a way to organise it for effective use. With each entry on a separate card, sorting and interfiling the information became relatively easy. Multiple cards for a single item allowed materials to be searched by author, title, subject, call number and other access points. Metadata changed to accommodate this growing new use, with call numbers, subject headings, added authors, series information, additional forms of title and numerous other fields added.

Phase 2: the card catalogue
Creative Commons License Phase 2: the card catalogue |
Author: Jason Pearce

As computers became more readily available, libraries took advantage of the new capabilities they provided. Early automation in libraries focused not on discovery, but on inventory control. The first widespread automation systems supported circulation, and usually not much else. There were some early forays into automated acquisitions, but these were separate from the circulation systems. Both types of systems were used exclusively by the library staff.

As the concept of an integrated library system (ILS) emerged, so did the primary principle of using a single record to describe an item from ordering until end of life. As the importance of a single record grew, increasingly detailed and consistent standards were developed for encoding cataloguing data, including MARC, AACR1 and 2, and more recently, RDA.

The idea of making the information in the ILS available to library patrons came later. The first Online Public Access Catalogues (OPACs) were clunky and limited, but they augmented the card catalogue, leveraging the investment in technology. The systems were originally designed for back office inventory control and the need to track materials through acquisitions, cataloguing, and circulation. This dictated both the way the data was stored and organised, and the required format and content of the metadata. Still, OPACs were incredibly powerful compared to the card catalogue, and many libraries ceased maintenance of the card catalogue and focused more resources on their OPAC.

Printed journal indexes were converted to electronic databases and interfaces were created to search this new class of metadata. Expert searchers were trained to search multiple arcane journal indexes to uncover information. At first there was no integration between these interfaces and the library systems. Customers and vendor developers worked together to define standard methods for searching data, which led to the ability to search multiple resources with a single search, or 'federated searching'. Integrating the results from diverse indexes and library catalogues was and is as much an art as a science. This is because different systems respond at vastly different speeds, with different metadata, and sometimes through different protocols.

Libraries have been cooperating for many years through Interlibrary Lending (ILL). By permitting patrons to borrow materials from other libraries, all participants benefit. While a library's main collection consists of items it has acquired and which are stored on its shelves, its 'virtual collection' includes the collections of other libraries, whose catalogues can be easily searched remotely.

 City of Canada Bay Library, NSW
Creative Commons License City of Canada Bay Library, NSW |
Author: State Library of NSW Public Library Services

More recently, libraries have begun to replace their ageing OPACs, which provide limited flexibility and customisability and do not reflect the advances in computer-user interaction learned from experience with the World-Wide Web. New discovery platforms 'harvest' the data from the ILS and other data sources, augment it and provide a robust, flexible, customisable and inclusive interface. These platforms incorporate modern user interface technology, such as search scoping, facets, images, integrated external links and easy customisation - features which would require most OPACs to be rewritten from scratch. They also include metadata for materials that are not covered by an ILS, such as digitised and born digital materials, licensed e-journals and pay-per-view eBooks, providing a single platform from which to search virtually everything the library has access to.

Significant advances in computer storage technology and computing power have made it possible to create what have been termed 'mega-aggregate indexes' to replace slow and problematic federated searching. These indexes harvest metadata and full-text content for hundreds of millions of items from a plethora of databases covering journal, eBook, newspaper, publisher and other content. The data is transformed into a consistent format and indexed, then integrated seamlessly into discovery systems to providing searchers with massive amounts of data in a single interface.

This brief overview shows how the increasing size and scope of library collections and the availability of new technologies have played off one another in the shaping of library catalogues and the metadata they contain. As more is possible technologically, patrons come to expect and demand easier and more comprehensive access to information, and as the size and scope of library collections increases, new solutions must be developed to manage these collections and the growing requirements of their users.

The shifting purposes of metadata

Increased availability of materials in electronic form has thrown a profession that grew up around the concept of collecting materials into confusion. When people at home can have access to most of the sources they can access within the walls of the library, the utility of physical libraries is called into question. When the full text of a vast array of materials can be searched through the web, the purpose of metadata becomes less clear as well.

The purpose of library metadata needs to shift to fulfil the evolving needs of information seekers. When many web searches return more than a billion results based on the full text of web pages, documents and eBooks, discoverability of materials is no longer a primary concern. The primary purpose of metadata today must be to help users find the most relevant, authoritative, best quality materials available. A web searcher needs clues to help them determine which hits to pursue and which to ignore. The most helpful clues are those that indicate that a reliable source has examined the material and rated it highly. While reader reviews are helpful, searchers are more likely to trust metadata from a library than from an individual.

Quick reference: the purposes of metadata
Quick reference: the purposes of metadata

Library metadata is created according to standards which can make searches more focused. Material with subject headings from a controlled vocabulary can be discovered more reliably than material where the full text is searchable, but which has no metadata. Books which use a variant form of a title can be easier to find when metadata includes a uniform title. Authors whose names have been normalised through a name authority file can be more readily identified than those with only an uncontrolled form of name.

Finally, metadata that links related resources together is invaluable in winnowing down a huge set of results to manageable proportions. For example, if metadata existed that collated books and other resources applicable to a particular Australian Curriculum code, it could save teachers time in identifying and evaluating resources. Peer reviewed articles and award winning books could be identified by their metadata and thus brought together through an appropriate search interface.

Some of these uses for metadata are possible, but some of them may not be easily achieved within current metadata content and formatting standards. As new purposes for metadata emerge, the content and format for metadata will eventually morph to support these new purposes. Formats (such as MARC) and content standards(such as RDA) will be extended to accommodate them, or new formats and standards will be introduced to meet the changing needs.

The role of SCIS in this evolving environment

SCIS is in a good position to provide leadership in this area for a number of reasons. As part of Education Services Australia, SCIS has organisational support for making changes that support the educational mission of Australian schools. The organisation is small enough to allow staff to consult, to decide on best practices and to monitor their implementation, and adapt them as needed. The fact that SCIS subscribers are school libraries allows metadata practices to be effectively targeted to an audience with common requirements and priorities. School libraries tend to have common characteristics:

  • They take an active role in fulfilling the school's mission.
  • They provide materials that are closely tied to the curriculum.
  • They focus strongly on promoting learning and literacy.
  • They participate in supporting schools' vision of a diverse but unified student body and community.
  • They protect their young patrons from materials with inappropriate subject matter, treatment, or audience.
  • They lag behind the cutting edge of technology adoption and tend to be underfunded.
  • They serve learners who generally do not require a specific resource; rather any resource that covers the subject will usually do.

Decisions made by SCIS and its members will automatically and immediately affect a large number of school libraries. Building on its long history of providing high-quality cataloguing services for its members, SCIS could experiment with linking library resources to specific curriculum codes and grade levels, and to link to related resources on the web. Items that meet specific criteria could be marked as providing rich multicultural perspectives suitable for children and youth. It would be worth exploring the possibilities for incorporating quality ratings into the metadata.


The content and form of metadata has been continually evolving due to the interplay between the increasing scope and size of library collections and rapidly changing technology. This transformation of metadata has a great deal to do with the purposes for which it has been used, which have been changing as well. With massive quantities of information readily available at everyone's fingertips, metadata serves to validate, authenticate, categorise, limit and evaluate resources. With its tight focus, lean staffing and close ties to Australia's educational infrastructure, SCIS has a unique opportunity to introduce new standards and practices in cataloguing that will help its members achieve their missions more efficiently and effectively. The learning that will arise out of this leadership will prove invaluable to the entire library profession, as it seeks to remain relevant in a world of too much information.

This article records and expands on themes covered in the author's presentation at the SCIS Asks forum in November 2013. Presentation slides available at:


 Alan Manifold

Alan Manifold

From Manager of Library Enterprise Applications at Purdue University to Primo and Voyager Support Manager at Ex Libris to Digital and Library Applications Manager at the State Library of Victoria, Alan has more than 30 years experience in library automation. Off-hours, he bakes, sings, composes and arranges choral music, and serves as an officer of the Banyule Baha'i Local Spiritual Assembly.