Index Data will be in Chicago at ALA Annual
booth number is 624. We have half an island all to ourselves in a sea
of exhibit booths. Please stop by to say hello and check out our latest
developments. Hope to see you in Denver!
Relais International and ID Announce Partnership
International and Index Data announce a partnership to offer a next
generation standards based Resource Sharing solution. The combination
of Index Data’s meta- search engine with Relais International’s
renowned resource sharing software will result in a ‘seamless end to
end discovery to delivery’ solution for Resource Sharing. Users
will be able to search simultaneously across multiple catalogs and
databases. Once results are displayed, the availability for items of
interest will be checked and the user may submit a request. For
libraries that may need to place limits on what can be requested,
additional checking will be done based on the user profile.
Data ApS is in the middle of moving its headquarters to a new location.
We expect all services to be stable again by September 1.
Note: This is an OCLC press release. It can also be read here .
DUBLIN, Ohio, USA, 26 June 2008—OCLC
and Index Data, a software development and consulting enterprise that
specializes in information retrieval and metasearch solutions, are
working together to extend the discovery capabilities of WorldCat Local
to include all licensed and full-text resources of a library.
Local is the service that combines the cooperative power of OCLC member
libraries worldwide with the ability to use WorldCat.org as a solution
for local discovery and delivery services. WorldCat Local provides a
powerful discovery environment that presents localized results most
relevant to the library user while at the same time allowing the user
to search the entire WorldCat database of more than 100 million records.
continues to work with database producers to add article-level metadata
to WorldCat.org to enrich the search experience and make collections
from libraries more visible on the Web. Index Data will help OCLC
incorporate metasearch into WorldCat Local for searching databases that
are not indexed in WorldCat.org.
"The integration of metasearch
and locally indexed results will help OCLC to provide a single search
through WorldCat Local for licensed resources across the library,” said
Robin Murray, Vice President, OCLC Global Product Management. "We're
confident that Index Data will help us achieve that integration."
Data believes that the future of information discovery lies in a
combination of many different technologies including broadcast
searching and local indexes. Our expertise and modular technologies are
a perfect fit for OCLC's demanding production environment," said Lynn
Bailey, CEO of Index Data.
OCLC recently announced that
article-level metadata from H.W. Wilson and MLA will be added to the
more than 50 million articles indexed from NLM MEDLINE, the Department
of Education's ERIC database, the British Library Inside serials, the
GPO Monthly Catalog and the OCLC ArticleFirst® database to expand
access and discovery of authoritative content through the WorldCat.org
platform. The work with Index Data will help to ensure that libraries
can provide access to their full collections.
in 1967 and headquartered in Dublin, Ohio, OCLC is a nonprofit library
service and research organization that has provided computer-based
cataloging, reference, resource sharing, eContent, preservation,
library management and Web services to 60,000 libraries in 112
countries and territories. OCLC and its member libraries worldwide have
created and maintain WorldCat, the world's richest online resource for
finding library materials. For more information, visit www.oclc.org.
About Index Data
Data is a software development and consulting enterprise with offices
in Copenhagen, the UK and the United States. They are experts in
networked information retrieval and management, and have a
fourteen-year history of releasing their software as open source. Their
mission is to use their skills and creativity to build bridges between
people, technologies and organizations, and to empower information
providers to deliver better, more integrated services. For more
information, visit www.indexdata.com.
OCLC, WorldCat, WorldCat Local and WorldCat.org are trademarks/service
marks of OCLC, Inc. Third-party product, service and business names are
trademarks/service marks of their respective owners.
The Ability To Choose the Federated Search Interface And Content Of Your Choice Using Open Standards.
11, 2008 – Blacksburg, VA. – CARE Affiliates™ announced a new product
today, in conjunction with its strategic partners, Index Data™ and
WebFeat®, called OpenTranslators™. OpenTranslators is intended to
reshape the way libraries select and use federated search and
OpenTranslators will allow
libraries to use the federated search interface of their choice to
access over 10,000 databases using SRU/SRW/Z39.50. The databases
consist of: licensed databases, free databases, catalogs, Z39.50,
Telnet and proprietary databases. Libraries that already have a Z39.50
client in their OPAC will be able to connect to, not only library
catalogs, but also thousands of additional databases. Those libraries
that are building or already using an open source federated search tool
will now be able to expand the world of information that can be
accessed. Finally, for those institutions/organizations building new
mashup clients, this will allow them to access and use vast amounts of
Use of the WebFeat translators
in this solution provides a unique capability in that this technology
structures and parses unstructured citations, even for databases that
don’t natively support such functionality. This means it is possible to
sort results by date, title and author and support citation exports
into a variety of formats. In addition results usage can be tracked in
compliance with the COUNTER standards.
the WebFeat Administrative Console (WAC), which is well known for its
ease of use, flexibility and speed in managing translators, is included
in the solution. It takes what could be a tedious and time-consuming
task for library staff and virtually eliminates it.
has combined these capabilities with Index Data’s expertise and
resources to provide a SRU/SRW/Z39.50 gateway to the WebFeat
translators. Index Data, long a pioneer in Information Retrieval
technology, designed and built a sophisticated gateway that provides a
seamless level of connectivity for end users.
is a hosted service, so subscribing to the service is easy. New
databases can be added on request and there are no servers or software
About CARE Affiliates, Inc.
Affiliates provides services and products based on open source software
to libraries and information centers around the world. These services
include consulting, selection, packaging, integration, installation,
support, hosting (SaaS), maintenance and continued development.
Products include repositories, federated search tools, preservation and
archiving solutions. CARE Affiliates is an international organization
with offices in Chicago, Illinois, Blacksburg, Virginia and Toronto,
Canada. Learn more about CARE at http://www.care-affiliates.com
About Index Data
Data is a consulting and software-development enterprise headquartered
in Copenhagen, Denmark with offices in the UK and United States. Index
Data LLC, its United States subsidiary, is headquartered in West
Hartford Connecticut with offices in Keene, New Hampshire and
Winchester, Massachusetts. They are experts in information retrieval
and management in a networked environment and have a thirteen-year
history of releasing their software as open source.
(U.S. patent #6,807,539) is the developer of the WebFeat™ and WebFeat
Express™ search solutions, used by over 16,500 leading public,
academic, government and Global 1000 libraries and information
centers—including one third of the largest 100 U.S. academic and public
libraries, 17 statewide library systems, and 2 out of every 10
Association of Research Libraries (ARL) institutions. WebFeat offers
next-generation federated search and e-resource management tools to
meet the needs of any library. Products include: the WebFeat Express
federated search solution, WebFeat Enterprise Edition for multi-library
networks, WebFeat Custom API service and SMART™, WebFeat’s usage
tracker. WebFeat Menu Manager™ and WebFeat Proxy™ provide tools for
access and management of e-resource collections. WebFeat provides
multi-language support. WebFeat has earned industry recognition from
leading organizations such as Library Journal and the Gartner Group.
Learn more about WebFeat at http://www.webfeat.org.
is a registered trademark of WebFeat, Inc., “WebFeat Administrative
Console”, “WAC”, “SMART”, “WebFeat Custom”, “WebFeat Enterprise
Edition”, “WebFeat Menu Manager”, and “WebFeat Proxy” are trademarks of
OpenTranslators and CARE Affiliates are trademarks of CARE Affiliates, Inc.
Index Data is a trademark of Index Data, LLC.
Sebastian Hammer has appointed Lynn Bailey as Index Data's new CEO to
keep the company nimble and well-governed through what is expected to
be a period of substantial growth. Hammer assumed the role of President
and will devote more time to software development and technology
"Libraries are demanding significant improvements in information
retrieval services, and I want to focus my efforts on ensuring that
Index Data meets those expectations," explained Hammer. "I sense the
library community as a whole is beginning to recognize that the
combination of commercial open source software, open standards and
modular architecture offers an attractive alternative to proprietary,
closed and monolithic solutions. The company needs a CEO with Lynn's
abilities to focus on our business strategy and manage our growth
wisely. I am delighted she's on board.”
Hammer was the original developer of YAZ, the company's first major
software application and now the world's leading toolkit for building
Z39.50/SRU client and server applications. Recently Hammer developed
Pazpar2, the core metasearch engine behind Index Data's MasterKey™
service. He is currently focusing his efforts on integrating metadata
harvesting, local indexing, metasearching and other information
retrieval technologies to develop a unified and simple discovery
process based on open source software.
Bailey comes to Index Data after a successful 15 year career at
AT&T where she focused on Emerging Technologies, Internet Solutions
and Data Networking. While at AT&T, she held a variety of positions
in Management, Marketing and Technical Sales. She was highly regarded
for her work with key clients in transforming the nature of AT&T’s
business relationships from ‘supplier’ to ‘strategic business partner,’
resulting in mutually beneficial long-term alliances. Bailey holds a
B.S. in Management Systems from Rensselaer Polytechnic Institute and
lives in the Boston area.
Asked why she accepted the offer to lead a small open source
software company in the library market, Bailey responded, "Heading up
Index Data is an appealing challenge. The company has superb technical
and personnel assets but is under recognized in its market. Its ability
to develop and support high quality open source software far exceeds
its current market penetration. I have a strong desire to use my
experience to help the company grow and position it for success. This
includes not only continuing to improve our core technologies but also
developing partnerships and expanding our market reach. My goal is to
keep Index Data's solutions at the leading edge of the information
landscape. Keep your eyes on Index Data."
Come and meet Lynn at the joint Index Data/ CARE Affiliates booth (#230) at ALA Midwinter in Philadelphia.
Data is pleased to announce that it is the primary technology partner
in a grant awarded yesterday by the Institute of Museum and Library
Services (IMLS) to the New England Law Library Consortium (NELLCO) to
build a "Universal Search Solution" for NELLCO libraries and their
The NELLCO "Universal Search Solution" will be based on open standards
and open source software, and will result in the creation of a physical
master index of material, including participating library catalogs, as
well as subscription-based databases and open content, special
collections, and other resources that a participating library wishes to
make discoverable to its patrons.
This grant award breaks important new ground in the delivery of digital library services.
Technologically, the Universal Search Solution will combine multiple
technologies (consolidated indexing and data storage; metadata
harvesting; and metasearching) to put together a single window to
disparate resources. This fusion of technologies could serve as a model
for future library search services, and indeed, the technologies used
will be made available as open source software.
The grant also offers a new model for libraries to obtain affordable
software services that are under their control. It combines the
powerful financial leverage of IMLS and the organizational capabilities
of NELLCO and its membership with the software development expertise of
Index Data to bring cutting edge open source software and services to
all types and sizes of libraries--affordable and with commercial
support--but without vendor lock-in.
Index Data is proud that it was selected by NELLCO and IMLS to build
the Universal Search Solution for the library community. We look
forward to other research and higher education communities initiating
similar commercial open source projects in the future.
The Institute of Museum and Library Services is the primary source of
federal support for US libraries and museums. Through its grant making,
convenings, research and publications, the IMLS empowers museums and
libraries to provide leadership and services to enhance learning,
sustain cultural heritage, build twenty-first-century skills, and
increase civic participation. (http://www.imls.gov)
Founded in 1983, NELLCO is a consortium of 101 law libraries dedicated
to the promotion and facilitation of resource sharing on behalf of its
members. NELLCO is comprised of 25 full members, over 65 affiliate
members from 33 states across the United States, and 7 international
affiliate members from Canada and the U.K. (http://www.nellco.org)
are pleased to announce release 1.2.1 of CQL-Java, the free CQL parser
for Java. This release, sponsored by the National Library of Australia,
supports version 1.2 of CQL, including the new sorting syntax. It is
distributed under the GNU LGPL (Lesser General Public Licence) and can
be freely downloaded from http://zing.z3950.org/cql/java/
We're delighted to announce the release of another new product: Simple2ZOOM,
a sort of universal Swiss Army gateway that proxies between Z39.50,
SRU, SRU/POST and SRW. Although nearly all testing so far has been with
Z39.50-to-SRU configurations, it ought to work with pretty much any
combination of these protocols on the front- and back-ends.
Simple2ZOOM is free-as-in-freedom, open source,
software. It is distributed under the same terms as Perl, that is,
either under the GNU GPL (General Public Licence) or the Artistic
Licence -- your choice.
Simple2ZOOM is implemented in Perl, as a tiny script
that calls the Net::Z3950::Simple2ZOOM Perl module. It is this module
that is distributed, and it's freely available on CPAN at
We would like to gratefully acknowledge the National Library of Australia for providing funding that enabled us to add lots of the functionality and bring this product up to a releasable standard.
Six days ago, Seb wrote that "It isn't every day that we release a
brand-new piece of software". It's true: our record at the moment is
once every six days :-)
We're pleased to announce the release of IRSpy, a repository of
service-description records for standards-compliant Information
Retrieval services (Z39.50 and SRU/SRW). As you'd expect, it uses
ZeeRex records as described at
to describe the services, since ZeeRex (ANSI/NISO Z39.92) is itself
the standard format for this kind of thing.
IRSpy uses a Zebra database as its backend, and includes both
command-line tools and a Web UI. The repository is itself accessible
using both Z39.50 and SRU. Best of all, it can run tests on services
to see what facilites they support, what access points work, what
record syntaxes can be returned, etc.
This software was developed in co-operative between Index Data and the
National Library of Finland. We'd like to gratefully acknowledge
their sponsorship of this project.
It's written in Perl, using the ZOOM-Perl module to provide the Z39.50
and SRU client functionality. IRSpy's Web UI is built on the
isn't every day that we release a brand-new piece of software, and this
is one we're pretty excited about. We sincerely hope you will like it,
too. Pazpar2 (pronounced like 'passe-partout', if it please
ya) can be viewed either as a high-performance metasearching middleware
or a Z39.50 client with a webservice interface, depending on your
perspective and needs. It is a fairly compact C program -- a resident
daemon -- that incorporates the best we know how to do in terms of
providing high performance, user-oriented federated searching. The
original design began in my head back when Google maps first came on
the scene, and I started to wonder how browser-based logic could be
used to provide a better search experience, and what kind of back-end
tool we would need to support that. The last 10 months or so has seen
much furious development, and we are really excited to be sharing the
results with everybody. One cool thing it does is search many
databases in parallel, and do it fast, without unduly loading up the
user interface.. in fact, we find it will search more than 100 targets
in parallel and 'feel' faster the more data you throw at it. It
retrieves a set of records from each target, and performs merging,
deduplication, ranking/sorting, and pulls browse facets from them. It
does it really fast, and because this functionality is exposed through
a pretty simple XML webservice, it is really simple to build very cool,
lightning-fast user interfaces on top of it -- interfaces that tend to
make people go 'no WAY is that Z39.50!'. It doesn't know
anything about data models, so you can handle exotic data sources if
you need to.. you use XSLT to normalize data into an internal model --
we provide examples for MARC21 and a DC-esque internal model, and
configure ranking, facets, sorting, etc., from that. Let us know if you
try other things. More details at http://www.indexdata.com/pazpar2/ Isn't
Z39.50 a little old-school? Perhaps so, but it's also widely supported
-- probably one of the most vital and viable information retrieval
standards out there today. Most library catalogs and many of the
larger, commercial databases support it. It is trivial to gateway SRU/W
to it. You can use our SimpleServer to turn anything into a Z server.
If you need to do large-scale metasearching across non-standard
resources, talk to us -- we have relationships and tools with
traditional metasearch vendors and can help enable their 'database
connectors' for use via this tool. It's GPL -- use it, don't
abuse it. We offer the usual services and support to people, and we
encourage anyone contemplating commercial deployment to work with us;
make sure we know your needs, and help us develop this further and keep
it viable for the future. For anyone else; have at it. We call it
beta.. issues are bound to pop up when new people play with it in
different environments.. share your experiences, your observations,
etc. We can offer installation assistance, training, support, even
turnkey development and hosting. Enjoy, --Sebastian
YAZ 3.0.2 is available. This version changes
and simplifies the API of YAZ. Zebra
2.0.14 and PHP/YAZ has important
bug fixes and use the updated YAZ API.
Index Data launches Open Content service
has long been on our mind to find ways to better leverage the growing
repositories of free, open content within libraries and related
organizations. We feel that open standards have a key role to play in
this; to make it easy to integrate such resources into the information
landscapes that we are all building for our users.
To pursue this, we have launched a new service, described here.
The opencontent service consists of SRU and Z39.50 enabled indexes for
some of the most important open content resources: Ebooks from the Open
Content Alliance and Project Gutenberg, Wikipedia, the Open Directory.
We will continue to seek out new resources that we can enable for
access in this way, and we would welcome suggestions and participation
You can use these services from
metasearch tools, from personal citation managers, or you can build
your own applications, or integrate these resources into websites and
services. What the resources have in common is that they provide
searchable metadata for high-quality stuff which can be made freely
available to your users.
We encourage people to let us
know how they are using these services, let us know if you have ideas
for new resources, or if you can help, with data, assistance, or in
Sebastian Hammer, CEO, Index Data
is happy to announce that we have just released Keystone/1.5.7, which
is available the usual places. Including as Debian sarge/etch packages.
Some of the high-lights are:
- Integration of the tinyMCE
editor framework in the Keystone XML editor. Credits to: Horsens
Library, Denmark and University of North Texas, USA for sponsoring
money and code.
- The ' named entity is no longer double-encoded.
- The xml editor (Xform2) now supports XML schema include.
2006-11-30Release 1.14 is the latest in a series of ZOOM-Perl
releases, providing Perl programmers with ZOOM-compliant searching of
IR resources such as Z39.50 targets and SRW/U Services. It's freely
available from CPAN.
YAZ 2.1.36 is available. Besides fixing
a number of bugs since last release, the license changed to the
Revised BSD License. Wording is different from the YAZ License
but semantics should be unchanged.
Major New Release of Zebra Server
2006-10-05Index Data is pleased to announce a major upgrade
of Zebra, its Open Source database server and
indexing engine. Zebra 2.0 represents an
extensive upgrade that makes index profiling much
easier, supports increased tuning of search
results, incorporates XML technology into core
functionality, and dramatically increases performance speed.
Some of the highlights of the many improvements
of Zebra 2.0 over the 1.3 version are:
• A 64-bit based index structure
• Elimination of the 2GB limit on register file size
• New on-disk format providing increased stability
and faster indexing and retrieval
• New record filter using XSLT transformations
to drive both indexing and retrieval
• Improved logging and analysis of external traffic
• Revised and expanded documentation
Zebra 2.0 replaces the previous versions' tight
coupling to the Z39.50 BIB-1 attribute set with a
new XML friendliness, making Zebra easy to use
for such XML-based formats as Dublin Core, MODS,
METS, MARCXML, OAI-PMH, RSS, etc. The software's
new plug-in architecture allows the skilled user
to write his or her own record indexing and
retrieval filters as loadable modules.
The performance enhancements incorporated into
version 2.0 mean that Zebra can now index and
search even faster than version 1.3. In a test of
Zebra 2.0, the software was able to build a 31 GB
database of very large records in four elapsed
hours on a 1800 GHz Dual AMD box. It processed an
average of 2.2 MB of data per second.
Zebra 2.0 offers more precise logging of external
traffic, access and indexing, and log messages
are now printed in a style similar to Apache
server logs. These enhancements greatly improve
users ability to mine the log data for patterns of use.
Overview of Zebra
Adam Dickmeiss and Sebastian Hammer, the
co-founders of Index Data, began developing Zebra
in the mid 1990s. It was originally designed as a
native Z39.50-based indexer of MARC bibliographic
records, but has evolved into an XML-based text
indexing engine and database server that can
handle full text as well as any form of
structured metadata. It is used by hundreds of
libraries and library vendors around the world
for databases with upwards of 70 million records.
For more information on Zebra, visit www.indexdata.com/zebra.
2006-07-27We are delighted to announce
our toolkit for building IR servers in Perl. Release 1.02
more reliably provides authentication tokens, and allows back-end
servers to indicate the schema used by the records they provide.
Keystone 1.5.5 available
Indexdata is happy to announce that we have released Keystone version 1.5.5. High-lights from the changelog include:
- The document editor's (Xform2) display is now customizable on a portal to portal basis using XSLT stylesheets.
- Field indentation in the multipage views in the document editor now works in Firefox too.
- Improved error reporting in the administration interfaces.
Apart from this, a lot of small bugs have been fixed. Confirm with the changelog.
8, 2006 - West Hartford, CT -- The Taylor & Francis Group, a
leading international publisher of scientific, technical and medical
literature, has contracted with Index Data to provide a
Z39.50/SRU/SRW-capable server for CRCnetBASE, Taylor and Francis'
family of online STM reference databases, thereby making it easier
for libraries to access this full text service via single search
services and standards-based search clients.
and Francis is one of a growing number publishers that recognize the
advantages of exposing online content using standard search
protocols, and that are calling on Index Data to make it happen.
"Federated Search represents a critical part of our overall
product strategy," said Robert J. Morris, Director of
Development at the Taylor and Francis Group, LLC. "Index Data's
team has worked with us to design a custom solution that allows for
search integration across our entire enterprise. This system
represents a key component in our efforts to expand the volume and
types of content contained within our electronic product offerings."
are delighted that Taylor and Francis is making it possible for
researchers to easily access their online reference services by using
open standards." said Sebastian Hammer, Index Data's CEO. "As
more content providers support Z39.50 and SRU/SRW, implementation and
maintenance costs for single search services drop, freeing up more
funds for actual content."
Data provides standardized access to publishers' content in one of
a custom gateway that mediates between the publishers' server and
remote clients, translating between standards-based information
retrieval clients and the local content management system employed by
using Index Data's Zebra server, which indexes content to the
publisher's specifications and serves it up via standard protocols.
Zebra gives publishers maximum control and flexibility over search
and retrieval capabilities.
YAZ version 2.1.18 has been released today. The most important changes are:
ASN.1 compiler 0.4: generates brief Doxygen file header for
generated .h/.c files.
Utility yaz_iconv now supports conversion to MARC-8.
New facilities for the MARC module. The reading - and writing of
content are separate methods for the yaz_marc_t handle. The following
read functions are available: yaz_marc_read_iso2709 (Reads MARC in
ISO2709 format), yaz_marc_read_xml (reads MARC in MARCXML/MarcXchange
format). Write functions have prefix yaz_marc_write_.. The existing
utilities yaz_marc_decode_wrbuf and yaz_marc_decode_buf are still
available. Removed support for OAI-MARC and simplexml (not the
simplexml from PHP5).
Added a new ZOOM event type ZOOM_EVENT_END which signals no more events
to be returned for this connection.
We are delighted to announce the new release 1.5 of
the ZOOM-Perl module
for building IR applications in Perl, using the standard protocols
Z39.50, SRU and SRW.
The big change in release 1.05 is the inclusion of support for asynchronous
multiplexing - that is, the ability to build metasearching applications in Perl.
This functionality has been in the underlying ZOOM-C code all along, but is
now wired out to the Perl level for the first time. The 1.05 distribution includes
example programs using the new facilities, and for the very first time the ZOOM
metasearching facilities are
documented, albeit briefly.
Data has just signed a contract with Helsinki University Library, The
National Library of Finland, to develop an information retrieval target
monitoring and analysis tool. This software, which is being released
under the name Keystone Target Manager, will be designed to meet the
specifications of the NISO draft standard Z39.92: Information Retrieval
Service Description, which is based on an earlier community standard
The Keystone Target Manager will perform the following services:
- Automatically monitor a collection of Z39.50/SRW/SRU targets,
maintaining statistics for availability over time.
- Provide configurable alerting services when targets become
- Automatically gather information about the search capabilities
of targets that use standard search protocols, such as
Z39.50/SRW/SRU, for use in configuring IR clients of different
- Make the gathered information available in a software-readable
form, as a database of ZeeRex (ANSI/NISO Z39.92) records.
(This will enable sharing of this information via the Internet.)
Juha Hakala, Director of Information Technology
at Helsinki University Library and Chair of the NISO Committee which
developed the NISO Z39.92 draft standard, points out that "up to now
information about Z39.50 or SRU/SRW targets has usually been discovered
and documented manually, which is a laborious and time consuming
process. Also, there has been no efficient means of exchanging this
data among metasearch applications such as portals. While NISO Z39.92,
solves the latter problem, the Keystone Target Manager will simplify
the process of creating accurate target descriptions and keeping them
"Given the expertise Index Data has in
implementing Z39.50 and SRU/SRW applications," Hakala continued, "I am
confident that the service descriptions generated by the Target Manager
will be useful. It will be interesting to see how quickly the IR portal
vendors and maintenance agencies such as libraries will adopt this
target management tool, and start sharing on a global scale the target
descriptions harvested with it."
Pat Stevens, the Interim Executive Director of
NISO, was delighted to learn about this agreement: "NISO thanks
Helsinki University and Index Data for their collaborative work on this
project," she stated. "It comes at just the right time as organizations
are now looking for an effective way to implement Z39.92."
The Keystone Target Manager will be a major
step forward in standardizing and reducing the costs of maintaining
target gateways for metasearch services. Content providers that support
standardized search interfaces, and federated search services that take
advantage of this soon-to-be-released Open Source/Open Access target
management service, will together be able to reduce the cost of
providing federated search services to libraries. If these lower costs
are passed on to libraries in the form of reduced gateway maintenance
fees, libraries will be able to shift costs from access mechanisms to
YAZ 2.1.16 available.
2006-03-24We are delighted to announce
our toolkit for building IR servers in Perl. The hot news with the 1.x
series is that SimpleServer now supports SRU and SRW as well as Z39.50.
Index Data at SRU Meeting and Integration Workshop
2006-02-23Index Data's Mike Taylor and Adam Dickmeiss, both members of the
Editorial Board, will be in The Hague (Netherlands) from 1st-3rd March for the 2006.
SRU Meeting and Integration Workshop.
the SRU meeting, we will be proposing an important change to how
sorting is done, bringing sort specifications right into the query
language CQL; and in the Integration Workshop, we will be presenting
this query language to a broader audience.
New Index Data Software
2006-02-08Index Data is pleased to announce the release of ZOOM-Perl, a new Z39.50/SRW/SRU client API based on the YAZ toolkit.
is based on the ZOOM abstract API (http://zoom.z3950.org/). Because it
is based on the YAZ toolkit, it inherits the following key benefits
from that toolkit:
- Transparent support for Z39.50, SRW, snd SRU
- Client-side CQL-parsing, so that CQL can be used as a query language against both Z39.50 and SRW/SRU targets
- Mapping of MARC records to MARCXML
- Mapping of MARC-8 and other character sets in retrieval records to UTF-8
- Support for remote database administration in compatible servers (e.g. Index Data's Zebra)
is intended to replace the Net-Z3950 API, but incorporating more of the
benefits of the YAZ/ZOOM implementation. See also the introduction on
the Net-Z3950 mailing list at
Download and documentation from http://search.cpan.org/~mirk/Net-Z3950-ZOOM-1.01/
Meet Index Data at ALA Midwinter 2006
this year Index Data is exhibiting at the ALA Midwinter Conference in
San Antonio, Texas. Come and see us at booth 1472 and learn about new
developments and services.
Scholastic deploys Index Data Gateway
CONTENT PROVIDER MOVES TO IMPLEMENT STANDARDS
9, 2006 - West Hartford, CT -- Scholastic Library Publishing, a
division of Scholastic, the global children’s publishing, media and
education company, has employed the services of Index Data to
implement standards-based information retrieval gateway, thereby
making it easier for libraries to access Grolier Online, the leading
subscription-based online reference resource for schools and
via single search strategies.
Library Publishing is pleased to announce the release of Z39.50
compliant versions of all of the encyclopedia databases that are
included in Grolier Online,” said Trudy Tuffy, Senior Director of
Grolier Online. "We are very excited about this new method of
access to our encyclopedia databases, as it helps to make them
available to a wider range of educators and students."
of our specialties is to build gateways between proprietary database
servers and standards-based clients," said Sebastian Hammer,
Index Data's CEO. "In a time when libraries everywhere are
seeking to implement single search interfaces to multiple resources
for their patrons, it is in the interests of content providers to
make it an easy matter for metasearch software vendors to access
their content, and ease of access is best achieved through standards
such as Z39.50 and its Web services offspring, ZING."
Scholastic Library Publishing Scholastic Library Publishing
creates a multitude of products, both in print and online, for the
school and public library market under the imprints of Grolier,
Children's Press, Franklin Watts, and Grolier Online. Children’s
Press offers primarily non-fiction series for elementary and middle
schools, while Franklin Watts focuses on non-fiction for the high
school market. Grolier publishes encyclopedias and is best known for
the brands Encyclopedia Americana and The New Book of
Knowledge. Grolier Online is a self-contained,
non-commercial electronic gateway offering thousands of resources for
student research, including the Encyclopedia Americana online,
The New Book of Knowledge Online and the Spanish language
encyclopedia La Nueva Enciclopedia Cumbre - serving more than
18 million students and teachers.
8, 2005 - Athens, OH and West Hartford, CT - LibLime, the recently
formed company that is marketing and supporting Koha, the world's most
widely used Open Source library management system, and Index Data, the
eleven year old library software development company that specializes
in Open Source information retrieval toolkits and end user
applications, announced today that the next release of Koha will use
Index Data's Zebra database server.
2005, the Koha project development team began looking at ways to
improve MARC support and overcome scalability limitations in the Koha
2.x series. After extensive evaluations of the best of the Open Source
textual database engines--including MySQL full-text searching,
PostgreSQL, Lucene and Plucene--the team selected Zebra.
completely eliminates scalability limitations, because it can support
tens of millions of records." explained Joshua Ferraro, LibLime's
Technology President and Koha's Project Release Manager. "Our
performance tests showed search results in under a second for databases
with over 5 million records on a modest i386 900Mhz test server."
also includes support for true boolean search expressions and
relevance-ranked free-text queries, both of which the Koha 2.x series
lack. Zebra also supports incremental and safe database updates, which
allow on-the-fly record management. Finally, since Zebra has at its
heart the Z39.50protocol, it greatly improves Koha's support for that critical library standard."
the bibliographic database will be moved to Zebra, Koha 3.0 will
continue to use a relational SQL-based database design for the
'factual' database. "Relational database managers have their strengths,
in spite of their inability to handle large numbers of bibliographic
records efficiently," summed up Ferraro, "We're taking the best from
both worlds in our redesigned Koha 3.0.
is a full-featured Open Source library management system developed
initially in by Katipo Communications Ltd in New Zealand and first
deployed in January 2000 for Horowhenua Library Trust. It is currently
maintained by a team of software providers and library technology staff
from around the globe. Several companies around the world support Koha,
providing libraries with a full array of vendor services including
installation, migration assistance, data integrity testing, staff
training, software maintenance, and development of new features. To
learn more about what services are available visit
ABOUT INDEX DATA
Data, a software development and consultancy firm, with offices in
Copenhagen; Gloucestershire, England; Keene, NH and Middletown, CT, is
the world’s leading distributor of Open Source library application
software. It’s flagship product is the Keystone Digital Library Suite,
which includes content management and metasearch facilities. Other
products include the YAZ family of toolkits and applications for
information retrieval and interlibrary loan; Zebra, an XML/MARC/text
indexing engine with a Z39.50 front end; and YakPac, a young people’s
search interface. For more information, visit www.indexdata.com or call
860-389-1568.ABOUT LIBLIMELibLime's mission is to make Open Source software accessible to libraries across the globe. To
that end, LibLime develops and markets affordable and customizable Open
Source library technology solutions like Koha ILS. LibLime also
provides full vendor services on these software products including:
migration assistance, staff training, and software maintenance, support
and development. For more information, visit www.liblime.com or call
YAZ 2.1.10 and PHP/YAZ 1.0.5 Released
2005-11-29It's been quite a while since last YAZ release. YAZ 2.1.10 includes new
features such as SRU/POST support, SRW/U named result sets,
SearchResult-1 ZOOM support (hit counts for individual terms), ZinG
Update support (experimental).
Also a few bugs were fixed.
Special thanks to Ko van der Sloot for implementing new features.
See the a more complete list at the YAZ web site..
And the PHP/YAZ site:
Index Data and the Open Content Alliance
2005-10-27At Index Data, we applaud the Open Content Alliance for its use of open
standards, for its respect for copyrights issues, and for its
dedication to make the full text of books available to anyone,
anywhere. We think that the Open Library, which is the vision that the
OCA is working to realize, may well be seen years from now as one of
the pivotal accomplishments of humankind, and we are absolutely
delighted to be helping the OCA realizing this vision.
During 2005, Index Data has worked with the Internet Archive, one of
the founding and central members of the Open Content Alliance (which
also includes Yahoo!, Microsoft, HP, Adobe, and many others), to ensure
that structured metadata, open standards, and copyright issues are
handled well. The project is extremely well-suited to the diverse
skill-sets of Index Data, and we're very excited to be a part of it.
Data is pleased to announce that Paratext, the developer and
distributor of Reference Universe, 19th Century Masterfile and other
online reference databases, is employing the services of Index Data to
implement a standards-based information retrieval gateway, thereby
making it easier and less expensive for libraries to access their
content via single search strategies.
"One of our specialties is to build gateways between proprietary
database servers and standards-based clients," said Sebastian Hammer,
Index Data's CEO and CTO. "In a time when libraries everywhere are
seeking to implement single search interfaces to multiple resources for
their patrons, it is in the interests of content providers to make it
an easy matter for metasearch software vendors to access their content,
and ease of access is best achieved through standards such as Z39.50
Eric Calaluca, President and CEO of Paratext, adds, "As an
electronic content provider we want to serve libraries in the best way
possible. Investing to conform to information retrieval standards will
make it easier and cheaper for libraries to access our services.
Implementing a standard gateway to our products is a win-win for
content providers, libraries and metasearch vendors."
Index Data has been employing metasearch technology since the mid
1990's, even before the term was being used. The company's metasearch
service, Keystone Retriever, is among the most advanced in the
industry. Index Data's business model is to distribute its software
under an open source license, which gives its customers greater control
over the software they use, while still preserving the option of
Library software vendors as varied as Ex Libris, Fretwell-Downing,
Geac, ISACSOFT, Koha and Polaris use Index Data's technology in the
software they provide to libraries. And libraries around the world,
including the Library of Congress and numerous national libraries in
Europe, use the company's technology and services to meet demanding and
sophisticated information retrieval challenges.
2005-09-02 Our paper from the IDDI 2005 conference
on Integrating Data Mining, Databases and Information Retrieval
explains how we plan, as a part of the Alvis project, to provide
peer-to-peer search services through a proxy.
YAZ Proxy at IFLA 2005: an introduction to the proxy for Library people
14th-18th August 2005, we exhibited at IFLA 2005, an international
library conference. During that event, we made a very well-recieved
presentation on how the Library of Congress is using YAZ Proxy to
deliver MARCXML records over SRU. You can now read the presentation, and the paper that it was based on (also available in a French translation).
SRU Explained and Championed
Lease Morgan tells it like it is. In a July 2005 post to Web4Lib, Eric
outlined in clear and straightforward prose - along with a few lines of
code - why the Search and Retrieve URL Service (SRU for short) is such
an important standard for librarians to implement. We think his
thoughts are worth passing on, so we have made them available here.
Keystone 1.5.3 released
The web framework Keystone version 1.5.3 has been released today.
It is important to notice that Keystone portals running on pre 1.5.3
versions might need to be updated in the way XSLT handlers are called,
due to dependencies on the new Index Data patched Sablotron libraries.
Please study the
before updating your Keystone installation.
The Online Music Research Library of Denmark, in danish DVM, Det virtuelle Musikbibliotek, is a pilot project for the humanities area under the auspices of DEF
(Danmarks Elektroniske Forskningsbibliotek - Denmark's Electronic
Research Library), and driven by a consortium with participation from
the State and University Library, the Royal Library in Copenhagen,
Odense University Library and the Danish Music Information Centre (MIC).
The new, much extended version of the portal is build by Index Data on the web framework
Keystone version 1.5.3 .
June 21, 2005 - Austin, Texas - The Library of Texas (LoT), an innovative federated searching service and Web portal
created jointly by the Texas State Library and Archives Commission and Index Data, a company specializing in networked
information retrieval services for libraries, is saving time and money for libraries and library users across Texas. New
collaborative efforts with Houston Public Library and the El Paso Area Library Consortium are demonstrating new uses and
new benefits for this program in its second year of operation.
Authenticated patrons of Texas public and academic libraries can search groups of library collections and commercial
databases from a single, easy-to-use Web site at www.libraryoftexas.org. Through this federated searching approach,
people can use a single search to find reliable information from diverse information resources to use in making their
important business, medical, consumer, and other decisions. Some Texas libraries are now expanding the program in
The Houston Public Library is one of the largest library systems in the country. They had been paying up to
$60,000/year for a proprietary and patented service that provided federated searching across multiple commercial
databases. With the end of some grant funding and the perennial tightening of library budgets, the library needed to
find a less costly solution that still met all their requirements. Because Index Data had built the Library of Texas
using open standards and had delivered the software to the Texas State Library and Archives Commission under an Open
Source software license, the Commission was able to configure the LoT service for use by the Houston Public Library at
no cost to the library.
Judith Hiott, Assistant Coordinator in the Office of Materials Selection at Houston Public Library, explained that
"while cost was the primary motivation for our change to Library of Texas, we also wanted to advocate for standards and
support our State Library. LoT is completely standards based in its approach to federated searching and that was an
important factor in our decision."
In the near future Houston Public and other Texas libraries will be able to add their local database subscriptions to
the set of state-wide subscriptions available through the Library of Texas, significantly expanding its value to the
large public and academic libraries that subscribe to many additional databases.
The El Paso Area Library Consortium includes a major university, a community college, a large public library, a small
public library and three independent school districts. These libraries had looked for a way to load their collection
databases into a single "union catalog" to make it easier for library users to discover all of the information resources
of the community. Unfortunately the cost of building and maintaining such a union catalog was prohibitive. Once again,
the Library of Texas project was able to provide a comparable service at no charge. Using the existing LoT software and
the ANSI/NISO Z39.50 standard, the State Library was able to provide a Web-based virtual union catalog encompassing most
of these library collections immediately. The smaller remaining collections will be added as soon as their local library
automation systems implement a Z39.50 server.
You can use the Library of Texas site at LibraryofTexas.org or read more about the project at the Texas State Library
and Archives Commission site at www.tsl.state.tx.us. For more information about Keystone, the underlying software, visit
the developer's site at www.indexdata.com.
2005-03-14 YAZ 2.1.0 and YAZ Proxy
1.0. The latest version of YAZ/YAZ Proxy has facilities for OPAC XML
handling. The YAZ server library has new features for SRW/SRU
adaptation and multi-homed servers.
Zebra 1.3.24 is available. This version fixes a few bugs found in the previous version.
Embedding the Zebra -- new Whitepaper
Zebra server is not only a powerful Z39.50 server. It is also a
high-performance indexing and retrieval engine for structured documents
which can be embedded into a number of different applications.
To provide some inspiration, and to whet your appetite for
further research, we've put up a whitepaper describing some of the
possibilities. You can Download it from here.
We do hope you'll find it useful, and we encourage you to get in touch if you'd like
more information about the possibilities.
Library of Congress begins use of Keystone and Zebra, and extends use of YAZ Proxy
September 8, 2004 - Middletown, CT - The Library of Congress signed a
second contract with Index Data last month for development and
consulting services in conjunction with the use of three Index Data
Open Source library application software packages: Keystone Digital
Library Suite, YAZ Proxy and Zebra.
LC will use the Keystone Digital Library Suite (Keystone Organizer and
Keystone Retriever) in a pilot project designed to provide broader
access to LC resources that are currently only available to the public
through separate HTML interfaces. The Zebra indexing engine will
complement the use of Keystone by making some files directly
searchable that formerly were not. Index Data developers will also
build customized Z39.50/SRU/SRW search and retrieval gateways for each
type of resource.
The pilot project is designed to provide federated search services
(also called metasearch services) with easy and rapid access to
resources such as LCs American Memory collections. The custom-built
gateways will translate standard Z39.50/SRU/SRW queries that all
federated search systems support into non-Z39.50 search protocols used
by various software tools (e.g. inQuery/Aurora, Lucene and MySQL).
LC selected Index Data for this project because of the companys proven
expertise in building Z39.50 gateways to structured databases, and
because of the companys commitment to an Open Source distribution.
As part of the contract, Index Data will also enhance the Librarys YAZ
Proxy front-end to its Voyager Integrated Library System databases to
support Z39.50/SRU/SRW retrieval of records with holdings, which will
then be sent to the searcher in an XML format. The enhancement will
subsequently be incorporated into the YAZ Proxy OSS distribution
available for free download (and optional installation and support
contract) from the Index Data web site.
University of Toronto Libraries is now enabling access to its catalog
via SRW services using YAZ Proxy. The University is using YAZ Proxy to
enable users of Microsoft applications to search its catalogue from
within those applications by using Microsoft's ResearchPane to
formulate a query, and then using other software to translate that
query into the SRW protocol, which is then sent on to the YAZ Proxy.
Darnell, the Applications Programmer/Analyst who is coordinating the
YAZ Proxy installation, explained, "We would prefer it if Microsoft
supported SRW directly. The next best thing would be for YAZ Proxy to
support ResearchPane directly." Darnell is currently discussing with
Index Data the best approach to using YAZ Proxy to support access to
library catalogs via web services.
Search/Retrieve Web Service) Protocol was developed by the Z39.50
Implementor's Group to enable non-library resources that do not speak
MARC or Z39.50 to directly access library catalogs via a standard
XML-based protocol that uses Z39.50 semantics. YAZ Proxy, the most
advanced Z39.50 server software available to libraries, fully supports
For more information on YAZ Proxy, visit www.indexdata.com/yazproxy.
those who missed our booth at ALA, here's a treat: You can now download
and print our brochure right from the website, just go here.
Please share it with anyone in your organization who might be interested.
2004-06-25For the first time, Index Data is represented with a booth at the ALA Exhibition, in Orlando, FL. Stop by and visit us at booth #2987 during June 26-29. There's free software!!
SimpleServer with threads
2004-06-14SimpleServer now runs with threads. This means that you can run SimpleServer on Windows. Read more about how in the documentation.
YAZ proxy and YAZ++ available
collaboration with the Library of Congress, we have extended and
improved our YAZ proxy (part of the YAZ++ package) to help the Library
deal with the significant load experienced by their Z39.50 server.
Arguably the most popular Z39.50-enabled resource in the world, the
server has been struggling under the load of several operations every
second. The YAZ proxy enables the Library to implement load sharing
among multiple backend servers, and improves diagnostics and logging
capabilities. In addition to addressing performance issues, the proxy
also supports entirely new functionality, such as on-the-fly mapping of
records to XML and access via the ZiNG protocols.
See post on the YAZ mailing list here: http://www.indexdata.dk/pipermail/yazlist/2004-March/000857.html).
We expect to release an updated version of the proxy, including
functionality developed for the Library within a few days, under an
Open Source license. We will be offering installation assistance and
full support for the package for libraries that wish to make use of
this new functionality.
Version 1.1 of the SRW/SRU protocols have now been released. From the website:
SRW is "an XML-based protocol designed to be a
low-barrier-to-entry solution for searching and other information
retrieval operations across the internet. It uses existing, well
tested, and easily available technologies, such as URI, XML, SOAP,
HTTP, and XPath. All SRW records are transfered in XML. Record schemas
used in SRW include Dublin Core, Onix, MODS, and MarcXml. Support for
Dublin Core is strongly encouraged; other schemas can be defined
locally. The protocol may be carried via SOAP or as parameters in a
URL. When carried via SOAP, it is referred to as SRW; via URL, as SRU.
More information at http://lcweb.loc.gov/z3950/agency/zing/srw/
Index Data staff has been heavily involved in the work on the
SRW/SRU standard, and the YAZ toolkit is the first protocol toolkit
available to support both Z39.50, SRW, and SRU in a single package.
More info at http://www.indexdata.dk/yaz/.
Additional coverage of the release at http://xml.coverpages.org/ni2004-03-04-a.html
2, 2004 - Copenhagen, London and Middletown, CT - Index Data has
initiated two new product lines: Keystone™, a suite of federated search
products and services; and YakPac™, a multi-lingual catalog client for
young library users.
Keystone will consist of open source software based on international
standards and communication protocols, as well as support and
development services to augment that software. "We believe the time is
right for Index Data to launch a major open source software development
effort to bring sophisticated digital library meta-search services
within reach of all the world's libraries, whatever the level of their
resources," says Sebastian Hammer, co-founder of the company. "Our goal
is to develop and sustain the highest quality information discovery
software that, by virtue of its being open source, becomes part of the
shared intellectual capital of the world-wide library community. We are
now actively seeking partners among both libraries and vendors to
achieve that goal."
YakPac is a multi-lingual extension of a youth-oriented catalog
client/Z39.50 gateway that Index Data developed for the Danish public
library market. It uses Macromedia Flash and XML technologies to
achieve fun and functionality that go well beyond any other
youth-oriented library catalog interface currently available. The
company plans a world-wide general release of YakPac at the ALA 2004
Annual Conference in Orlando, Florida.
About Index Data
Index Data has grown from small beginnings in Copenhagen a decade
ago into one of the world's most respected software houses in the area
of networked information retrieval. With offices in Copenhagen, London
and Middletown, CT USA, Index Data works with partners throughout
Europe and the US to meet the networked searching needs of library
users everywhere. All Index Data software products support
multi-lingual functionality through UNICODE implementation. To the
greatest extent possible, Index Data software products are based on
open source, open standards, and modular architecture. The company's
free and open source products, as well as its development and support
services, are used by the Library of Congress, the State of Texas,
major library system vendors, and numerous academic libraries around
the world. For additional information, visit our Web Site at
http://www.indexdata.com or call David Dorman, US Marketing Manager, at
version 1.4.4 is available. This version fixes a bug with
configure/makefile where Tcl was not enabled even though configure
XML web framework TKL version 1.4.4 released
2003-12-03We have recently released a new version of our popularXML web development framework TKL. Some of the improvements made in this release are
- OAI serveren is now adhering to the OAI standard
- Better install and adminstration scripts for Debian/GNU Linux
- Many small bug fixes
- Better statistics and URL check functionality
Index Data was profiled by the online publication Open Magazine. They
highlight our commitment to standards and some of our latest projects.
The story can be found at
new development version of the popular 'Harvest' web spider has been
released, now with improved integration of Index Data's Zebra
information server. The use of Zebra offers improved performance in
updating and searching, as well as support for open standards for
You can read more about the Harvest project here:
2003-11-05We have redesigned our web site and it is now powered by Keystone Organizer, our own XML-based website/content management system. Please contact us if you have any trouble with our new site.Welcome!
ID Sponsors NISO OpenURL/Metasearch Workshop
On October 29-30, NISO
held two one-day conferences on the OpenURL standard and on issues
related to Metasearching. Index Data, a voting NISO member,
co-sponsored the event, and presented some of our latest projects in
the area of cross-searching portals. Standards related to information
retrieval are of critical importance because they lower the cost of
integrating information resources, and ultimately they improve the
quality and utility of applications, by allowing developers to focus on
functionality and usability.
More information at http://www.niso.org/news/events_workshops/MS-2003_workshop.html
2003-09-29 ZAP! version 1.4.2 is available. ZAP! now supports both Apache 2 and Apache 1.3. This version supports sorting on the client-side.