5. The Backend Functions

For each service of the protocol, the backend interface declares one or two functions. You are required to provide implementations of the functions representing the services that you wish to implement.

5.1. Init

bend_initresult (*bend_init)(bend_initrequest *r);
    

This handler is called once for each new connection request, after a new process/thread has been created, and an Initialize Request has been received from the client. The pointer to the bend_init handler is passed in the call to statserv_start.

This handler is also called when operating in SRU mode - when a connection has been made (even though SRU does not offer this service).

Unlike previous versions of YAZ, the bend_init also serves as a handler that defines the Z39.50 services that the backend intends to support. Pointers to all service handlers, including search - and fetch must be specified here in this handler.

The request - and result structures are defined as

typedef struct bend_initrequest
{
    /** \brief user/name/password to be read */
    Z_IdAuthentication *auth;
    /** \brief encoding stream (for results) */
    ODR stream;
    /** \brief printing stream */
    ODR print;
    /** \brief decoding stream (use stream for results) */
    ODR decode;
    /** \brief reference ID */
    Z_ReferenceId *referenceId;
    /** \brief peer address of client */
    char *peer_name;

    /** \brief character set and language negotiation

    see include/yaz/z-charneg.h
    */
    Z_CharSetandLanguageNegotiation *charneg_request;

    /** \brief character negotiation response */
    Z_External *charneg_response;

    /** \brief character set (encoding) for query terms

    This is NULL by default. It should be set to the native character
    set that the backend assumes for query terms */
    char *query_charset;

    /** \brief whether query_charset also applies to records

    Is 0 (No) by default. Set to 1 (yes) if records is in the same
    character set as queries. If in doubt, use 0 (No).
    */
    int records_in_same_charset;

    char *implementation_id;
    char *implementation_name;
    char *implementation_version;

    /** \brief Z39.50 sort handler */
    int (*bend_sort)(void *handle, bend_sort_rr *rr);
    /** \brief SRU/Z39.50 search handler */
    int (*bend_search)(void *handle, bend_search_rr *rr);
    /** \brief SRU/Z39.50 fetch handler */
    int (*bend_fetch)(void *handle, bend_fetch_rr *rr);
    /** \brief SRU/Z39.50 present handler */
    int (*bend_present)(void *handle, bend_present_rr *rr);
    /** \brief Z39.50 extended services handler */
    int (*bend_esrequest) (void *handle, bend_esrequest_rr *rr);
    /** \brief Z39.50 delete result set handler */
    int (*bend_delete)(void *handle, bend_delete_rr *rr);
    /** \brief Z39.50 scan handler */
    int (*bend_scan)(void *handle, bend_scan_rr *rr);
    /** \brief Z39.50 segment facility handler */
    int (*bend_segment)(void *handle, bend_segment_rr *rr);
    /** \brief SRU explain handler */
    int (*bend_explain)(void *handle, bend_explain_rr *rr);
    /** \brief SRU scan handler */
    int (*bend_srw_scan)(void *handle, bend_scan_rr *rr);
    /** \brief SRU record update handler */
    int (*bend_srw_update)(void *handle, bend_update_rr *rr);

    /** \brief whether named result sets are supported (0=disable, 1=enable) */
    int named_result_sets;
} bend_initrequest;

typedef struct bend_initresult
{
    int errcode;               /* 0==OK */
    char *errstring;           /* system error string or NULL */
    void *handle;              /* private handle to the backend module */
} bend_initresult;
    

In general, the server frontend expects that the bend_*result pointer that you return is valid at least until the next call to a bend_* function. This applies to all of the functions described herein. The parameter structure passed to you in the call belongs to the server frontend, and you should not make assumptions about its contents after the current function call has completed. In other words, if you want to retain any of the contents of a request structure, you should copy them.

The errcode should be zero if the initialization of the backend went well. Any other value will be interpreted as an error. The errstring isn't used in the current version, but one option would be to stick it in the initResponse as a VisibleString. The handle is the most important parameter. It should be set to some value that uniquely identifies the current session to the backend implementation. It is used by the frontend server in any future calls to a backend function. The typical use is to set it to point to a dynamically allocated state structure that is private to your backend module.

The auth member holds the authentication information part of the Z39.50 Initialize Request. Interpret this if your server requires authentication.

The members peer_name, implementation_id, implementation_name and implementation_version holds DNS of client, ID of implementor, name of client (Z39.50) implementation - and version.

The bend_ - members are set to NULL when bend_init is called. Modify the pointers by setting them to point to backend functions.

5.2. Search and Retrieve

We now describe the handlers that are required to support search - and retrieve. You must support two functions - one for search - and one for fetch (retrieval of one record). If desirable you can provide a third handler which is called when a present request is received which allows you to optimize retrieval of multiple-records.

int (*bend_search) (void *handle, bend_search_rr *rr);

typedef struct {
    char *setname;             /* name to give to this set */
    int replace_set;           /* replace set, if it already exists */
    int num_bases;             /* number of databases in list */
    char **basenames;          /* databases to search */
    Z_ReferenceId *referenceId;/* reference ID */
    Z_Query *query;            /* query structure */
    ODR stream;                /* encode stream */
    ODR decode;                /* decode stream */
    ODR print;                 /* print stream */

    bend_request request;
    bend_association association;
    int *fd;
    int hits;                  /* number of hits */
    int errcode;               /* 0==OK */
    char *errstring;           /* system error string or NULL */
    Z_OtherInformation *search_info; /* additional search info */
    char *srw_sortKeys;        /* holds SRU/SRW sortKeys info */
    char *srw_setname;         /* holds SRU/SRW generated resultsetID */
    int *srw_setnameIdleTime;  /* holds SRU/SRW life-time */
    int estimated_hit_count;   /* if hit count is estimated */
    int partial_resultset;     /* if result set is partial */
} bend_search_rr;
    

The bend_search handler is a fairly close approximation of a protocol Z39.50 Search Request - and Response PDUs. The setname is the resultSetName from the protocol. You are required to establish a mapping between the set name and whatever your backend database likes to use. Similarly, the replace_set is a boolean value corresponding to the resultSetIndicator field in the protocol. num_bases/basenames is a length of/array of character pointers to the database names provided by the client. The query is the full query structure as defined in the protocol ASN.1 specification. It can be either of the possible query types, and it's up to you to determine if you can handle the provided query type. Rather than reproduce the C interface here, we'll refer you to the structure definitions in the file include/yaz/z-core.h. If you want to look at the attributeSetId OID of the RPN query, you can either match it against your own internal tables, or you can use the OID tools.

The structure contains a number of hits, and an errcode/errstring pair. If an error occurs during the search, or if you're unhappy with the request, you should set the errcode to a value from the BIB-1 diagnostic set. The value will then be returned to the user in a nonsurrogate diagnostic record in the response. The errstring, if provided, will go in the addinfo field. Look at the protocol definition for the defined error codes, and the suggested uses of the addinfo field.

The bend_search handler is also called when the frontend server receives a SRU SearchRetrieveRequest. For SRU, a CQL query is usually provided by the client. The CQL query is available as part of Z_Query structure (note that CQL is now part of Z39.50 via an external). To support CQL in existing implementations that only do Type-1, we refer to the CQL-to-PQF tool described here.

To maintain backwards compatibility, the frontend server of yaz always assume that error codes are BIB-1 diagnostics. For SRU operation, a Bib-1 diagnostic code is mapped to SRU diagnostic.

int (*bend_fetch) (void *handle, bend_fetch_rr *rr);

typedef struct bend_fetch_rr {
    char *setname;             /* set name */
    int number;                /* record number */
    Z_ReferenceId *referenceId;/* reference ID */
    Odr_oid *request_format;   /* format, transfer syntax (OID) */
    Z_RecordComposition *comp; /* Formatting instructions */
    ODR stream;                /* encoding stream - memory source if req */
    ODR print;                 /* printing stream */

    char *basename;            /* name of database that provided record */
    int len;                   /* length of record or -1 if structured */
    char *record;              /* record */
    int last_in_set;           /* is it?  */
    Odr_oid *output_format;    /* response format/syntax (OID) */
    int errcode;               /* 0==success */
    char *errstring;           /* system error string or NULL */
    int surrogate_flag;        /* surrogate diagnostic */
    char *schema;              /* string record schema input/output */
} bend_fetch_rr;
    

The frontend server calls the bend_fetch handler when it needs database records to fulfill a Z39.50 Search Request, a Z39.50 Present Request or a SRU SearchRetrieveRequest. The setname is simply the name of the result set that holds the reference to the desired record. The number is the offset into the set (with 1 being the first record in the set). The format field is the record format requested by the client (See Section 2, “Object Identifiers”). A value of NULL for format indicates that the client did not request a specific format. The stream argument is an ODR stream which should be used for allocating space for structured data records. The stream will be reset when all records have been assembled, and the response package has been transmitted. For unstructured data, the backend is responsible for maintaining a static or dynamic buffer for the record between calls.

If a SRU SearchRetrieveRequest is received by the frontend server, the referenceId is NULL and the format (transfer syntax) is the OID for XML. The schema for SRU is stored in both the Z_RecordComposition structure and schema (simple string).

In the structure, the basename is the name of the database that holds the record. len is the length of the record returned, in bytes, and record is a pointer to the record. last_in_set should be nonzero only if the record returned is the last one in the given result set. errcode and errstring, if given, will be interpreted as a global error pertaining to the set, and will be returned in a non-surrogate-diagnostic. If you wish to return the error as a surrogate-diagnostic (local error) you can do this by setting surrogate_flag to 1 also.

If the len field has the value -1, then record is assumed to point to a constructed data type. The format field will be used to determine which encoder should be used to serialize the data.

Note

If your backend generates structured records, it should use odr_malloc() on the provided stream for allocating data: This allows the frontend server to keep track of the record sizes.

The format field is mapped to an object identifier in the direct reference of the resulting EXTERNAL representation of the record.

Note

The current version of YAZ only supports the direct reference mode.

int (*bend_present) (void *handle, bend_present_rr *rr);

typedef struct {
    char *setname;             /* set name */
    int start;
    int number;                /* record number */
    Odr_oid *format;           /* format, transfer syntax (OID) */
    Z_ReferenceId *referenceId;/* reference ID */
    Z_RecordComposition *comp; /* Formatting instructions */
    ODR stream;                /* encoding stream - memory source if required */
    ODR print;                 /* printing stream */
    bend_request request;
    bend_association association;

    int hits;                  /* number of hits */
    int errcode;               /* 0==OK */
    char *errstring;           /* system error string or NULL */
} bend_present_rr;
    

The bend_present handler is called when the server receives a Z39.50 Present Request. The setname, start and number is the name of the result set - start position - and number of records to be retrieved respectively. format and comp is the preferred transfer syntax and element specifications of the present request.

Note that this is handler serves as a supplement for bend_fetch and need not to be defined in order to support search - and retrieve.

5.3. Delete

For back-ends that supports delete of a result set, only one handler must be defined.

int (*bend_delete)(void *handle, bend_delete_rr *rr);

typedef struct bend_delete_rr {
    int function;
    int num_setnames;
    char **setnames;
    Z_ReferenceId *referenceId;
    int delete_status;      /* status for the whole operation */
    int *statuses;          /* status each set - indexed as setnames */
    ODR stream;
    ODR print;
} bend_delete_rr;
    

Note

The delete set function definition is rather primitive, mostly because we have had no practical need for it as of yet. If someone wants to provide a full delete service, we'd be happy to add the extra parameters that are required. Are there clients out there that will actually delete sets they no longer need?

5.4. Scan

For servers that wish to offer the scan service one handler must be defined.

int (*bend_scan)(void *handle, bend_scan_rr *rr);

typedef enum {
    BEND_SCAN_SUCCESS,  /* ok */
    BEND_SCAN_PARTIAL   /* not all entries could be found */
} bend_scan_status;

typedef struct bend_scan_rr {
    int num_bases;      /* number of elements in databaselist */
    char **basenames;   /* databases to search */
    Odr_oid *attributeset;
    Z_ReferenceId *referenceId; /* reference ID */
    Z_AttributesPlusTerm *term;
    ODR stream;         /* encoding stream - memory source if required */
    ODR print;          /* printing stream */

    int *step_size;     /* step size */
    int term_position;  /* desired index of term in result list/returned */
    int num_entries;    /* number of entries requested/returned */

    /* scan term entries. The called handler does not have
       to allocate this. Size of entries is num_entries (see above) */
    struct scan_entry *entries;
    bend_scan_status status;
    int errcode;
    char *errstring;
    char *scanClause;   /* CQL scan clause */
    char *setname;      /* Scan in result set (NULL if omitted) */
} bend_scan_rr;
    

This backend server handles both Z39.50 scan and SRU scan. In order for a handler to distinguish between SRU (CQL) scan Z39.50 Scan, it must check for a non-NULL value of scanClause.

Note

If designed today, it would be a choice using a union or similar, but that would break binary compatibility with existing servers.