Building a simpler HTTP-to-Z39.50 gateway using Ruby-ZOOM and Thin

Inspired by Jakub’s posting yesterday, I wondered how easy it would be to build an HTTP-to-Z39.50 gateway similar to his in Ruby, my language of the moment. Different languages offer different tools and different ways of doing things, and it’s always instructive to compare.

Ruby libraries are generally distributed in the form of “gems”: packages analogous to the .deb files used by the Debian and Ubuntu GNU/Linux operating systems, so before we start installing the Ruby libraries that we’re going to need, we have to install the rubygems package that does that installation. The gems system is like Debian’s dpkg and apt-get all rolled into one, as it knows how to fetch packages as well as how to install them; and it has something of the BSD “ports” mechanism about it, too, as gem installation can include building, most usually when the gem is not pure Ruby but includes glue-code to an underlying C library – as in the case of Ruby-ZOOM.

While we’re installing the gems system, we may as well also ensure that we have Ruby itself installed, plus the header-files needed for building gems that include a C component, and Ruby’s OpenSSL and FastCGI support. On my Ubuntu 9.04 system, I used:

$ sudo apt-get install ruby1.8 ruby1.8-dev rubygems1.8 libopenssl-ruby1.8 libfcgi-dev

Once that’s been done, we can install the Ruby gems that we need for the Web-to-Z39.50 gateway. These are:

  • rack is the web-server API, which works with various different specific web servers, invoking application-specific code to handle requests.
  • thin is one of several specific Ruby-based web servers that we could use – it’s roughly analogous to something like Tomcat.
  • zoom is the Ruby implementation of the ZOOM abstract API .

Depending on how your Ruby installation is set up, you may or may not already have the various prerequisite gems that thin needs; it’s harmless to ask for an already-installed gem to be installed, so you may as well just ask.

So to install the necessary gems, then, I did:

$ sudo gem install rack thin zoom test-spec camping memcache-client mongrel

We’re all done with installing – now it’s time to write the web server. And here it is!

require ‘rubygems’
require ‘thin’
require ‘zoom’

class <span class="caps">ZOOMC</span>lient
  def call(env)
    req = Rack::Request.new(env)
    headers = { ‘Content-Type’ => ‘text/plain’ }
    zurl = req[‘zurl’] or return [ 400, headers, “no zurl specified” ]
    query = req[‘query’] or return [ 400, headers, “no query specified” ]
    syntax = req[‘syntax’] or return [ 400, headers, “no syntax specified” ]
    maxrecs = req[‘maxrecs’] ? Integer(req[‘maxrecs’]) : 10

    res = []
    res << “<span class="caps">SEARCH</span> <span class="caps">PARAMETERS</span>“
    res << “zurl: #{zurl}”
    res << “query: #{query}”
    res << “syntax: #{syntax}”
    res << “maxrecs: #{maxrecs}”
    res << ”

    <span class="caps">ZOOM</span>::Connection.open(zurl) do |conn|
      conn.preferred_record_syntax = syntax
      rset = conn.search(query)
      res << “Showing #{maxrecs} of #{rset.size}” << ”
      [ maxrecs, rset.size ].min.times { |i| res << String(rset[i]) << ” }
    end

    [ 200, headers, res.map { |line| line + “\n” } ]
  end
end

app = Rack::<span class="caps">URLM</span>ap.new(‘/zgate’  => <span class="caps">ZOOMC</span>lient.new)
Thin::Server.new(nil, 12368, app).start! 

This doesn’t need wiring into an existing web-server installation: it is a web-server, listening on port 12368 and ready to accept requests in the /zgate area. Despite its name, thin is not a trivial piece of software: it’s highly secure, stable, fast and extensible. It’s also a lot easier to wire into than some other servers cough Tomcat cough.

The sweet thing about the Rack API is just how simple it is. You provide a class for each web-application, which need have only one method, call: and that method accepts a request object and returns a triple of HTTP status-code, hash of headers, and content. Of course, much, much more sophisticated behaviour is possible, but it’s nice that doing the simple thing is, well, simple.

So, anyway – to run our server just use:

ruby web-to-z.rb

And now we can fetch URLs like http://localhost:12368/zgate/?zurl=z3950.loc.gov:7090/voyager&query=@att...

And we’re done!

4 Comments

Good to see use of the ZOOM gem

Nice post. People may also be interested in the acts_as_zoom plugin for Ruby on Rails. It's a wrapper for the zoom gem and both include ZOOM extended services (i.e. being able to write to your Zebra instance).

We use a modified version of acts_as_zoom for the open source Ruby on Rails app Kete. We're actually looking for collaborators on the acts_as_zoom codebase if anyone is interested. The big thing is to generalize the newer (improved) code that is with the Kete repository and roll it back into the canonical plugin.

Cheers,
Walter

Still learning Ruby, not ready for Rails yet!

Hi, Walter, good to hear from you. At this stage, I have only really begun playing with Ruby, and its use is not at all widespread within Index Data. But when I get up to Rails, I'll be sure to check out Kete and acts_as_zoom.

Re: Still learning Ruby, not ready for Rails yet!

No worries. Just putting that out there. I should note that we haven't migrated any of the plugins, including acts_as_zoom, to Rails 3 (still in Beta) yet.

We are planning to (so if you see this a year from now chances are good it they have), but we are letting others be the pioneers on this one and we'll hopefully have an easier migration as a result. In the meantime, if anyone wants to play with Kete or the plugins, use the latest in the Rails 2.3.x series (or feel free to contribute the migration code for the plugins).

I raise this because Rails 3 has nice facilities for running a Rack based app "within" a larger Rails app. So you could take your Rack app for returning Z39.40 requests above and run it from the same host URL by "dropping it into" the Rails app.

This actually also exists in a feature called "Metal" in the 2.3.x series, but in a slightly different manner.

Anyway, just points of information. Good to have a new member to the Ruby Z39.50 community. Fun to follow your progress.