Author: Ross

Fairly recently on the Web4Lib mailing list, a thread started by Jim Campbell (of UVA) and David Walker (of Cal State San Marcos) (and others) prompted me to ask what the role of OPAC is in the modern library.

Outside of “Inventory Control System”, I don’t feel like I got a very good or meaningful response.

I have been thinking a lot about something that Karen Schneider had written a while ago about the need for search interfaces to be search/browse. By this, I mean you begin your session by typing some words in a box and your interface adapts itself contextually to the results and what you should be looking at, so your “browse” options would be logical based on the context of your results.

If your terms were to bring back government documents, say, you would also have the ability to browse our GovDocs research guide or email our GovDocs librarian. If your search brought back a database (for example, ABI/Inform), then the page should also link to the subject guide that includes ABI/Inform (in this case, the Business guide).

This, of course, requires that the “library website” be in a format that makes it potentially servable in this manner. For our site, I have proposed that our content be broken down into small sections (rather than pages) that can be classified and served as necessary.

If you are an undergrad and one of your results happens to be one of your reserves items (which I’ll get to in a minute), there’s not much need to see the faculty policies for placing something on reserve. There is a use, however, in seeing the circulation policies regarding said reserve as appropriate to an undergrad.

If your search results in a journal that we get through an aggregator that sucks (meaning Lexis-Nexis or Factiva or their ilk), present tutorials on getting to the journal through that aggregator (or just a tutorial, in general).

Searches should be weighted contextually, as well. Objects that appear in your reserves lists or subject disciplines should have more relevance than other things. Circulation/clickthroughs should boost relevance (although I realize that non-circulating items present a problem here).

The important thing I want to see is the relationship between objects and content. My search brings back a journal. Besides the obvious information I want to know about thing (esp. things not included, like, what is it about?), tell me what databases index this thing; what other journals are similar; what is the current ToC (if available via RSS); are there preprints from this journal in our institutional repository/ETDs; etc. If there is any library created content related to a particular object, I want that, too.

I want to break down the silos between our resources and content and different collections.

And, yes, I think articles and other database content should be included in that as well (if you have the credentials to view them – if not, an indication of what you’d see if you were logged in).

If we don’t include the entirety of our collections, I am not entirely sure what the purpose of the catalog is.

I notice a lot of the ‘blogs I read regularly have recently had a similar posting to this one:

Wow, it’s been so long since I last posted. Well, it’s time to catch up.

I suspect for many, summer is the busy time of year. This is when you have to pack in all of the important projects before fall semester begins. It’s also conference season and, hopefully, you might be able to sneak in a vacation (oh well, two out of three for me).

Anyway, here’s to getting this blog beast out of hibernation.

Yesterday I posted my truck, and it sold in 20 minutes. 20 minutes.

No wonder the print media is losing classified ad revenue left and right to craigslist.

ed. Never mind about clicking the link… I just had to remove the posting, because despite the fact that the ad said “SOLD” all over it, people were still contacting me about it.

Emory University Libraries has released Reserves Direct(2). I left for Georgia Tech during the redesign process and it far exceeds anything I could have imagined. It shares little to nothing from 0.9 (I was always too self-concious to label it a 1.x release).

One of the bigger changes is that it’s now in PHP5/MySQL. PostgreSQL, for several reasons, was a rather large barrier to adoption for many libraries. The database has been completely redesigned (that was really the only part I participated in), since the old version was built upon a very awkward legacy data model (see the history for more on this). They also consolidated the three interfaces (staff, faculty and student) into one which should clear up much of the confusion faculty members would have when they couldn’t edit their class from the student interface (see the demo). Actually, the new interface is so much more user-friendly, it’s obvious that Maurice York (who did most of the actual interface design) put quite a bit of work into it. Jason White did the bulk of the actual coding, which I’m sure will be an improvement over the mess of code that passed for a reserves system previously. I’ll know more about that soon, since I’m supposed to add the Voyager integration module (it currently only works with SirsiDynix Unicorn). My take is that this job should be a whole hell of a lot easier, thanks to Jason et. al.

One thing I find pretty slick about their site is the promotional video. That’s Andy Ditzler doing the voiceover (and I’m assuming he directed it). It’s seldom you see an OpenSource project with a marketing video, and even less by one sponsored by a library. I mean, who cares if the documentation is spotty if there’s a slick promotional campaign, right?

I don’t mean to sound so much like a shill for Reserves Direct, but it’s really amazing to see something that started by being disappointed at an Eres demo turn into this.

So, last week, before I left for SMUG, Mike Rylander (of Evergreen-ILS), Joshua Ferraro (of Liblime/Koha) and I began talking about OpenSearch interfaces for our respective catalogs. The only reason I was able to really contribute to the conversation was the fact that I had my little python CGI, but I hadn’t thought much about it since I wrote it.

I had largely given up on targeted searching within OpenSearch due to the fact that targeted searching would be wasted on A9. A9 is basically only keyword and phrase searching, so more sophisticated queries would really only produce results in the catalog column. You would remove any advantage of A9’s cross searching plus you’d remove any advantage of the catalog’s native interface. Since I hadn’t seen anyone else implementing OpenSearch, there was no point in pursuing this.

Well, until last week. Mike was talking about implementing OpenSearch in Evergreen and was interested in including results from Koha catalogs. Since I had already created my OpenSearch widget, it also seemed like a natural target. More natural, in fact, since the advantages of including the Georgia academic libraries in a search of Georgia public libraries (and vice versa) seemed so obvious.

This meant that targeted searching needed to work, though. I made the suggestion that queries should assume keyword and respond to CQL if it is supplied. CQL is just so intuitive that it seems silly not to use it, plus the added bonus of not having to manipulate it in any way before I send requests to yazproxy. Always looking for ways to get out of doing work, you see. So with that statement, I packed up and headed to SMUG and its lack of internet access for the rest of the week.

When I got back, not only had Mike created a proof-of-concept search interface, he had also created an extension to OpenSearch to account for relevancy and merged sets. He proposes adding the namespace xmlns:openIll=”http://open-ils.org/xml/openIll/1.0″ and using an <openill:relevance> tag to display relevance. With this, result sets can be merged and sorted by relevance, and any search targets that don’t include this will appear as columns like normal.

There are some issues with this method, the most obvious being “what is relevance?” For example, I have no abilities to sort or get relevancy rankings from Voyager’s Z39.50 server. In order to make Mike’s proof-of-concept work, I had to “fake” a relevancy ranking based on order (which is always reverse chronological by creation date). I take the remainder of 100 minus ((the quotient of 100 divided by the number of results) and multiplied by the (result number minus 1)) (wow, refreshing my arithmetic vocabulary). My point here is, that’s a crappy algorithm. For queries that produce thousands of results, you’ll wind up with upwards of 50 hits with greater than 98% relevancy, despite the fact that the second result may not be relevant at all. Of course, I’d never put this in a production system, but it still goes to prove that relevancy is relative. The other problem is that in a merged search of two or more OpenSearch targets, the results on page three of a given target may be considerably more relevant than, say, the third result of the other target, yet you will still get all of the less relevant results in the pages in between. While, this is possibly a valid argument, it sort of misses the point of OpenSearch which I view merely as a resource exposure tool rather than a robust search protocol (at this stage, anyway).

The good news is that when Art and I have our OPAC mirror set up, we will actually be able to do relevance properly. The even better news is that we don’t even have to have a concept of how the public interface is going to work to get this functionality. We’ll have the data, we’ll have the indexer, we just need to point Z39.50 queries to it and then direct them to the real OPAC (for now).

One thing I want to point out, though, is how cool the Evergreen and Koha results are. Not only are they deduping (we’ll be able to do that later), but they display a sort of brief holdings (x copies available) and provide links on author and subject. I wasn’t even thinking about providing subjects! I will have to see how easy it is to add this sort of functionality to our search results.

I hope to be able to point our OpenSearcher at our DSpace repository soon, too. We’re currently trying to install the OCLC SRW web app and this could go a long way in providing exposure to DSpace from other resources like WAG the Dog or our catalog.

So I’m sitting in the Adele K. Stamp Student Union at the University of Maryland waiting for the shuttle to take me to the Metro and the airport (and out of this hell-furnace they call D.C.) logged into their campus wireless with some username and password that got passed around at the SMUG (SFX/Metalib User’s Group) conference.

All in all, this was a really useful conference (and I don’t throw those words around lightly). One major problem, however, was that there was really no internet access for the people here (to be fair, there was a lab in the library that we could use — but our schedule didn’t allow us to get there much). Even worse, I was staying in the dorm. The dorm rooms have nothing but a bed, really… so it was imperative to avoid the dorm at all costs until I wanted to fall asleep.

But back to the conference: this was my first SMUG, and I really didn’t know what to expect. The morning started off (early!) with the typical rah-rah fest of the vendor announcements (this time through the Ex Libris filter, rather than Endeavor). The last of these, though, was a bit inspirational.

Ex Libris has named Oren Beit-Arie their Chief Strategy Officer and he spoke about upcoming development and goals for the company. The amazing part here is he talked about how Ex Libris was going to focus on interoperability. He added that Ex Libris wants to create products that are modular and can be used with other technologies, whether Ex Libris is the backend or something else is and Ex Libris is used as the front end (or vice versa). Wow! This is exactly what I asked Endeavor to do (they won’t).

Dear University System of Georgia: can we please move to Ex Libris as our primary vendor?
If not, Dear Ex Libris: are you hiring?

Helping put the smug into SMUG, oddly my name was mentioned by two of the speakers before 10AM on the first day (Oren and Roy Tennant). The last time my name called out before 10 AM in a room full of my peers, I had to do 3-5 (thank you folks, I’m here until 3PM!). To be completely honest, though, it was always addressed as “Peter Binkley and Ross Singer”. This is actually the way it should be done, Peter should always come before me. Still… flattering that Oren had any idea who I was. Roy, of course, is on the beer (scotch, wine) list.

Roy was next to talk about the pros and cons of Google Scholar. This was very informative, made even more so by the fact that Anarug Acharya (principal engineer, Google Scholar) arrived in the middle of Roy’s presentation.

Then Carla Lillvik of Harvard spoke of their Metalib installation. I’m sorry, but it flat out sucks. Worse, it has taken 2 years to get to this level of suckery. Metalib–

Next up was a panel on Metalib usability testing. If anyone needs to read Polishing the Turd, it’s these folks. It seems the problems with Metalib’s user interface are so deep (and customization so difficult) that they resort to things like “changing colors of tabs” to help the users. Metalib–

Anurag spoke after lunch and talked about the mechanics and direction of Google Scholar. I feel a little better about the holdings issue now, but I still am a bit uncomfortable about the advantage that gives Google Scholar over other database search engines. Anurag is an interesting fellow. He’s very confident (some may take it as arrogant), intelligent, and while he wants to work with libraries, he doesn’t really want to deal with our b.s., either.

At this point the program changed dramatically and my presentation with Selden was moved up to be next. I hadn’t actually gotten a chance to get to a computer to see if my presentation was working (it’s, of course, a live presentation). After a scary moment without any network connectivity, we were up and running. Frank Cervone introduced us and mentioned my heavy metal hair from high school. As usual, I forgot half of the things I wanted to say while onstage.

It was quite cool that Anurag stayed as long as he could through my presentation.

Afterwards, I spoke to Oren and he got me on the notion of “Context Objects” vs. “Latent OpenURL Autodiscovery”. I will definitely investigate this more, but Oren (and Herbert van de Sompel) are probably right. This is probably a much better approach.

I need to start wrapping this up so I can catch the shuttle… but I have quite a bit more that I want to say… mainly how by day two (thanks to Roy, Mike McKenna and David Walker, not to mention Karen Groves) I am very high on Metalib again, but only through the X-Server. I got to spend a lot of time with David Walker (in fact, he was inducted into the beer list) and I’m really glad I did. He’s doing some amazing stuff at Cal State San Marcos and we can all learn a lot from him.

So, Metalib++?

I also have several orders for WAGgers. I am thinking of maybe moving it to being a hosted service.

Also, thanks to Katie Gohn for supplying the entertainment at our table at the Dinner/Reception Thursday night.

When people ask me why I “retired” from acting a couple of years ago, I always tell them, “Because I don’t like other actors”.

So why I decided to get back into playing music is a bit beyond me. While actors and theatre people have their constant drama and their singing songs from musicals at cast parties, in many ways it pales in comparison to the juvenile society and culture that exists around rock and/or roll.

This is my first real band since The Province of Avocado (Ashley Proffitt doesn’t count) which was, like, back in 1999 (so last millenium…) so I hadn’t bothered to maintain my equipment. After dealing with a shorted out instrument cable for a month (and a shorted speaker cable for even longer), I finally decided it was time for new cables.

I really hate guitar shops. They are crowded, expensive, staffed by former members of my high school heavy metal band, and one has to hear (repeatedly) some kid (badly) playing either:

• Metallica’s Seek and Destroy
• Metallica’s Master of Puppets

or, if they want to show how sensitive they are

• Metallica’s Welcome Home (Sanitarium)

It almost makes you nostalgic for the poorly played Stairway to Heaven you used to hear.

So, a couple of months ago, with the Benz overheating, I drove across Atlanta to Guitar Center to buy some cables. It was everything I feared and worse. To boot, their cables were really freaking expensive, and since Selena’s reluctant about my spending $35/mo. on rent for our practice space, she would really freak out if I spent $100 on cables.

So I went home and bought all of my cables for somewhere around $30 from Musician’s Friend. This, naturally, put me on their mailing list.

So when my first catalog arrived, I was transported back to high school. Oh sure, some of the companies are different (although not that many) and some of the gadgets are high-techier (although not much), but the vibe is still the same. And for a 32 year old, it is a vibe of shame.

Let me get some things clear before I continue:

1. I went to high school in Chattanooga, TN.
2. I graduated in 1990.
3. When I graduated, I had hair halfway down my back.
4. My ensemble routinely consisted of tight jeans, converse all-stars (or wrestling shoes) and either a Metallica t-shirt or a Queensryche t-shirt.
5. My first bass (a Peavey Foundation) was purple.
6. In my senior year, my band performed for the entire school some Tesla song and Ozzy Osbourne’s Crazy Train.

What I’m trying to say is that I’ve been in the belly of the beast of bad taste and it’s nothing to be proud of and it should not be perpetuated.

So imagine my horror, 15 years later, when I see this. And this. And this (Dave Mustaine?!). How about this one? Even moving over to basses, where I belong, and had hoped (erroneously) was more “dignified”, revealed this. At least there aren’t as many cheesy graphics.

The struggle for the average musician is navigating between the heavy metal death schwag and the jam band “kind” look. My current bass strap would make the unsuspecting bystander think I play in the “Dave Matthews Cover Band”, due to my frustration at trying to find a comfortable, padded bass strap a couple of years ago. I hate it, but I hate skulls and flames and snakes even more.

Dear “rock & roll industry”: Zakk Wylde, Joe Satriani and George Lynch weren’t that all that cool or interesting 15 years ago, please stop trying to sell me their crap now.

Also, I still have my Peavey, but it’s no longer purple and it sits in pieces in my shed.

I drove to work today. Normally, I take the train, but for several stupid reasons, I decided to drive.

Atlanta drivers have a disdain for “rules” and “traffic laws” that make me want to scream. They will pass you in an exit-only lane only to hold up that lane when they try to merge back into traffic 3 cars ahead. Four or five cars will plow through the intersection after the light has changed. The turn signal is apparently a sign of weakness.

So, because some fathead wants to get 6 car lengths ahead, we all sit and suffer in some of the worst traffic in the country. By actually following the rules, you:

1. Are forced to sit in the traffic being caused by those that break the rules
2. Are probably more of a liability on the road because if everyone is doing “wrong” things, you are the unpredictable one by being different.

Which brings me to metasearch (and a jarring segue). I am currently not sitting in a meeting in Macon to decide which metasearch product the state is going to go with. And, really, it doesn’t matter that much.

Although I won’t name any names, the candidates were down to two choices:

1. A “traditional” metasearch that uses standards like Z39.50, SRW/U, etc. to search
2. A metasearch that is based on screen scraping

While the research libraries in the state were leaning towards #1, there was something gnawing at us that was hard to deny.

#2 really mopped the floor with #1 as a federated search engine. Not only was it exponentially faster, but it is capable of searching over 95% of our databases (as opposed to #1 which is in the 30-40% range… and does that slowly).

Still, there is other functionality in #1 that still makes it desirable (mostly revolving around workflow and integration into an academic environment, integration with our link resolvers). As a cross database searcher, however, #2 is clearly the winner.

What this brings me to is… How did we get to this point? Why is it actually so much easier and brings better results when we “break the rules”? We have invested a lot of time, thought and energy into creating our standards… how can it possibly be easier to screen scrape results pages rather than use the tools we have created?

I blame libraries first. Securing access via Z39.50, XML gateway, API, etc. has never been a particularly high priority. Metasearch is not only a “systems” issue. It also needs to be looked upon as a collection development issue. If two vendors have “Compendex” and only one of them makes it available through means outside the native web interface, unless that vendor’s native web interface is “the suck” (technical term), they really should be considered the more desirable option. Along with a whole host of other factors, of course. Still, I think non-native access is a very low priority among collection development decisions.

I blame the vendors next. First of all, so many of them don’t even offer some sort of alternative access. Secondly, if they do, it’s an afterthought.

I have been toying with robcaSSon’s federated search project, unhelpfully supplying suggestions when he asks #code4lib for help on particular problems. What Rob has written so far is very cool (but unfinished and therefore not publically available) but it struck me how slowly it searched Academic Search Premier and Business Source Premier (that’s Rob’s “canned” query — those two dbs with the keyword search “hamid karzai and heroin”).

In the native interface, searching across those two dbs is nearly instantaneous… it’s basically just waiting for the browser to render the tables that takes any time. In Rob’s interface, it takes about 5+ seconds to do the same search (and this is with no load on Rob’s end, since it’s not in production… so real world performance would probably be lower). Now, as we learned from Metasearch product #1, this is sadly respectable in the metasearch arena. It’s still bad, though, and I wanted to figure out why it took so much longer.

Using indexdata’s handy yaz-client, I fired up a Z39.50 session to EBSCO’s Z39.50 server to investigate. Searching “hamid karzai and heroin” took a little over 4 seconds. Hmm. 4 seconds?! So I did a search for “female genital mutilation”. 0.2 seconds. Hmm. I did the original search again. 0.05 seconds. Wow. I exited out of yaz-client and then reopened the connection and did it all again. Basically the same thing.

So, apparently it’s the first search in a session that’s a problem. And that sucks. Inherently, every search in a metasearch is the first search in the session. Certainly some connections can be cached, but this definitely raises the complexity of the application and, no matter what, not everything can be cached all the time.

Now, yazproxy would be perfect for dealing with this. It could maintain the session information and at the same time transform the output to xml. Everybody wins! Well, except I can’t get it to work. I guess that’s a bit of a hindrance…

So, again, by trying to do right and follow the standards our community has set, we are left behind the sloppy, inexact searching of a screen scraping method. Ultimately, we all lose, though, because screen scraping can only go so far. The richness of services we can layer upon a screen scaper has far less depth than that of a structured search.

And laying on the horn doesn’t really help…

I’ve finally managed to rewrite quite a bit of the WAGger to run in Greasemonkey, a sort of crazy swiss-army knife for Firefox. My only experience with Greasemonkey up until now was modifying Peter Binkley’s Firefox extension to work with GM (which required… nothing on my part, really). My hesitation mainly revolved around my historical frustrations surrounding JavaScript and the amount of work I’d potentially have to put into something that works in a browser that nobody uses.

Now I’m pretty happy I invested some time.

First of all, I have the WAGger persistence that I really could never get from the PHP Localizer. Instead of requiring the user to know when to use the bookmarklet, WAG the Monkey knows when to wag for the user. Of course, this assumes it’s always on, which is less than ideal.

Another advantage is that it’s fast. Real fast. Like so much faster. Like really.

The best part is that I think I can use most of it for a better and faster bookmarklet style WAGger, as well.

Thank you, Greasemonkey, for making me get off my ass and learn JavaScript in ways that I should have a long time ago.

If you want to try it (for a Georgia Tech experience), install Greasemonkey, then this user script.

Now that I’ve overcome my fear of JavaScript, maybe I’ll take a look at Piggy Bank. RDF has always given me the willies…

Today, I attended GUGM 2005. GUGM stands for “GIL User’s Group Meeting”, and, unless you happen to be from the State of Georgia, you probably have no idea what that means. “GIL” stands for “GALILEO Interconnected Libraries” which is fancy name for the consortial deal the University System of Georgia has with Endeavor’s Voyager. All 34 USG schools use Voyager and we have a big, happy union catalog and universal borrowing and the whole works.

So once a year USG and Endeavor throw GUGM, which is a hodge-podge of Voyager semi-related sessions intermingled with Endeavor rah-rah propaganda.

I was in attendance because I was asked to do a presentation on WAG the Dog. I probably wouldn’t have gone otherwise. There were very few automation type sessions and learning about the problems with ILL and cataloging aren’t the best way I can spend my days.

Still, I actually had a pretty good time. I carpooled down (to Macon) with Crit Stuart and we had pretty good conversation – amazing since we left GT at 7:30AM. The keynote speaker was Kris Biesinger on GACollege411. This is a really pretty incredible service for high school students in Georgia to help track their path to college. Bravo! I certainly wish something like that had existed when I was in high school (and I wouldn’t have wasted a year and a half at UTC).

This is not the point of this posting, however.

The first session I attended (by process of elimination) was one of the aforementioned Endeavor rah-rahs given by Dave Richter, VP of Global Sales. It was a mostly dry overview of the “state of the company” and whatnot, but what stuck with me was an air of humility that permeated from his PowerPoint.

We have gained of reputation of being less responsive to our users.

We need to act more like partners with our customers.

We need to become technological innovators again.

Granted, this was coming from the VP of Global Sales, so if anyone should be able to cry crocodile tears it was he, but still, there was something to these admissions of culpitude that one rarely sees from a vendor, especially one associated with Reed-Elsevier.

He went to describe the dilemma of being a library vendor in that one may have to go the “novel” approach of working with another vendor to deliver a particular product. That this sort of “radical” thinking was what you sometimes “had” to do, but what to do if this particular deal would possibly favor a competitor’s product over another you or your parent company offers?

Here is where I had to interject. I tried to hold my tongue as long as I could and I kept telling myself, “Don’t sound confrontational… Don’t sound confrontational…”… but I think I sounded confrontational.

I said something along the lines of:

I understand that you have a business to run and you have to think of the bottom line across all your divisions… but ultimately that hurts us. We can’t expect any one vendor to supply all of our information needs, so therefore we need you to play nice with each other. Further, we need to be able to do anything we want with our data and the data we’re paying for. When we license Voyager from you, we don’t want proprietary features that make WebVoyage do things that we can’t do in any other way. We have ideas of what we can do with our database, and the limitations you set on what we’re able to do with it forces us into little silos of data. And that’s good for no one.

I went on to say some more about “speaking for the library developer community at large” (sorry library developer community at large) that we need vendors to relax a bit and open up their products more to us.

I talk a lot, I realize, but the crux of this is: We have to treat each other as “partners”. We (the libraries) are reliant on the ILS and database vendors to accomplish our tasks as libraries. Vendors are reliant on libraries to even have their freakin’ business. We both need to address the reliance we have on each other and actually work with each other as symbiotic entities.

Also, Memo to library vendors: Don’t worry so much about losing our business to a competitor. There is no reason to be overprotective. An ILS migration is probably the biggest nightmare that faces a library organization, so it would take an incredible act of negligence or bankruptcy to drive us away.

One last note. If you want to regain your place as a technological innovator, perhaps you should start fostering and endorsing a development community among your customer base. It could only serve to make your product and company stronger.