Sound and vision blog

Audio Search results page

OK, as promised, here's part two of the guide to the moving image applications available for testing at the British Library's Growing Knowledge exhibition. In the last post we covered Video Server, which is making BBC television news recordings from May 2010 available with a subtitle search facility. This involves some ingenious work underneath the bonnet, because TV subtitles comes as graphics, not text. The results are - we think - very impressive and other huge potential for integrating video content with other, text-based resources in a digital research environment. But what about video content that doesn't come with subtitles?

Many TV channels do not have a subtitles feature (generally in the UK it is only the terrestrial channels that do), so unless you have access to catalogue information, which can be scanty in any case, then you have to scroll through the video, or audio file, until you get to what you are looking for - if it is there at all. What is needed is software that converts speech to text. And that's what we've got.

MS Audio Search makes use of Microsoft Research Audio Video Indexing System, or MAVIS. This is speech-recognition software which takes an audio file and converts the sounds searchable text, by means of 'Large-vocabulary continuous speech recognition', which essentially means that it is underpinned by a dictionary and a language grammar controller. In practice you type in a search term, and up comes a range of videos with a column on the results page which gives those lines in the speech track where your search term was spoke. Click on any one of these and it takes you to a point roughly five seconds before the word is spoken and plays the video or audio.

For the purposes of Growing Knowledge we have made available around 120 hours of BBC television news programmes dating May-August 2010, plus around 500 hours of audio-only reording from the British Library's collection, covering oral history interviews with Jewish Holocaust survivors, talks at at London's Institute of Contemporary Arts, and interviews with British painters, sculptors, photographers and architects. The software isn't perfect - there's around 70% accuracy rate - and it is essential to cross-check with the actual audio track that you have selected because the 'transcript' alone cannot be trusted as an entirely accurate record of what has been said, which is the same with subtitle or indeed with any form of optical character recognition (OCR). The software is happiest with clearly-spoken English, and struggles a little when there are multiple speakers.

But even so, wow. The potential is huge. Audio or video recordings can become immediately discoverable instead or requiring a period of time to analyse their contents, so long as there is a speech track. Because a dictionary underpins the resource, you can build up a thesaurus and keywords, and develop rich linkages with other text-based resources. Although it is early days for such technology, it will in time open up audiovisual archives to an unimaginable degree.

You can try out MS Audio Search and Video Server at the Growing Knowledge exhibition, which runs until July 2011. We're unable to make the video content available remotely through the Growing Knowledge website, but you can test out the audio files from the British Library and other institutions experimenting with MAVIS at Microsoft's test site. Do have a go, and just imagine what the future for research might be.

It's taken a while, but at long last we've been able to go public with our first digital video service. From today the British Library is showcasing its digital video management system, or Video Server, in the Growing Knowledge exhibition. Growing Knowledge is a small exhibition about the future of research, and "aims to challenge our audiences on how research is changing and ask what they want to experience from the library of the future". In practice this means presenting users with an array of mostly web-based projects which are pointing the way forward for research, with new tools, partnerships, forms of interaction, and outcomes.

Should you visit Growing Knowledge - which runs until 16 July 2011 - you will be confronted by a startlingly white room looking much like the set of a 1970s sci-fi movie (Sleeper? THX 1138? 2001: A Space Odyssey? - you take your choice). There is a browsable 19th century panorama on a surface table and an ingenious 3-D object viewer to distract your attention, but the main business is at four 'pods', which offer different arrangment of screens and and some particularly comfortable chairs at which to test the resources on offer. There are twenty-five of them, most of which you can also test online through the Growing Knowledge website. One that you can test onsite only (owing to rights issues) is Video Server.

What we are offering is BBC television news pogrammes which we have been recording since 6 May 2010 (general election day), which you can access by searching across subtitles. This is rather more difficult than you might think, as the subtitles provided for television programmes are not text-based but graphics (bitmaps) and need to be processed through the video equivalent of optical character recognition to convert them to text. This is what Video Server does, and it makes the news programmes word searchable. Above is the results page for an individual programme (The Andrew Marr Show) showing the subtitles down the left-hand side, with the term searched for highlighted. Click on all line of text and it takes you automatically to that point in the video.

Below the video player and subtitles is another clever bit - the thumbnails. It displays a thumbnail image every five, ten or fifteen seconds, and by clicking on any image it takes you to that exact point in the video. Thus the time-based medium is made word-searchable and visible in its entirety on a single page, so that the researcher can see immediately where to go within the video without having to browse through for thirty minutes or more. The potential for improved searching, and for linking up video content to other resources (especially news-based resources) through word-linkages is huge.

There's also a Workspaces feature, which enables you to select programmes on whatever theme you might wish to investigate and to view them all together. Eventually, when Video Server becomes a public service the workspaces will be private, accessible by personal log-in. For the purposes of the exhibition they are open to all.

Video Server is still a work in progress, and we are keen to receive feedback on its functionality and utility as a research tool. Behind the scenes we are recording more channels that just BBC 1, 2, 4, News and Parliament (which are the ones featured in the exhibition). The next stage for Video Server will be when it becomes a British Library reading room service, which should be by autumn 2011, and where we plan to make available a greater range of content. At present we are recording 13 hours of TV and 6 hours of radio per day - we hope to up this to 25 hours of TV per day quite soon. We're also aiming to add digitised video material from our non-news collections. Meanwhile we continue to test, and to add content - all new BBC programmes that we record (and we are doing so on a daily basis) will be added to the service in the Growing Knowledge exhibiton.

But what about television news programmes that don't have subtitles? What about radio and other forms of speech recording which present a huge challenge for the researcher who needs to find subjects quickly and efficiently? Well, there is another solution on display in Growing Knowledge - Audio Search. More on that in the next post.

The British Library

I’ve already described the British Library’s moving image collection, and I’ll be coming back to particular aspects of the collection in future posts.

But what about our plans for the future?

The collection that we do have is relatively small (if you can call 40,000 titles small) and is very focussed on sound. It makes sense as part of the British Library Sound Archive, which is what it is, but it stands up a little less when viewed as what users of national library need to complement their research across a range of academic disciplines. The BL is keen to give the moving image a higher presence, for the simple reason that we see the future of research (and our business is research) as being ‘media agnostic’; that is, the researchers of the future won’t necessarily care whether something is a book, manuscript, a sound recording or a video – many are just going to want to have the necessary information in one place, in digital form, and easy to use. There must still be cinephiles, bibliophiles and those who cherish a medium for its own sake, but in the digital soup that we are all building it is the subject that is paramount.

Moving images are important for the BL's users, as our Content Strategy identified in 2007. But how are we going to acquire them? The traditional route of establishing a film archive, with all of its accoutrements and issues, isn’t a sensible option. There are many film archives out there already (the UK has around 380 film collections of one sort or another) and we don’t need to add to their number - we would rather work with them. Our intention is to work towards optimum moving image access for our users, underpinned by a core BL capability but working with collections elsewhere, to be integrated with the other kinds of resources that the BL holds.

Our plan has three main elements to it. Element one is to develop the delivery of moving images across particular subject areas. Already, thanks to the work of the Sound Archive, we are strong in areas such as popular music, drama, and oral history. We want to develop such specialisms further, to benefit researchers where we know there is particular need.

Element two is to develop a general moving image service and capability in line with other services that BL offers for texts, still images, maps, etc. By general I mean a service that can offer something for researchers across for many subject areas. To do so we will be establishing a digital video management system (or Video Server, as it’s being called). This will host some of our existing moving image collection, but its main business will be to record television news programmes. This will fulfil a real research need, and with the exciting possibility of integrating such a service with our newspaper holdings in some way.

We will be recording news programmes (television and radio) off-air, using a system called Box of Broadcasts which has been developed by Cambridge Imaging Systems. The intention is to concentrate on 24-hour news channels available through Freeview and Freesat, content which is generally not available anywhere else to researchers post-transmission. The system is being built now, and will be ready to start recording in April. However, it won’t become a public service until late next year at the earliest, not least because we want to have some time to build up a body of content. However, there will be some public tests of the service along the way. Access will be onsite only – we are able to record programmes to deliver access to them at our St Pancras site, but for copyright reasons not on the Web.

There's a lot of video out there, or so it seems

Element three is longer-term, and involves working with other institutions collectively to deliver extensive access to moving image content. You may feel that in an iPlayer and YouTube world that we are awash with video content already, but in truth only a tiny amount is really available for research. Much needs to be done before we are in a position where it is as easy for a researcher to gain access to a film or video recording at any time as it is a book or journal article. We have statutory deposit for the printed word in this country, and that means that – in principle – every book printed in this country now and in the past is available. You may have to go through some hoops to get hold of some titles, but the principle remains.

Moving images are specifically excluded from UK statutory deposit legislation, as are sound recordings. It says a lot about how moving images are viewed in our culture that this is so. There is a form of legal deposit for television, because under the Communications Act the British Film Institute records selected British television programmes from the terrestrial commercial television channels (the BBC archives its own output), but in general there is no guarantee that a film, video or programme that you would like to see for study purposes will be available to you, because no one is able, legally or technically, to keep it all.

But there is much that we can do to improve the situation, by having the major institutions work together. You may have read of the British Library and the BBC signing a memorandum of understanding at the end of 2009. A lot can come out of the signing of such agreements to co-operate.

Here’s a vision of how things could work in the future. You, the researcher, are interested in Hamlet. You want to read the play. You want to look up the critical literature. You would like to listen to interviews undertaken with actors who have taken on the role. You would like to see images of set designs or costumes. You would like to see films based on the play. You would like all of this to appear through one search facility and the results to be delivered to you on one screen. Some of the content would come from the British Library. Some of it would come from other places, identified as belonging to them, but available in this same digital place. You might then want it all on the Web or your mobile as well, and that will be harder to achieve – but on site, we could do it.

This isn’t about making research easy (good research won’t ever be easy). It’s about changing the nature of research, and about encouraging new discovery. If you find the video or the sound recording alongside the text, that leads to new forms of discovery, new conclusions, new research outputs. It’s a goal worth working towards.