Since being digitised, this collection of digitised books has been used for many different projects. This includes recent work to try and augment this dataset with genre metadata and a project using machine learning to tag images extracted from the books. The books have also served as training data for a historic language model.
This blog post will focus on two challenges of working with this dataset: size and documentation, and discuss how we’ve experimented with one potential approach to addressing these challenges.
One of the challenges of working with this collection is its size. The OCR output is over 20GB. This poses some challenges for researchers and other interested users wanting to work with these collections. Projects like Living with Machines are one avenue in which the British Library seeks to develop new methods for working at scale. For an individual researcher, one of the possible barriers to working with a collection like this is the computational resources required to process it.
Recently we have been experimenting with a Python library, datasets, to see if this can help make this collection easier to work with. The datasets library is part of the Hugging Face ecosystem. If you have been following developments in machine learning, you have probably heard of Hugging Face already. If not, Hugging Face is a delightfully named company focusing on developing open-source tools aimed at democratising machine learning.
The datasets library is a tool aiming to make it easier for researchers to share and process large datasets for machine learning efficiently. Whilst this was the library’s original focus, there may also be other uses cases for which the datasets library may help make datasets held by the British Library more accessible.
Some features of the datasets library:
Tools for efficiently processing large datasets
Support for easily sharing datasets via a ‘dataset hub’
Support for documenting datasets hosted on the hub (more on this later).
As a result of these and other features, we have recently worked on adding the British Library books dataset library to the Hugging Face hub. Making the dataset available via the datasets library has now made the dataset more accessible in a few different ways.
Firstly, it is now possible to download the dataset in two lines of Python code:
We can also use the Hugging Face library to process large datasets. For example, we only want to include data with a high OCR confidence score (this partially helps filter out text with many OCR errors):
One of the particularly nice features here is that the library uses memory mapping to store the dataset under the hood. This means that you can process data that is larger than the RAM you have available on your machine. This can make the process of working with large datasets more accessible. We could also use this as a first step in processing data before getting back to more familiar tools like pandas.
In a follow on blog post, we’ll dig into the technical details of datasets in some more detail. Whilst making the technical processing of datasets more accessible is one part of the puzzle, there are also non-technical challenges to making a dataset more usable.
Documenting datasets
One of the challenges of sharing large datasets is documenting the data effectively. Traditionally libraries have mainly focused on describing material at the ‘item level,’ i.e. documenting one dataset at a time. However, there is a difference between documenting one book and 100,000 books. There are no easy answers to this, but libraries could explore one possible avenue by using Datasheets. Timnit Gebru et al. proposed the idea of Datasheets in ‘Datasheets for Datasets’. A datasheet aims to provide a structured format for describing a dataset. This includes questions like how and why it was constructed, what the data consists of, and how it could potentially be used. Crucially, datasheets also encourage a discussion of the bias and limitations of a dataset. Whilst you can identify some of these limitations by working with the data, there is also a crucial amount of information known by curators of the data that might not be obvious to end-users of the data. Datasheets offer one possible way for libraries to begin more systematically commuting this information.
The dataset hub adopts the practice of writing datasheets and encourages users of the hub to write a datasheet for their dataset. For the British library books, we have attempted to write one of these datacards. Whilst it is certainly not perfect, it hopefully begins to outline some of the challenges of this dataset and gives end-users a better sense of how they should approach a dataset.
Every year the British Library host a range of PhD placement scheme projects. If you are interested in applying for one of these, the 2022 opportunities are advertised here. There are currently 15 projects available across Library departments, all starting from June 2022 onwards and ending before March 2023. If you would like to work with born digital collections, you may want to read last week’s Digital Scholarship blog post about two projects on enhanced curation, hybrid archives and emerging formats. However, if you are interested in Caribbean diasporic networks and want to experiment creating network analysis visualisations, then read on to find out more about the “Mapping Caribbean Diasporic Networks through correspondence (2022-ACQ-CDN)” project.
This is an exciting opportunity to be involved with the preliminary stages of a project to map the Caribbean Diasporic Network evident in the ‘Special Correspondence’ files of the Andrew Salkey Archive. This placement will be based in the Contemporary Literary and Creative Archives team at the British Library with support from Digital Scholarship colleagues. The successful candidate will be given access to a selection of correspondence files to create an item level dataset and explore the content of letters from the likes of Edward Kamau Brathwaite, C.L.R. James, and Samuel Selvon.
Photograph of Andrew Salkey, from the Andrew Salkey Archive, Deposit 10310. With kind permission of Jason Salkey.
The main outcome envisaged for this placement is to develop a dataset, using a sample of ten files, linking the data and mapping the correspondent’s names, location they were writing from, and dates of the correspondence in a spreadsheet. The placement student will also learn how to use the Gephi Open Graph Visualisation Platform to create a visual representation of this network, associating individuals with each other and mapping their movement across the world between the 1950s and 1990s.
Gephi is open-source software for visualising and analysing networks, they provide a step-by-step guide to getting started, with the first step to upload a spreadsheet detailing your ‘nodes’ and ‘edges’. To show an example of how Gephi can be used, We've included an example below, which was created by previous British Library research placement student Sarah FitzGerald from the University of Sussex, using data from the Endangered Archives Programme (EAP) to create a Gephi visualisation of all EAP applications received between 2004 and 2017.
Network visualisation of EAP Applications created by Sarah FitzGerald
In this visualisation the size of each country relates to the number of applications it features in, as country of archive, country of applicant, or both. The colours show related groups. Each line shows the direction and frequency of application. The line always travels in a clockwise direction from country of applicant to country of archive, the thicker the line the more applications. Where the country of applicant and country of archive are the same the line becomes a loop. If you want to read more about the other visualisations that Sarah created during her project, please check out these two blog posts:
We hope this new PhD placement will offer the successful candidate the opportunity to develop their specialist knowledge through access to the extensive correspondence series in the Andrew Salkey archive, and to undertake practical research in a curatorial context by improving the accessibility of linked metadata for this collection material. This project is a vital building block in improving the Library’s engagement with this material and exploring the ways it can be accessed by a wider audience.
The British Library is accepting applications for the new round of 2022 PhD Placement opportunities: there are 15 projects available across Library departments, all starting from June 2022 onwards and ending before March 2023. Two of the projects within the Contemporary British Collections department focus on Enhanced Curation as an approach to add to the research value of an archival object or digital publication.
“Developing an enhanced curation framework for contemporary hybrid archives (2022-CB-HAC)” will outline a framework for Enhanced Curation in relation to contemporary hybrid archives. These archival collections are the record of the creative and professional lives of prominent individuals in UK society, containing both paper and digital material. So far we have defined Enhanced Curation as the means by which the research value of these records can be enhanced through the creation, collection, and interrogation of the contextual information which surrounds them.
Luckily, we’re in a privileged position – most of our archive donors are living individuals who can illuminate their creative practice for us in real-time. Similarly, with forensic techniques, we’re capturing more data than ever before when we acquire an archive. The truly live questions are then – how can we use this position to best effect? What can we do with what we’re already collecting? What else should we be collecting? And how can we represent this data in engaging and enlightening new ways for the benefit of everyone, including our researchers and exhibition audiences?
Enhanced Curation, as we see it, is about bringing these dynamic collections to life for as many people as possible. In approaching these questions, the chosen student will engage in a mixture of theoretical and practical work – first outlining the relevant debates and techniques in and around curation, archival science, museology and digital humanities, and then recommending a course of action for one particular hybrid personal archive. This is a collaborative exercise, though, and they will be provided with hands-on training for working with (and getting the most out of) this growing collection area by specialist curatorial staff at the Library.
Floppy disk from the Will Self archive.
“Collecting complex digital publications: Testing an enhanced curation method (2022-CB-EF)” focuses on the Library collection of emerging formats. Emerging formats are defined as born-digital publications whose structure, technical dependencies and highly interactive nature challenge our traditional collection methods. These publications include apps, such as the interactive adventure 80 Days, as well as digital interactive narratives, such as the examples collected in the UK Web Archive Interactive Narratives and New Media Writing Prize collections. Collection and preservation of these digital formats in their entirety might not always be possible: there are many challenges and implications in terms of technical capabilities, software and hardware dependencies, copyright restrictions and long-term solutions that are effective against technical obsolescence.
The collection and creation of contextual information is one approach to fill in the gaps and enhance curation for these digital publications. The placement student will helps us test a collection matrix for contextual information relating to emerging formats, which include – but is not limited to – webpages, interviews, reviews, blog posts and screenshots/screencast of usage of a work. These might be collected using a variety of methods (e.g. web archiving, direct transfer from the author, etc.) as well as created by the student themselves (e.g. interviews with the author, video recordings of usage, etc.) Through this placement, the student will have the opportunity to participate in a network of cultural heritage institutions concerned with the preservation of digital publications while helping develop one of the Library contemporary collections.
Both PhD Placements are offered for 3 months full time, or part-time equivalent. They can be undertaken as hybrid placements (i.e. remotely, with some visits to the British Library building in London, St. Pancras), with the option of a fully remote placement for “Collecting complex digital publications: Testing an enhanced curation method”.
Applications for all 2022/23 PhD Placements close on Friday 25 February 2022, 5pm GMT. The application form and guidelines are available online here. Please address any queries to [email protected].
This post is by Giulia Carla Rossi, Curator of Digital Publications on twitter as @giugimonogatari and Callum McKean, Digital Lead Curator, Contemporary Archives and Manuscripts.
Congratulations to the 2021 New Media Writing Prize (NMWP) winners, which were announced at a Bournemouth University online event recently: Joannes Truyens and collaborators (Main Prize), Melody MOU (Student Award) and Daria Donina (FIPP Journalism Award 2021). The main prize winner ‘Neurocracy’ is an experimental dystopian narrative that takes place over 10 episodes, through Omnipedia, an imagined future version of Wikipedia in 2049. So this seemed like a very apt jumping off point for today’s blog post, which discusses a recent project where we added NMWP data to Wikidata.
Note: If you wish to read ‘Neurocracy’ and are prompted for a username and password, use NewMediaWritingPrize1 password N3wMediaWritingPrize!. You can learn more about the work in this article and listen to an interview with the author in this podcast episode.
Working With Wikidata
Dr Martin Poulter describes learning how to work with Wikidata as being like learning a language. When I first heard this description, I didn’t understand: how could something so reliant on raw data be anything like the intricacies of language learning?
It turns out, Martin was completely correct.
Imagine a stack of data as slips of paper. Each slip has an individual piece of data on it: an author’s name, a publication date, a format, a title. How do you start to string this data together so that it makes sense?
One of the beautiful things about Wikidata is that it is both machine and human readable. In order for it to work this way, and for us to upload it effectively, thinking about the relationships between these slips of paper is essential.
In 2021, I had an opportunity to see what Martin was talking about when he spoke about language, as I was asked to work with a set of data about NMWP shortlisted and winning works, which the British Library has collected in the UK Web Archive. You can read more about this special collection here and here.
The New Media Writing Prize was founded in 2010 to showcase exciting and inventive stories and poetry that integrate a variety of digital formats, platforms, and media. One of the driving forces in setting up and establishing the prize was Chris Meade, director of if:book uk, a ‘think and do tank’ for exploring digital and collaborative possibilities for writers and readers. He was the lead sponsor of the if:book UK New Media Writing Prize, and the Dot Award, which he created in honour of his mother, Dorothy, and he chaired every NMWP awards evening since 2010. Very sadly Chris passed away on 13th January 2022 and the recent 2021 awards event was dedicated to Chris and his family.
Recognising the significance of the NMWP, in recent years the British Library created the New Media Writing Prize Special Collection as part of its emerging formats work. With 11 years of metadata about a born digital collection, this was an ideal data set for me to work with in order to establish a methodology for working with Wikidata uploads in the Library.
Last year I was fortunate to collaborate with Tegan Pyke, a PhD placement student in the Contemporary British Publications Collections team, supervised by Guilia Carla Rossi, Curator for Digital Publications. Tegan's project examined the digital preservation challenges of complex digital objects, developing and testing a quality assurance process for examining works in the NMWP collection. If you want to read more about this project, a report is available here. For the Wikidata work Tegan and Giulia provided two spreadsheets of data (or slips of paper!), and my aim was to upload linked data that covered the authors, their works, and the award itself - who had been shortlisted, who had won, and when.
Simple, right?
Getting Started
I thought so - until I began to structure my uploads. There were some key questions that needed to be answered about how these relationships would be built, and I needed to start somewhere. Should I upload the authors or the texts first? Should I go through the prize year by year, or be led by other information? And what about texts with multiple authors?
Suddenly it all felt a bit more intimidating!
I was fortunate to attend some Wikidata training run by Wikimedia UK late last year. Martin was our trainer, and one piece of advice he gave us was indispensable: if you’re not sure where to start, literally write it out with pencil and paper. What is the relationship you’re trying to show, in its simplest form? This is where language framing comes in especially useful: thinking about the basic sentence structures I’d learned in high school German became vital.
Image by the author, notes own.
The Numbers Bit
You can see from this image how the framework develops: specific items, like nouns, are given identification numbers when they become a Wikidata item. This is their QID. The relationships between QIDs, sort of like the adjectives and verbs, are defined as properties and have P numbers. So Christine Wilks is now Q108810306, and her relationship to her work, Underbelly, or Q109237591, is defined with P800 which means ‘notable work’.
Q108810306 - P800 - Q109237591
You can upload this relationship using the visual editor on Wikidata, by clicking fields and entering data. If you have a large amount of information (remember those slips of paper!) tools like QuickStatements become very useful. Dominic Kane blogged about his experience of this system during his British Library student placement project in 2021.
The intricacies of language are also very important on Wikidata. The nuance and inference we can draw from specific terms is important. The concept of ‘winning’ an award became a subject of semantic debate: the taxonomy of Wikidata advises that we use ‘award received’ in the case of a literary prize, as it’s less of an active sporting competition than something like a marathon or an athletic event.
Ultimately we upload information to Wikidata so that it can be queried. Querying uses SPAQRL, a language which allows users to draw information and patterns from vast swathes of data. Querying can be complex: to go back to the language analogy, you have to phrase the query in precisely the right way to get the information you want.
One of the lessons I learned during the NMWP uploads was the importance of a unifying property. Users will likely query this data with a view to surveying results and finding patterns. Each author and work, therefore, needed to be linked to the prize and the collection itself (pictured above). By adding this QID to the property P6379 (‘has works in the collection’), we create a web of data that links every shortlisted author over the 11 year time period.
Getting Started
To have a look at some of the NMWP data, here are some queries I prepared earlier. Please note that data from the 2021 competition has not yet been uploaded!
Global warming does not affect only the environment, it affects the entire system we live in. We can’t think of it as detached from gender, social and racial inequalities. Neither as something separated from our cultural heritage. For this reason, when we think about actions we shouldn’t focus only on emissions reductions, but also think about how to preserve our cultural and artistic production and learn how this, with the aid of new technologies, can help us find new ways to shape our future.
Last year, during my spare time, with the help of Marco Magini (writer and environmental policy adviser), Paolo Nelli (writer) and Maddalena Vatti (producer) I started investigating what role digital technologies play in a city like Venice, which is notoriously under the threat of rising waters, and even more so with the increased global warming.
On the 13th of November 2019 an exceptional acqua alta (a high tide) hit the city bringing one of the worst devastation of the last century. Various archives, buildings, commercial activities, homes and cultural venues were damaged. This prompted a question: what can we understand from an event like this? Is the case of Venice an isolated one or is it a cautionary tale for humanity? After all Venice is not the only city which is sinking and where rising tides threaten to unravel the urban fabric. We should not simply mourn the devastation and start to repair the damage, we should consider the event as an opportunity to think about the direct impact of global warming on our cultural heritage and what we can do to reduce it.
While conducting interviews with scholars, experts, professionals and citizens with the aim of producing a podcast, we slowly came to understand the role and potential of digital technologies in the study of the evolution of a city in respect to changing climate and urban conditions, as well as the role these play in its preservation.
Digital preservation, 3D rendering and water sensors
A fantastic example of digital preservation is the one carried out between the 6th and 17th of July 2020 by a team from the Factum Foundation for Digital Technology in Conservation in collaboration with the Cini Foundation, EPFL and Iconem (https://www.factumfoundation.org/pag/1640/recording-the-island-of-san-giorgio-maggiore). They spent twelve days in Venice recording the Island of San Giorgio Maggiore in its entirety. The result was a virtual rendering of the island made using a mix of LID long-range LIDAR scanning to capture the overall shape of the buildings, external and internal views and high resolution photogrammetry to add the surface detail to that. The island was recorded from more than 600 different recording spots, from which a massive 60.000 million-point cloud was generated. The data acquired through photogrammetry is currently being merged with the point-clouds with the aim of creating a 3D model of the whole island.
This massive work enabled researchers to study the sculptures and the inscriptions that are high up on the facade of San Giorgio but also to analyse the way that the plaster covering the walls was being affected by salt and peeling off.
Thanks to these data it is now possible to carry out really detailed recording of the breakdown of a surface and also monitor the speed at which the cobalt coverings are being blown off by the salt, the speed of decay, to really look and create data to discuss how best to preserve the material heritage on the island.
Camera obscura, painting and digital image analysis: what can the past tell us about the present and the future
It is also possible to use paintings and buildings to look at the past to learn our present. In fact, these artifacts can unconsciously record events and phenomena that postdate their own creation, carrying them into the future.
The researcher in atmospheric physics and cultural heritage Dario Camuffo has conducted a scientific analysis of the works of Venetian painters, Canaletto in particular, depicting buildings and compared them with the state of the very same buildings today in an attempt to calculate the impact of land subsidence in Venice.
Canaletto (Venice 1697-Venice 1768) - The Grand Canal looking East from the Carità towards the Bacino
As professor Camuffo has written, “in general paintings provide a qualitative image, but in Venice’s case, a quantitative evaluation of the apparent sea level rise is possible, thanks to accurate paintings by Canaletto and Bellotto, drawn with the aid of the camera obscura. The paintings accurately reproduce all of the details with a high degree of precision, including the algae belt. […] By analysing these paintings, and comparing them with the algae level we see today, we can extend our knowledge of Venice’s submersion, reaching back in time almost as far back as three centuries.”
How many stories and information are buried in the archives? Deep learning image analysis can help to reveal them, we just need to think creatively.
Maps and algorithms, space syntax, literature and architecture
Maps and literature can also reveal more stories about a city than we think.
UCL/Bartlett Institute Professor Sophia Psarra, drawing inspiration from Italo Calvino’s Invisible Cities and Le Corbusier’s discarded project for the Venice Hospital, has studied the urban evolution of Venice computing the distribution and distances between bridges, calli (=tiny alleys), squares and wells over time. The analysis, which is based on the approaches developed within the world of space syntax, has shown that Venice has and still evolves as a system that resembles a highly probabilistic ‘algorithm’.
What seems a chaotic evolution is in fact the result of the interaction between space and social activity. Maps and data analysis can reveal the modularity of a city and the traces of how social activities have interacted and forged the space. These can help see new connections between literary imagination and the evolution of our society but also help us understand how we can imagine a future which is affected by growing uncertainties.
Digital technologies applied to our cultural heritage as these three examples have shown are an aid to study the past and imagine the future. They can help understand how we as a society can evolve, but also how all our cultural productions are sources of incredible information if we know how to look at them. We can measure the impact of global warming on our cultural artifacts and try to imagine a better future.
To know more on the role of Venice as a vantage point from where to look at the growing emergencies surrounding us –– environmental, cultural, social, and technological –– you can listen to the podcast The Fifth Siren (thefifthsiren.com) and join us for a British Library free online event on Monday 8th November with Professor Sophia Psarra and architectural artists Ila Bêka and Louise Lemoine. More info here: https://www.bl.uk/events/venice-tales-of-a-sinking-city.
Posted by Mahendra Mahey, former Manager of British Library Labs or "BL Labs" for short
[estimated reading time of around 15 minutes]
This is is my last day working as manager of BL Labs, and also my final posting on the Digital Scholarship blog. I thought I would take this chance to reflect on my journey of almost 9 years in helping to set up, maintain and enabling BL Labs to become a permanent fixture at the British Library (BL).
BL Labs was the first digital Lab in a national library, anywhere in the world, that gets people to experiment with its cultural heritage digital collections and data. There are now several Gallery, Library, Archive and Museum Labs or 'GLAM Labs' for short around the world, with an active community which I helped build, from 2018.
I am really proud I was there from the beginning to implement the original proposal which was written by several colleagues, but especially Adam Farquhar, former head of Digital Scholarship at the British Library (BL). The project was at first generously funded by the Andrew W. Mellon foundation through four rounds of funding as well as support from the BL. In April 2021, the project became a permanently funded fixture, helped very much by my new manager Maja Maricevic, Head of Higher Education and Science.
The great news is that BL Labs is going to stay after I have left. The position of leading the Lab will soon be advertised. Hopefully, someone will get a chance to work with my helpful and supportive colleague Technical Lead of Labs, Dr Filipe Bento, bright, talented and very hard working Maja and other great colleagues in Digital Research and wider at the BL.
The beginnings, the BL and me!
I met Adam Farquhar and Aly Conteh (Former Head of Digital Research at the BL) in December 2012. They must have liked something about me because I started working on the project in January 2013, though I officially started in March 2013 to launch BL Labs.
I must admit, I had always felt a bit intimidated by the BL. My first visit was in the early 1980s before the St Pancras site was opened (in 1997) as a Psychology student. I remember coming up from Wolverhampton on the train to get a research paper about "Serotonin Pathways in Rats when sleeping" by Lidov, feeling nervous and excited at the same time. It felt like a place for 'really intelligent educated people' and for those who were one for the intellectual elites in society. It also felt for me a bit like it represented the British empire and its troubled history of colonialism, especially some of the collections which made me feel uncomfortable as to why they were there in the first place.
I remember thinking that the BL probably wasn't a place for some like me, a child of Indian Punjabi immigrants from humble beginnings who came to England in the 1960s. Actually, I felt like an imposter and not worthy of being there.
Nearly 9 years later, I can say I learned to respect and even cherish what was inside it, especially the incredible collections, though I also became more confident about expressing stronger views about the decolonisation of some of these. I became very fond of some of the people who work or use it, there are some really good kind-hearted souls at the BL. However, I never completely lost that 'imposter and being an outsider' feeling.
What I remember at that time, going for my interview, was having this thought, what will happen if I got the position and 'What would be the one thing I would try and change?'. It came easily to me, namely that I would try and get more new people through the doors literally or virtually by connecting them to the BL's collections (especially the digital). New people like me, who may have never set foot, or had been motivated to step into the building before. This has been one of the most important reasons for me to get up in the morning and go to work at BL Labs.
So what have been my highlights? Let's have a very quick pass through!
BL Labs Launch and Advisory Board
I launched BL Labs in March 2013, one week after I had started. It was at the launch event organised by my wonderfully supportive and innovative colleague, Digital Curator Stella Wisdom. I distinctly remember in the afternoon session (which I did alone), I had to present my 'ideas' of how I might launch the first BL Labs competition where we would be trying to get pioneering researchers to work with the BL's digital collections.
God it was a tough crowd! They asked pretty difficult questions, questions I myself was asking too which I still didn't know the answer too either.
My first gut feeling overall after the event was, this is going to be hard work. This feeling and reality remained a constant throughout my time at BL Labs.
In early May 2013, we launched the competition, which was a really quick and stressful turnaround as I had only officially started in mid March (one and a half months). I remember worrying as to whether anyone would even enter! All the final entries were pretty much submitted a few minutes before the deadline. I remember being alone that evening on deadline day near to midnight waiting by my laptop, thinking what happens if no one enters, it's going to be disaster and I will lose my job. Luckily that didn't happen, in the end, we received 26 entries.
I am a firm believer that we can help make our own luck, but sometimes luck can be quite random! Perhaps BL Labs had a bit of both!
After that, I never really looked back! BL Labs developed its own kind of pattern and momentum each year:
hunting around the BL for digital collections to make into datasets and make available
helping to make more digital collections openly licensed
having hundreds of conversations with people interested in connecting with the BL's digital collections in the BL and outside
working with some people more intensively to carry out experiments
developing ideas further into prototype projects
telling the world of successes and failures in person, meetings, events and social media
launching a competition and awards in April or May
roadshows before and after with invitations to speak at events around the world
the summer working with competition winners
late October/November the international symposium showcased things from the year
'Nothing interesting happens in the office' - Roadshows, Presentations, Workshops and Symposia!
One of the highlights of BL Labs was to go out to universities and other places to explain what the BL is and what BL Labs does. This ended up with me pretty much seeing the world (North America, Europe, Asia, Australia, and giving virtual talks in South America and Africa).
My greatest challenge in BL Labs was always to get people to truly and passionately 'connect' with the BL's digital collections and data in order to come up with cool ideas of what to actually do with them. What I learned from my very first trip was that telling people what you have is great, they definitely need to know what you have! However, once you do that, the hard work really begins as you often need to guide and inspire many of them, help and support them to use the collections creatively and meaningfully. It was also important to understand the back story of the digital collection and learn about the institutional culture of the BL if people also wanted to work with BL colleagues. For me and the researchers involved, inspirational engagement with digital collections required a lot of intellectual effort and emotional intelligence. Often this means asking the uncomfortable questions about research such as 'Why are we doing this?', 'What is the benefit to society in doing this?', 'Who cares?', 'How can computation help?' and 'Why is it necessary to even use computation?'.
Making those connections between people and data does feel like magic when it really works. It's incredibly exciting, suddenly everyone has goose bumps and is energised. This feeling, I will take away with me, it's the essence of my work at BL Labs!
A full list of over 200 presentations, roadshows, events and 9 annual symposia can be found here.
Competitions, Awards and Projects
Another significant way BL Labs has tried to connect people with data has been through Competitions (tell us what you would like to do, and we will choose an idea and work collaboratively with you on it to make it a reality), Awards (show us what you have already done) and Projects (collaborative working).
At the last count, we have supported and / or highlighted over 450 projects in research, artistic, entrepreneurial, educational, community based, activist and public categories most through competitions, awards and project collaborations.
We also set up awards for British Library Staff which has been a wonderful way to highlight the fantastic work our staff do with digital collections and give them the recognition they deserve. I have noticed over the years that the number of staff who have been working on digital projects has increased significantly. Sometimes this was with the help of BL Labs but often because of the significant Digital Scholarship Training Programme, run by my Digital Curator colleagues in Digital Research for staff to understand that the BL isn't just about physical things but digital items too.
Browse through our project archive to get inspiration of the various projects BL Labs has been involved in or highlighted.
Putting the digital collections 'where the light is' - British Library platforms and others
When I started at BL Labs it was clear that we needed to make a fundamental decision about how we saw digital collections. Quite early on, we decided we should treat collections as data to harness the power of computational tools to work with each collection, especially for research purposes. Each collection should have a unique Digital Object Identifier (DOI) so researchers can cite them in publications. Any new datasets generated from them will also have DOIs, allowing us to understand the ecosystem through DOIs of what happens to data when you get it out there for people to use.
However, BL Labs has not stopped there! We always believed that it's important to put our digital collections where others are likely to discover them (we can't assume that researchers will want to come to BL platforms), 'where the light is' so to speak. We were very open and able to put them on other platforms such as Flickr and Wikimedia Commons, not forgetting that we still needed to do the hard work to connect data to people after they have discovered them, if they needed that support.
Our greatest success by far was placing 1 million largely undescribed images that were digitally snipped from 65,000 digitised public domain books from the 19th Century on Flickr Commons in 2013. The number of images on the platform have grown since then by another 50 to 60 thousand from collections elsewhere in the BL. There has been significant interaction from the public to generate crowdsourced tags to help to make it easier to find the specific images. The number of views we have had have reached over a staggering 2 billion over this time. There have also been an incredible array of projects which have used the images, from artistic use to using machine learning and artificial intelligence to identify them. It's my favourite collection, probably because there are no restrictions in using it.
Read the most popular blog post the BL has ever published by my former BL Labs colleague, the brilliant and inspirational Ben O'Steen, a million first steps and the 'Mechanical Curator' which describes how we told the world why and how we had put 1 million images online for anyone to use freely.
It is wonderful to know that George Oates, the founder of Flickr Commons and still a BL Labs Advisory Board member, has been involved in the creation of the Flickr Foundation which was announced a few days ago! Long live Flickr Commons! We loved it because it also offered a computational way to access the collections, critical for powerful and efficient computational experiments, through its Application Programming Interface (API).
I loved working with artists, its my passion! They are so creative and often not restricted by academic thinking, see the work of Mario Klingemann for example! You can browse through our archives for various artistic projects that used the BL's digital collections, it's inspiring.
I was also involved in the first British Library Fashion Student Competition won by Alanna Hilton, held at the BL which used the BL's Flickr Commons collection as inspiration for the students to design new fashion ranges. It was organised by my colleague Maja Maricevic, the British Fashion Colleges Council and Teatum Jones who were great fun to work with. I am really pleased to say that Maja has gone on from strength to strength working with the fashion industry and continues to run the competition to this day.
I am really proud of helping to create the international GLAM Labs community with over 250 members, established in 2018 and still active today. I affectionately call them the GLAM Labbers, and I often ask people to explore their inner 'Labber' when I give presentations. What is a Labber? It's the experimental and playful part of us we all had as children and unfortunately many have lost when becoming an adult. It's the ability to be fearless, having the audacity and perhaps even naivety to try crazy things even if they are likely to fail! Unfortunately society values success more than it does failure. In my opinion, we need to recognise, respect and revere those that have the courage to try but failed. That courage to experiment should be honoured and embraced and should become the bedrock of our educational systems from the very outset.
Two years ago, many of us Labbers 'ate our own dog food' or 'practised what we preached' when me and 15 other colleagues came together for 5 days to produce a book through a booksprint, probably the most rewarding professional experience of my life. The book is about how to set up, maintain, sustain and even close a GLAM Lab and is called 'Open a GLAM Lab'. It is available as public domain content and I encourage you to read it.
Online drop-in goodbye - today!
I organised a 30 minute ‘online farewell drop-in’ on Wednesday 29 September 2021, 1330 BST (London), 1430 (Paris, Amsterdam), 2200 (Adelaide), 0830 (New York) on my very last day at the British Library. It was heart-warming that the session was 'maxed out' at one point with participants from all over the world. I honestly didn't expect over 100 colleagues to show up. I guess when you leave an organisation you get to find out who you actually made an impact on, who shows up, and who tells you, otherwise you may never know.
Those that know me well know that I would have much rather had a farewell do ‘in person’, over a pint and praying for the ‘chip god’ to deliver a huge portion of chips with salt/vinegar and tomato sauce’ magically and mysteriously to the table. The pub would have been Mc'Glynns (http://www.mcglynnsfreehouse.com/) near the British Library in London. I wonder who the chip god was? I never found out ;)
The answer to who the chip god was is in text following this sentence on white on white text...you will be very shocked to know who it was!- s
Spoiler alert it was me after all, my alter ego
Mahendra's online farewell to BL Labs, Wednesday 29 September, 1330 BST, 2021. Left: Flowers and wine from the GLAM Labbers arrived in Tallinn, 20 mins before the meeting! Right: Some of the participants of the online farewell
Leave a message of good will to see me off on my voyage!
It would be wonderful if you would like to leave me your good wishes, comments, memories, thoughts, scans of handwritten messages, pictures, photographs etc. on the following Google doc:
I will leave it open for a week or so after I have left. Reading positive sincere heartfelt messages from colleagues and collaborators over the years have already lifted my spirits. For me it provides evidence that you perhaps did actually make a difference to somone's life. I will definitely be re-reading them during the cold dark Baltic nights in Tallinn.
I would love to hear from you and find out what you are doing, or if you prefer, you can email me, the details are at the end of this post.
BL Labs Sailor and Captain Signing Off!
It's been a blast and lots of fun! Of course there is a tinge of sadness in leaving! For me, it's also been intellectually and emotionally challenging as well as exhausting, with many ‘highs’ and a few ‘lows’ or choppy waters, some professional and others personal.
I have learned so much about myself and there are so many things I am really really proud of. There are other things of course I wish I had done better. Most of all, I learned to embrace failure, my best teacher!
I think I did meet my original wish of wanting to help to open up the BL to as many new people who perhaps would have never engaged in the Library before. That was either by using digital collections and data for cool projects and/or simply walking through the doors of the BL in London or Boston Spa and having a look around and being inspired to do something because of it.
I wish the person who takes over my position lots of success! My only piece of advice is if you care, you will be fine!
Anyhow, what a time this has been for us all on this planet? I have definitely struggled at times. I, like many others, have lost loved ones and thought deeply about life and it's true meaning. I have also managed to find the courage to know what’s important and act accordingly, even if that has been a bit terrifying and difficult at times. Leaving the BL for example was not an easy decision for me, and I wish perhaps things had turned out differently, but I know I am doing the right thing for me, my future and my loved ones.
Though there have been a few dark times for me both professionally and personally, I hope you will be happy to know that I have also found peace and happiness too. I am in a really good place.
I would like to thank former alumni of BL Labs, Ben O'Steen - Technical Lead for BL Labs from 2013 to 2018, Hana Lewis (2016 - 2018) and Eleanor Cooper (2018-2019) both BL Labs Project Officers and many other people I worked through BL Labs and wider in the Library and outside it in my journey.
Where I am off to and what am I doing?
My professional plans are 'evolving', but one thing is certain, I will be moving country!
To Estonia to be precise!
I plan to live, settle down with my family and work there. I was never a fan of Brexit, and this way I get to stay a European.
I would like to finish with this final sweet video created by writer and filmaker Ling Low and her team in 2016, entitled 'Hey there Young Sailor' which they all made as volunteers for the Malaysian band, the 'Impatient Sisters'. It won the BL Labs Artistic Award in 2016. I had the pleasure and honour of meeting Ling over a lovely lunch in Kuala Lumpa, Malaysia, where I had also given a talk at the National Library about my work and looked for remanants of my grandfather who had settled there many years ago.
I wish all of you well, and if you are interested in keeping in touch with me, working with me or just saying hello, you can contact me via my personal email address: [email protected] or follow my progress on my personal website.
Happy journeys through this short life to all of you!
Wikimania is often the highlight of any Wikimedian’s calendar. Hosted by the Wikimedia Foundation, Wikimania is a conference like no other. A large number of participants take part in the annual celebration of open knowledge and Wikimedia projects. Previous events have taken place in Stockholm (2019), Cape Town (2018), Montreal (2017) and Italy (2016). Due to the ongoing global pandemic situation, this year's conference being held 13-17 August 2021 is taking place entirely online, something Wikimania is ideally suited for!
In addition to more traditional conference sessions, Wikimania will be running an Unconference, a Community Village, and a community Hackathon. Communication is encouraged through a variety of channels including Telegram, IRC and Wiki talk pages.
A photograph of an old telegraph key by Sandra Tan on Unsplash
Looking at the programme, so many interesting topics are on the table for presentation and discussion: from copyright reform, to innovation and community development, there’s a wide spectrum of material to interest all Wikimedians of every level. Handily, events are rated in terms of their suitability for beginners, to make things as welcoming as possible. There is a whole strand of presentations devoted to Wikidata, which you can view here.
I am very excited to be presenting remotely at this conference on behalf of the British Library. I will be introducing the work of Tom Derrick on the Bengali Books Wikisource Competition, and Dominic Kane (UCL) on the India Office Records project. We have shaped our panel to show what GLAM institutions can do to promote and effectively utilise Wiki platforms for public engagement with library and archive collections. Our panel will run on Sunday 15th of August at 8.15pm (7.15pm UTC).
Wikimania is free to attend online, 13-17 August 2021, registration is open until midnight on Thursday 12th August. We hope to see you there!
The New Media Writing Prize is awarded annually to interactive works that use technology and digital tools in exciting and innovative ways. Organised by Bournemouth University, the prize is now in its 12th year and open for entries until 26th November 2021.
The British Library hosted a Digital Conversations event to celebrate the 10th anniversary of the prize in 2019 and as part of our work on collecting and preserving emerging formats, last year we started building a special collection to archive all shortlisted and winning entries to the prize in the UK Web Archive. Thanks to Joan Francis for her valued support adding targets and metadata into the Annotation and Curation Tool, at the moment of writing, the collection stands at 226 websites, including not only all the works that were web-based and live at the moment of collection, but blog posts, press kits, online reviews and author’s websites as well. This kind of contextual information (like the data recorded on the ELMCIP Knowledge Base website) is especially valuable in those instances where the work itself couldn’t be captured, due to the limitations of web archiving tools, or the fact that it had already disappeared from the Internet. More information on how the collection was conceived and developed is available in the Collection Scoping Document on the British Library Research Repository.
In order to improve access to the collection and assure quality for the websites we captured, a PhD placement project started at the beginning of this June. Tegan Pyke, from Cardiff Metropolitan University, is working on the collection to identify best captures for each of these works and is also developing a creative response to the collection.
Tegan writes:
From the New Media Writing Prize shortlists, a total of 78 works have been captured, with each work averaging 13 instances to compare and contrast. Each instance represents a web crawl undertaken by the team from the Emerging Formats project.
A screenshot showing the instances collected for Serge Bouchardon’s 2011 Main Prize winning piece, "Loss of Grasp".
One of the most difficult aspects of this work has been deciding what, exactly, constitutes an ‘acceptable’ capture. By nature digital works are highly complex—featuring audio, visual, and kinetic assets—and using bespoke platforms, formats, and code. These attributes are heightened by the speed at which technology changes; what was acceptable a decade ago may be entirely defunct today, as is the case with Adobe removing their Flash Player support.
After an initial overview of the collection, I came to the conclusion that a strict set of criteria wouldn’t be appropriate. Nor would the capture of all aspects of a work, as many—such as Amira Hanafi’s What I’m Wearing and J R Carpenter’s The Gathering Cloud—make use of external links or externally hosted image and video files. If these lie outside the UK Legal Deposit’s scope, capturing them in their entirety becomes more difficult and sometimes impossible.
Instead, I decided to focus on narrative, asking three questions as I approached each instance:
Can viewers complete the narrative?
Does the theme remain understandable?
Is the atmosphere (the overall mood of the piece) intact?
If an instance fulfils these questions, it’s acceptable, with the most complete of those captures being identified as suitable for display in the archive.
At this point, I’m half-way through comparing instances for the collection. Of the pieces captured, just less than half meet the criteria above. Out of these, most can be improved by additional crawls that capture the missing assets. Those that cannot be improved have, for the most part, been affected by software deprecation or EOL (end-of-life), where support has been completely removed.
I’m aiming to finish my review of the collection over the next couple of months, at which point I hope to provide further insight into the process. I’ve also started a collaboration with the BL's Wikimedian-in-Residence, Lucy Hinnie, to plan a Wikidata project related to the collection aiming to make use of contextual data points collected during its creation—I’m sure you’ll read about this work here soon!
This post is by Giulia Carla Rossi, Curator of Digital Publications on twitter as @giugimonogatari and Tegan Pyke, a PhD student at Cardiff Metropolitan University currently undertaking a placement in Contemporary British Published Collections at the British Library.