By Jason Webber, Web Archiving Engagement Manager

 

We at the UK Web Archive (UKWA) have recently taken part in a new initiative called WARCnet led by the University of Aarhus in Denmark (and funded by Independent Research Fund Denmark).

“The aim of the WARCnet network is to promote high-quality national and transnational research that will help us to understand the history of (trans)national web domains and of transnational events on the web, drawing on the increasingly important digital cultural heritage held in national web archives.”

 

WARCnet logo

 

The majority of participants are researchers currently using web archives as part of their studies, many with extensive experience and others new to the field. This makes this an exciting project to be part of as it is an excellent way for content holders such as UKWA to be able to work closely with a group of researchers and try and understand their needs and challenges. The project had a kick-off meeting in May 2020 that was originally intended to be in person but took place virtually. All the speakers pre-recorded their talks which does now mean that these are now all available (including one by myself). I’d particularly recommend viewing the two keynote speakers Matthew S. Weber and Ian Milligan.

 

Title slide for Jason Webber's WARCnet presentation

 

Working Groups
It is intended for any outcomes from WARCnet to be driven by the participants themselves and to this end four working groups have been formed:

 

• Working Group 1 - Comparing entire web domains
• Working Group 2 - Analysing transnational events
• Working group 3 - Digital research methods and tools
• Working group 4 - Research data management across borders

 

The UKWA team is involved with each of the first three working groups, all of which have met in the last weeks to see how we can take this project forward. You can read more about each group here.

There are at least three more small conferences planned (currently as in person), one later this year in Luxembourg and two next year in London and Aarhus.

Look out for updates on our involvement with this initiative on this blog and through our twitter account @UKWebArchive and @WARC_net.

 

 

Stamps issued by Greece in 1896, the Universal Postal Union Collection, Philatelic Collections, The British Library.

 

Join our panel discussion to discover more about researchers' experiences when navigating archives, as well as the collection policies related to Olympics/Paralympics of GLAM organisations. This event is a collaboration between the British Society of Sports History (BSSH) and the British Library Web Archive team.

 

Register here to receive the joining details:

https://forms.gle/Tjzikxgjvr3FofSr8 

Date:           19 June 2020

Time:          3-4:30pm (BST) / 10-11:30am (EST)

Location:    Zoom

Twitter hashtag: #ResearchingtheGames

 

Presentations

Heather Dichter, De Montfort University - Finding Olympic history in non-sport archives

Laura Alexandra Brown, Northumbria University - The heritage of the Games: Interpreting urban change in Olympic host cities

Robert McNicol, Librarian, Wimbledon Lawn Tennis Museum - Researching the Olympics/Paralympics at Wimbledon

Helena Byrne, Curator of Web Archives, British Library - Preserving the Olympics/Paralympics online

 

What to expect

There is a broad mix of physical, digitised and born digital resources will be covered in the presentations. The Curator of Web Archives, Helena Byrne will be discussing the UK Web Archive collections related to the Olympics/Paralympics as well as the collaboration with the International Internet Preservation Consortium (IIPC).

The year 2020 was originally an Olympic/Paralympic year before the outbreak of the coronavirus pandemic. It is also a significant milestone for the UK Web Archive and the IIPC. It marks 15 years since the first UK Web Archive collections were published and also 10 years since the IIPC first started archiving the Olympics.

 

https://www.webarchive.org.uk/en/ukwa/collection

 

The UK Web Archive and sports

The UK Web Archive has been archiving sports related websites since it was established in 2005. However, it wasn’t until 2017 when dedicated sports collections were established. There are three broad collection groups Sports Collection, Sports: Football and Sports: International Events. The subsections of the Sports: International Events includes two summer and two winter Olympic/Paralympic collections from 2010, 2012, 2014 and 2016. The largest of these collections is the Olympic & Paralympic Games 2012 collection as the Games were hosted in the UK.

 

Access and reuse

Under the Non-Print Legal Deposit Regulations 2013 (NPLD) access to archived content is restricted to a UK legal deposit library reading room. However, if we have permission from the website owner, we can make the archived version of their content open access along with government publications under the Open Government Licence. This is why if you browse through the collections on our website, most of the links to archived content will direct you to one of the UK legal deposit libraries for access but some of the content you can view from your personal device.

 

IIPC and the Olympic/Paralympics

The UK Web Archive is made up of the six UK legal deposit libraries, two of those libraries, the British Library and the National Library of Scotland are also members of the International Internet Preservation Consortium (IIPC) which was founded in 2003. In 2010 the IIPC started its first collaborative collection on the Winter Olympics 2010 and has covered every Olympic/Paralympic Games since. Since the formation of the IIPC Content Development Group (CDG) the collections have started to include a broader range of subjects on and off the playing field.

 

Get Involved

The UK Web Archive aims to archive, preserve and give access to the UK web space.

If you see content that that should be included in one of sports collections then please fill in our online nomination form.

This blog post is is by Nicola Bingham, Helena Byrne, Carlos Lelkes-Rarugal and Giulia Carla Rossi

Introduction to Webrecorder

The UK Web Archive aims to capture the whole of the UK web space at least once a year, and targeted websites at more frequent intervals. We conduct this activity under the auspices of the Legal Deposit Regulations 2013 which enable us to capture, preserve and make accessible the UK Web for the benefit of researchers now and in the future.

Along with many cultural and heritage institutions that perform at-scale web archiving, we use Heritrix 3, the state of the art crawler developed by the Internet Archive and maintained and improved by an international community of web archiving technologists.

Heritrix copes very well with large scale, bulk crawling but is not optimised for high fidelity crawling of dynamic content, and in particular does not archive social media content very well.

Researchers are increasingly turning their attention to social media as a significant witness to our times, therefore we have a requirement to capture this content, in certain circumstances and in line with our collection development policy. Usually this will be around public events such as General Elections where much of the campaigning over recent years has been played out online and increasingly on social media in particular. 

For this reason we have looked at alternative web archiving tools such as Webrecorder to complement our existing toolset. 

Webrecorder was developed by Ilya Kreymer under the auspices of Rhizome (a non-profit organisation based in New York which commissions, presents and preserves digital art), under its digital preservation program. It offers a browser based version, which offers free accounts up to 5GB storage and a Desktop App. 

Webrecorder was already well known to us at the UK Web Archive although we had not used it until recently. It is a web archiving service which creates an interactive copy of web pages that the user explores in their browser including content revealed by interactions such as playing video and audio, scrolling, clicking buttons etc. This is a much more sophisticated method of acquisition than that used by Hertrix which essentially only follows HTML links and doesn’t handle dynamic content very well. 

What we planned to do

The UK General Election Campaign ran from the 6th of November 2019 when Parliament was dissolved, until polling day on the 12th of December 2019. On the 13th of December 2019 the UK Web Archive team, based at the British Library attempted to archive various social media accounts of the main political party leaders. Seventeen political leaders from the four home nations were identified and a selection of three social media accounts were targeted: Twitter, Facebook and Instagram. Not all leaders have accounts on all three platforms, but in total forty four social media accounts were archived. These accounts are identified in the table below by an X. 

Image credit: Carlos Lelkes-Rarugal

 

 

How we did it

On the 13th of December, 2019 we ran the Webrecorder Desktop App across twelve office PCs. Many were running the Webrecorder Autopilot function over the accounts, but we had mixed success, in that not all accounts captured the same amount of data. As the Autopilot functionality didn’t work well on all accounts, a combination of automated and manual capture processes were used where necessary. It took the team a lot longer than expected to archive the accounts therefore some were archived on a range of dates the following week.    

 

Large political party’s vs smaller party’s social media accounts

The two largest political party leaders, Jeremy Corbyn and Boris Johnson, have many more social media followers than the other home nations party leaders. This meant that it was more difficult to get a comprehensive capture of Corbyn and Johnson’s Twitter accounts than, for example, Arlene Foster’s. The more popular Twitter accounts took many hours to crawl; Corbyn’s took almost ten hours to archive thirteen day’s worth of Tweets (which only took us up to 1st December). 

 

Technical Issues

We experienced several technical issues with crawling, mainly concerned with issues around  IP addresses, the app crashing, and Autopilot working on some computers and not others. It was hard to get the app restarted after it crashed, so some time was lost when this happened. Different computers with the same specs ran differently. The Autopilot capture for Jeremy Corbyn’s and Boris Johnson’s Twitter accounts were started at the same time but Corbyn’s ran uninterrupted while Johnson’s crashed when it reached 475 MB. Although Corbyn’s account was crawled for nearly ten hours it only collected 93 MB of data. In contrast, Nigel Farage’s Twitter page was crawled for over four hours and only produced 506 MB. It is important to check the size of crawled data, as the hours the Webrecorder Desktop App is running on Autopilot does not necessarily translate into a high fidelity crawl. 

 

Added complications when using multiple devices with the same user profile:

Complications arose mainly from the auditing and collating of WARC files; performing QA and keeping track of which jobs were successful and those that were not. 

Initially, all participants in this project had planned to use their own work PC or work laptop and a local desktop installation of Webrecorder. However, an hour or so into the process(early in the day), it soon became apparent that there would not be enough time to archive all of the social media accounts within our time frame, given the volume of social media accounts and the unanticipated time it would take to archive each one. For example, it took one instance of a desktop Webrecorder application almost ten hours to archive Jeremy Corbyn’s Twitter account (only able to capture Tweets up to a month prior to the day of archiving).

It was then decided that we could potentially, and experimentally, run multiple parallel Webrecorder applications across a number of office desktop PCs; PCs that were free and available for us to use. This was possible because of the IT Architecture in place, allowing users to log into any office machine with the correct credentials and making their personal desktop load up along with all their files and user settings, regardless of the PC they log into. 

The British Library’s IT system, which incorporates a lot of the Windows ecosystem, gives each user their own dedicated central work directory where they are given a virtual hard drive and  their own storage space for all their documents and any other work related files. This allowed one user to be logged into several office PCs at the same time and therefore run a separate desktop Webrecorder application running on each machine. This was indeed very helpful as it allowed each machine to focus on one particular social media account, which in many cases took hours to archive. 

Having multiple Webrecorder jobs greatly increased our capacity to archive by removing the previous bottleneck, that was, one webrecorder job per user. Instead, this was increased to several webrecorder jobs per user.

Image credit: Carlos Lelkes-Rarugal

 

 

Having multiple Webrecorder jobs added complications down the line, not necessarily impacting the archiving process, but rather, complicating the auditing and collating of WARC files. When a user had several Webrecorder jobs running concurrently, each job would still be downloading to the same user work directory (the user’s virtual hard drive). So if a user had many parallel jobs running, this would create multiple WARC files in the same folder (but with different names, so no clashes), WARC files being produced by the different desktop PC that the user had logged in to. This was quite an elaborate setup because once a job had completed, the entire contents of the Webrecrder folder (where the WARCs were stored) was copied to a USB so that an initial Quality Assurance (QA) could be performed on the completed job on a more capable laptop. The difficulty was in finding the WARC file that corresponded to the completed job, which was somewhat convoluted as there would have been multiple WARC files with this type of file-naming convention:

 “rec-20191213100335021576-DESKTOP-AOCGH38-7B5SEXKS.warc.gz”. 

As you can imagine, taking a copy of Webrecorder’s folder contents not only has the completed job, but also the instances of other WARC files from other incomplete jobs. Coupled with multiple jobs per PC, and multiple PCs per user; keeping track of what had completed and which WARCs were either corrupted or not up to standard, was quite demanding. 

 

Review of the data collected 

Image credit: Carlos Lelkes-Rarugal

 

How to access this data

The archived social media accounts can be accessed through the UK General Election 2019 collection in a UK Legal Deposit Library Reading Room. The UK Legal Deposit Libraries are the British Library, National Library of Scotland, National Library of Wales, Bodleian Libraries, Cambridge University Library and Trinity College Dublin Library.  

The 2019 collection is part of a time series of UK General Elections dating from 2005. They can be accessed over the Internet on the Topics and Themes page of the UK Web Archive website. All the party leaders' social media accounts are tagged into the subsection UK Party Leaders Social Media Accounts (access to individual websites depends on whether we have an additional permission to allow ‘open’ access). More information about what is included in the UK General Election 2019 collection is available through the UK Web Archive blog. 

 

Conclusion

Overall, undertaking this experiment was an interesting experience for our small team of British Library Web Archive Curators. Many valuable lessons were learnt on how best to utilise Webrecorder in our current practice. The major takeaway was that it was a lot more time consuming than we expected. Instead of taking up one working day, it took nearly a whole week to archive our targeted social media accounts with Webrecorder. Our usual practise is to archive social media accounts with the Heritrix crawler, which works reasonably well with Twitter but is less suited to capturing other platforms. For a long time, we were unable to capture any Facebook content with Heritrix, mainly due to the platform’s publishing model, however the way the platform is published has changed recently allowing us limited success. Archiving social media will always remain challenging for the UK Web Archive, for myriad technical, ethical and legal reasons. The sheer scale of the UK’s social media output is too large for us to capture adequately (and indeed, this may not even be desirable) and certainly too large a task for us to tackle with manual, high fidelity tools such as Webrecorder. However, our recent experience during the 2019 UK General Election has convinced us that using Webrecorder to capture significant events is a worthwhile exercise, as long as we target selected, in scope accounts on a case by case basis. 

 

By Jennie Grimshaw, Curator for Official Publications, The British Library

The 2019 general election was a turning point in British political history.  It saw the resurgence of the Conservative vote from 8.8% at the May 2019 European Parliament election to 23.6% eight months later. It saw the collapse of the “Red Wall” in the North and the Midlands as seats such as Sedgfield,  Labour since 1935, and Workington, Labour 1918-1976 and 1979-2019, turned Conservative. It saw the breaking of the Parliamentary deadlock over Brexit with the return to power of pro-Leave Conservative Prime Minister Boris Johnson, with a majority of 80, which then enabled him to “Get Brexit Done”

To help researchers trace how use of the Internet for political campaigning and communication has evolved over time, the British Library, the National Libraries of Scotland and Wales, and the Bodleian Library have collaborated to create a web archive collection for all UK general elections since 2005, using more or less the same categories – candidates web presence, national and local political party websites, online news and commentary, interest group manifestos and comment and analysis by think tanks. This collection is the fifth in the time series and is complemented by the EU Referendum and Brexit collections. In 2019 Northern Ireland political party and candidate sites were selected by the Public Record Office of Northern Ireland (PRONI).

Sadly, our ability to harvest social media sites is very limited due to technical and legal issues. We can gather Twitter feeds, but not Facebook pages as the site deliberately blocks the crawls and it has proved impossible to negotiate access.  Due to the limitations of our crawl software, we cannot always gather dynamic content, Wix-based sites, documents stored in the cloud or videos. This may explain why some worthy candidate sites you might expect to see have not been archived.

However, the UK Web Archive General Election 2019 does offers researchers a respectable total of 2237 sites, including:

• UK–based news and comment sites , such as Politics Home, Political UK, the Commentator, Unherd, Reaction, CAPX and  CAPX 2019 Election Archive as well as Twitter feeds of selected  political journalists. We include satirical sites such as the spoof Conservative Manifesto created by a company called Concerned Citizens Ltd and the Daily Reckless, which offers satirical songs by Tommy Mackay.
• Candidates campaign websites and Twitter feeds and local constituency party sites. The collection includes all candidates standing in Scottish and Welsh constituencies. Due to staff resource limitations, we can only capture a sample of the websites and Twitter feeds of candidates standing in English constituencies. We cover three inner London and three outer London boroughs, and one rural and one urban constituency from each of the English regions.  We have used the same English constituencies for every election since 2005.
• Websites and Twitter feeds of 100 national political parties, ranging from fringe groups such as the Animal Welfare Party, Arthur Horner of the Welsh Communist Party, Britain First and the anarchist Class War Party to the major national parties (Conservative, Labour, Liberal Democrats, and Greens) and political parties in the devolved administrations (Plaid Cymru, Scottish Nationalists and the Northern Ireland parties). We also capture major political party blogs such as Conservative Home, Conservative Woman, Labour List and Liberal Democrat Voice.
• Social media sites of the main party leaders captured in depth using Web Recorder. We can use this software only very sparingly as operating it is very resource and time intensive, but it can capture sites our regular crawler cannot reach.
• Interest groups, seeking to influence party policies through engagement with candidates and publication of manifestos and lists of “asks”.  We have selected about 340 sites, ranging from the manifestos of campaigning charities such as Age UK and trade associations such as Airlines UK and the Association of the British Pharmaceutical Industry to unions, religious groups, such as the Evangelical Alliance and Muslim Council of Britain, and pressure groups such as the Campaign to Protect Rural England, Actionaid and Anti-Slavery International. Health charities and environmental groups are particularly prominent at this election and professional associations such as the medical royal colleges are also well represented. The voices of disabled people and minority groups are also heard through manifestos and comment from Leonard Cheshire Disability, SCOPE, Disability Rights UK, MENCAP, RNIB, Operation Black Vote and the Muslim Public Affair’s Committee’s Operation Muslim Vote 2019.
• Thank tanks and academic research centres providing in-depth comment and analysis. We have sought to include both right- and left-wing views, and comment from political, legal and economic viewpoints. Targets include the Centre for Labour and Social Studies (CLASS), the London School of Economics British Politics and Policy blog, the Centre for Constitutional Change, Demos’ Manifesto for consensus politics, the Democratic Audit blog, the British Future think tank, Full Fact, the Institute for Fiscal Studies the King’s Fund and the Institute for Government, etc.

We hope that this collection will preserve the voices and illustrate the concerns and priorities of a wide spectrum of UK society and help to show how political parties and candidates engaged and  responded at this pivotal moment of UK history.

General Election 2019 collection. Note that you can view what is in this collection but many of the actual websites can only be viewed in the reading room of a UK Legal Deposit Library.

By Jennie Grimshaw, Curator of Official Publications, The British Library

The vote to leave the EU on June 23^rd 2016 by 52% to 48% bitterly divided the nation. Remainers argued that the margin of victory was narrow, and that Leave voters had been misled. They saw Brexit as a self-inflicted wound and campaigned for a second referendum.  Leavers accused the Remain camp of seeking to undermine democracy and frustrate the clearly expressed will of the majority of voters. They saw Brexit as an opportunity to throw off what they viewed as the shackles of the EU and become a proud, independent sovereign nation once again.

The Brexit web archive collection follows on directly from the EU Referendum Collection and traces the course of the increasingly bitter battle to either overturn Brexit or “get it done” from June 24^th 2016 to Brexit Day on January 31^st 2020.  It was created by web archivists and curators at the British Library, the Bodleian Library, and the national libraries of Scotland and Wales working in collaboration over three years.  It seeks to achieve balance by including:

• News reports, indexed at the level of the individual article, from pro-Brexit, anti-Brexit and neutral newspapers and broadcasters. Pro-Brexit sources include the Sun, the Daily Mail and the Express newspapers. Overtly anti-Brexit sources are represented by the Guardian and the Independent.  Coverage of broadcast news sources focuses on the BBC, ITV and Sky News. The aim is to document every twist and turn of the unbelievably complex and convoluted process of delivering the EU Referendum result in order to guide enquirers through the labyrinth.
• A range of online only news and comment services, indexed at the site level, including Politics Home, Politics.co.uk, Huffingtom Post, CAPX, The Conversation, Spiked and Unherd. Unfortunately Politico had to be excluded as it operates outside of the UK web domain.
• Legislation and legal commentary, covering bills, Commons and Lords debates,  briefings on them by the Commons and Lords Libraries, legal challenges and case law, and comment in blogs such as EU Law Analysis and by professional groups such as Lawyers for Britain
• Government and Parliamentary action, offering Commons and Lords Select Committee reports, analysis by the Commons and Lords Libraries, and speeches, press notices, letters and guidance from GOV.UK.
• The EU view on developments as reported by UK-based news sources. The constraints of the UK Non-Print Legal Deposit regulations meant that we could not include EU institutional websites based in Brussels!
• The voices of politicians of all parties, heard through their websites, Twitter feeds, and blogs such as Conservative Home, Conservative Woman, Labour List and Liberal Democrat Voice.
• Comment by pro- and anti-Brexit pressure groups such as Best for Britain and the People’s Vote Campaign (anti-Brexit) and Brexit Central and Better Off Out (pro-Brexit)
• The voices of a wide range of trade and professional associations,  charities, business organisations and trade unions lobbying government and speaking up for and advising their members. Some voices are sadly silenced due to technical difficulties, such as the website of The3Million, which speaks for EU citizens in the UK, but which cannot be archived because it is Wix-based.  Others which are heard range from business associations MakeUK  and the CBI to charities like Age UK and professional bodies such as the British Veterinary Association.
• Research and analysis by pro- and anti- EU think tanks such as Centre for European Reform, the Federal Trust,  Policy Exchange, the Institute for Fiscal Studies, the Red Cell, Briefings for Brexit,  Demos, IPPR  and Centre for Policy Studies, the Institute for Government, Global Britain, Politeia,  etc.
• Sub-Collections on the impact of Brexit on Wales and Scotland built up energetically by colleagues at their respective National Libraries.
• Sub-collections on the impact on Northern Ireland and the Republic of Ireland, especially the vexed question of the Irish land border.

The path leading to Brexit day on January 31^st was long and winding. The debate polarised society and strained the UK constitution to breaking point as Remainers in a hung Parliament fought across party lines to delay or prevent Brexit  and Leavers went to equally extreme lengths to deliver it. The process saw the birth of Lawfare as Remainers used legal challenges to seek to block it.  Brexit also destabilised the devolution settlement and reignited calls for Scottish independence. We hope that by documenting the debate as it played out on the Internet and on social media we can help researchers to gain understanding in retrospect of this tumultuous period of our history.

By Nicola Bingham, Lead Curator of Web Archiving, The British Library

Note: This post was originally published on the Digital Preservation Coalition (DPC) blog.

The UK Web Archive, a partnership of the 6 UK Legal Deposit Libraries* (LDLs), has been collecting UK websites since the early 2000’s. As well as archiving snapshots of the whole UK Web Space we have dozens of curated collections focussing on a wide range of topics, themes and events reflecting all aspects of UK life.

Collections are instigated by a broad range of curators – in this context, ‘curator’ is not necessarily synonymous with job title - including LDL staff, academic researchers, various UK GLAM organisations (e.g., Jersey Heritage, Hampshire Archives and Local Studies, Wimbledon Lawn Tennis Museum) and local community groups. Collections may focus on a researcher’s area of interest, align with an institution’s collection policy or reflect diverse political, sporting or topical events such as the London Olympic Games, Brexit or Climate Change. Below are the members of the Web Archiving team at the British Library.

We have a particularly strong time-series of collections focusing on UK General Elections having archived every campaign since 2005. For each event we have used more or less the same categories – candidate’s web presence, national and local political party websites, online news and commentary, interest group manifestos and comment and analysis by think tanks.

Structuring the collections with consistent sub-categories enables curators to distribute web archiving more efficiently, as does dividing selection broadly along the lines of the geographical interest of the 3 National Libraries that belong to the UKWA.

We hope that our General Election collections will preserve the voices and illustrate the concerns and priorities of a wide spectrum of UK society and help to show how political parties and candidates engaged and responded at pivotal moments in UK history.

It is interesting to note how use of the Internet for political campaigning and communication has evolved over time. In 2005 very little social media existed and politicians were just beginning to explore its capabilities, whereas by 2019 campaigners were making little or no use of websites, concentrating almost exclusively on using social media.

The (somewhat) scheduled nature of UK General Elections, especially since the Fixed-term Parliaments Act of 2011, allows us to plan election web archiving strategies ahead of time. Having said this, we have been tested in recent years with snap elections in June 2017 and December 2019! And of course candidates are only announced a couple of week’s before polling day which means we have to react at that point to archive candidate’s websites, or official, publicly facing social media accounts.

Rapidly unfolding events such as natural disasters or terrorist attacks require a different approach. However, even here we have some experience, having archived collections about the London Terrorist Attack 2005, Grenfell Tower Fire, and Pandemic Outbreaks such as Avian Flu and Swine Flu over the years.

For the past few weeks we have been actively collecting the UK perspective of the Coronavirus (COVID-19) Pandemic. We are clearly facing one of the severest threats in our lifetimes, certainly one of the fastest and most clearly devastating, and while Librarians might not (yet) be members of the Emergency Services, we feel the act of recording the outbreak as it plays out online is a crucial one.

Websites are being selected by a cohort of curators across the LDLs and beyond. We have also been ably assisted by colleagues at the Royal College of Nursing Archives who are nominating health-related websites. However due to the unpredictable, fast paced nature of the outbreak and the consequent deluge in online information, it is more important for us to harness the crowd to elicit website nominations. For this reason, we will canvas for website nominations much more widely among our colleagues, the library and archive community and the general public when responding to rapidly unfolding events. We will also visit targeted websites much more frequently than we would usually to capture frequently edited web content.

The collection is not public yet while we concentrate on acquiring the websites. Once we’re finished, it will take time to prepare the collection for publication by performing quality assurance and clearing permissions for open access. In due course, the Coronavirus collection will be available here under the Pandemic Outbreaks Collection. The top-level heading reflects the fact that we have previously collected around Avian Flu and Swine Flu and acknowledges that, sadly, we will be collecting about future outbreaks.

In terms of getting involved, we welcome submissions from colleagues in the DPC community - and in fact from any member of the public. Details of how to nominate websites for inclusion are here: www.webarchive.org.uk/nominate. Alternatively, please email nominations to [email protected]
We’re also working on an international collection with the International Internet Preservation Consortium (IIPC). Details of how to contribute to this collection are here: netpreserveblog.wordpress.com/2020/02/13/cdg-collection-novel-coronavirus/ (non-English language websites are particularly welcome here).

If your organisation has not previously done any web archiving and you would like to capture your own institution’s or communities’ response to Coronavirus, plenty of tools exist that can be used remotely. Webrecorder is a good place to start as it can be used in a browser, free of charge up to a 5GB data limit. Of course web archives such as the UKWA and Internet Archive would also be very happy to preserve your websites free of charge (see details above).

*The UK Legal Deposit Libraries: Bodleian Libraries, Oxford University, British Library, Cambridge University Libraries, National Library of Scotland, National Library of Wales, Trinity College, Dublin

By Pete Hebden, Phd Student Placement, Newcastle University

One of the great features of the UK Web Archive is its series of curated collections, which can be found on the UKWA Topics and Themes page. Each collection centres on a specific topic, some responding to particular events, such as Brexit or the First World War Centenary, others drawing upon the knowledge of contributors to create a set of in-depth examples around a particular subject. During my time at the British Library, I spent some time contributing to the Poetry Zines and Journals collection, originally started in 2016 by previous PhD placement student Joe McCarthy. The collection contains an amazing range of UK-based online outlets for poetry, encompassing blogs, Twitter accounts, online journals, and the personal websites of some poets, where there is a significant amount of the poet’s creative work on the site.

Poetry writing

 Although the collection was already very well curated when I came to it, it had not been significantly updated in several years, so many newer publications were not included in the collection. The past few years have seen a serious increase in the number of high-quality online literary journals – a trend that this collection was very astutely responding to when it was first created – and so there were a number of recent but well-established poetry titles that I could add to the list. One example is perverse, an online-only poetry journal started by the poet Chrissy Williams in 2018.

Along with roughly a third of the Poetry Zines and Journals collection, the archived version of perverse is only accessible on-site at the reading rooms of the UK’s legal deposit libraries. The rest of the content, for which open access permission has been obtained, can be viewed from anywhere. The Rialto and Porridge magazine are two examples of recently added sites that are open access, and the links here lead to the archived versions of those sites.

My other choices for inclusion in the collection were guided by some of my own specific areas of knowledge and interest. I included several online journals that are based in, or focus on, the north of England and Scotland, as these are literary scenes that I am more familiar with. Butcher’s Dog and Another North are two relatively recent literary journals based in the north of England. Another North is entirely digital, while Butcher’s Dog is a print journal with a strong online presence. I also added several more websites for print magazines that feature a significant amount of poetry on their site. For example, Popshot and The Rialto, both print magazines, regularly feature poetry from their most recent issues on their websites and/or social media, which gives readers an idea of the journal’s editorial policy and marks a significant change in the way that poetry is distributed by these publications thanks to the internet.

One interesting problem that I encountered during the process was around the formats that some digital publications use for distribution. While most online poetry journals choose to publish in a standard website or blog form, some distribute each issue as a downloadable file, such as a PDF or EPUB. This method of delivery presents a problem when attempting to archive the content, as the web crawler is not necessarily able to access and download these files, meaning that the poetry itself goes unrecorded. For these journals, we had to use alternative ways of recording their poetry in the collection. For example, perverse (mentioned above), as well as distributing each issue as a PDF download, also posts each poem individually to Twitter, and so we set up a regular capture of their Twitter account in order to record all of the poetry. Many other journals use social media in a similar way, and so in these cases I was able to use this as a way of archiving the journal’s output.

Over the past few decades, the web has provided an exciting platform for a diverse range of poets and publishers to showcase their work and it has been a very enjoyable challenge to contribute to the cataloguing of this transformation. I hope that my work on the Poetry Zines and Journals collection will help other readers and researchers exploring the breadth and variety of UK poetry available online today. 

If you know of any websites that should be included in this collection or in the general UK Web Archive, please nominate it.

 

By Jason Webber, Web Archiving Engagement Manager, The British Library

Like many public places at the moment, the reading rooms of the UK Legal Deposit Libraries are going to be shut for some time. What does this mean for the UK Web Archive? Well as some of you might know we try to collect every UK website at least once a year and this is done under the provision of the Non-Print Legal Deposit Regulations 2013. A condition of these regulations are that content collected can only be viewed on library premises. Never fear though, we still have lots for you to do!

UKWA website home page

Discover millions of websites
At the end of 2018, we launched our new service for searching the whole UK Web Archive catalogue from anywhere. Go to our website: www.webarchive.org.uk and search for a web address (URL) or word/phrase. You will get results showing all of our resources that you can access from anywhere. Tick the box 'At Libraries' to see everything that we have collected. Access thousands of websites
Over the 15 years that we have been archiving websites we have frequently sought permission from owners to make their sites publicly viewable outside of library premises. In that time we have received permissions from over 15,000 website owners These websites have been selected because they relate to a  specific topics or event, for their importance, or because they were about to go offline. Lots to see!

Browse 'Topics and Themes'
You can browse over 100 different topics and themes. From the extensive 'Brexit' collection to 'Web Comics' there is something for everyone. As a starter, check out 'Online Enthusiasts' and discover many of the  hobbies and societies in the UK.

SHINE service
UK Web Archive holds a collection of all the .uk websites that were archived by the Internet Archive between 1996-2013.  The service includes a 'trends' feature that we highly recommend that you try.https://www.webarchive.org.uk/shine/graph

You can enter a word or phrase (in speech marks) to see the relative popularity in a given year. Enter different terms separated by a comma and you can compare popularity e.g. tom,jane. See who is 'best',  cat or dog or the emergence of words such as 'iphone', 'emoji' or phrases such as 'credit crunch'.

Do tell us what you find!

Trends - the use of the term 'loungewear' in the UK web

Nominate websites
We are still able to add websites to the archive and welcome nominations! We want to archive every single UK website and your help is invaluable. Make your suggestions here: www.webarchive.org.uk/nominate

Stay safe everyone!

@ukwebarchive