What are newspapers made of?

One could say that they are made of accounts of current events, collected in the form of a document for the interest of a particular readership. One could equally say that they are made of assumptions that covertly or overtly express an ideology of one kind or another. One could be literal and say that newspapers are made of paper, generally derived from a rag or wood pulp confection, cut to a set shape and overlaid with ink in the form of recognisable objects. Newspapers are, most simply, made of words, numbers and pictures - but mostly words.

Participants at British Library newspaper data visualisation workshop, 30 October 2019

Newspapers are also made of data. Data is a newspaper's underlying code. Beyond the plain text there exist underlying collections of terms from which we may discovers ideas, clues, connections and patterns which may reveal all the more for us what a set of newspapers has to say.

The digitisation of historic newspapers is creating not just a digital simulacrum of our physical newspaper archives, but a vast collection of data that can be derived from the digitisation process. Most users of digitised historic newspapers will be aware of optical character recognition, or OCR, the process by which the text on an old newspaper page, which we see as words but which a machine understands to be images, is converted by that machine into words that it can recognise. Some will also know that older type, or poor quality microfilmed newspapers, can lead to inaccurate OCR, as the machine struggles to interpret the muddy images it sees into words that we will recognise. Some may also know that specialist software enables a digitised newspaper page to be broken down into its constituent parts, such as article, illustrations or advertisements.

But there is much more than can be done when we digitally analyse this initial layer of information still further. Software programmes can highlights proper names (people, places, organisations), word frequencies, patterns of words and recurrent phrases. It is a form of indexing, as though someone had read through an entire run of a newspaper and produced an index. Indexes to annual volumes of newspapers were not uncommon in the nineteenth century, when  people would visits clubs or newspaper reading rooms to leaf their way through past newspapers, the best known being Palmer's Index to The Times. Now software can do this basic work but also much more. It can reveal those patterns which may reveal an underlying history; hidden truths just waiting for the right software programme to bring them to the surface.

The opportunities created by derived data are exciting a growing number of data science and digital humanities scholars, who find newspapers an especially fruitful source of enquiry for their large numbers, their consistency of form and their geographical, social or political identity. Digitised newspapers reflect the flow of time, turning news into history.

So the specialists are being well catered for, but what about the rest of us? Here at the British Library, as we digitise more and more newspapers and so create an ever greater reservoir of re-usable data, we are interested in opening up newspaper data to other kinds of users. We shouldn't all have to be experts in specialist file formats or programming languages to get something out of newspaper data. We should be thinking equally of those who would just like to have a spreadsheet with a clear set of fields that they can sort (by place, date, title etc.), and maybe some guidance on easy-to-use visualisation tools that enable anyone to produce a graph, pie-chart or stylish map.

All of this we are going to do. We will have news about what we are going to be making available to all soon. But we are also looking at ways in which such data might inspire creativity. In partnership with the London College of Communication, we have organised some trial workshop in newspaper data visualisation. 

The first of these took place early in October, a report on which was published on this blog. For this workshop we had a mixed group of volunteers, though several were newspaper history specialists. We gave them some sample nineteenth century British newspaper stories and invited them to rethink what they saw in visual terms, with reference to the data that could be derived from the stories, either by machine or human.

The results were fascinating, at times inspiring (we now know that inside some newspaper historians lies an artist just waiting to be set free). However, for a second workshop at the end of October we changed tack. Instead of giving the volunteers stories we gave them one of four sample newspapers from the nineteenth century but asked them to concentrate on sets of terms, phrases and story headlines that we had generated from an entire year of the newspaper (we chose 1880 and the newspapers the Illustrated Police News, Hull Packet, Newcastle Courant and Manchester Weekly Times). Analysing an entire year yielded more meaningful results from which we expected the volunteers to be able to create their own visual impressions, rough sketches inevitably, but with the hope showing the potential.

Our volunteers were another mixed group, this time with more people from the creative side of things (including some art and design students), but again some with expertise in newspapers. What we saw were a rich set of different responses to the data. Some worked with the terms as presented. One create an idea for a headline generator, focussing on disturbing stories reporting on violence towards women, from the Illustrated Police News. Another, who was a poet, uncovered patterns within the terms that revealed found poetry.

Named entities from the Hull Packet for 1880 (names and organisations)

Others worked with the data to visualise the newspaper form differently, one that made its underlying messages more apparent, or at least arranged in a new light. One design student reinvented her newspaper as an unfolding square, with its messages on the outside leading to greater discovery within. One group extracted the major components of their newspaper (advertising, law reports, local news, entertainments) which they laid out on the floor, with lengths of string indicating which kinds of newspaper component were most prominent (advertising had by far the longest string). 

Another group started with an academic research question (they wanted to know how they could find out how much advertising space was being paid for different products) and imagined a form of digital analysis which measured space alongside subject, extrapolating the potential for visual analysis in a most interesting way. Another participant saw the opportunity for presenting nineteenth century newspapers in a twenty-first century format, and likewise twenty-first century subjects in a nineteenth century newspaper format, to make that which could appear alien to a young audience of today more meaningful, and revelatory.

I noted three things in particular. The first was that data on its own was not helpful. It appeared to lack meaning. A set of terms only became meaningful to the volunteers when they could see it in the context of the newspaper from which the data was generated. To understand and value derived data, we need knowledge of its roots.

Secondly, it is noticeable how much people did not so much work with the data as use it as a springboard for their own analyses of what was significant about the newspaper before them. The data encouraged creative thinking without necessarily being used directly as the basis of the creative object. Derived data can form the building blocks of a new kind of historical enquiry, but it can also - quite literally - inspire. It encourages to think, and to visualise, analytically.

Lastly, people see things differently because they are different, and in this variety lies such opportunity. A newspaper historian see the patterns of news. A designer sees how a raw idea may be made both beautiful and practical. A poet sees poetry. 

We will be exploring this area further. Although our primary goal in making more historical newspaper data available is to assist academic, as well as general researchers, we want to see where the creative impulse may take us. It could lead to different kinds of newspapers being digitised, or derived data being made in forms most suitable for creative inspiration. It could lead to beautiful things.

 

 

Historical newspapers are not always easy to digest. Mostly without illustrations or photography, the ‘wall of text’ of a nineteenth-century newspaper can be intimidating or difficult to engage with, and we’ve been thinking about how we can solve this.

Last Wednesday evening we ran an experimental workshop in conjunction with lecturers in design from London College of Communication. Our aim was to learn how techniques from design could help the understanding of nineteenth-century newspaper articles, and we hoped to learn how we could make the information within our newspapers more accessible to Library users. We were very lucky to have a diverse group of historical newspaper users to help us out in our experiment.

We gathered historians, general library users with a passing interest in newspapers and newspaper data, artists, and designers together and asked them to take part in a workshop where they would ‘visualise’ a newspaper in an innovative way, using art materials rather than computer software.

The evening started with a brief overview of The British Library’s newspaper resources, an outline of our Heritage Made Digital programme (which will result in a set of openly available historical newspaper resources and the underlying data), and some pointers for those looking to learn how to do data analysis. We gave a very quick description of some of the tools we use, including R, Python, Jupyter Notebooks, https://voyant-tools.org/, and Palladio, and where to learn how to use them: we recommend checking out https://programminghistorian.org/ and  https://software-carpentry.org/.

Participants then worked with art materials, including hand-printed riso paper, to visually communicate an aspect of a newspaper article they found interesting. We gave several articles to choose from: one on the ‘Trial of Queen Caroline’, one about the burglar and murderer Charles Peace, one about the first commercial passenger railway journey (which also includes the first railway fatality), as well as a page of advertisements and a page of letters to the editor.

Participant showing their visual representation of categorical information found in a newspaper article

Participants took a range of approaches and styles: some took to chopping up paper straight away, whereas others were more cautious. Some chose to focus on the content of the article, and others looked at the visual or structural elements of the page. One participant color-coded their article according to gender. Another noticed a faint outline of an illustration on a page and based their work around this. Several attendees picked up on the significance of the advertisements, and the potential historical information within. By the end we had an impressive array of visualisations, including an entire three dimensional dolls’ house.

A dolls’ house based on the details of the ‘Trial of Queen Caroline’

There’s definitely more work to do. The workshop was exploratory and we already have some ideas on how to improve our next iteration. But it was an interesting, stimulating experiment, and we think a good stepping-point in the goal of making historic newspapers more accessible.

The introductory slides from the workshop are available at https://www.slideshare.net/lukemckernan/data-visualisation-workshop

We are organising a second newspaper data visualisation workshop for  October 30, between 17:00 and 19:00, at the British Library in London. If you’re interested to participate, please contact [email protected] for more information.

Yann Ryan, Curator Newspaper Data

Interested in data visualisation, information design or historic newspapers? We’re looking for a small group of volunteers to take part in some trial workshops in October, being run in collaboration with London College of Communication. You’ll learn how to work with data in a creative, hands-on way, while getting an overview of the Library’s digital newspaper collection and how you might use its data.

Volunteers from any background are welcome: you don't need to have expertise in working with data. All you need is some enthusiasm, ideas, and an interest in learning more about one of the areas above. We’ll provide all the necessary supplies, including the tea and biscuits.

The workshops will be held at The British Library in London on October 2 and October 30, between 17:00 and 19:00. 

If you’re interested and can make one or other of these dates, please contact [email protected] for more information.

The world of news is changing, and at the British Library we are responding to that change - in how we collect, preserve, describe and present our news collections. Our goal is to transform what we hope is a world-class newspaper service into a world-class news service. This post outlines the Library's News Content Strategy for 2019-2023 with our plans for the next five years.

Inside the National Newspaper Building at Boston Spa

The British Library holds many millions of newspaper issues, and thousands of news websites, radio broadcasts and television programmes. Because it is a legal deposit library, it regularly collects thousands of news-related UK websites for its web archive and continues to receive the range of UK newspapers in print, including foreign language news published in the UK. It also subscribes to news services from across the world, providing a first class research experience for its readers. Together these form one of the greatest historical collections in the world, underpinning research into centuries of UK life and events, and to those of further afield.

In the last decade the Library has transformed its preservation of news, building state of the art facilities to store its historical newspapers collection in excellent environmental conditions and putting in place the first key elements of digital storage for ‘born digital’ news. It has greatly upgraded its service offer for news, making its content available in its reading rooms in London and Yorkshire, including a specially dedicated Newsroom at St Pancras. But there is much more to be done for the discovery of news onsite and online.

Medium	Size of collection	Weekly intake
Newspapers	60 million issues	1,200 issues
Web news	500,000 captures	2,000 websites
TV news	90,000 programmes	200 programmes
Radio news	50,000 programmes	170 programmes

Figures for the current news collection at the British Library

The Library works in partnership with other bodies to develop in-depth understanding of news and the events it describes. Working with family history company Findmypast the Library has provided most of the digitised newspapers on the British Newspaper Archive website, helping researchers and the general public to view rare newspapers from the comfort of their home or workplace.  With over 30 million pages digitised, many online readers exploring their family history will already be familiar with the resource. It is proving invaluable for a huge range of academic research topics as well.

That said, there is still so much to do. The digitisation challenge is vast: 93% of our newspaper content remains undigitised

One key to transforming our news offer is through data. News data is of particular value to researchers for its range across so many subjects and time periods, and for the regularity of its published outputs. It has huge potential for furthering our expertise in the data sciences. Our digitised newspaper archives are already being used by several ‘big data’ projects; in particular our historical archives underpin the major UKRI-funded ‘Living with Machines’ collaboration between the British Library and the Alan Turing Institute. Through the Heritage Made Digital programme we are building up a substantial body of out-of-copyright newspaper data which will greatly improve the service we offer to digital scholarship.

Finally, data forms the building blocks by which we will bring together the different news media to deliver an integrated news service that best serves future needs. 

Our commitment is to the news, not to the newspaper. This shift in thinking follows the direction in which the news media themselves have gone, and will trigger great changes in storage, access and use. It will ensure that the British Library continues to offer the best news research service, for researchers now and in the future.

Over the next five years, the Library will concentrate on four areas of its news collections:

Transforming discovery of news 

We will greatly improve the ease with which readers and the wider public can access our news offer, and respond fully to the big data opportunities of our historic news collections.

Collecting contemporary news 

We will collect UK contemporary news digitally as a matter of course and regularly review our selective approach to overseas news.

Protecting at-risk historical news

We will greatly increase our preservation of historical newspapers, digitising to rebalanced priorities, including at-risk titles.

Planning the next major phase of our strategic storage of news

Our large secure digital store will take audiovisual and digital news as business as usual, and save on physical storage space by switching to digital versions for a majority current UK newspapers; but we will still need to plan for new physical storage.

Masthead for The News and Sunday Globe, 2 July 1837, one of the titles being digitised by the Heritage Made Digital programme

Many activities relevant to the News Content Strategy are already underway. As we approach the 400th anniversary of the first newspaper available in Britain (1620) and the first newspaper published in Britain (1621), the British Library is responding to the profound changes taking place in the world of news today. At the same time we aim to revitalise how researchers may use and understand the news of yesterday. Look out for some significant announcements over the next year. Good news is on its way.

 

 

In Spring 2021 the British Library will be hosting a major exhibition on the history of news in Britain.

The aim of the exhibition is to explore the history, present and future of news in Britain over 400 years since the first newspaper was published in this country, asking what makes the news what it is, and what this means for us. 

The exhibition will trace how news for the diverse audiences of this country has been produced, distributed and read over four centuries, through news sheets, news books, broadsides, newspapers, newsreels, radio, television, the internet and social media. The exhibition will encourage questions about the role of news in society. It will look at the ways in which news is changing as we ourselves change. It will invite to us to consider vital issues of choice, interpretation, truth and trust.

Planning for the exhibition, provisionally entitled Making the News, has got underway. To help us put it together, we are advertising for a two-year exhibition project curator.

Working with a curatorial team and the Library’s exhibition team, the post-holder will contribute to the development and delivery of the exhibition. They will contribute to the administration of the curatorial content of the exhibition; will prepare external visits and show-and-tells; will promote the exhibition on social media and to visitors; and will apply research expertise in one or more areas of British news history in support of the selection and curation of content for the exhibition.

Details of the vacancy and how to apply can be found on the British Library's careers site. The deadline for applications is 23 June 2019.

One of the most rewarding aspects of working on the Heritage Made Digital Newspapers, the British Library’s project to digitise a number of 19th century British newspapers, is uncovering the hidden, forgotten or lesser-known stories surrounding the newspapers we have selected. One of the aims of the project it to begin to fill gaps in the digital archive, and therefore many of the newspapers that we have chosen are from the early 19th century, were short-lived, and are generally less well known (though not necessarily less important) than those that have been digitised by other projects and organisations. This means that there is often limited information about these newspapers, and in researching the publication histories of each title, we sometimes stumbled across interesting news stories, images and accounts of events; and even on occasion a juicy story that directly involves the newspaper itself. A perfect example of this is the Beacon (1821).

The [Edinburgh] Beacon, vol. 1, no. 1, 6^th January 1821, p. 1.

The Edinburgh Beacon

The Beacon was published in Edinburgh between January and September 1821, and is one of the few newspapers based outside of London that has made it on to our project shortlist. It came to our notice as it had links to a title that had already met some of our selection criteria (in that it was London based and out of copyright), a later newspaper also called the Beacon (1822), which had been identified as being of particular interest due to the involvement of the famous author and poet, Sir Walter Scott  (below, 1822 portrait by by Sir Henry Raeburn). However, after digging a little bit deeper, it became clear that these two separate titles, the Edinburgh Beacon and the London Beacon, had been muddled in some of the core resources on 19^th century newspapers, and Walter Scott was, in fact, linked to the Scottish and not the London newspaper.

It also became clear that the Edinburgh Beacon’s short existence was mired in controversy and scandal, and while there is only a limited amount of information and scholarship on the subject, what there is tells a hugely interesting story. The Beacon’s founding and financing set the stage for political turmoil; the way it was run led to libel action and heightened political and press antagonism; and the ultimate outcome of the newspaper’s existence was a fatal duel between Sir Alexander Boswell and James Stuart in March 1822. John Chalmers, who wrote a book on the surrounding scandal, suggests that the impact of events was far reaching and significant, crediting it with ‘a move toward more responsible journalism and a contribution towards the popular demand for parliamentary reform.' (John Chalmers, Duel Personalities: James Stuart versus Sir Alexander Boswell). These are no small claims. So how did a newspaper, which ran for only a few months, have such a huge impact?

The Beacon was established as a Tory counterpoint to the popularity of the Whig supporting Scotsman (1817-2004), and also as a reaction to the coverage in the Whig and Radical Press of the Queen Caroline Affair in 1820. However, the newspaper quickly caused trouble, printing personal and vicious attacks against those it disagreed with. In particular it levelled frequent abuse at a small number of Whig politicians, including Francis Jeffrey, James Stuart, James Gibson and Lord Archibald Hamilton. The Beacon was closed down suddenly in September 1821, after several cases of libel were brought against the publication, and its printer fled to the continent.

Two things elevated the Beacon’s notoriety to scandalous levels. The first was the involvement of a large number of well-known figures, many of them in positions of authority or with links to the Establishment. In general the financial backers of a newspaper during this period remained anonymous, and could therefore deny any involvement if trouble arose. However, the money to publish the Beacon had been raised via a subscription, and many of those involved, rather than paying the money up front, signed a bond of security with a bank which advanced the money, therefore creating documentary evidence of their ties to the publication. One of those named on the bond was Sir William Rae, Lord Advocate of Scotland, and his involvement with the publication sparked discussions about the close relationship between politicians and the press, and ultimately resulted in a debate before Parliament. Others named on the bond included the Solicitor General for Scotland, the Lord Provost of Edinburgh and the MP for Stirling. Sir Walter Scott, whose involvement initially drew our interest, was also a signatory on the bond. He was one of the driving forces behind the launching of the newspaper, providing both funds and advice, however much of his advice was ignored, and he quickly became disillusioned with the venture, describing it as a ‘blasted business’. He attempted to distance himself from the paper as the scandal took hold, but this was largely unsuccessful and his reputation took a significant hit as a result.

The second particularly scandalous element came about because of the attacks the paper printed against the Whig politician, James Stuart (right, portrait by Daniel Macnee), who was a favourite target for abuse. Stuart took exception to several of the items written about him, especially those that he saw as attacks on his social standing. The result was a prolonged dispute with the paper’s publisher, Duncan Stevenson, with Stevenson refusing to disclose who had written any of the offending articles. This largely played out in the pages of the Beacon, but also in one public event where Stuart horsewhipped Stevenson. Unable to find satisfaction via the publisher, Stuart turned his attention to Sir William Rae, the Lord Advocate of Scotland, after discovering his name listed on the bond. It was in Stuart’s public revealing of the connection between the newspaper and the Lord Advocate of Scotland, which led to the scandal and the ultimate demise of the paper. Stuart may have felt that the affair was laid to rest with the shutting down of the Beacon, but this was not to be the case.

Challenged to a duel

Upon closing, the Beacon’s cause was instantly taken up by a Glasgow paper, The Sentinel (1821-1823). Not only were several of the contributors shared between the two papers, but Sir William Rae had once again lent his support to the founding of the newspaper, under its former title The Clydesdale Journal (1820-1821), although he was more circumspect in that instance and did so privately. In particular the Sentinel continued in publishing personal attacks against those Whigs singled out by the Beacon, with a sustained campaign of articles, letters and poems levelled at James Stuart. This time, due to a dispute between the two publishers of the Sentinel, Stuart was able to get hold of original copies of the articles and identity their author. It was revealed to be Sir Alexander Boswell, son of Samuel Johnson's biographer James Boswell, a distant relation of Stuart’s, and a well-known and well-liked member of the Tory Establishment in Scotland. Stuart believed his only recourse was to challenge Boswell to a duel, and therefore sent his appointed ‘Second’ to issue the challenge.

The Sentinel, vol. 1, no. 1, 10^th October 1821, p. 1.

The duel was fought on 26^th March 1822, and while neither side intended to cause actual harm, Boswell was fatally wounded by Stuart’s mis-aimed shot. The duel and Stuart’s subsequent trial for murder gained intense public interest, much of it whipped up the extensive media coverage of events. Many of those accounts linked back to the original scandal surrounding the downfall of the Beacon, with more focus placed upon that than the role of the Sentinel in the dispute. Stuart was acquitted, as so frequently happened at trials for duelling during this period, and came out of the affair with little damage to his reputation.

Many of the others involved suffered more long term consequences, particularly those who were directly linked to the two newspapers involved. Notably, the printer of the Beacon, John Nimmo, who despite being only a very minor figure in the operations of the newspaper, was briefly accused of being its proprietor. He fled to France, fearing prosecution, and remained in exile there for the rest of his life. William Borthwick, one of the publisher/proprietors of the Sentinel, also had his life completely turned upside down by the affair. It was his dispute with his former colleague, Robert Alexander that led to the revealing of Boswell’s authorship of the offending articles, but as a result of his actions he was arrested for stealing papers that may or may not have rightfully been his property. Borthwick’s circumstances were complicated because the papers he was accused of stealing also played a role in James Stuart’s trial, and his own prosecution was repeatedly delayed. Upon Stuart’s acquittal, Borthwick was suddenly released from jail without trail, but the legal wranglings and imprisonment had left him penniless and unemployed. He spent much of the rest of his life bitterly campaigning for compensation for the miscarriage of justice against him, claiming, not unreasonably, that he had been used as a pawn in the political dispute between the Scottish Tories and Whigs.

An important factor is all of this once again brings Sir William Rae, Lord Advocate of Scotland, to the fore. As Lord Advocate, Rae was the only public prosecutor in Scotland, and therefore oversaw all Crown prosecutions in the country. This meant that he oversaw not only James Stuart’s trial, but also William Borthwick’s, but in both cases he was personally connected to the case via his involvement with the Beacon and the Sentinel. His close relationship with scandalous elements of the Tory press would be debated in parliament, but alongside this he was also accused of using his political power and privilege to interfere with the legal proceedings surrounding the two cases, and particularly of illegally extending Borthwick’s detention, while having no real intention of prosecuting him. While a parliamentary vote narrowly acquitted him of these charges, allowing him to continue in the role of Lord Advocate, Rae’s reputation was severely dented. He continued in high office until his death in 1842, but Rae was not raised to the bench, as was customary for most Lord Advocates, indicating that his involvement in the scandal had long-term consequences for his career.

The London Beacon

Finally, we turn to the title which had originally drawn the attention of our project, the London Beacon, which was founded in April 1822. It boldly declared its connection to the original Beacon, introducing itself with a fiery statement:

‘The Beacon in London’, The [London] Beacon, vol. 1, no. 1, Sunday 21^st April 1822, p. 1.

However, it lasted for only seven issues, perhaps indicating that the public had lost interest and patience with the personal attacks and antagonistic writing style that were the Beacon’s trademark. It might also speak to the geographically specific audience that the original publication had engaged with, meaning a London based successor did not have a solid base of readers to rely upon. Or it might tell us something of the wariness of investors to put money into a venture that might be both a political and a financial catastrophe for them.

While Chalmers credit’s the Beacon scandal with beginning a move away from the personal libellous attacks and irresponsible journalism that had characterised the period, there was certainly no sudden change in tone or attitude in either the Scottish or English press at this time. The shift was slow and subtle, and libel cases were frequent for many years afterwards, and even the occasional duel resulting from the fraught political and publishing environment of the time. However, it was a memorable case, drawing well-known individuals into a scandal that played out across the media, and may perhaps be seen as the starting point for the slow emergence of a more thoughtful approach to political journalism.

Whatever its impact, the Beacon, and those titles closely linked to it, provide interesting and amusing additions to the collections of newspapers being digitised by Heritage Made Digital. They remind us of the importance of detailed research into the titles we are presenting, and also the delight of coming across a hidden gem in the diverse range of material we are working with.

Further reading

• Chalmers, John. Duel Personalities: James Stuart versus Sir Alexander Boswell. Edinburgh: Newbattle Publishing, 2014.
• Schoenfield, Mark. ‘The Trial of James Stuart (1822): ‘Abuse of the Press, and Duelling.'’ BRANCH: Britain, Representation and Nineteenth-Century History. Dino Franco Felluga (ed).  Extension of Romanticism and Victorianism on the Net. http://www.branchcollective.org/?ps_articles=mark-schoenfield-the-trial-of-james-stuart-1822-abuse-of-the-press-and-duelling [accessed 15/04/2019].

 

Beth Gaskell, Curator Newspaper Digitisation

 

Conspiracy theories capture the public’s interest and imagination. It’s evident in the documentaries about flat-earthers on Netflix, BBC podcasts about the anti-vaccination movement, and the panic surrounding the ‘Momo challenge’. The anti-vaccination movement, in particular, has been getting a lot of coverage lately, because of high-profile sympathisers and the potential damage to society’s health. There’s a lot of public and media interest in understanding this very modern-seeming phenomenon. But conspiracy theories are not new and neither are anti-vaccination movements.

Vaccination stories from 19th century British newspapers

The 19th century had an anti-vaccination movement which organised meetings, wrote letters and even paid the fines of those convicted of refusing to have their children vaccinated. They wrote letters denouncing enforced vaccination, arguing that it was an encroachment by the government on civil liberties, and that the vaccination was as or more dangerous than the disease it sought to prevent. They produced pamphlets and political cartoons. The movement spoke to fears about overreaching state power and technology encroaching on personal freedom and an imagined pastoral idyll. On the pro-vaccination side, the debate used science and statistics to prove that vaccines were necessary, and argued that they were compulsory because they ensured the safety of all, especially the weak.

It was long known that infecting patients with a mild dose of smallpox led to them developing resistance to the deadlier strains (apparently some places had a tradition of blowing powdered smallpox scabs up the noses of patients to inoculate them - another reason to be grateful for the advances of science). In 1796 Edward Jenner ‘discovered’ that those infected with cowpox (a very mild disease) also developed resistance to smallpox. He developed the world’s first vaccination: the word comes directly from the cowpox method used – vacca is the Latin word for cow. Jenner’s vaccine spread in popularity and was made compulsory in several European countries, including England in 1856. Children were to be vaccinated within six or seven months of birth, and a fine of up to £2 would be given in the event of failure. Failure to pay the fine could mean, eventually, a prison sentence. In 1867, another bill was introduced requiring re-vaccination after puberty. It was at this point that the anti-vaccination movement took hold.

The debate played out in the newspapers: there were articles and letters to the editor arguing both sides. The controversy even affected newspaper advertisements: entrepreneurs advertised ointments which supposedly eased the skin complaints of those recently vaccinated:

Nairnshire Telegraph and General Advertiser for the Northern Counties, 28th September 1859, via British Newspaper Archive

But the same advertisement is found in papers all over the country:

Coventry Evening Telegraph, 23 May 1892, via British Newspaper Archive

It’s hard to imagine a movement of this type existing without easy access to mass communication. Letters to the editor, for example, proved an easy way for those with fringe views to put their opinions on an even footing with more commonly-held opinions. This access to a platform allowed the movement to assume an authority it would not otherwise have had: access to the same media as mainstream material can mean that both sides of an argument are equally valid, even when then isn’t the case.

But how big was the conversation about vaccines, exactly? Looking at a large sample of newspapers published over the period can give us some clues. This data is from a set of around 62,000 19th century newspaper issues held by The British Library and digitised with JISC funding. It’s a simple approach: counting the relative frequency of a word over time can give an idea of how important the topic was at any time, although it doesn’t tell us anything about why it was being discussed or in what way. It also misses out alternative spellings or mis-spellings. But it can help us to identify general trends.

Unsurprisingly, we see some spikes. There are some small spikes in mentions of vaccination at the time the compulsory bill was introduced in 1856, and again for the re-vaccination bill in 1867. The interest in vaccination itself doesn’t really come until about 10 years later: an interesting indication, perhaps, of the lag between the conversation about the disease taking off, and policy (in the form of a compulsory vaccination bill) being formed. The first real spike of interest is in the early 1870s, and here I think we can detect the anti-vaccination movement. The mentions of vaccinations in this second spike are more related to the debate – on both sides. There are times, around 1888 and again in about 1896, when mentions of the disease are not really followed by mentions of vaccination. These may be times when enthusiasm for anti-vaccination groups falls on account of fears for the disease itself.

The debates themselves played out on the pages of the regional and national newspapers. They were bitter, and echoed those of today. A reprinted letter from The Lancet sums up some of the frustration on the side of the pro-vaccination:

The members of this league have some “talents for mischief,” not from the facts which the adduce, which are too insignificant to be noticed, nor from the arguments which they employ, which if they were only addressed to reasoning minds, would assuredly be recognised as puerile and contemptible, but these gentlemen wield more powerful arguments in support of the cause which they advocate. These are the hackneyed appeals to the ‘liberty of the subject: the resistance to a tyrannous enactment, and the publication of “striking” and dreadful cases of disease, and even death, as the results of vaccination.

Then, as now, the scientific and medical communities were frustrated by arguments invoking more abstract ideas: those that appealed to emotion over reason.

The insensibility of many persons to the danger of smallpox, and to the value of vaccination as a preventive, appears to arise from two causes; of which one is total ignorance of the horrors of the past, and the other is scepticism as to the representations of those who are well informed.

The author of an article in the Edinburgh Medical Journal, Dr. John Gairdner, used historical arguments to appeal to reason. He searched the archives to produce a list of royal family members who had died from smallpox. The influence of the monarchy on ordinary people was also used in other ways to promote vaccination: In February 1871 the Manchester Evening News reported that “The Queen has been revaccinated and wishes it to be generally known”. Perhaps these more narrative-focused, non data-driven arguments were seen to have more influence than statistics.

The anti-vaccination side had three main tactics. First was picking statistics which supported their argument. Second was appealing to arguments about personal freedom. In 1882, one letter to the editor of the Derby Chronicle tried to reason that vaccination should not be compulsory because the disease didn’t affect those already vaccinated:

When Mr. Cotteman has proved that doctors have a moral right to scratch us with a pin from which evil effects may follow, he may be able to prove that they have a right to insist upon vaccination. Yet this would be superfluous, since vaccination is a protection in his estimation. The protected being safe, why compel objectors?

This argument, of course, overlooked those who were unable to get vaccinated for health reasons, or the small percentage on which the vaccination had no effect.

The third tactic was supplying anecdotal evidence of individual cases where the vaccine had disastrous consequences. A writer to the Leicester Chronicle wrote in to describe a child that had been recently vaccinated, saying that it had been ‘fine, fair and healthy looking’ but after vaccination was covered all over with sores, “so much so that it is repulsive to see the poor thing”.

These groups were often hyper-local. Groups like the ‘Darlington Anti-Vaccine League’ had regular meetings and advertised them in local papers. The debate played out in the pages of the regional papers, rather than through national, official channels.

We can use news data to get some insight into the changing perceptions of the word ‘vaccination’. These word clouds illustrate the words that most commonly appear in sentences with the term:

In 1856, the words are mostly related to the financial and administrative aspects of vaccination. Thirty years later, the mostly commonly associated words have become a mix of administrative-type words, and some terms which clearly relate to suspicion and controversy surrounding compulsory vaccination. The conversation in the newspapers about vaccinations clearly changed in the intervening years. Now vaccination is mentioned with ‘child’ and ‘children’. It doesn’t prove that the conversation was negative, but it does show that newspapers were commenting on the more human element of vaccination. It’s a personal as well as a public conversation.

Compared to the fear of cholera, the attention given to smallpox by the newspapers was small, and despite spikes at the end of the century (when a ‘conscientious objector clause’ was inserted into a new vaccination bill), generally interest in the controversy surrounding vaccinations waned. What a good conspiracy really needs is air: studies have shown that more we are exposed to an idea, the more likely it is we’ll believe it is true, regardless of the evidence we’re given. It’s possible that the anti-vaccination movement lost steam because it wasn’t being talked about in the newspapers any more.

The debate surrounding smallpox vaccination tells us something about the ways in which information and communication can be used to spark debates that previously would have stayed hidden. Cultural movements, however small, are often facilitated by the expansion of access to new technology (such as newspapers in the latter half of the 19th century, or the internet at the beginning of the 21st). When these technologies reach a critical mass, they expand the ‘public sphere’ to take in the viewpoints of the minority - even when those views cause us discomfort. Opposition to anti-vaxxers proved difficult: work like Gairdner’s book might have helped to counter the movement in a way that statistics themselves didn’t seem to. Time was the best opposition: in the long run, it seems that the movement against smallpox vaccination simply petered out. Smallpox vaccinations continued, and a worldwide programme led to the virtual eradication of the disease by 1980.

It may be surprising to see such strong opposition to vaccination in a world with such a terrible problem with disease. Today these diseases can seem far removed from our lives, but in the 19th century the evidence was so incredibly clear: smallpox infection rates plummeted in areas with vaccinations. People lived with the fear and threat of infectious diseases, and most families would have been affected, at some point, by diseases like smallpox. Despite this, there was still resistance to compulsory vaccination. Even in the face of overwhelming evidence, when the alternative was a very real chance of disfigurement or death, illogical viewpoints can take hold.

Without an outlet like a regional newspaper or Reddit forum, these fringe viewpoints can often stay buried. It’s only when a place is found for them to be debated that the ideas can really spread. Regional newspapers allowed the debate to reach all parts of the United Kingdom and helped the creation of hyper-local interest groups. Today, the internet allows for the spread of ideas to any part of the world, in a very short space of time. Fringe movements can reach a critical mass even though their number in any one area may be tiny. Do new technologies breed conspiracy theories? Is the debate related to the ease with which people can communicate over long distances, to a large group of people? Does the democratization of media bring together communities of like-minded individuals, and what consequences does this have for society? These are crucial questions of both the 19th century and our own.

"The race of mankind would perish”, wrote a correspondent to the Isle of Wight Observer in 1856,

did they cease to aid each other. From the time that the mother binds the child’s head, till the moment that some kind assistant wipes the death-damp from the brow of the dying, we cannot exist without mutual help. All, therefore, that need aid, have a right to ask it of their fellow-mortals; no one who holds the power of granting can refuse it without guilt.

Those in favour of vaccination would argue that herd immunity ensures the safety of all: claiming a right personally to refuse vaccination means increasing the danger to those who are unable to get protection through no fault of their own. The debate about personal freedom and public good still continues.

Links:

• There has been some interesting work on disease done by researchers at Lancaster, mapping mentions of cholera in 19th century: https://www.lancaster.ac.uk/fass/projects/spatialhum.wordpress/?page_id=652
• Article about opposition to smallpox vaccination, with historical information about the disease: https://www.researchgate.net/publication/233898083_Smallpox_vaccination_and_opposition_by_anti-vaccination_societies_in_19th_century_Britain

Yann Ryan

Curator, Newspaper Data

 

How do I love thee? Let me %>% group_by (ways) %>% count()

Counting is very simple. We’ve been doing it for 50,000 years. One of the first things we learn as a child is how to count: before or at the same time we learn the alphabet, we learn to count to ten. First we learn to count on our fingers, perhaps next we count on an abacus. Eventually we graduate to counting on a calculator or a computer. Computers are very good at it, too, which is useful. Give a computer some text, and it can really quickly count lots of things for you: things like the total number of words, the total number of characters or the number of unique words. Counting helps us do lots of useful things. Counting can help us to break codes or compress data. Samuel Morse counted the average frequency of letters in the English language and assigned the most frequent ones to shorter dot-dash combinations. Your computer is doing the same thing when it zips or unzips a file.

Corpus analysis is the study of lots and lots of words of a particular type. Google N-Gram browser finds words or short phrases in millions of digitised books. EEBO N-Gram browser does the same for millions of transcribed texts from the 17^th and 18^th century. At this scale, simple counting becomes really powerful. Using these tools, researchers can count the frequency of words, which can be the starting point for understanding how words were used and how ideas gained or lost momentum over time. These tools count the relative frequency of words: how unusual is it to have this word here? Are there many more instances of a word appearing than one would expect from the usual frequency? Simply counting can tell us the importance of terms, ideas, concepts in particular texts, or at particular times.

We can divide things up and then count them: How many times did a particular phrase appear in a particular location? At a particular time? In a particular title?

We can count counts: How many titles were printed in a particular year, and how many words did each of those titles contain?

What else can we count? How about whole documents: how many newspapers were printed in the 19^th century? How many titles? How many times was the word ‘Gladstone’ mentioned, vs ‘Disraeli’? Did mentions of ‘steam’ overtake mentions of ‘horse’? Counting can be a blunt tool, but it’s a starting point.

A couple of crude word searches using millions of pages of text from selected 19th century British newspapers

To take a concrete example: let’s do some counting on a single issue of one of the newspapers we’re digitising as part of our Heritage Made Digital project. We’ve taken the text of this issue and uploaded it to a web app called Voyant Tools. Voyant Tools takes text files and gives statistics and visualisations of the words within. What are the counts in this issue? This single issue has 29,734 words. It has 7,793 unique words, which could tell us something about the type of audience, or the ‘footprint’ of the author or title. What are the most common words?

Let’s quickly think about some of these words and their implications.

Mr tells us that news is, unsurprisingly bias towards reporting about one gender.

Street, house and place are intriguing, if not surprising. News is so much about space and place. Without a sense of time, news ceases, really, to be news. Perhaps the same can be said about news and space? 

Which leads into the next word: Jan (the abbreviated version of January). This is a newspaper from 6 January 1821. This, alongside Dec (December shortened) tell us something about the age of news. Would you expect more or less mentions of December once news is transmitted via telegraph? There’s also day and time. It’s unlikely these words would be so common in, say, a novel, or a scientific paper. Can counting tell us something about genre?

We can count the counts: Can the words be divided into categories and counted?

What does this tell us? Well, it probably tells us more about the makeup of each individual page than anything else. We could probably guess the front page by looking its unique words. The front page was often mostly advertisements, and contact details would include words like street and Mr. It also confirms our belief that news is about information in space and time: clearly there’s a focus on place, time and people, in a way that would presumably not be so apparent in, say, a novel. If we counted the change in common words over time, we could get a picture of the changing makeup of the front page, as it moved from advertisements to headline news.

Counting is a most natural human urge and one that can have very interesting outcomes. It’s a start for all sorts of interesting research: a way to make all sorts of (often wrong) assumptions. Because counting is dangerous. It attempts to put numbers on things that may not be enumerable. We may find our attempts at counting frustrated by the stubborn fuzziness of the world, stymied by our need to put order on disorder. Over the coming months we hope to show some of the interesting things that can be done with the millions of pages being digitised by Heritage Made Digital, and lots of this research will involve, at its core, counting.  

In digital scholarship, it sometimes feels like there is a move away from counting to produce results. Machine learning seems at a great distance from a chart of the most-commonly used words in a bunch of text. But machine learning still often takes a simple count as its raw material. The ‘features’ (the attributes of things we feed machine learning algorithms to make predictions about those things) are often elements like the total count of words in a particular document, or the count of unique words. No matter how sophisticated these methods get, they still, in the end, rely on counting.

Yann Ryan,

Curator, Newspaper Data

Here is a remarkable image. It looks, at first sight, like a collection of prosperous mid-Victorian gentlemen: perhaps a gathering of local politicians, or merchants of some kind, preening themselves with civic pride. But then we see the caption that accompanies the image: ‘The Great Pugilists of England’. They are boxers, presented in their Sunday finest. But look again to the right, and there is something we might not expect: one of them is black, presented without qualification as an Englishman among Englishmen, someone to be admired.

'The Great Pugilists of England', Illustrated Sporting News and Theatrical and Musical Review, 14 February 1863

The image comes from Illustrated Sporting News and Theatrical and Musical Review, 14 February 1863, one of the newspapers we are digitising as part of our Heritage Made Digital programme. The man is Bob Travers, a lightweight, one of a number of black boxers who fought at this time when the sport was bare-knuckle, the brutal contests could last for hours, and rounds could exceed a hundred (a round ended when a fighter fell and had to come up to a line, or scratch, for the next to begin - if one failed to 'come up to scratch', the contest was over). The police were never too far away, ready to break up what were viewed as riotous assemblies. Most, if not all the black boxers who fought in Britain in the 19^th century were, in fact, American, born slaves or the free-born sons of slaves who managed to cross the Atlantic to try their chances in a land where pugilism had an avid following.

The first such fighter may have been the man billed as the ‘Black Dynamite’, mentioned in a report in The Times on 27 April 1786. He was followed by the celebrated Bill Richmond, who came to Britain in 1777, became a servant to a peer, bought a pub near Leicester Square with his winnings as a fighter, and established himself as a trainer. His greatest protégé was another former American slave, Tom Molineaux, whose two fights with Tom Cribb in 1811 (both won by Cribb) was considered the very peak of the regency era of prize fighting. Other black fighters that followed in Britain through the 19th century, of whom we sometimes know little more than a name or a nickname, included John Augustus Edward Plantagenet Green, Frank Craig, ‘Massa’ Kendrick, Bob Smith, Sambo Sutton and James Wharton.

Bob Travers, from Illustrated Sporting News and Theatrical and Musical Review, 15 August 1863 (left) and 14 February 1863

Bob Travers was American too. Some confused contemporary sources tried to suggest that he was British born, but he was born in Norfolk, Virginia in 1832, though he came to Britain as a child, when his father moved to Truro in Cornwall and ran a shop selling crockery. He was, apparently, Charlie Jones then, but a talent for fighting, nurtured by English middleweight Nat Langham, saw him enjoy some success over a 10-year career as a pugilist under the name Bob Travers. His earliest known bout was in 1854, his last in 1863. He is best known to boxing historians for his 1860 bout against the great Jem Mace, when Travers displayed some dubious tactics (falling without a blow having been landed) and was eventually disqualified. Referred to rather disparagingly in some sources as ‘Langham’s Black’, he was more impressively billed elsewhere as the ‘Black Wonder’.

There was only a very small black population in Britain in the 19^th century. Few black figures came to the forefront of British consciousness: among them were political radical William Davidson, hanged for his part in the Cato Street conspiracy of 1820; the Chartist William Cuffay; Crimean War businesswoman Mary Seacole; circus entertainer Pablo Fanque (of 'Being for the Benefit of Mr Kite' fame); and the composer Samuel Coleridge-Taylor. Prominent black American visitors included abolitionist speakers such as Frederick Douglass and Ida B. Wells, and the Shakespearean actor Ira Aldridge. 

What seems notable about Travers, and some other black sportsmen of the mid-19^th century that were reported on in the pages of the Illustrated Sporting News and Theatrical and Musical Review, Bell’s Life and other sports-focussed papers that sprung up at this time (Sporting Life, Sporting Telegraph, Sporting Gazette), is how they were taken entirely at their own merits. Of course in real life they were subjected probably to daily racial abuse. Someone such as Bill Richmond, with his social connections (he even served as one of a group of pugilists ushers at the coronation of George IV in 1821), was unusual. For those such as Travers who toured the country, appearing before rough audiences, there would have been much to endure. Even friendly accounts in boxing memoirs of the period use language to describe him that we would now find offensive.

But in the pages of the Illustrated Sporting News and Theatrical and Musical Review things are different. Travers is one of us. More than that, he is an admired figure, a notable exponent of pugilistic science, an exponent of sporting virtue, despite what censorious authorities might say about the rough world of bare-knuckle fighting. Certainly his colour is referred to - he is variously labelled as 'The Black' or 'The Ebony Gentleman' in the characteristically florid style adopted for fight descriptions - but for most of the time he is simply Travers, or Bob.

The Illustrated Sporting News and Theatrical and Musical Review was founded in March 1862 to capitalize on the growing public enthusiasm for sport. Established by the London printer Edward Harrison, producer of numerous penny serials and weekly periodicals, the sports and games it covered included athletics, hunting, yachting, cricket, rackets, bowls, billiards, wrestling, the ring, pedestrianism, aquatics, golf, billiards, chess, cribbage, and coursing. True to its title, the newspaper also included coverage of various kinds of theatrical performance, increasingly so in its later years until the title closed in 1870.

Bob Travers in fighting attitude and private costume, Illustrated Sporting News and Theatrical and Musical Review, 19 July 1862

It specialised in wood engravings of sportsmen, many of them derived from photographs, which were sometimes sold separately and lined the walls of many a home and public house. They were literally the pin-ups of their time. Such was the importance of the illustrations that a separate version of the journal with images only, Gallery of Engravings (not held by the British Library) was also published, while the journal maintained an index of its illustrations for easy reference. Travers was illustrated on several occasions, both in fighting mode and in elegant private dress.

Travers was not a great fighter, but he was a dogged opponent, and a smart one. He was not averse to taking a fall when the fight was no longer going his way. He certainly seems to have had a sound sense of self-preservation, particularly in his latter years as a fighter, when retirement beckoned. One indication of the dangerous world in which he operated is his fight with Jem Dillon in 1863, when the newspapers reported the shocking news that Travers had died of his injuries following fifty-three brutal rounds. Two weeks later, Travers enjoyed a Mark Twain-like opportunity to tell the press by a letter that reports of his death had been exaggerated – throwing in an advertisement for his pub for good measure (like Bill Richmond, he had opened a pub, the Sun and Thirteen Cantons, off Leicester Square).

Bob Travers's comments on reports of his death, Western Gazette, 5 September 1863, via British Newspaper Archive

Travers retired from the ring at some point in the mid-1860s. Meanwhile boxing began to move away from its lawless roots. The Marquess of Queensbury’s rules, first set down around 1867, brought in some semblance of order and replaced bare knuckles with gloves. Travers was still around when the first world heavyweight championship fight, held in America between John L. Sullivan and James Corbett (the victor) in 1892, was fought with gloves. 16 years later Jack Johnson became the first black world heavyweight champion.

Maybe Bob Travers lived long enough to hear the news. We do not know where or when he died; he is last heard of in 1904. But we see him now in his prime, held up as an exemplar in the pages of the Illustrated Sporting News and Theatrical and Musical Review, taking his place among the great of England in his field. In the backwaters of the British sporting newspapers of the mid-19th century we can find some inklings of a society based not on race or class, but personal merit.

[Update: It is now known that Bob Travers died in 1918]

Notes

There is more information on Bob Travers on the Cyber Boxing Zone website, and in Kevin Smith, Black Genesis: The History of the Black Prizefighter 1760-1870 (iUniverse, 2003). On sporting newspapers of the period, see Mike Huggins, The Victorians and Sport (2004). On black Britons of the 19th century, see David Olusoga, Black and British: A Forgotten History (2017). On boxing generally, see Kasia Boddy, Boxing: A Cultural History (2008).

“I hear new news every day”, wrote the scholar Robert Burton in 1628, “and those ordinary rumours of war, plagues, fires, inundations, thefts, murders, massacres, meteors, comets, spectrums, prodigies, apparitions, of towns taken, cities besieged in France, Germany Turkey, Poland, daily musters and preparations, and such like.” For Burton, this firehose of news amounted to a “vast confusion”, though his attitude seems to have been one of wonder rather than fear.

Burton was an Oxford man, but made regular trips to London. There he would have paid a visit to the Exchange, gathering up news and gossip from the merchants crowding the surrounding streets, before moving on to St. Paul’s Churchyard, perhaps stopping to buy a pamphlet from a hawker on the way. On front of the Cathedral he might have picked up some more pamphlets from the many booksellers lining the border of its square, or a copy of Nathaniel Butter and Nicholas Bourne’s new news publication, an innovative weekly format copied from the continent, although, somewhat disappointingly, it wouldn’t have contained any domestic news.

This short walk helps us understand how Burton perceived a world of overwhelming information. But what would he have made of the 21st century? Indeed, what would he have made of the 19th? Had he been writing, say, 250 years later, in 1872, Burton would surely have been overwhelmed by the number of titles available to him on a daily basis.

A late-seventeenth-century London coffee house (Usage terms: Creative Commons Attribution Non Commercial Share Alike licence. Held by © Trustees of the British Museum)

The 19th century is a new world for me, coming from a background of 17th century newspapers. And it is a different world. There’s the name, for one thing: the Oxford English Dictionary records the first use of the word ‘newspaper’, to mean a publication of regular, periodical news, in 1688. My own work is on the first half of the 17th century, when the word ‘news-book’ was most common, as was a host of words and phrases like ‘coranto’, ‘weekly news-sheet’, ‘weekly pamphlet’ and ‘Mercuries’, with overlapping, shifting and slightly different meanings.

This naming change can be useful – it helps us to grasp the real intellectual and material differences between the news world of the 17th century and that of the 19th. Although the change was gradual and not always linear – changes and innovations often moved backwards as well as forwards – the march of progress was did eventual pick up pace. 17th century news looked very different, much like a few sheets of A4 paper folded in half, with news in a single column. It was called a news-book because it looked like a small book. The way information was organised was different, too: early 17th century news-books contained a series of paragraphs each from a particular place, recording all the news collected from that place. The invention of the ‘article’, a unit of news based on one particular subject or event, was not to happen for some time.

The evolution from one to eight columns

This categorical divide also continues with the data. I estimate there are 1,000,000 words in Early English Books Online’s entire periodicals collection. The British Library’s collection of 19th century news runs to hundreds of millions of pages (we wrote recently that the collection consists of 60 million issues, 450 million pages... perhaps four trillion words... twenty-six trillion characters…). The other seismic change is that a computer can be taught to read (with varying accuracy) 19th century news. For the 17th, it’s still very difficult.

This Optical Character Recognition is what allows me to load up the British Newspaper Archive and check if my great-great-granddad committed any crimes in 1839 (still can’t find anything), for example, or check Limerick hurling scores from 1887. This difference isn’t just trivial: it represents a complete step-change in the way we approach newspaper history. For one thing, the datasets increase in size, by orders of magnitude. I have created a dataset of about 15,000 rows, manually collected, by reading 17th century news and noting down bits of information in a spreadsheet. 15,000 rows, from about 400 newspaper issues, which took many months to create. Yesterday, a few hours, I created a dataset of N-Grams (basically combinations of words) from a single issue of one 19th century title.  It contained 150,000 rows.

150,000 rows of generated data, from one issue. Multiply that by about 250 for a weekday title, then by hundreds of titles, then by 200 years and the potential for ‘big data’ is rather astonishing. Of course, this data is not as rich with information as my humble spreadsheet, nor does it record any kind of fine-grained detail, but it does change the types of processing, computing power and storage needed, and most importantly, the types of intellectual questions that are and are not answerable. My 17th century dataset is like interviewing everyone in a small town, in some detail; the 19th century datasets we’ll be working with on our Heritage Made Digital newspapers project records the cosmos – albeit from far away. We don’t know much, but we know it about an enormous number of things. But the differences extend past volume: there is also a step-change in readership and scope.

The 19th century newspaper was everywhere. Some of the most popular 17th century newsbooks were probably printed in weekly runs of about 2,000; by 1863, the Daily Telegraph had a circulation of 120,000 per day. In 1628 Burton was overwhelmed by information in London and Oxford but elsewhere the firehose could be a drip, or a drought. By the 19th century news surged through the country’s arteries, veins and capillaries: at first everywhere within the reach of the train; eventually the telegraph, information finally travelling at the speed of light, in dots and dashes. It was the most pervasive cultural object of the century.

Newspaper titles held by the British Library, year by year, 1621-1900

Even accounting for the reuse and sharing of copies this is a fundamentally very different type of cultural artefact. If I analyse every page of news in the early 17th century, I have a vast record of events, and the thoughts and feelings of a select group of people. In the 19th century, the newspaper is a reasonable proxy for the way society thinks. To me it seems as though news in the 19th century captures a good proportion of a collective consciousness. It is a reasonable (though problematic) way to infer societal change. Through the newspaper’s great reach we can understand historical forces. The articles and personalities in the 19th century newspaper can tell us about structures of power. Its advertisements identify trends, economic forces and the changing roles within the family. The words themselves and their frequencies can help us understand the use of language, or uncover drifts in sentiments towards political movements, ideologies and so forth. In the 17th century the readership is so small, such a small part of the diet of information ingested by both important and ordinary people, that the questions we ask of its remains are different. Not less important, certainly not less interesting, but surely of a different kind.

Yes, the 19th century news world feels like a different one to the 17th. A mostly new world, with some evidence of the ruins of its earlier civilisation: the old towers are fallen, though echoes of their presence remain. The vast confusion had been replaced with one infinitely greater. Our job is to find, research and understand the new techniques that are necessary to make sense of this information overload.

Yann Ryan

Curator, Newspaper Data