Great Britain Historical Database Introduction

Scope

As explained below, the first version of this database was constructed with a fairly specific focus: measures of localised economic distress in the period 1850-1914. Even that quite limited theme, which began with research into Trade Union-run Unemployment Insurance schemes, means that the database also includes:

• Official unemployment statistics, both pre-WW1 and inter-war
• Poor Law Statistics, again pre-WW1 and inter-war
• Records of Small Debt Cases
• Data from the Registrar General's Reports on numbers of marriages
• Wage Rates in individual towns
• Some limited information on Short-Time Working

Contextualising economic distress meant that we also gathered Employment Statistics from the census, while a separate project to create an Atlas of Industrial Protest (Charlesworth et al, 1996) led to a major database of strikes being assembled.

More recently, new collaborations and new interests have led to the addition of:

• Demographic statistics from the census, especially on age and sex structure
• Mortality statistics
• Electoral statistics

Geographical Coverage

The geographical coverage of tables in the database depends on the original source. Where the same report covered Scotland or Ireland, we have included them, but so far we have not actively sought out Scottish or Irish sources to supplement data on England and Wales. In general, data gathered by trade unions or the pre-1914 Board of Trade covers the entire British Isles; data gathered by the inter-war Ministry of Labour covers Great Britain; and data gathered by the linked census, vital registration and poor law systems covers England and Wales. In the longer term, we aim to add systematic data for Scotland but we have no intention of duplicating the work of the Database of Irish Historical Statistics at Queen's University, Belfast.

History

Until 1996 the system was referred to as the Labour Markets Database and was originally developed as part of research on the regional economies of pre-1914 Britain funded by the Leverhulme Trust between 1989 and 1991, which built on Humphrey Southall's doctoral research. The first version of the database was constructed using INGRES on the QMW Geography department's MicroVAX by David Gilbert, and was named simply 'gilbert'. Due to the limited power of the MicroVAX, each original report from the source material was held as a separate table in the database, leading to considerable complexity in the construction of time series and to the database containing over 600 tables. When the MicroVAX was closed down in the summer of 1992, this database was moved to beta, an ICL DRS-6000 Unix mini-computer operated by QMW Computing Services, and reconstituted as the 'qmwlmdb' database without modifying its structure. Note that the closure of the MicroVAX meant that we could no longer support the Hypercard-based data access method described in D.Gilbert and H.R.Southall, 'Data Glasnost: a user-friendly system for access to research databases across wide-area networks', History and Computing, Vol.3 (1991), pp.119-28.

The 'lmdb' database was created by Humphrey Southall in December 1993 as a supplement to and eventual replacement for the 'qmwlmdb' database. There were three reasons for its construction:

• The number of tables in the 'qmwlmdb' database made it difficult to manage, and difficult to interrogate; in particular, the construction of time series was complex because values of the same variable for different dates were generally in many separate tables. In general, the current database stores all records of a given type in a single table, with additional fields indicating the year and sometimes month to which the data refer. It is still possible to retrieve data in 'raw' form by selecting the year and month and requesting fields in the order they appeared in the original.
• The gazetteer system used in the original database has proved too crude, and a more sophisticated system is being implemented. This is described elsewhere.
• In the course of the transfer from the MicroVAX to beta, a number of errors and in particular missing observations became apparent; this is much more of a problem when concern is with time series for single towns rather than with cross-sectional data for regions or counties. Given that checking such an extensive system is inevitably a protracted process, the safest solution was to create a wholly new system which contains only checked data and whose tables were fully documented as they were created.

From 1995 onwards, new funding permitted the addition of substantial new bodies of data: a range of economic statistics from the inter-war period, funded by a 1995 grant to us from the Nuffield Foundation; a variety of demographic data, much of it provided by other researchers; and various tables linked to the construction of the GIS.

In the summer of 1996, the database moved again, from INGRES running on 'beta' to the Oracle SQL database running on a Sun workstation in the QMW Geography department. This move was made necessary by changes in hardware and software availability, but should permit closer integration with the GIS. As part of the move the database was renamed to reflect the wider range of data now included, and particular the addition of electoral statistics and cause of death data which took it clearly beyond a focus on labour markets. At the same time, copies of many of the tables were supplied to the History Data Service for incorporation into the Great Britain Historical Database Online, a new on-line information system being created under their JISC-funded 'History Special Collections' programme.

What's missing from the database

• Almost all of the data input in the original Leverhulme-funded project on the origins of the depressed areas is now held in 'lmdb'. The principle exception is unemployment data for the United Society of Boilermakers. We also have a long run of photocopies of unemployment statistics for the Friendly Society of Ironfounders which have yet to be input.
• A substantial body of data concerning trade unions and strikes was input as part of research for the Atlas of Industrial Protest, using funding from the Nuffield Foundation. This has still to be added to 'lmdb'.
• We expect to be adding additional census data for 1861 and 1871 supplied by David Gatley, and 1951 census data supplied by Daniel Dorling.
• Data from the Steam Engine Makers' Society reports, and related sources, are held in a separate database, 'sem'. There is also a large collection of census enumerators' book data for Bolton and the Isle of Dogs which it is not intended to add to 'lmdb'.

What's not described here

This document aims to describe all tables in the database which contain original data, or meta-data such as gazetteers. It does not, in general, describe tables created entirely from data held in other tables, many of which have been created in the course of generating results from the data.

The QMW Historical Geographical Information System

Since October 1994, the ESRC have funded the construction of a GIS closely associated with the Great Britain Historical Database. Existing historical GISs generally relate to a specific date, or to a series of discrete dates such as census years. However, the database contains statistics which were recorded for large numbers of separate geographical units once or twice a year, and we have therefore had to construct a system which continuously tracked changes in the network of reporting units. In doing this, we have relied heavily on reports of changes in the boundaries of Registration Districts which appeared in the Registrar General's reports, and have been greatly aided by a Wellcome Trust-funded project at the ESRC Cambridge Group which input many of these lists of changes. This information, specifying changes to the precise day, forms part of the GIS and permits it to construct appropriate base maps for any date within the period covered.

More documentation of the GIS is provided elsewhere.

Making use of the database

We want to work with other researchers interested in the regional economies of 19th and early 20th century Britain and related topics which require access to our data for the whole country, not just a particular locality. We will be happy to either supply data or arrange for access to our system over the Internet once we have arranged some basis for collaboration:

• If you wish to use our data to provide divisors, explanatory variables or suchlike in research whose central concern is with data you currently hold, we would expect that any resulting publication would mention the Great Britain Historical Database as the source of the information and acknowledge both ourselves, Humphrey Southall, Ian Gregory and David Gilbert, and the relevant funding body (the Leverhulme Trust for pre-1914 data, the Nuffield Foundation for inter-war data and the ESRC for the GIS). We would also expect to be sent copies of any such publications - this matters a great deal to our attempts to obtain further funding to expand the system.
• If it appears to us that a principal focus of your proposed research would be the data we were supplying, we may ask to be included as co-authors of any resulting publications. In return for this, we would be supplying not just our data but also our expertise on what the data mean: most of the data we hold is full of pitfalls for the unwary, only a few of which can be explained here.

In either case, we would expect that any related data you hold would be made available to us on similar conditions, directly or through the History Data Service. This applies in particular to data from printed census reports, the Registrar General's Annual Reports and Decennial Supplements, reports of the poor law system, labour market data from trade unions or the Board of Trade/Ministry of Labour, and electoral statistics. A wealth of geographically-located data is available for Britain for the 19th and 20th centuries, and many important lines of research can only be followed by combining data from different sources. Potential uses of machine-readable data are often not apparent to the researcher who originally organised input; for example, two of our future research plans require data on Registration District boundaries to study voting behaviour, and mortality data for research on migration. Given the very limited resources available for historical research in the U.K., the repeated inputting of the same large datasets by different researchers is an unacceptable waste, even if funding bodies will pay for it; for further discussion see the introduction to our paper 'Data Glasnost', cited above. However, fears that others will simply plunder datasets which have taken years or decades to assemble are understandable, and for this reason we seek to establish a 'circle of co-operation' within which researchers will feel able to more freely exchange data. The Great Britain Historical Database and GIS, implemented using 'industrial strength' software running on Internet-accessible multi-user systems, can hopefully make a large contribution to promoting such a collaboration.

Anyone wishing to discuss possible collaborations should contact Humphrey Southall.

Funding

Parts of the database date back to Humphrey Southall's work as a research student, funded by the Social Science Research Council. Most pre-1914 statistics were gathered through a project on 'the Origins of the Depressed Areas' funded by the Leverhulme Trust. Inter-war unemployment and poor law statistics were input using funding from the Nuffield Foundation. The Economic and Social Research Council funded construction of the GIS.

Other Acknowledgments

The existing database and GIS have already benefited from many people's efforts. We are particularly grateful to Daniel Dorling (Newcastle), Eilidh Garrett, Alice Reid and Jenny Wood (ESRC Cambridge Group), David Gatley (Staffordshire), Paul Johnson (LSE), and Clive Lee (Aberdeen) who have generously provided data, in most cases without imposing any restrictions on subsequent use. Most data entry at QMW has been the work of Pat Furniss and Maureen Watters, and the size of tables such as 'ase_mr' and 'lui' is a tribute to the speed of their fingers. The GIS is largely the creation of Ian Gregory, who has done a remarkable job of constructing a coherent system from a strange array of historical sources.

(c) Humphrey Southall, David Gilbert and Ian Gregory, 1996.

GBHD Online Documentation Home Page