Why it’s not enough to simply restore the long-form census

2015/11/07 Leave a comment

Kevin Milligan on the longer term questions regarding how we should leverage more administrative data for future Censuses:

But how exactly should we go about repairing the damage? Census questionnaires need to be thoroughly tested, and then they must be printed. You can’t do this in a few months. According to the Huffington Post, for 2016 the government will use the already-tested questionnaire for the planned 2016 National Household Survey and simply make its completion mandatory rather than voluntary. While 33 per cent of Canadians were requested to fill in the 2011 NHS, apparently only 25 per cent will be asked in 2016. Still, if compliance rates go back to 2006 levels, this should yield a larger number of completed surveys. But much more importantly, the sample should be much cleaner because we won’t have the skewed non-completion problems that plagued the 2011 NHS.

This strategy strikes me as sensible battlefield medicine. Time is short, so the government is constrained in what can be achieved in the few short weeks before sending the census forms to the printer. Making the 2016 NHS mandatory solves the largest problem we had with the 2011 NHS. However, I hope this is just the beginning of a new conversation on the census—and data in general—rather than a one-off restoration of past practices.

In the United Kingdom in 2010, the newly-elected Conservative government also had some concerns about their census. But, instead of acting impetuously, they put in place a process to rethink how governments ought to be collecting data in the 21st Century. The initial report of this process came out in 2014, and a new “Census Transformation Programme” is at work on plans for the 2021 UK Census.

What should Canada do next? Well, the main recommendations of that 2014 UK report were to make greater use of existing data already being collected for administrative purposes and greater use of internet-based census forms. Canada was already doing both those things in 2006. But, I believe there is room for much more innovation.

I gave a guest lecture a year ago to a meeting of data librarians outlining my thoughts on the future of data in the social sciences, the notes from which can be found here. I remarked that we have more and more administrative data, such as tax, employment insurance and immigration records, at the same time as surveys (like the census) are becoming harder to conduct. If we move to greater use of administrative data, we need to be sure we properly balance privacy concerns, researcher access, cost, and data accuracy.

Restoring the mandatory basis for the 2016 survey was necessary, but also easy. The true test of the resolve of the new government on data will come in the actions they take as we begin to plan the 2021 census.

Source: Why it’s not enough to simply restore the long-form census

Filed under Government, Multiculturalism Tagged with , ,

Did losing the long-form census weaken Canada’s jobs data?

2015/02/18 Leave a comment

toronto-census-nhs-756x1024More on the Census and National Household Survey, and Kevin Milligan’s analysis of how the shift has degraded the quality of the Labour Force Survey:

I say ‘presumably’ because the most recent complete methodology document I can find is from 2008. In that 2008 methodology document, it is very clear that the Census plays a very important role in the methodology of the Labour Force Survey. In fact, a quick ‘control F’ search of the document reveals the word ‘census’ to appear 127 times. Looking through these instances, you can see the Long-Form Census was used in a variety of ways. Sometimes, it was to cross-check an assumption or a decision they made in designing the Labour Force Survey. In other places, it is clear they used the Long-Form Census explicitly to pick which households get surveyed. To give one precise example, page 19 of the methodology document explicitly references income taken from the 2001 Census—and income is not available on the Short-Form Census.

In short, I think Mr. Smith’s ‘myth’ is a miss.

I have complete confidence in Mr. Smith’s claim that the 2015 Labour Force Survey methodology now uses only the Short-Form Census for selecting and weighting households. However, it is equally clear that until now the Labour Force Survey relied on the Long-Form Census. This change raises several important questions: Why make a change? Why was the National Household Survey discarded? Is the Labour Force Survey improved by ignoring the National Household Survey? Why and how?

I have a pretty good guess why Statistics Canada discarded its own expensive National Household Survey for designing the Labour Force Survey. The reason: the response rate from the National Household Survey is low, and varies strongly within regions. As one example, above is a map made by Dwight Follick that compares the response rate at the ‘dissemination area’ level for Toronto from the 2006 Long-Form Census and the 2011 National Household Survey. Similar maps are available from Mr. Follick for Montreal, Ottawa, London, Calgary, Edmonton, and Vancouver. The pictures aren’t pretty—the response rate fell dramatically, rendering the results of the National Household Survey much less reliable.

Mr. Smith might be correct that the Labour Force Survey methodology changes aren’t large enough to make much difference. But we will never actually know, since we can’t compare the new results to the previous methodology because the 2011 Long-Form Census doesn’t exist so we can’t check.

I will continue to use and trust the Labour Force Survey, myself. However, the switch to the National Household Survey has degraded some of the tools that Statistics Canada has used until now to make the Labour Force Survey—and other Statistics Canada products—reliable.

Did losing the long-form census weaken Canada’s jobs data? – Macleans.ca.

Filed under Government Tagged with , , ,