Garbage in, garbage out: Canada’s big data problem

2016/08/24 Leave a comment

A reminder that despite the restoration of the Census, there still remain significant gaps in the collection, methodologies and dissemination of statistical data by the government:

In a recent article in the Toronto Star, Paul Wells lays out what he sees as Prime Minister Trudeau’s game plan for slowing Canada’s brain drain and making science pay. “Over the next year,” he writes, “the Trudeau government will seek to reinforce or shore up Canada’s advantage in three emerging fields: quantum tech, artificial intelligence and big data and analytics.”

As he should. If that’s the plan, it’s a good one. Canada’s future prosperity depends on our ability to innovate and retain the best talent in those three fields.

What we call “big data analytics” works by finding previously unknown patterns in the huge blocks of data that very large organizations — governments, for example — grow around themselves constantly, like coral. Finding those patterns can point the way to new efficiencies, new ways to fight crime and disease, new trends in business. But as with any complex system, what you get depends on what you put in. If the inputs aren’t accurate, the results won’t be, either. So before we embrace the “big data revolution”, we may want to look first at the worsening quality of the data our federal government produces, and that businesses, activists and social planners use.

Take something as basic as divorce. Statistics Canada first started reported marriage rates in 1921, divorce rates in 1972; it stopped collecting both data streams in 2011, citing “cost” concerns.

Marriage and divorce rates are exactly the kinds of data streams consumers of big data want collected, because they affect so many things: government policies, job markets, the service sector, housing starts — you name it. Having abandoned the field now for five years, StatsCan’s data volume on marital status isn’t nearly as useful as it might have been.

Take wildlife conservation. Recently an Ontario provincial backbencher proposed a private members bill to allow for unlimited hunting of cormorants. The bill’s proponent says the species is experiencing a population explosion. And we don’t know if he’s right or wrong — because the feds stopped collecting that data in 2011.

open quote 761b1bCanada used to publish statistical reports that were every bit as good as the Americans’ — in some cases, better. Then we stopped.

Here’s another big data blind spot: gasoline imports. After having reported data on gasoline imports regularly since 1973, StatsCan has been suppressing the numbers since 2013 due to what it calls “privacy” concerns. In the last reporting year, 2012, a staggering amount of imported gasoline came into the country — almost 4 billion litres.

Now, if you were thinking of expanding your oil refinery, or wanted to know more about how dependent this country is on foreign fuel, this would be pretty precious data — the kind you’d probably pay for. But the data aren’t reliable — any more than the StatsCan data on gasoline demand by province, which we use to work out whether carbon taxes are actually reducing demand for gasoline. It’s bad data; it has been for years. You’d think someone in the higher echelons of the federal or provincial governments would get annoyed.

Combing through StatsCan’s archive of reports can be a bewildering experience, even for experts. Its online database, CANSIM, is easy enough to use. It’s the reports themselves that sometimes fail you.

Say you want to understand trends in Ontario’s demand for natural gas. You’d start by looking at CANSIM table 129-0003, which shows an increase in sales of natural gas in 2007 over 2006 of 85 per cent. “Ah,” you think to yourself, “that must be because of the conversion of coal-burning plants to gas.” But no, that change occurred years later. Ask StatsCan and they’ll tell you that they changed their methodology that year — but didn’t bother re-stating the previous years’ numbers under the same methodology. Individually, the numbers are accurate — but the trend stops making sense.

StatsCan changed its methodology again this year; it now warns researchers to take care when comparing current and historical data. That’s an improvement over changing the methodology without telling anyone but it isn’t very helpful for understanding long-term trends.

And this isn’t just StatsCan’s problem. The National Energy Board published an excellent report showing where Canada’s crude ends up in the United States. Industry analysts use the numbers to understand the reasons why light and heavy crude are selling for what they’re selling for south of the border.

The NEB stopped reporting the data after September 2015. Ask why, and this is the response you get: “The Board has decided to discontinue publication of this data while we re-evaluate our statistical products.” That, of course, was a year ago.

Source: Garbage in, garbage out: Canada’s big data problem

Filed under Government Tagged with , , ,

Public servants scramble to fill data deficit on Liberals’ priorities

2016/06/23 Leave a comment

Understandable given difficult cut choices recommended by the public service and approved at the political level (with the previous government’s anti-evidence and anti-data bias), with predictable impact on the quality of analysis:

If Prime Minister Justin Trudeau really is a data geek, he couldn’t have been encouraged by what some federal departments had on hand.

Internal documents obtained by the Star suggest years of belt tightening has led to a data deficit in Ottawa, gaps that may “create challenges” in delivering on the Liberal government’s priorities.

Early childhood learning and child care, expanding parental leave, increasing youth employment, and expanding training for apprentices and post-secondary students all figured prominently in the Liberals’ election platform.

But as of November, the department responsible for making good on those promises was worried they didn’t have enough concrete data to deliver.

“Spending on surveys has been reduced over the last several fiscal years and has been concentrated on priority areas to help manage financial pressures,” read documents prepared for the senior public servant at Employment and Social Development Canada (ESDC).

The Liberals have made “evidence-based decision-making” a watchword for their early days in office, and senior staff in the Prime Minister’s Office are known for their attachment to data-driven strategy.

A spokesperson for Families, Children and Social Development Minister Jean-Yves Duclos said the issue is government-wide, not isolated in their department.

“This is an issue that all ministers are facing right now. We do know that there are gaps in the data the government owns,” Mathieu Filion told the Star in an email.

“There are many discussions on the matter with different minister’s offices as to see what will be done to acquire more data.”

According to the November documents, Statistics Canada was largely preoccupied with the restoration of the long-form census, but had identified a number priority files.

Along with ESDC, StatsCan was looking to revive “longitudinal surveys” to fill in gaps. Longitudinal surveys are more expensive and time consuming than other methods of collecting data, but the documents suggest they can give greater insight into “the dynamics of life events” and have a greater payoff when continued over a number of years.

StatsCan’s wish list includes greater labour market information (specifically aboriginal participation, unpaid internships, temporary foreign workers, and worker mobility), better information on children’s physical and mental health development, and more data on Canada’s aging population and the resulting effect on the economy and the health-care system.

The agency says the digital economy remains largely in the dark, as well.

“The use of digital technologies is an important and growing phenomenon and stakeholders are increasingly demanding statistical products to address questions on the topic,” the documents read.

“While the agency has been doing some feasibility work on Internet use by children, the incidence of cybercrime amongst Canadian businesses, and has developed some questions for the inclusion in various surveys, there remain important data gaps.”

ESDC is also interested in learning more about Canadians’ “computer literacy” and use of the Internet.

Source: Public servants scramble to fill data deficit on Liberals’ priorities | Toronto Star

Filed under Government Tagged with , ,

How the Big Red Machine became the big data machine: Delacourt

2016/05/24 Leave a comment

As someone who likes playing with and analyzing data, found Delacourt’s recounting of how the Liberals became the most data savvy political party interesting:

The Console, with its maps and myriad graphs and numbers, was the most vivid evidence of how far the Liberal party had come in its bid to play catch-up in the data war with its Conservative and NDP rivals. Call it Trudeau 2.0. Just as the old Rainmaker Keith Davey brought science to the party of Trudeau’s father in the 1960s and 1970s, the next generation of Trudeau Liberalism would get seized with data, science and evidence in a big way, too.

And in the grand tradition of Davey, Allan Gregg and all the other political pollsters and marketers who went before them, this new squad of strategists set about dividing Canada’s electoral map into target ridings, ranked according to their chances of winning in them. In a 21st-century-style campaign, though, the distinctions would be far more sophisticated than simply “winnable” and “unwinnable” ridings. Trudeau’s Liberals divided the nation’s 338 electoral districts into six types, named for metals and compounds: platinum, gold, silver, bronze, steel and wood.

Platinum ridings were sure bets: mostly the few dozen that the Liberals had managed to keep in the electoral catastrophe of 2011. Gold ridings were not quite that solid, but they were the ones in which the party strategists felt pretty certain about their prospects. Silver ridings were the ones the Liberals would need to gain to win the election, while bronze ridings, the longer shots, would push them into majority government territory. Steel ridings were ones they might win in a subsequent election, and wood ridings were the ones where the Liberals probably could never win a seat, in rural Alberta for instance.

The Console kept close track of voter outreach efforts on the ground, right down to the number of doorsteps visited by volunteers and what kind of information they had gathered from those visits — family size, composition, political interests, even the estimated age of the residents. By consulting the Console, campaigners could even figure out which time of day was best for canvassing in specific neighbourhoods or which voters required another visit to seal the deal.

When the Liberal team unveiled the Console to Trudeau, he was blown away. He told his team that it was his new favourite thing. He wanted regular briefings on the contents of the program: where it showed the Liberal party ahead, and where fortunes were flagging and volunteers needed to do more door-knocking. Actually, he wondered, why couldn’t he be given access to the Console himself, so that he could consult it on his home computer or on his phone while on the road?

And that, Trudeau would say later, was the last he ever saw of the Console. “My job was to bring it back, not on the analysis side, but on the connection side — on getting volunteers to go out, drawing people in, getting people to sign up,” Trudeau said. Clearly he was doing something right on that score — Liberal membership numbers had climbed from about 60,000 to 300,000 within Trudeau’s first 18 months as leader.

Volunteers for the party would learn — often to their peril — that the leader was fiercely serious about turning his crowd appeal into useful data. Trudeau wasn’t known for displays of temper, but the easiest way to provoke him was to fall down on the job of collecting data from the crowds at campaign stops. Few things made Trudeau angrier, for instance, than to see Liberal volunteers surrounding him at events instead of gathering up contact information. “That was what I demanded. If they wanted a visit from the leader they had to arrange that or else I’d be really upset,” Trudeau said.

Source: How the Big Red Machine became the big data machine | Toronto Star

Filed under Government, Polling Tagged with , , ,

Charts: No, the Y-Axis Doesn’t Always Need to Start at Zero | Re/code

2015/12/09 Leave a comment

Having spent more than a year on finding the right charts for my book, Multiculturalism in Canada: Evidence and Anecdote, I liked this little video on when to use the 0 and when not (also applies to the x-axis, where for median income data, starting at 0 made no sense):

If you write things on the Internet, angry email is inevitable. If you make charts on the Internet, angry email about those charts is inevitable. Especially if your charts sometimes use a y-axis that starts at a number other than zero. You see, an old book called “How to Lie With Statistics” has convinced people that truncated axes are a devilish tool of deception.

The truth is that you certainly can use truncated axes to deceive. But you can also use them to illuminate. In fact, you often have to mess with the y-axis in order to craft a useful image — especially because data sometimes reaches into negative territory and sometimes goes nowhere near zero. The right principle is that charts ought to show appropriate context. Sometimes that context includes zero, but sometimes it doesn’t. It’s long past time to say no to y-axis fundamentalism.

Source: Charts: No, the Y-Axis Doesn’t Always Need to Start at Zero | Re/code

Filed under Other Tagged with ,

Long-form census could be reinstated for 2016, experts say

2015/10/23 Leave a comment

An early test  of the incoming Liberal government, one that looks like it could be done:

The return of the long form, promised by Justin Trudeau during the election campaign, would yield vastly more reliable data and cost less than running another national household survey, the former heads of the agency say.

“It should be possible. I am certainly very hopeful. But [the decision] needs to be done very soon. It’s an enormous logistical operation,” said Ivan Fellegi, chief statistician from 1985 to 2008.

It’s “no problem” to reintroduce the long form in time for the 2016 census, said Munir Sheikh, head of the agency from 2008 to 2010. The questions needn’t change, he said – just the instructions at the top. “All they need to do is put on the front page that this is mandatory.”

The other step is for “cabinet to approve it as a census, which they can do at any time – it would take a matter of seconds.”

Researchers are already pressing for action. “Undoing these mistakes cannot wait; the time for action is now as Statistics Canada is on the cusp of launching the 2016 census,” says a letter signed this week by 61 academics and directors of research centres, including Statscan’s former assistant director Alain Bélanger.

Issuing an immediate order in council “is the only way to implement the long form in time for the census six months from now,” they said. “This must be one of the first moves made ​​by the Liberal government of Mr. Trudeau. It would mark a clear break with the previous government and ensure that future social policies can be made on scientific grounds rather than ideological dogmatism.”

….The Liberal platform pledges to “immediately” restore the mandatory long form – and make Statistics Canada “fully independent.”

Mr. Sheikh, who resigned over the controversy in 2010, said having the agency operate at arm’s length to the government is an even more crucial step. “I would say that is more important than restoring the long-form census, because that really was the cause of the problem, that the government can interfere with Statscan on issues like this.If you have an independent agency, the census in the future wouldn’t be the cabinet or minister’s problem, it would be the chief statistician’s problem.”

Mr. Sheikh said “anyone who uses data” will benefit from the return of the census. The biggest beneficiaries would be governments at all levels, “which have to base their policies on reliable data. And then of course researchers, who use this data to determine social outcomes, the condition of households in terms of income, poverty, unemployment, the state of housing, transportation needs, the needs of ethnic minorities, language, the employment equity act. Any kind of social and economic policy issues you can think of really are related to the census.”

As well, “the census provides an anchor to all other surveys, will have much more reliable data to check all other survey results against that.”

Both former chief statisticians said the switch could save money by reducing printing costs and expenditures associated with the labour required to administer and analyze the separate household survey. The NHS was sent to about 4.5 million Canadian households while the 2006 long-form census was sent to 2.5 million dwellings. Running any census is a massive undertaking that typically takes years to plan. The total projected budget for the 2016 census – which had been planned as a mandatory short form and voluntary NHS – is $701.8-million.

Statistics Canada wouldn’t comment on whether it’s possible to make the changes in time for the 2016 census. “It’s a policy matter, and we can’t comment,” said spokesman Peter Frayne.

Other experts say it can be done. “It is inherently easier to return to a well-tested methodology” such as the traditional census, said Ian McKinnon, chair of the National Statistics Council. “If any statistical agency in the world can do it, Statistics Canada can.”

Reinstating the census “soon, both sends a signal of change of policy, and interest in basing policy on evidence – evidence-based decision-making, which I think is very healthy,” said Charles Beach, professor emeritus at Queen’s University and head of the Canadian Economics Association. Moreover, “doing something that is both cost effective and more useful, it’s an economic no-brainer.”

Source: Long-form census could be reinstated for 2016, experts say – The Globe and Mail

Filed under Government, Multiculturalism Tagged with , , , , ,

The Data Behind Radicalization

2015/01/26 Leave a comment

Findings of a recent study of 1,500 individuals radicalized in the US since WW II:

Compared to violent domestic terrorists on the Far Left and Far Right, Islamists stand out. They’re more likely to be young (between 18 and 28 years old), unmarried and unassimilated into American society. They are also more likely to be actively recruited to an extremist group.

But in other important ways, Islamist extremists in the U.S. as a whole — violent and nonviolent — are not so different from other extremists. People in the three groups were equally likely to have become radicalized while serving time in prison — complicating the narrative that Muslim prisoners are unusually likely to commit to extremism from behind bars — and to be composed of individuals who have psychological issues, are loners, or have recently experienced “a loss of social standing.”

“Social networks are incredibly important to radicalization, but that’s not unique to Islamists at all,” [researcher Patrick] James said. “There’s almost always a facilitator — a personal relationship with a friend or family member who’s already made that leap.”

The Data Behind Radicalization « The Dish.

Filed under Radicalization Tagged with , ,

Newer posts →