In praise of induction – The Washington Post

Another way at looking at the difference between evidence and anecdote, and the merits and utility of each, by Daniel W. Dresser of  the Fletcher School of Law and Diplomacy at Tufts University:

One of the tensions that explains the fraught relationship between politics and the academy is that academics are big fans of deductive thinking and politicians are not.

At the risk of exaggerating the gap, academics like to think deductively — i.e., start from theory and then test whether that theory explains parts of the real world. When I was in graduate school, my professors talked a lot about the perils of thinking inductively — i.e., building a general theory from looking at a particular case. The obvious danger was to build a theory from a particular case, and then use that case as evidence of the theory’s power — the very definition of a tautology.

Politicians preternaturally think in an inductive manner. They build from experience, narrative and analogy to articulate what they think matters in the world of policy. For politicians, this makes a great deal of sense, because they trust their own experiences far more than abstract data, and because they know that narratives resonate far more with voters and citizens than abstract theories. Consider, for example, Chris Christie’s moving discussion of how to treat drug addicts. It’s a brilliant demonstration of a politician using a particular narrative to make a deeper point on policy.

In splitting the world like this, I’m simplifying things a lot. One could argue that Barack Obama’s problem as a politician is that he is too abstract and not inductive enough. Similarly, most scholarship emerges from the interplay of deductive and inductive thinking. But still, I think there is some truth in this dichotomy.

My reason for bringing this up is to point out that my own tribe of academics still looks down on the inductive method of theorizing as a flawed approach that is prone to error. And those flaws are real. But I fear that this has blinded many academics to the virtues of induction, because they exist. Indeed, twice in the past week, it’s come up in policy debates.

Source: In praise of induction – The Washington Post

Former CIC mandarin says several public policies came from minister’s anecdotes | hilltimes.com

Article from Hill Times today on the occasion of my book launch. Open event, The 3 Brewers, Bank and Sparks, today between 5 and 7 pm. Look forward to seeing many Ottawa-based people there. Best price for paper version of book ($15, HST and shipping included).

Andrew Griffith offers an insider’s account about the major cultural shift in the public service when the Conservatives formed government back in 2006.

When the Conservatives won government in 2006, the federal public service was not prepared for the ideological change to public policy-making, says a former top mandarin and author of the new book Policy Arrogance or Innocent Bias: Resetting Citizenship and Multiculturalism.

“One of the funny things about the relationship between the political level and official level is that we’re both equally certain in our own truth,” said Andrew Griffith, a former 30-year veteran of the public service, in an interview with The Hill Times. “A party comes in, they’ve developed a platform, they’re absolutely convinced they’re right and that they have the truth and they were elected on that platform and, similarly, we in the public service are convinced that we’re absolutely right, we have the studies, the research, the evidence—how can anybody disagree with us?”

Mr. Griffith, a former director general at the Canadian Heritage Department who worked on multiculturalism policy, is launching his new book in Ottawa on Sept. 23 at The Three Brewers, 240 Sparks St., from 5 to 7 p.m.

He moved over to the Citizenship and Immigration department when Jason Kenney (Calgary Southeast, Alta.) was named the minister in 2008 and took the multiculturalism files with him. Using his experience with implementing multiculturalism and citizenship policy, Mr. Griffith wrote an insider’s account about the major cultural shift in the public service when the Conservatives formed government.

“In this particular transition, the perspective, or worldview, of both sides was so different. We had the Calgary crowd—by and large the Conservative Party wanted smaller government, less government intervention and was more skeptical of the power of government to actually do good,” Mr. Griffith told The Hill Times in a phone interview last week. “We live in the Ottawa bubble, Central Canada, and, by and large, civil servants are small ‘l’ liberals. You know, you don’t join government because you want to shrink it generally, maybe the people in Finance do, but, generally speaking, the people who join government have a belief in the power of government to do good. It doesn’t mean they’re big government people, it’s just a different world view.”

Mr. Griffith said the differing worldviews “sharpened tensions” between the public service and the new government.

“Previous transitions hadn’t had, I don’t think, such a sharp tension. I don’t recall that during the Mulroney government, because, again, it was more of a Central Canadian government,” he said. “They came with strong ideas and knew what they didn’t like.”

In the case of multiculturalism and citizenship policy, he said, the Conservative government’s worldview was a complete departure from that of former prime ministers Pierre Trudeau, Brian Mulroney, and Jean Chrétien.

“They didn’t like much of the traditional approach in multiculturalism and everything like that, sort of the old-style focusing on visible minority issues. On citizenship, it was very clear they wanted a stronger reference to Canadian history, military, Crown, etc., and so the way they would come at the issues is we’d have a meeting, and they’d say, ‘Here’s what we want,’ and we’d initially figure it out. In many cases, it appeared very foreign to us in terms of what we knew about Canada, so it took us time to absorb it and react to it and find a way to say, ‘Now we understand it so we can actually work with you,’ ” he said.

Mr. Griffith said several of the policies generated were based on anecdotes that the minister or his staff would bring back and attempt to fix.

For example, in Policy Arrogance, he outlined that in the case of making changes to citizenship rules around “birth tourism”—or dealing with people who planned trips to Canada so that their baby would be born on Canadian soil and be granted automatic citizenship—anecdotes “trumped” evidence he said, because there was very little data to begin with.

“The minister admitted that he did not know the extent of the problem even as he made the case to crack down on birth tourism,” Mr. Griffith wrote. “Officials struggled with this lack of hard numbers as stories emerged in the Quebec and B.C. media.”

Mr. Griffith wrote that the CIC later engaged with medical associations and hospitals to “ascertain the extent of the issue,” but did not consult with provincial health systems that would have allowed them to see how many births were paid or not paid through the public system for which citizens and permanent residents are eligible.

“Such analysis would help quantify the extent of the issue, and help inform cost-benefit analysis of any change to citizenship legislation to align Canadian policy with other jurisdictions that no longer allow automatic citizenship upon birth,” Mr. Griffith wrote. “In developing policy and program advice, the paucity of data and analysis made it hard to provide advice on the likely impact of any policy changes. More, the minister’s wishes for early implementation meant there were limits to appropriate due diligence.”

Mr. Griffith told The Hill Times that public servants couldn’t discount Mr. Kenney’s anecdotes, however, because he went to at least 20 community events three weekends out of four.

“His anecdotes had a reasonable amount of weight,” he said, noting that officials did not take the anecdotes wholly; as the people Mr. Kenney was seeing was not entirely representative of the Canadian population.

“He was more in touch with the communities than we were. Our evidence tended to be large-scale research and surveys, which are very valid, and his evidence tended to be anecdotal, but it was such a large base of anecdotes that it was something that we actually had to take into account.”

When it came time to rewrite the citizenship guide, Discover Canada, the public servants working on it “didn’t get it right at all,” so the ministerial and political staff “actually wrote it for us” and the department went from there, Mr. Griffith said.

“Normally that isn’t done,” he said, adding that later, the minister’s office would have “a challenge session” going through each page one by one. “We were able to understand why they wanted it and the why is actually more important than the what because if you understand the why, then you can figure out a way to make it work. It would be difficult at the beginning … and then as you got through those discussions, you could get to more pragmatic ‘okay, now that we understand what you want, we can move in this direction.’ It served as a bit of a dance.”

Mr. Griffith said that while he was “never afraid” to give advice under these circumstances, his four years at Citizen and Immigration Canada was a “real learning experience.”

Writing that experience down “was actually satisfying and cathartic,” he said.

“My intent was actually to provoke a bit of a discussion initially within the public service about the relationship issue between the government and the public service because my sense was that we didn’t manage the relationship very well at the beginning,” Mr. Griffith said about writing the book.

“We weren’t responsive enough to the change in direction of the government so we appeared obstructive at best or resisting or even disloyal perhaps to the incoming government so I think there were some lessons learned for the public service in terms of how we manage that transition that hopefully by having a more open discussion about how we actually deal with a situation where we have an incoming government that has a very different worldview from our worldview in a way that actually doesn’t exacerbate tensions, but actually sort of helps develop a more normal working relationship.”

There was a difficult line between the public servants giving “fearless advice” and putting into practice the “loyal implementation” role, he said.

In the end, Mr. Griffith said, he felt at CIC that public servants were able to balance both, despite going through the “Kübler-Ross stages of grief and loss—denial, anger, bargaining, depression, and acceptance—in dealing with the traumatic challenge to their role, as well as to the long-standing consensus between previous Liberal and Conservative parties on citizenship and multiculturalism issues.”

Mr. Griffith told The Hill Times that, for the most part, Mr. Kenney was “actually quite good” at listening to advice, although “he wouldn’t necessarily accept it.”

While he couldn’t say whether this was widespread in other departments, Mr. Griffith said politicians are likely more drawn to anecdotes than scientific evidence and statistics because they are people’s people.

“This government is more ideological than previous governments. This government does tend to discount evidence. This government does actually tend to cut things that do provide evidence, like the census. All that’s on the public record,” Mr. Griffith said.

“How it works in individual departments, I’m not close enough to know that. I do know from some people that yes, some ministers are more receptive to listening to advice but again that always gets run by ‘The Centre’ [the PMO]. In the end, whether the minister listens or not is almost less important than whether ‘The Centre,’ i.e. the PMO, listens to it,” Mr. Griffith said.

As for whether things will change if and when a new government is elected, Mr. Griffith said it would likely be easier under a non-Conservative government.“My sense is that this Conservative government situation with the public service is probably fairly unique,” he said, noting that if the Liberals or NDP formed a government, they would likely have more confidence in the public service. “But either way, the public service has to be prepared to respond to whatever decision Canadians make at the polls. That’s always the bottom line in terms of the loyal implementation part.”

Bea Vongdouangchanh, The Hill Times, 23 September 2013

Former CIC mandarin says several public policies came from minister’s anecdotes | hilltimes.com.

Half a cheer for Jason Kenney’s revolution in immigration policy | Toronto Star

Natalie Brender in The Star on Policy Arrogance or Innocent Bias: Resetting Citizenship and Multiculturalism, focussing on the risks and limits of anecdotes for decision-making. Nice to see words like epistemological  (theory of knowledge – yes, I had to look it up too!) to capture the issues and dynamics.

In the end, I am more in the camp of anecdotes and evidence, understanding the strengths and weaknesses of each one, but using both to ensure the best possible policy outcome.  Article as follows:

Andrew Griffith, a retired senior official at Citizenship and Immigration Canada, has just published a book about the tense period beginning in 2007 that saw minister Jason Kenney bring a tidal wave of change to two federal departments. Among the many virtues of Griffith’s book, Policy Arrogance or Innocent Bias: Resetting Citizenship and Multiculturalism, is a striking commitment to epistemological modesty and self-reflection.

Throughout his case studies of various policy issues, Griffith underlines how officials working on multiculturalism and citizenship issues under Kenney were forced to confront their own latent ideologies and grapple with challenges to their expertise under a regime that broke starkly from the approach of previous governments.

From vocabulary to policy priorities to the deepest questions of what counted as sound evidence for policy-making, the Conservatives upended decades of received wisdom. For instance, Griffith reports, Kenney and his staff held in particular odium the blame-laying perspectives taken by “downtown activists” and researchers in analyzing mainstream discrimination toward cultural minorities.

An organization’s use of terms such as “oppression,” “white power” and “racialized communities” became grounds for striking it from a pool of grant applicants. This aversion was part of the minister’s larger distaste for the issue of barriers facing visible-minority Canadians, and his desire to shift focus toward discrimination within and among minority communities.

Because Griffith writes as a consummately professional public servant, he doesn’t pass explicit judgment on the policy shifts effected during the Kenney years. As he notes, it’s the job of elected officials to decide government priorities, and the job of public servants to be loyal implementers of those decisions.

On the other hand, it’s also the job of public servants to provide expert insight and advice to their ministers, who are supposed to take that advice into account in making policy decisions. It’s on this score that some of the book’s most revealing insights lie, since there was an unprecedented parting of ways between Kenney and officials on the question of what counted as sound evidence.

Multiculturalism and citizenship officials had long been used to basing their insight on social scientific research such as large-scale surveys and data collection on a range of standard topics. In Kenney, they were confronted with a minister who took his bearing from first-person anecdotes gathered from tireless meetings across Canada. (Such a minister, in the words of another official quoted by Griffith, was “like Halley’s comet, only coming by once every 76 years.”) Through the nuggets of information gained from his unmatched ear-to-the-ground contact with the nation’s increasingly suburban ethnic communities, Kenney was confident in his knowledge of their realities and concerns.

That confidence accompanied what Griffith alludes to as “the minister’s (and the government’s) general skepticism about social policy research,” and their disdain for the “downtown activists” who had forged deep ties with multiculturalism staff. Two starkly different “evidence bases,” as he puts it, were being drawn on by the political and bureaucratic levels.

Notably, Griffith does not depict the outcome as an unmitigated disaster from a policy-making perspective. Kenney was indeed gleaning real insights into experiences and concerns within different communities, which could not be captured in large national surveys or data sets. He gathered anecdotal reports on topics it had never occurred to officials to investigate systematically – for instance, on violations of citizenship integrity within certain immigrant groups in matters such as cheating on citizenship tests or so-called “birth tourism.”

Expert officials sometimes found to their surprise that the minister’s revamped multiculturalism priorities met with approval among diverse communities in the department’s focus group testing. And in Griffith’s own judgment, the anecdotal evidence that Kenney gained sometimes did produce worthwhile new directions in policy and programming (such as initiatives to address discrimination within and among ethnic groups).

For these reasons, Griffith writes, “officials had to learn to listen to — and respect — the key messages and insights coming from the minister, reflecting his anecdotes and conversations from his extensive community outreach.” It was a wrenching adjustment for many to have their expertise challenged and world views dismissed. Eventually, though, most staff took on board the insights that anecdote could offer, and worked to incorporate them into programming and policy.

There is no indication that Kenney and his staff reciprocated in the epistemological modesty department. In one exceptional instance, Griffith reports, officials found studies that managed to persuade them that racism and discrimination indeed pose real barriers to the success of certain ethnic groups in Canada. But other than that, the learning and broadening of world views seems to have been entirely one-sided.

And in the bigger picture, even anecdotes reflecting a partial reality give precious little for policymakers to go on. Stories of fraud whispered in the minister’s ear don’t tell policy makers how widespread the incidence of citizenship-test cheating or birth tourism is. They don’t tell policymakers what the relative dollar costs of taking action or keeping the status quo will be; nor do they predict what side effects might come from dramatically changing current policy.

Only careful data collection and analysis can do that. And that’s precisely what the Kenney regime (and the Harper government) couldn’t be bothered with in their haste to get tough on “abusers of Canadians’ generosity.”

Writing as a loyal civil servant, Griffith doesn’t say it explicitly, but the lessons of his book are clear. Anecdote is a lousy basis for policymaking, and modesty and self-reflection are not virtues to be expected only on one side of the relationship between the public service and politicians. As Chris Alexander takes over these files as minister of immigration, he could get a fine start by bearing those truths in mind.

Half a cheer for Jason Kenney’s revolution in immigration policy | Toronto Star.

Column: What’s the evidence for evidence-based policy?

William Watson raises some valid and important points about evidence-based policy and the limits. While some data and evidence is largely neutral and firm (e.g., Census data) other evidence can be subject to confirmation and other biases, in addition to the limits of our understanding of the complexity of society and behaviour. Evidence is still better than anecdote, but it limits also need to be understood. #W2P #GOC

Column: What’s the evidence for evidence-based policy?.