Appears more evidence-based than only bias in the examples and countries highlighted but the CBSA review and analysis of the technology and possible bias or limitations encouraging. And the kiosks are such a vast improvement over the old paper forms that always left me scrambling for a pen:

The Canada Border Services Agency is conducting a series of tests to learn if its human agents, and its passport-reading machines, are prone to discriminating against certain kinds of travellers.

The CBSA’s research to date, obtained by CBC News through Access to Information requests, suggests that most of the discrepancies in the treatment of different nationalities and ethnicities at Canada’s international airports are driven by procedures, rather than prejudice.

But border service officers did use their discretion to order secondary inspections for travellers from the Middle East, Africa and the Caribbean at far higher rates than for travellers from the U.S. or Western Europe.

The surveys also confirmed that border officers are more likely to look twice at the customs declaration of a returning Canadian traveller than that of a U.S. visitor.

And internal CBSA communications suggest that the new Primary Inspection Kiosks (PIK) that read passports at some Canadian airports may have higher error rates when processing people with certain ethnic backgrounds.

One analysis obtained by CBC News was undertaken by the border agency in response to a CTV News report in May 2018.

“The news report implied that referrals for secondary inspection were biased with respect to travelers from certain countries and regions,” says the analysis report. “As a result of the news report, the CBSA formed a task force to analyze the accuracy of these findings.”

The Air Traveler Referral Analysis was delivered to Citizenship and Immigration Minister Ahmed Hussen in July of last year.

The results seemed to bear out the news report, says the document: “The CBSA determined that the conclusions drawn by the media could be inferred from the assessment of the information provided through the ATIP (access to information) request.”

In 2017, about 3,500,000 travellers arriving at Canadian airports were flagged for secondary inspections — for immigration purposes, for customs, to pay taxes and fees or to meet other requirements regarding health or imports.

The analysis report reveals that very few travellers are selected randomly for secondary inspection — just 70,000 out of 4.2 million in 2017, less than 2 per cent of the total.

Iranians v. Icelanders

The CBSA analysis found that the rates at which travellers were referred to secondary inspection differed wildly depending on their countries of origin.

For example, Iranian travellers arriving in Canada in 2017 were on average about twenty times as likely to be referred to secondary inspections for Customs purposes — and about six times as likely to be referred for immigration purposes — as were visiting Icelanders.

A Jamaican visitor was about ten times as likely as a Dane to face a secondary inspection for Customs purposes, and almost ten times as likely to be followed up for immigration purposes.

A CBSA graph compares the rates at which air travelers of different nationalities were ordered into secondary inspection for customs purposes or to satisfy requirements of other government departments (“OGD”) such as the Canadian Food Inspection Agency. (CBSA)

Canada openly treats travellers differently depending on their passports, of course. Iranians and Jamaicans require a visa to visit Canada as tourists. Icelanders and Danes do not.

But CBSA’s analysis suggests that the discrepancies in the rates of secondary inspections are not a matter of policy.

“While Jamaican and Iranian nationals were referred more often for inspection than some other foreign nationals, looking at a macro level analysis, this report found no systematic evidence of bias,” says the CBSA analysis.

It notes that about 10 per cent of all travellers are referred and, of the 4.2 million referrals it examined, “the vast majority of referrals were for mandatory reasons, with the largest proportion having been referred by a kiosk.

“The percentage of individuals referred as a result of an ‘on the spot’ exercise of judgement by a border services officer was low in comparison.”

Many of the referrals were triggered automatically because of something in the traveller’s documents, declarations or immigration status.

A graph showing how different nationalities were referred to secondary inspection because of questions on their immigration status. (CBSA)

For example, a permanent resident arriving in his or her new country of residence for the first time is automatically referred to a secondary immigration inspection in order to confirm residency, and is also subject to a customs referral to document the personal effects they are bringing to Canada. A tourist or business traveller is not subject to either of those requirements.

Iranians travelling to Canada are more likely than Icelanders to be coming here to settle. Icelanders also are more likely to be tourists on short-term visits.

Booths flagging more people

Only about 140,000 out of two million secondary customs inspections were actually ordered by human agents acting on their own discretion. The rest were automatic, mostly ordered by machines.

The primary inspection booths flag any traveller who presents a customs declaration with irregularities — such as a person who checks the box saying they are bringing in food.

Because CBSA is aware of the tendency of kiosks to refer people to secondary inspection unnecessarily, a human officer is given the task of “referrals management”. For example, the referrals management officer might let through the passenger who checks the food box because of a candy bar, while ensuring someone bringing in raw meat is inspected.

The analysis found that those human officers overrule six out of every ten machine referrals for customs purposes.

Last spring, CBSA began adding more questions about food and plants at the automated booths — a step that the report credits with reducing by half the number of passengers being sent to secondary inspections for those reasons.

Immigration stops more ‘selective’

The 2.2 million immigration-related secondary inspections were far more likely to be ordered on a “selective” basis than customs-related cases, which were mostly “mandatory” orders for things like food inspections or payment of duties.

But even the “selective” immigration inspections were more likely to be ordered by machines (88 per cent) than by human agents (12 per cent). The main reasons booths order inspections are problems like incomplete information, stays of more than six months’ duration, or permanent resident cards close to their expiry dates.

CBSA found that an Iranian or Jamaican flagged for secondary inspection was no more likely to have been selected for a secondary inspection by a human officer exercising discretion than a French or South Korean passenger (in fact, they were more likely to have been picked by a machine).

But when looking at the total number of people arriving from each of those countries, the French and Korean travellers were much more likely to sail through the airport without being flagged — by either an automated kiosk or a CBSA officer — than their Iranian or Jamaican counterparts.

Machines bias-free?

“Officer selective referral is the only type of referral that requires ‘on the spot’ officer discretion or judgement which could potentially involve personal bias,” says the CBSA analysis.

But internal CBSA communications hint at problems that may affect kiosk machines’ even-handedness in dealing with different ethnicities.

Emails obtained by CBC News through Access to Information discuss the roll-out of electronic inspection booths at Canadian airports and early efforts to measure their accuracy.

CBC News also obtained a report entitled “Facial Matching at Primary Inspection Kiosks” that discusses ‘false match’ rates. False matches include ‘false positives’ — innocent travellers incorrectly flagged as posing problems — and ‘false negatives’ — a failure by the machine to detect such problems as fake documents or passport photos that don’t match the individual.

The documents released were heavily redacted, with entire pages blanked out. “The CBSA will not speak to details of this report out of interests of national security and integrity of the border process,” the agency’s Nicholas Dorion said.

‘I thought maybe it was just the press’

While all discussion of Canadian findings was redacted from the documents CBSA released, the documents do include some revealing emails in which the evaluation team discusses U.S. findings.

Referring to articles that suggested facial recognition technology had serious problems reading darker-skinned faces, one of the evaluation team wrote:

“I thought maybe it was just the press making a fuss and actually it’s not an issue. However … you do see that (U.S. agency) NIST has found a similar bias.

“The false match rate shows a massive increase for visa images when the imposter is from South Asia region, etc.”

“I never thought it was just press,” responds a colleague, sharing a link with another U.S. study that shows that facial recognition algorithms are wildly more inaccurate when dealing with dark-skinned travellers than with light-skinned travellers, and are also worse at assessing women.

That study found that two of the main facial recognition technologies available — from Microsoft and IBM — misidentify gender in dark-skinned individuals at 18 and seven times the error rate the two technologies experience, respectively, when assessing light-skinned individuals.

The MIT study evaluated three commercial face-scanning systems and found that while the maximum error rate for classifying the gender of light-skinned men was 0.8 per cent, the same systems produced error rates of up to 34 per cent for dark-skinned females.