Can COVID testing save the airline industry?
Air Canada “encouraged” by initial passenger COVID-19 testing results
The airline industry has pinned its hopes of getting rid of the COVID-19 travel restrictions and quarantine requirements on getting governments to agree on a comprehensive testing regime.
A few days ago, Air Canada reported “encouraging initial results” from a study they are running in collaboration with health science partner, McMaster HealthLabs. International passengers arriving in Toronto Terminal 1 are being asked to volunteer for a testing programme in which three tests are carried out, one at the airport, one 7 days later and a final one after 14 days.
What can be figured out from the results that they reported, one month into the study?
Lies, damn lies and statistics
Air Canada reported that 13,000 tests had been carried out and of those, fewer than 1% tested positive. More than 80% of those testing positive were identified in the initial airport test. The remainder were identified in the second test seven days after arrival. No new cases were identified in the 14 day tests.
That 80% figure is highlighted and on the face of it seems to imply that an airport only test would catch 80% of people who will eventually test positive, perhaps supporting the policy that many in the industry have suggested of doing away with quarantine requirements completely for anyone with a clear COVID test taken on arrival, or just prior to departure.
The fact that no new cases were identified after 14 days was certainly positioned by Air Canada as evidence that the quarantine period could be shortened, if clear tests at both days 0 and 7 are obtained.
I will admit to being initially very encouraged after reading this. But, being me, I had to take a closer look behind the headlines to see what the numbers really tell us.
From percentages to actual numbers
In understanding statistics, it is often very helpful to translate the percentages into actual numbers.
We know they carried out 13,000 tests in September, but how did these break down into the 0, 7, and 14 day tests? McMaster HealthLabs also issued a press release on the same day and they focused on the fact that 8,000 passengers had been enrolled in the study. That seems to imply that 8,000 took the airport test, leaving only 5,000 tests done at day 7 and 14 combined. If we also assume that people signed up evenly over the four weeks, 6,000 people should have taken the second test (anyone who signed up in the first three weeks) and 4,000 should have taken all three. That’s a total of 10,000 for the second and third tests combined, implying a 50% drop-out rate after the first test.
To summarise, we get the following estimates for the breakdown of the number of tests:
| Test | Expected | Estimated |
|---|---|---|
| Day 0 | 8,000 | 8,000 |
| Day 7 | 6,000 | 3,000 |
| Day 14 | 4,000 | 2,000 |
| Total | 18,000 | 13,000 |
Hopefully you can now see the flaw in the 80% statistic. Only 3,000 day 7 tests have been carried out compared to 8,000 day 0 tests. “80% of the positive tests were at the airport” is an incredibly misleading figure.
If 1% or 80 passengers test positive at day zero , we are told that they found another 20 new ones at day 7. But that is based on only 3,000 tests, so the actual percentage of people testing positive at day 7 after a clear day 0 test would be 20 / 3,000 = 0.7%, almost as high as the rate testing positive at the airport.
How many cases would airport testing alone miss?
Using the figures we have worked out above, if we test 8,000 people, get 80 testing positive at the airport and then do second tests for all 8,000, we expect to find another 53. So airport testing alone will miss around 40% of the cases based on these results.
That is not a surprise, it is what we would expect. As I explained in this article, the tests are not very effective during the incubation period. Missing 40% is better than the UK government’s much derided claim that “testing would only catch 1 in 14 cases”, but it is still quite significant.
How reliable are the figures anyway?
The other thing that it is easy to miss is that 13,000 tests sounds like a big, reliable, sample. But it really isn’t, for reasons I will explain..
I did a quick analysis of international flights into Toronto Terminal 1 during September. Flights from the US accounted for about 30% of seats, followed by Germany/Switzerland at 25% and the UK at 15%. 20% was made up of a range of smaller European markets and 10% from the rest of the world. It is worth pointing out there would have been very few passengers from the “high COVID” European countries of France and Spain, since Air France flies into another terminal, Air Canada didn’t operate to France in September and there were no flights from Spain.
Overall, I would estimate that the passenger infection profile was probably similar to the UK. The US is somewhat worse but Germany/Switzerland is better. The UK runs a regular programme of random testing of the population which allows us to estimate what proportion of airline passengers are likely to test positive. The most recent survey done in mid September found that 1 in 500 people randomly selected “in the community” (i.e. not in hospital) tested positive.
So, if 8,000 passengers with a risk profile similar to the UK population are tested at the airport, you would expect only 16 to test positive. If that matches what they found in the study, the number who tested positive in the second test was only 4 people.
The fact that they found nobody testing positive at the 14 day point is completely unremarkable. With 80% of people developing symptoms within 7 days and only 2,000 14-day tests having been carried out, pretty much any reasonable estimate would round to zero.
McMaster HealthLab are clearly not comfortable with the size of the sample so far. In their release they said that “No definitive conclusions can be drawn without greater enrollment and follow-up”.
So what?
I think the results so far of this study seem to be in line with what we would have expected. Maybe they are a little better, but if you are in government and responsible for public health, you might well conclude that airport testing alone will be only about half as effective as making everyone quarantine and therefore continue to reject it as an alternative. You might be convinced to allow people to end quarantine early if they get a clear test at day 7. But that is hardly going to reignite the world’s tourist business.
On the other hand, the airlines will point to the extremely low number of cases that quarantine is currently catching, both in absolute terms and compared to what could be achieved with airport testing alone. They would contrast that with the massive economic damage being done to the industry.
If the international airline business was operating at 50% of normal volumes (which it isn’t), you might get 100,000 international air arrivals a week into Canada. Based on the Air Canada test results, replacing quarantine with airport testing would risk an additional 50 imported cases a week. And even that assumes that quarantine is 100% effective, which it isn’t. That compares to Canada’s 2,000 total new cases last week.
if you work in the travel and tourism industry, the risk of a 2.5% increase in cases will probably seem like a small risk to take in order to save an entire industry.
Pubs versus planes
Many other industries are making similar cases and I’d like to think that policies would be set based on transparent and objective criteria.
The reality of course is that it is politicians that are making these decisions and closing borders against infectious foreigners probably seems a lot more palatable than closing pubs and restaurants to their own voters.
It is no surprise that the airline business has begun to shout louder in an attempt to get their case across.
