New paper: how well can we predict AMR in tuberculosis samples?

This paper just published in Microbial Genomics examines how well our software tool, gnomonicus, predicts to which antibiotics a clinical sample that has been whole-genome sequenced is resistant. To do so, it implements the second edition of the WHO catalogue of resistance-associated mutations (WHOv2) which in turn we had to, in effect, translate from the original Excel and PDF report. Hence we are partly checking our implementation of WHOv2.

Now we had originally wanted to call the tool gnomon as that is the part of a sundial that casts the shadow (thereby following our timekeeping theme: clockwork, sundial) but unfortunately by the time we registered the project on PyPi that name was taken so we ended up with gnomonicus which is a bit of a mouthful.

To characterise its performance, we created a dataset of 2,663 samples that is large enough to give a reasonable idea of performance partly by ensuring there is a good mix of resistance and susceptibility for all 15 antibiotics included in WHOv2. As a further check, we also compared the results of our approach with the results of TB-Profiler which is currently probably the best known TB tool. What was very gratifying was that out of the 13,821 predictions made by both tools, only 63 were truly discrepant (in the sense that one tool called Resistant and the other Susceptible for a drug) and when you dig into these you find that they are nearly all due to differences in the variants called by the upstream pipelines, rather than differences in how we have interpreted WHOv2.

That said, there are some clear differences in interpretation; for example we chose to return a result of Unknown if a mutation in a gene associated with resistance was detected that is not listed in WHOv2 as there is a non-zero chance that it confers resistance. This really illustrates the lack of guidance in the WHOv2 report on how to apply the catalogue in a real world scenario.

Our AMR prediction software stack (including gnomonicus and our implementation of WHOv2) is included in the Myco pipeline which is deployed in EIT GPAS — you can sign up and try it out here.