By VASANTH VENUGOPAL MD and VIDUR MAHAJAN MBBS, MBA
What can Artificial Intelligence (AI) do?
AI can, simply put, do two things – one, it can do what humans can do. These are tasks like looking at CCTV cameras, detecting faces of people, or in this case, read CT scans and identify ‘findings’ of pneumonia that radiologists can otherwise also find – just that this happens automatically and fast. Two, AI can do things that humans can’t do – like telling you the exact time it would take you to go from point A to point B (i.e. Google maps), or like in this case, diagnose COVID-19 pneumonia on a CT scan.
Pneumonia on CT scans?
Pneumonia, an infection of the lungs, is a killer disease. According to WHO statistics from 2015, Community Acquired Pneumonia (CAP) is the deadliest communicable disease and third leading cause of mortality worldwide leading to 3.2 million deaths every year.
Pneumonias can be classified in many ways, including the type of infectious agent (etiology), source of infection and pattern of lung involvement. From an etiological classification perspective, the most common causative agents of pneumonia are bacteria (typical like Pneumococcus, H.Influenza and atypical like Legionella, Mycoplasma), viral (Influenza, Respiratory Syncytial Virus, Parainfluenza, and adenoviruses) and fungi (Histoplasma & Pneumocystis Carinii).
Can radiologists find the infectious agent for pneumonia?
While lab tests like sputum microscopy and culture, rapid antigen tests and gene amplification tests are the cornerstone for the identification of causative organisms in pneumonia, there have been attempts to do this differentiation based on CT scans of the chest. The commonly referred to “typical” bacterial pneumonias are known to present with findings such as classic lobar consolidation, air bronchograms, centrilobular nodules and in some cases, pleural effusions. There is a great degree of overlap between findings of the less common “atypical” pneumonias including Mycoplasma pneumonia, Legionella pneumonia and the plethora of viral pneumonias. The findings can include one or more of the following: bronchopneumonia pattern with centrilobular nodules, bronchial wall thickening, ground glass opacities in diffuse, bilateral patchy distribution or in some cases like varicella pneumonia randomly distributed nodules and also the more rememberable “crazy paving pattern”. The CT findings of the COVID-19 pneumonia that have been described in literature, mostly from hospitals in China, are a subset of these findings with not a single specific diagnostic or “AuntMinnie” finding. For even an experienced chest radiologist, these patterns are not enough to suggest the causative etiology.
So, can AI help?
This inability to distinguish causative organism by looking at a CT scan, is seen by many – more commonly by proponents of deep learning – as a limitation of the pattern recognition capabilities of human readers (refer to above – a task AI can do, but humans can’t). As shown by a recent study titled “Artificial Intelligence Distinguishes COVID-19 from Community Acquired Pneumonia on Chest CT” published in the leading journal Radiology, Artificial Intelligence might be able to distinguish pneumonia caused by COVID-19 from CAP on Chest CT scans. The key strength of this study is the use of a large number of RT-PCR proven COVID-19 scans (1,165 scans), CAP (1,560 scans) and non- pneumonia scans (1,193 scans) for training a 3-dimensional convolutional neural network, aptly called ‘CovNet’. The results of CovNet on their test set, which comprised of a random sampling of 10% cases from the overall dataset, are very impressive with a sensitivity of 90% and specificity of 96% for diagnosing COVID-19 (with an Area Under the Curve (AUC) of 0.96) and a sensitivity of 87% and specificity of 92% for CAP diagnosis (AUC of 0.95). Naturally, a system with such high ability to diagnose COVID-19, should be used unequivocally everywhere. But as they say, the devil almost always lies in the details.
But, why can’t it help?
There are two main problems with this study, the first pertains to the case mix of the data itself. The attached figure shows the breakdown of the training and test data by etiology – note that the COVID-19 cases (1,165) far outweigh the number of confirmed viral pneumonia (24) and confirmed bacterial pneumonia (72) cases. In fact, most of the CAP dataset is comprised of ‘unknown’ etiology, i.e. 1,212 cases. A review by Zhu et al. shows that viruses (in the pre-COVID-19 era) cause ~15% of CAP in China – implying that most of the 1,212 cases were probably of non-viral etiology. So, as pointed out in this Twitter post, is the algorithm simply learning to distinguish viral from non-viral pneumonia? If so, it raises serious concerns about the claim of AI being able to ‘diagnose COVID-19’.
The second issue is the lack of testing on a ‘truly’ independent test set – a dataset from an separate institution which has not been ‘seen’ by either the algorithm or the researchers who have built it. Overfitting of algorithms to the training data, leading to subsequent high performance on randomly sampled test data (from that very same data pool) is a known problem within deep learning circles – and can also be inferred from the fact that algorithms developed as part of numerous competitions and challenges involving medical imaging have never seen the light (or darkness) of a radiologist’s reporting room. This is not to take away anything from the researchers who have toiled hard during the difficult times to bring out this work – putting together such a massive dataset, then training a deep learning algorithm on it, and subsequently publishing the results, when the world around you is collapsing – is no easy feat.
It must help somewhere…
For what its worth, both ACR and CDC have advised against the use of CT as a first-line tool for the diagnosis / screening of COVID-19. But does that mean CT has no role to play? Of course not! CT is an essential tool in the armory of pulmonologists and intensivists and serial CT scans will obviously be done for patients with any kind of severe pneumonia – so it would be silly to assume that CT no role to play in the clinical course of a COVID-19 patient. And this is precisely where AI can help.
While diagnosing COVID-19 on CT might be hard (as yet) for AI, it would be interesting to see how the dynamics change when going through an epidemic of COVID-19 – is every viral pneumonia suspected to be COVID-19? Does the incidence of other viral pneumonias reduce? These are not easy questions to answer. That said, picking up signs of viral pneumonia, or even pneumonia in general, and quantifying them over serial CT scans of the same patient, would be truly value additive, especially in resource-constrained locations or situations. Another potential use case, which is again something that humans may not be able to do, is predict prognosis and/or clinical course of certain patients based on serial CT scans – this would be able to not only guide treatment but also help triage cases in a situation where healthcare systems are over-burdened.
In summary
The role of CT in COVID-19 is constantly evolving – when RTPCR kits were not available, it was part of the initial workup / screening of patients with suspected COVID-19, but now with the increased availability of RTPCR kits, CT’s role is moving to estimation of disease severity and progression. AI tools built to diagnose COVID-19 infection need to be evaluated robustly since current radiological knowledge does not have specific features that can help identify COVID-19 infection on CT. That said, AI tools have a vital role to play in quantifying disease progression and possibly predicting disease outcomes.
Vasanth Venugopal, MD is a radiologist who leads imaging research at CARING with expertise in developing novel validation and deployment methodologies for Artificial Intelligence algorithms for medical imaging.
Vidur Mahajan, MBBS, MBA is a physician-MBA who leads research & development for CARING who is passionate about bringing advanced technologies in imaging and genomics into the clinical workflow through a single unifying technology interface.
The post Can AI diagnose COVID-19 on CT scans? Can humans? appeared first on The Health Care Blog.
from The Health Care Blog https://ift.tt/2QFGR7B
via IFTTT
No comments:
Post a Comment