Is Babylon Health’s AI on par with a human diagnostician? Claim questioned in ‘The Lancet’.

In July, Babylon Health released the results of their testing against the MRCGP (Member of the Royal College of General Practitioners) exam based on publicly available questions. As we reported at the time, its AI system passed the exam with a score of 81 percent. A separate test where Babylon worked with the Royal College of Physicians, Stanford University and Yale New Haven Health subjected Babylon and seven primary care physicians to 100 independently-devised symptom sets. Babylon passed with an 80 score.

Now these results are being questioned in a letter to The Lancet. The authors–a medical doctor and two medical informatics academics–argue that the methodology used was questionable. ‘Safety of patient-facing digital symptom checkers’  shows there ‘is a possibility that it [Babylon’s service] might perform significantly worse’. The symptom checking methodology was questioned for not being real world–that the data in the latter test was entered by doctors only, not by patients or other clinicians. While the authors commended Babylon for being open about their research, they felt there was an “urgent need” for improvements in evaluation methods. “Such guidelines should form the basis of a regulatory framework, as there is currently minimal regulatory oversight of these technologies.” Babylon promises a response and additional improvements, presumably from its $100 million investment in AI announced in SeptemberDigitalHealth (UK), Mobihealthnews

The underweight may be at higher risk of dementia: UK study

With a personal connection. This study, just released in The Lancet, has created headlines because it contradicts previous research and conventional wisdom that the obese are at the highest risk for dementia. Of nearly 2 million aged 40+ (median age 55), developed from the UK Clinical Practice Research Datalink (CPRD) over 15 years and screened for body mass index (BMI), the highest risk–34 percent– was found among those with a BMI lower than 20, which would include low-normal and underweight, versus those in a normal weight range with BMI 20-25 percent. The very obese, with BMI over 40, had a 29 percent reduced risk versus the normal range. This leads of course to why this is, for others to study. So it looks like you should eat right and exercise, but you don’t have to be a social X-ray. Also The Telegraph, The Guardian. Cheers to one of the researchers, statistician Michelle E Johnson, MSc, of OXON Epidemiology, London–the daughter of TTA founder and EIC Emeritus Steve Hards.