Introduction
This series of posts covers some work I have been doing over the past three months: attempting to answer the question of how best to improve the perception by clinicians and patients of the efficacy of health-related apps. This work has been done for the i-Focus project, part of the Technology Strategy Board’s dallas programme.
Part I attempted to summarise the EU regulations covering health-related apps. The point was made that any health-related app must comply with data protection and consumer protection requirements, irrespective of whether the risk level is sufficient for it to be classified as a ‘medical device’. Where an app is classified as a ‘medical device’ it also has to be classified so that the appropriate adjudication work can be determined for it to receive a CE mark (Class I, lowest risk, requires least investigation; Class III, highest risk, requires greatest investigation).
This post summarises the principal findings from discussions with a very wide range of potential stakeholders, from patients to consultants, and from individual app developers to chief executives of app curation companies. The final post will draw conclusions. As different considerations can apply to those apps with only a limited health or wellbeing involvement, for these two posts, the focus is very much on serious apps – what I have chosen to term ‘medical apps’. These are the sort of apps that your GP might in future advise you to consider, or a clinician might use in a medical context. Although not necessarily fitting the description of ‘medical devices’, they nevertheless are likely potentially to play an important role in delivering improved patient outcomes.
Little evidence of efficacy
The single most important point raised by most people spoken to is the uncertainty over the efficacy of medical apps. There has been very little research on the benefits to date. Where it has been done, it has been done in small trials by developers, rarely to the RCT-type standard that clinicians have been taught is the gold standard of evidence. (Prof Jeremy Wyatt tells me he is aware of just one such trial, of a weight-loss app, the results of which compared very favourably vs a slimming internet site and diary, particularly notable being the far higher adherence level for the app).
This issue is particularly relevant to clinicians, for whom it feeds their innate conservatism. They in turn feel unable to recommend medical apps to their patients.
It is however worth pointing out that when pushed, most professional people in positions of power and responsibility concede that they expect apps to offer positive benefits; given the relatively low cost of mass-used apps, they are expected still to be beneficial even if the improvements achieved are small. Public health practitioners are most positive. There have also recently been a range of pieces in both the UK and US press on the potential benefits of medical apps, and calls for greater evidence of efficacy.
Bogus apps
What has unfortunately been researched quite widely are apps that are either ineffective or, worse, make unjustified claims. Dr Richard Brady has published a review of opioid converters that showed a 15x difference in results between different apps and a lack of reference to sources which is a clear indicator of major problems. In another paper recently submitted to the BMJ, Prof Jeremy Wyatt revises data on smoking cessation apps to show that efficacy is inversely proportional to popularity. Examples of entirely bogus claims are easy to find among the huge number of apps on Google Play and the Apple Store; some have been removed by the FTC in the US.
Both patients and clinicians echo this concern. Developers also recognise these apps do the sector no good at all.
Safety
Third in this group of three linked issues is safety. There appears to be a significant number of what might be termed potentially ‘dangerous apps’ with the clear characteristics of medical devices that have not been put through the appropriate tests – either the NHS’s ISB 0129 clinical risk management standard or the full checks appropriate to their appropriate MDD/IVDD classification. In research for this submission, the author was shown apps being used particularly by hospital doctors to make dosage calculations and assist in diagnosis that have not apparently been through any approval process yet an error in them would seem to be life-threatening.
The perception is that no-one is policing this EU legislation. This is potentially extremely serious as one death shown to be due to an unapproved app could threaten their use and set back this area of likely substantial medical benefit by many years.
As might therefore be expected, safety is an issue stressed by all the curation organisations, including the NHS Apps Library, MyHealthApps, OurMobileHealth, Happtique and such like. However, subjectively, like data protection (see below), it seems lower in most users’ ranking than questions about efficacy and concerns about false claims.
There is a final important point about safety which has profound implications for medical apps. This is that, unlike drugs, the formulation of an app can change dramatically over time. At the very least, a new operating system might well interfere with its operation. In reality, in order to keep it high in the rankings, enhancements will often be made. Therefore there is a need to keep testing and monitoring apps over their lifetime, to ensure that they continue to operate safely and continue to deliver benefits, which requires a revenue stream to pay for this. Equally important it should not be possible to continue to use a medical app with potentially serious consequences from malfunction if it is no longer supported by the supplier. Fortunately, the app itself can assist with this, as long as it is designed in. The Mersey Burns app for example – believed to be the first to receive a CE mark – does a multitude of checks, when started, to ensure that it is still sitting on the operating system for which it was designed.
Process
Another major area of concern is the process by which medical apps are approved, and by whom. The triad of fear, uncertainty and doubt are very prevalent here, especially among app developers and app retailers, although there is also an almost complete lack of understanding among clinicians and (perhaps more understandably) patients too. When talking to clinicians it is common to be asked what a CE mark is; and there remains widespread ignorance that software can be classified as a medical device. Sharp-eyed readers will also have spotted a potential overlap in Part I between the EU’s medical device legislation and R+TTE Directive
As there is no single record of type approvals, it is not possible to be sure how many apps have achieved CE certification. It is believed that, in addition to the Mersey Burns Unit mentioned above, there are at least two others in the UK: ONCOassist and Medopad.
No-one has reported that the process of achieving CE certification is transparent or simple. Clearly there is an urgent need for clarification of the process with guidance provided, as the FDA has already provided in the US, as to which apps require greater scrutiny. This should include clarification of the role of the R+TTE Directive as regards medical apps.
(Note that UKTI New York via the KTN has organised a free webinar on FDA regulations for medical devices. It is on Wednesday 22 January 2014, 15:00 – 16:00 GMT.)
Marketing
Clearly to drive up app downloads/usage will require the ability to make claims for the efficacy of medical apps that fall within the EU consumer protection legislation. Currently there seems to be little understanding of this legislation with some developers making unsubstantiated/unsubstantiable claims whereas others with the necessary evidence are concerned it is not strong enough. A typical example of the latter is a pharmaceutical company that has developed an app that in a small trial showed an improvement in medication adherence from 45% to over 80%, yet was unwilling to market it on the basis of this claim, as to provide what they understood to be the required level of evidence would they believed have required a large randomised controlled trial (RCT).
Picking up on earlier comments about demonstrating efficacy, clearly given the current pace of development, an RCT that was as protracted as the Whole System Demonstrator would be wholly inappropriate. Therefore there is a need for creativity and ingenuity in evidence gathering, possibly taking a lead from the banking and airline industry who optimise their apps in real time using techniques such as A/B testing.
Data protection
This is the ‘elephant in the room’ still to become a serious concern to most stakeholders, although a recent exposé of the iPharmacy app and Happtique’s decision to suspend app evaluation, because of data security problems found with apps already approved in the US, (both of which we have previously covered) suggests that this could become a huge concern if well publicised. As we reported recently, there appears to be a particularly worrying lack of concern among patients about the importance of data privacy, although there is no shortage of advice to developers on what to do. The current hiatus on finalising the EU data privacy regulation, now likely delayed to 2015, creates greater uncertainty in the EU too.
Sales
There is a widespread lack of understanding of how the NHS, for example, can pay for and provide apps to patients, although at least one CCG is known to have handcrafted a way of doing that. Making enquiries about how the NHS might generally enable the prescribing of medical apps has exposed that very few have considered the matter yet, even though there are now a few prescription-only apps in the US (primarily for dosage management for diabetes). One requirement that emerges though is that some form of official approval of the app as cost effective would be a pre-requisite for widespread dissemination.
As per Part I, observations, and suggestions for omissions & additions would be hugely welcomed – either as comments to this post, or as emails to me at charles.lowe@btinternet.com.
Most Recent Comments