In an article published in Clinical Pharmacology & Therapeutics, regulators and academics discuss factors still limiting the regulatory acceptability of clinical data coming from sources different than classical randomised trials (RCT). The authors, among which are EMA’s Executive Director Guido Rasi, the Senior Medical Officer, Hans-Georg Eichler, and the Chair and Vice-Chair of EMA’s human medicines Committee (CHMP), focused on how to adapt the design of clinical studies to consider the new sources of real-world data. The suggestion is for companies to work on the validation of innovative methods for clinical development, while looking for support from regulators to be dispensed through EMA’s methods qualification advice procedure.
The new era of not-randomised data
The increasing diffusion of new data sources (i.e. smart or wearable devices) and analytical algorithms for their interpretation is deeply changing how healthcare services are provided to patients. The new scenario also includes pharmaceutical development, where the increased diffusion of personalised medicine corresponds to the need for clinical studies to be run on smaller and more selected groups of patients. Post-authorisations phases are also often requested by the authorities in order to monitor the impact of a new treatment in real-world conditions.
The Clin. Pharmac. & Ther. paper analyses from a regulatory perspective the current limits to acceptability of not-randomised analytical methods (e.g. statistical, epidemiological, etc.). Randomised clinical trials traditionally represent the preferred way to validate outcomes of a development project; they are based on highly selected cohorts of patients, and results are obtained through statistical analysis and interpretation of data.
The method is not useful to validate real-world trials, where the considered population has not been selected against strict criteria, including for example the possible presence of co-morbidities. New statistical and analytical methods are thus needed to support the decision process during which regulatory authorities evaluate the data submitted with the application.
Not an easy goal to be achieved
As for every kind of analytical method, also those used for real-world data analysis need to be tested and validated before they can be applied for regulatory purposes. Data coming from electronic health records (e-HR), or from sharing results of terminated studies, from cross-trail analysis or through the combination of real-world data with the e-HR’s or insurance ones, are just few of the many possible sources yet available.
According to the authors, two main factors are currently limiting the true usefulness of this type of data: the availability of proper technical-operative skills and the lack of a clear governance for data. Other possible bias may result from data coming from not-randomised studies. Randomised trials should thus continue to represent the main, standard tool to develop new medicinal products, says the paper. Emerging approaches would constitute just a complementary possibility, to be used when the traditional design for the trial is not possible or acceptable, for example due to ethical reasons.
Experts also claim that “we see various degrees of methodology aversion in all stakeholder groups within the pharmaceutical ecosystem”: a quite open criticism to those who say regulators are unwilling to adopt novel statistical (and other) methods for data analysis while, on the other hand, they would also work to “recklessly abandoning the ‘gold standard’ RCT”. “We concur that unfounded methodology aversion is a potential roadblock to making the best use of new data sources”, writes the authors.
A pre-agreed plan for validation
The suggestion made by the experts of regulatory affairs is to validate the new analytical methods in the same way it would be done for a medicinal product, in a prospectively and well‐controlled manner and following a plan pre-agreed with regulators. EMA’s procedure for methods qualification advice may be of help with this respect, together with an active role of health technology assessment bodies, payers and patients’ associations. The proposed modalities would represent, from the European regulators perspective, an efficient and transparent platform for the development and validation of study design.
The article also includes many examples that may follow under the proposed new vision, from the “borrowing of data” from trials already terminated, to the use of external control groups and threshold crossing, from the indirect comparison of the relative efficacy to the possibility to fully replace randomised trials with the real-world model. Methodology evaluation may be run in parallel to the standard way of development (both in the pre- and post-marketing phase), is the suggestion coming from the article, and it would require investments judged by the authors “not prohibitive” if compared to costs and times typically needed to generate data by mean of interventional studies. These activities, they further suggest, may be also financed by the European Union through the IMI Initiative.
The CHMP opinion on the method
According to the proposed approach to validation, the CHMP may also release opinions on the acceptability of a certain method on the basis of specific scientific questions. From this point of view, a hot issue is the possibility to completely substitute randomised clinical trials with real-world studies, so to allow great savings both in terms of costs and time. But the track record of not-randomised comparative trials is still not enough convincing to support this type of transition, say the experts, and many are the reported cases of divergent results obtained with the two type of experimental design on both the size and direction of the observed effect of the experimental drug.
The prospective and structured evaluation of real-world studies would help limiting the risk of “adjusted” results, in the case outcomes of randomised trials are already available. Some bias might occur at the level of different geographical areas, warn the authors, thus making difficult, for example, the direct extrapolation in Europe of data produced in the US (or viceversa).
An open and agnostic mind-set is the ultimate key to achieving credibility, write the regulatory experts: pharmaceutical companies should not limit to test new approaches making “dry runs with old products”. “We emphasise that if developers want trial assessors to accept novel methods, they will have to expose some of their experimental drugs to methodology development exercises”, is the call to the industry made in the conclusions. A firewall should be put in place to ensure the complete separation of method and product evaluation: a very challenging target to be achieved, that requires the strong collaboration of all different actors involved in the development and regulatory approval of a new medicine, including also funding agencies those investments are needed to support the long-term sustainability of the transition.