Common deviations related to data-handling, etc. in investigator-initiated trials

Updated 08 September 2021

Contents

The GCP inspectors of the Danish Medicines Agency have completed a project focusing on the handling of systems and data in investigator-initiated trials. The project covered many areas of which the most important were subcontracted tasks concerning, for example, the provision of systems for randomisation and data collection and the documentation of processes after the last trial participant has left the trial such as data analyses, unblinding and the documentation of activities in the Trial Master File.

The aim of the project was to increase the researchers’ and the Danish regions’ awareness on obtaining valid conclusions from clinical trials and to provide guidance and standards in difficult areas that have not been inspected as frequently in the past, e.g. :

  • The use of computerised systems in clinical trials. Today, computerised systems are more often than not used in both company-initiated trials and investigator-initiated trials. But the computerised systems used in investigator-initiated trials tend to be different than the ones used in company-initiated trials.
  • Data-handling after the last trial participant has left the trial. As far as investigator-initiated trials are concerned, this area has not been inspected as frequently as in company-initiated trials, and the help available to researchers may be limited depending on the offers from, for example, the Danish regions. The Danish GCP units are usually not involved in this process. 

The number, nature and severity of the identified deviations have shown that measures are needed in several areas starting with this guidance. The project identified deviations in areas such as: contracts, quality management system (including training), validation (including change control), audit trail and audit trail review, user administration (including the investigator’s control over data), IT security, randomisation/blinding, escalation/reporting of serious non-compliance and security incidents, handling of electronic data (identification, data entry, alteration of data, copying, review, transfer, migration), data analyses (including communication on these), filing (Trial Master File) as well as backup/recovery.

The following sections will go through the areas that the project found especially problematic. We will look at the Danish Medicines Agency’s requirements in the area and the deviations that are commonly seen during inspections.

Contracts 

Written agreements or contracts must exist setting out the agreed delegation and distribution of tasks between the responsible parties in a clinical trial (principal investigator and sponsor) and their contractors, cf. ICH GCP, 1.17, 5.4.1, 5.2 and 8.2.6.

Many and serious contract-related deviations are observed at inspections. As a result, the EU GCP inspectors have made a Q&A on the topic. See question 8 of https://www.ema.europa.eu/en/human-regulatory/research-development/compliance/good-clinical-practice/qa-good-clinical-practice-gcp

The deviations have many causes of which these are the most important:

  • missing contracts, contracts concluded too late, status on contracts (expired contracts and contracts that are not maintained/updated).

Also, some contracts are unclear due to missing details on:

  • delegation of tasks between sponsor and contractor, e.g. in the form of a ‘split of responsibility’. This is to ensure that each party (principal investigator/sponsor) has the information needed at any time to ensure they can act according to their delegated responsibilities in the trial. 
  • a description of who has been assigned the task of generating and storing the respective elements of the Trial Master File (e-mails, minutes, ongoing agreements made and documentation of agreement/approval, system documentation such as validation/evidence of qualification, evidence of any reconciliation (comparison of exchanged information between the parties), evidence of coding of, e.g. randomisation algorithms, standard operating procedures (SOPs) and staff and training documentation, cf. the GCP Executive Order, sections 3 and 6 and schedule 2 (1.6) and ICH GCP, 2.13, 5.2. and 8.2.6.
  • sponsor’s access to the contractor’s non-trial specific documentation (e.g. validation documentation and evidence of implemented IT security), e.g. in connection with inspections and audits
  • investigator’s control over and continuous access to trial participant data, including the process to be activated if the database is decommissioned and access is provided in some other way than the ‘live’ database
  • standards followed by the contractor (ICH GCP, etc.)
  • requirements for compliance with the protocol and a description of how it is ensured that the contractor is informed of any relevant changes to the protocol
  • the sponsor’s right to audit contractors, and the contractor’s obligation to provide access to documentation for the authorities’ inspections (conducted by the Danish Medicines Agency or a corresponding foreign authority)
  • handling and escalation of serious non-compliance with GCP/the protocol to the sponsor (with deadlines)
  • storage of data (e.g. in the cloud)
  • addressing downtimes and temporary solutions in case the system is unavailable
  • the possibility for contractors to subcontract tasks and the terms under which they can do so
  • contractor’s provision (during and at the end of the trial), e.g. of data exports from the database, files for investigators, etc. This must include documentation such as metadata, specific query types, audit trail of reported data (CRF, Patient Reported Outcome (PRO) etc.), status on change of users and their access rights, delivery of TMF, etc. Assurance that the formats supplied are sufficient to, for example, maintain the dynamic functionality throughout the storage period
  • any arrangements regarding the decommissioning of the database to ensure that it is possible throughout the entire storage period to provide access to the data (e.g. for inspections) so that the access to the original database compared to the subsequent access is not materially different for the inspectors (e.g. access to audit trail, system log and dynamic search options).

In the event that a sponsor chooses to base the validation documentation on a contractor’s validation/qualification documentation, please see question 9 of the EU GCP Q&A:  https://www.ema.europa.eu/en/human-regulatory/research-development/compliance/good-clinical-practice/qa-good-clinical-practice-gcp).

EudraCT forms

The sponsor must ensure that the information provided in the clinical trial application in EudraCT is correct and complete, remembering also to provide updates in case of subsequent changes. 

Whereas section G.5 of the form ought to contain information on whether electronic data entry and/or randomisation had been transferred to a contract research organisation (CRO) (cf. the application form and sections 2, 3 of the Danish executive order on clinical trials of medicinal products in humans), the project found that this section often did not reflect the actual circumstances.

Quality management systems of contractors 

Pursuant to ICH GCP, 5.2.1, a sponsor may transfer tasks to a contractor, but the responsibility for the quality and integrity of the trial data remains with the sponsor. Likewise, pursuant to ICH GCP, 5.2.3, the sponsor must maintain an overview of the trial, including the tasks of a contractor. Thus, the sponsor must also ensure that the contractor has robust processes in place and that these processes are described in a quality management system and are subject to quality control and quality assurance (ICH GCP, 5.2.2).

On several occasions, our inspections of contractors’ IT services found that the contractor either had no quality assurance system at all or the system used did not sufficiently ensure that the subcontracted tasks were carried out in compliance with the rules of the ICH GCP. We issued serious or critical deviations in response to the total absence of quality management systems and deviations related to missing procedures for concrete processes that are deemed essential. These include the generation of randomisation codes, system validation, access control, training of staff in relevant legislation, etc. There has been a tendency that IT contractors used in investigator-initiated trials lack knowledge of ICH GCP.

Validation of computerised systems for collection of data in clinical trials

Any computerised system used to collect data in clinical trials must be supported by processes to ensure that the system consistently performs as intended (specified) and is suited to the intended purpose (validated). The validation should establish reliability of the documentation and the generated data from design to decommissioning of the system. 

The sponsor (and sponsor-investigator) are responsible for controlling this process regardless of whether the validation/qualification is done by the sponsor or another party, e.g. the party having developed the system.

We very often see major or even critical deviations related to the validation of systems used to collect data in clinical trials. This may concern data collection, e.g. in electronic Case Report Forms (eCRF), Patient Reported Outcome systems (ePRO) or randomisation systems, but other computerised systems are also involved. We have also seen deviations related to changes to such systems.

Some deviations were related to lack of overview of system releases, lack of formalised processes, lack of documentation of requirement specifications, lack of or insufficient test documentation, lack of change control, etc. 

The general requirements appear partly from ICH GCP, 5.5.3 (including addenda) and partly from the EU GCP inspectors reflection paper https://www.ema.europa.eu/en/documents/regulatory-procedural-guideline/reflection-paper-expectations-electronic-source-data-data-transcribed-electronic-data-collection_en.pdf. Please see Topic 1 regarding validation.

Work is in progress to update/upgrade this reflection paper to an actual EU guidance document which has recently been submitted for consultation: https://www.ema.europa.eu/en/documents/regulatory-procedural-guideline/draft-guideline-computerised-systems-electronic-data-clinical-trials_en.pdf

The validation documentation must be filed and must include requirement specifications (usually User Requirement Specification (URS) and Functional Specification (FS)), qualification documentation (e.g. Installation, Operational and Performance (IQ/OQ/PQ)) as well as documentation of authorisation prior to system release. Documentation of ongoing changes to the system, including a risk assessment, must also be available.

Investigator-initiated trials often make use of shareware that is made available to the researchers by central units in the Danish regions. In such cases the validation/qualification activities are often caried out by these units, and documentation of the sponsor-investigator control of validation status can therefore be gathered here. If this is not the case, the individual sponsor is responsible for ensuring and documenting that the system used is in a validated state.

In the event that a sponsor chooses to base their validation documentation on a contractor’s validation/qualification documentation, please see question 9 of the EU GCP Q&A: https://www.ema.europa.eu/en/human-regulatory/research-development/compliance/good-clinical-practice/qa-good-clinical-practice-gcp. 

In 2021, we expect to publish guidelines of a more general nature on the requirements for validation of IT systems in all GXP areas. They will be posted on our website.

Adjusting the system to the specific trials

In particular, it must be ensured in connection with the trial-specific part of the validation that the system is an accurate reflection of the protocol applicable at any given time, and it must be ensured that the system is designed to collect exactly the data defined in the protocol. 

In the event of protocol changes, it must be decided if changes to the IT system are required. Any such changes must be controlled and tested just like the initial application. Documentation on the release of each version used during completion of the trial must be kept on file and stored for as long as the Trial Master File.

It will often be necessary to embed validation rules for entered data (edit checks). Usually, these are trial-specific and must also be controlled and validated. The documentation of edit checks must be filed together with the trial’s other documentation. 

Trial-specific configuration of a system must be validated in the same way as basic system functionality (i.e. that a check mark put for a specific functionality is qualified, and that it returns the expected/specified result in testing).  

If new code is written, e.g. to establish an interface to other systems (e.g. between eCRF and randomisation system), that interface must also be validated.

User administration in clinical trials

Lists must be maintained of the individuals who are authorised to access the system and enter data (ICH GCP, 5.5.3 e). Also, a security system must be maintained to prevent unauthorised access to data (ICH GCP, 5.5.3 d), and the blinding must be maintained (ICH GCP, 5.5.3 g).

User access management is thus essential to the data integrity of clinical trials. 

During inspections, we found major and critical deviations related to user administration. The problems were related to lack of traceability of the creation and deletion of users as well as lack of robustness in the allocation of roles to the effect that persons who should be given read-access only also had write access, that blinded staff had access to unblinded data, and that the sponsor-investigator was able to edit the data of other investigators, etc.

In 2021, we expect to publish guidelines of a more general nature on the requirements for user administration in IT systems in all GXP areas. They will be posted on our website. This is also expected to be covered in a future GCP EU guidance document on computerised systems and data.

The expected outcome is that users should only be given rights pursuant to the tasks they are to perform in the system, that the allocation and removal of rights should take place pursuant to a robust procedure and that it should be assessed regularly if persons still have the correct accesses and rights. Stricter terms apply to systems in which incorrect accesses might unblind the trial.

The project found some cases that had used programs such as SharePoint for critical processes (investigator’s Trial Master File and the like). Please be aware that insufficient user management could be non-compliant with relevant legislation, the reason being that certain programs permit the allocation of user rights, e.g. for the creation of other users, management of the audit trail level and status, and more. 

Handling of electronic data

The requirements regarding the handling of electronic data in clinical trials are not described in detail in the ICH GCP guideline because the requirements in ICH GCP do not specifically distinguish between the types of media used (ICH GCP, 2.10. 2.13 and 5.5.3). The EU GCP inspectors elaborate on the expectations in the following reflection paper: https://www.ema.europa.eu/en/documents/regulatory-procedural-guideline/reflection-paper-expectations-electronic-source-data-data-transcribed-electronic-data-collection_en.pdf

This reflection paper is in the process of being updated to an actual EU guidance document (see above).

Deviations related to the handling of data are often classified as major or critical, the reason being that data handling deviations could quickly impact the data integrity and the trial results reported by the sponsor to the authorities in the clinical study report. The observed deviations (in addition to those related to validation, audit trail and review (which are covered in separate sections) and IT security) include definition of source data, review and quality control of data and metadata. The concerned deviations covered a range of aspects including: uncertainty about where the source data could be found, lack of source data, lack of documentation of data transfer involving a third party, lack of documentation of timely review of data, lack of charters/procedures of committees assessing endpoints and safety committee, lack of documentation of blinded review of endpoints, use of Excel sheets without risk analysis and mitigating actions against lack of audit trail, discrepancies between source data and data in analysis set and lack of quality control of data sets. 

Maintenance of Trial Master File

 

According to ICH GCP, 8.1, a Trial Master File (TMF) must include all the documents that collectively permit the reconstruction of the trial and an assessment of the quality of the generated data. 

The EU GCP inspectors have published a guideline on the content, handling and filing of the Trial Master File that we generally refer to:

https://www.ema.europa.eu/en/documents/scientific-guideline/guideline-content-management-archiving-clinical-trial-master-file-paper/electronic_en.pdf

In particular, we wish to highlight that GCP inspections record many and serious deviations related to Trial Master Files in both company- and investigator-initiated trials. This has led to the above EU guidance. Major or critical deviations related to the Trial Master File were reported in the vast majority of the investigator-initiated trials that were subjected to an inspection of the processes related to the trials’ data quality.

Among the deviations were:

  • lack of definition of what the TMF should contain
  • lack of definition of/agreement on where the documentation is filed (at the sponsor-investigator, at the CRO or at the investigator and the physical and logical storage location), 
  • inconsistent or missing filing of documentation such as:  
    - contracts, 
    - communication between contractor and sponsor, 
    - documentation of validation, 
    - documentation of coding, 
    - documentation of data handling, 
    - documentation of data analyses, unblinding, etc. 
  • Furthermore, some essential documents were stored in personal mailboxes, on shared drives of the organisation, etc. 

Statistical analyses

The statistical analyses must be pre-specified in the protocol. It is an underlying principle in GCP to pre-specify statistical analyses and hypotheses without knowledge of the data being collected in the trial. Further guidance on statistical analyses is provided in ICH E9.

It is often advisable to also make a supplementary statistical analysis plan (SAP) to provide a more detailed elaboration of the analyses and to make it clear exactly which analyses are planned in the trial. The statistical analysis plan (SAP) must be final before the data are unblinded, and a signed, dated, and version-controlled SAP must, like the protocol, be kept in the Trial Master File (TMF). It is recommended that the person(s) preparing the SAP are not given access to data, particularly not the unblinded data.

Before the documented and pre-specified statistical analyses can be made, a read-only and cleaned database of the collected clinical data must be obtained. A cleaned database is obtained by subjecting the blinded data to a data review process in which all outstanding data queries are solved. In addition, based on the blinded data, all decisions must be made regarding which trial participants and data to include in the statistical analyses and how to handle missing data. When all data queries/decisions have been clarified/made, the database can be locked (set to read-only), realising a so-called database lock (DBL). The data cannot be unblinded before this point. It is crucial that all decisions on data are documented and can be easily obtained and that there is a clear indication of the time/date of the decisions and the person having made them as well as the time when the DBL was realised. Documentation of these decisions and processes must therefore appear from the TMF.

The running of statistical programs and generation of output cannot start before the database has been locked and cleaned. It is important that clear traceability is ensured for the statistical analyses, so that it is possible to document for any analysis precisely which data set and which statistical analysis program were used, the output that was generated and not least when the analysis was made. These three items, dataset, program and output, must be stored and made read-only.

Likewise, interim analyses must be pre-specified (described in the protocol and, if relevant, in the statistical analysis plan) and documented.

Should it become necessary to alter the pre-specified statistical analyses, it must be clearly indicated in the documentation and in the final report and any publications why the analyses were altered in the process, indicating also that they are post hoc analyses.

GCP inspections have often found the TMF documentation to be insufficient with respect to the above processes. This creates uncertainty about the actual course of events of these specific processes, which are of vital importance to the credibility of the trial results. As a result, they are often classified as critical deviations and may have serious consequences for, for example, publications, applications for marketing authorisations, etc. 

Unblinding

The blinding of the individual trial participant’s treatment in a clinical trial is crucial to avoid bias. So, in cases of planned unblinding, or situations when it becomes necessary to unblind parts of the data during the trial, or if unintentional unblinding occurs, it is important to document the following:

  • the reason for the unblinding, 
  • the time of unblinding, 
  • who requested the unblinding (for planned unblinding),
  • who gained access to the unblinded data, 
  • and in the case of unintentional unblinding: what consequences followed from the unblinding and any mitigating actions taken. 

This documentation is important for the sponsor and the authorities to assess the consequences of unblinding and ultimately the credibility of the results of the clinical trial. If the unblinding progressed as planned, it is also important to provide evidence of the above and thus that the planned blinding of the trial was as expected. 

Should it become necessary to unblind some of the staff at the clinical site or at the sponsor/CRO during the course of the trial, it is crucial to document that the staff was in no position to influence the treatment of the individual trial participants, influence the trial’s blinded staff, or in some way could have influenced the collected trial data. This is achieved, among other things, through robust processes on the delegation of tasks and rights, including user creation and deletion (and ongoing review of rights) in computerised systems.

The safety of trial participants and/or the reporting of SUSARs may make it necessary to unblind the treatment given to specific trial participants. In such situations, it is important to have an efficient and well-documented process describing who may request and perform unblinding and how the unblinding is going to take place. It is also vital that this process can be effected instantly at the investigator site to safeguard the safety of the trial participant. It should also be possible to effect unblinding of one trial participant without revealing the treatment of other trial participants in the clinical trial.

GCP inspections have often found the TMF documentation of especially the final planned unblinding to be insufficient. This raises doubt about the credibility of the trial results. For this reason, they are often classified as critical deviations, and they may have serious consequences for, e.g. publications and applications for marketing authorisations. Please also see the section about statistical analyses.

Randomisation

Randomisation of the individual trial participant’s treatment in a clinical trial is crucial to avoid bias in a clinical trial. For this reason, it is important to document that the treatment given to each trial participant was decided by actual drawing of lots/randomisation. It is also important that the randomisation codes are kept in a safe place, that as few persons as possible have access to the codes, and that it is absolutely clear who and how they can access the codes. This applies to physical randomisation codes and electronic codes. There can be no overlap of roles in terms of those who generate and store these randomisation codes and those who are operationally involved in the trial. 

In case of a blinded trial, the processes on the handling of randomisation codes must also ensure that blinding is maintained. Please also see the section about unblinding.

During our GCP inspections, we have seen several cases of deviations related to randomisation and/or blinding. Among them were coding errors resulting in stratification errors, unblinding of sponsor staff, lack of documentation of the blinded completion of important assessments (e.g. assessment of endpoints) as well as lack of documentation of communication between the manufacturer and supplier of randomisation lists. Randomisation processes are essential in a clinical trial, which is why any deviations in this area are often classified as critical. 

Audit trail and metadata review 

An audit trail is essential to ensuring that changes in data are traceable, as otherwise the data integrity is not preserved. To be compliant with GCP, the audit trail must show the initial value and every subsequent changes (value – old and new) of what (field, data ’identifier’), made by who (user name, role, organisation), when (date/time stamp) and why (when relevant, e.g. in the case of changed values). 

Audit trails should be robust, and it must not be possible for ‘general’ users to deactivate an electronic audit trail. If an audit trail can be deactivated by an admin user, traceability must be provided in, for instance, an event log/audit trail. The audit trail must be protected against alterations, deletion and changes of rights. The audit trail must be an integral part of the system. 

The responsible investigator/sponsor/inspector must be able to review a readable audit trail. 

Risk-based procedures are expected to be in place for audit trail review. The audit trail review must focus on critical data and systems. These procedures are typically trial-specific as critical data and systems vary between different trials. The completion of an audit trail review should generally be documented. A data review is expected to be made on an ongoing basis, unless otherwise justified. This may be a manual process and/or a process based on technologies facilitating large volumes of data. 

A data review can be used for example to identify missing data, discover signs of data manipulation, identify unexpected/abnormal data/’outliers’, discover the use of alternative source data (if direct entry into the system of data is expected), identify incorrect data processing (e.g. non-automatic calculations), discover system errors, and identify any need for extra training of users, e.g. patients or site personnel. 

The audit trail can also cover user logs, event logs, etc. 

Principal investigators should also be able to access the audit trail of their own data, and should, if necessary, be instructed on how to gain access. Audit trails must be visible at field level ‘live’ in the system, but the full audit trail must also be available as a dynamic data set. This is fulfilled if, for example, the audit trail can be exported to an Excel file. This makes it possible to identify problems of a more systematic nature across patients and sites. 

During our GCP inspections, we found major and critical deviations related to audit trail/metadata. For example, in a given period, a sufficient audit trail was missing, the investigator trial sites could not access the audit trail, data on monitor queries were missing and the audit trail for users who had been active in the system compared to their allocated accesses was missing. 

Serious non-compliance 

The Danish GCP Executive Order and the coming regulation on clinical trials stipulate that the Danish Medicines Agency must be informed (immediately) in the event of serious non-compliance. During our inspections, we occasionally see that serious non-compliance has not been reported to us or reported to us too late. We have also seen cases of serious non-compliance that were not escalated to the trial's sponsor, who is obliged to report this to the authorities and therefore relies on the rapid reporting from, for example, contractors. Examples related to computerised systems could be lost access to data (temporarily or permanently) caused by server breakdowns or ransom attacks combined with inadequate backup and lack of reporting of randomisation/stratification problems. 

Please also see the information on the Danish Medicines Agency’s website on:

Notification of serious or repeated non-compliance