Prerpints.org logo
Preprint
Article

Blockchain-Enabled Quality Improvement Digital Twin for Clinical Trials

Altmetrics

Downloads

217

Views

105

Comments

0

Submitted:

23 May 2023

Posted:

24 May 2023

You are already at the latest version

Alerts
Abstract
Clinical trials are research studies performed on people that are aimed at evaluating a medical, surgical, or behavioural intervention. It is a primary way that researchers find out if a new treatment, like a new drug or diet, or medical device is safe and effective in people. In this paper blockchain technology is embedded in a digital twin aiming to improve the quality of clinical trials through the boost of data integrity among the quality-related activities, and to promote participants' safety. First, a blockchain-embedded quality-enabled digital twin is proposed and the interactions among the enablers and peers are highlighted. Then, a prototype of the proposed system is developed, and the data of a pilot trial is used to justify the applicability of the system. The results showed that the proposed system is efficient, and hence it is feasible to adopt into a clinical trial. With the successful development of the proposed system, it is promising to provide effective data integration and knowledge management for improving participant safety.
Keywords: 
Subject: Computer Science and Mathematics  -   Artificial Intelligence and Machine Learning

1. Introduction

The estimations reported that the costs of developing a new drug exceed 1.2 billion dollars and take an average of 12 years from creation to market, causing pharmaceutical companies to experience a pronounced “profitably gap” [1]. As pressures on the pharmaceutical industry continue to mount, quality and safety issues become a greater concern. Companies should build safety, quality, and efficacy into their new pharmaceutical products as early as possible [2]. Participant safety in clinical trials (CTs) is the pivotal aspect of quality in the drug development process and all stakeholders attempt to collaborate proactively to ensure the safety of participants. The evaluation and management of the risks and minimization strategies are fundamental requirements for regulating authorities [3]. Data integrity in CTs is the next complementary piece of quality to provide reliable and accurate data since missing data have seriously compromised inferences in CTs [4]. Some articles have promoted several approaches to quality improvement in CTs. Mainly they focused on the monitoring process based on Good Clinical Practice (GCP) [5,6,7]. GCP is the global quality standard to ensure the quality conduct of CTs [8]. Although this standard has been defined for the design, conduct, monitor, audit, analysis and report the CTs, serious concerns related to the structure and delivery of GCP remain. To build quality into a clinical process rather than ensuring the quality of clinical service through audits and inspections that performs in GCP, Quality by Design (QbD) has been introduced [9].
QbD is a systematic approach to development that begins with predefined objectives and emphasizes product and process understanding and process control, based on sound science and quality risk management [10]. It emphasizes building quality into a process from the beginning and has been successfully utilized in drug manufacturing [11]. The major enabler in implementing QbD is to extract, review, and use the data coming from diverse sources and then assess quality attributes and improve conduction strategies. Therefore, integration among the Information technology (IT) and Operation Technology (OT) are required and is a key factor for the successful implementation of QbD [12]. The introduction of the QbD approach into the clinical and pharmacovigilance areas still is in its infancy [http://www.wsqms.com/]. With the growing emphasis on enhanced process understanding for quality improvement, the need for establishing an efficient QbD is more important than ever [14].
The integration of IT and OT through the digital twin (DT) might be a significant opportunity to improve quality and increase efficiency. A DT is a virtual representation of a physical object, system, or process that is designed to mimic the real-world counterpart in a digital environment. DT should create using data collected from sensors and other sources, and it will be used to monitor, analyze, and simulate the behavior of the physical system. [13]. To gain greater efficiencies and higher quality in clinical research, QbD needs to embrace newly merged technological tools and approaches to well manage generated knowledge before, during, and after the CTs conduction. Perhaps the most important prerequisite to innovation is employing Blockchain Technology (BCT) and DT together to leverage QbD productivity and improve the quality of clinical trials. It might play an important role in the successful implementation of QbD and enhance the quality of CTs by facilitating efficient transfer and utilization of knowledge.
This work provides a blockchain-enabled quality embedded digital twin in the clinical trial stage of drug development aiming to improve the quality of CTs by first; highlighting the factors that affect the safety of the participants, defining measurement approaches and safe domains/states for the selected factors; second, conduct the trials, and achieve the amounts of the factors for the participants; third, performing root-cause analyses; and finally develop safety control strategies. The DT improves the quality of the CTs by enhancing patient safety through the use of immutable safety-related knowledge on the blockchain and boosting data integration among the QbD activities. A case study simulation is used as a “proof of concept”.

2. Literature review

In the next subsections former, the building block of blockchain technology is presented and some applications of this technology in clinical research are reviewed; later, the QbD approach is stated and the importance of DT in QbD is highlighted.

2.1. Blockchain Technology and its Applications in clinical research

Blockchain is a huge, public, secure, and decentralized data store of ordered records [15,16]. It is a distributed ledger (database), structured as a chain of blocks that each holds transactions. The ledger is owned by no one and is controlled by the nodes of the distributed network but not any centralized authority or trusted third party. The data of the ledger is immutable and auditable and cannot be hacked, erased, or stolen since a copy of the ledger is stored over the nodes (users) of the decentralized network, authentications are verified through cryptographic techniques, all the transactions are timestamped and accomplished through the use of cryptographic hashing [17,18]. By hashing process, data with variable length is altered into a fixed-length digest which is called a Hash. If any change in the input data happened the hash will be changed unpredictability. Any block in the ledger holds the hash of the previous block and the transactions which are used to generate the block’s hash. In case a change happened in the any of previous blocks (the hash of the block will be changed) the subsequent hash would no longer be valid. A new transaction can be added to the block if the majority of nodes reach into consensus. Trust is provided by the community of the nodes through encoding the smart contracts [19].
There are mainly four types of blockchain ledgers, namely, public, private, consortium, and hybrid. The former is an open-source ledger that allows anyone to participate in the network, private ledger restricts who is allowed to enter in the network, in consortium ledger more than one node can manage the entrances in the network, the latter is a combination of private and public ledger [20].
The integrity of data in a CT is essential, but the current data integration is too complex and highly labor-intensive. Massive volume and variety of data generated during multiple phases and multi-year studies in CTs bring several challenges such as; privacy issues, costs, results in reproducibility, data integrity and sharing, patient enrolment and recruitment, and protocol compliance [21,22,23,24,25]. Hirano, T. et.al [2020] developed and tested a blockchain-based data management system of clinical in breast cancer to validate a system that enables the security of medical data in a CT using blockchain technology [27]. Gordon, W. J. et al. [2018] look at how blockchain technology might facilitate a transition to Patient-Driven Interoperability through five mechanisms: (1) digital access rules, (2) data aggregation, (3) data liquidity, (4) patient identity, and (5) data immutability [28]. To tackle these obstacles, pharma, and biotechnology companies are started to move from currently employed clinical data management systems (e.g., Oracle Clinical software, SigmaSoft’s DMSYS Software) towards new cyber-physical systems [26]. Blockchain technology-enabled digital twin have the potential to address some of these challenges.
This technology is applied by Santos, J. A. et al. (2021) as a mechanism to protect Electronic Health Records (EHRs), patient health data, and mobile health (m-Health) [29]. Cheng et al. (2020) proposed a network model of a Medical Cyber-Physical System (MCPS) based on blockchain to design a secure medical data sharing scheme [30]. Blockchain technology may overcome many issues of medical data (EHR and EMR) such as security challenges [31], data storing, managing and protection [32], integration and access controlling [33], and privacy of users [34,35] in the eHealth world. This technology is a safe platform for storing sensitive information including electronic healthcare records, clinician notes, e-prescribing, analysis results, and protocols. BCT might enhance drug development research by empowering the research community with a secure network of sharing data. The convergence of BCT, CPS and CTs is still in its infancy and very limited literature is available on this subject. At a high level, BCT is used in patient recruiting and enforcement of human subject regulations [36]. To manage and monitor data in multi-site CTs a framework has developed by Choudhury et al. [37]. A system named “BlockTrial” has been developed to facilitate users to execute trial-related smart contracts on a private network [38]. A blockchain-based solution is introduced to manage consent information between patients and stakeholders [39]. To deal with the dynamic nature of consent management a blockchain-enabled framework has been presented [40]. In summary, the review of studies highlights that the BCT may be an appropriate approach and transparent, immutable, and secured way to manage data in e-health and CTs.

2.2. Quality by Design (QbD)

QbD approach is an approach aiming to guarantee the quality of designing, developing, and manufacturing processes of medicines by employing statistical, analytical, and risk-management methodology [41]. According to the ICH Q8 guideline, it is “A systematic approach to development that begins with predefined objectives and emphasizes product and process understanding and process control, based on sound science and quality risk management.” [42]. QbD has been performed in many aspects of biotechnological and pharmaceutical developments including but not limited: biopharmaceuticals [43], bioprocessing [44], abbreviated new drug applications (ANDAs) [45], drug formulation and delivery [46,47,48], pharmaceutical products [49], analytical methods [50,51,52], biotechnological products [53], drug development [54], nano pharmaceutical products [55,56], monoclonal antibodies [57], and CTs [58,59,60].
Pharmaceutical QbD includes designing production processes to guarantee predefined formulation quality and enhance development capability, speed, and formulation design [61]. Quality in QbD should be designed into a product which was supported by Guidance for Industry PAT-a framework for innovative pharmaceutical development, manufacturing, and quality assurance with “quality cannot be tested into products; it should be built-in or should be by design” expression [62]. The progression of QbD principles in the pharmaceutical industry can be monitored with the guidelines based on regulatory processes (ICH PAT Guideline (2004) [63], ICH Q8 (R2) (Pharmaceutical Development, 2014) [42], ICH Q9 (Quality Risk Management, 2014) [64], ICH Q10 (Pharmaceutical Quality System, 2009) [65], ICH Q11 (Development and Manufacture of Drug Substance, 2011) [66], and ICH Q12 (technical and regulatory considerations for pharmaceutical product lifecycle management, 2020) [67]. The main objectives of QbD are to obtain predefined quality specifications based on clinical performance, reduce formulation variability, develop post-approval change organization by improving product and process design, understanding, and control to prevent product variability that causes rejects, scrap, and re-processing [68].
For decades, QbD has played a vital role in drug design and manufacturing, however, it has just begun to take a role in CTs. Recently QbD has been used in some clinical researches and trials for improving the safety, quality, efficacy, cost, and time to market a potential new drug. In CTs, principally, quality is the sponsors’ judgment about the overall credibility of the results. The common approaches for quality improvement are focused on the monitoring processes based on GCP [69] and according to the ICH E8 (R1) Guideline, QbD in clinical research proactively ensures that the quality of a study and the reliability of the results will be improved by considering participants safety and risk management approaches [70].
Plan-Do-Check-Act (PDCA) is a framework upon which the QbD in CTs is based [71]. Plan; a) identify the factors that are critical to quality and must be met during the conduct of the CTs, b) determine the procedures that will enable quality measurement for the predefined quality factors, and c) systematically identify the safe domains of factors of the planned CT, aiming to identify and prioritize risks to quality. Do; implement quality risk management plans during and after the conduct of the CTs. Check; measure/monitor quality performance to assess whether quality factors are being met to enable identification of the risks, their levels, and roots. Act; respond to quality issues with appropriate corrective and/or preventive control strategies.
The flow and integration of data among the PDCA in QbD play a vital role in the producibility of the approach and boosting the quality [72,73]. Real-time data exchange among the activities in an integrated manner can be realized through incorporating a KMS into the QbD architecture. It plays a key role in facilitating efficient transfer and utilization of information among the QbD activities and it facilitates to review the knowledge coming from a disparate variety of sources and then use it to assess the criticality of the various quality attributes. QbD enabled with KMS might be effective in managing the flow of information towards the development of process strategies, and technology transfer as well as use this information towards continuous improvement of the processes [14]. A commonly centralized data sharing system is used for exchange and sharing knowledge among the activities and the involved enablers of QbD that are not quite effective [74]. These kinds of repository systems may not operative since the enablers are geographically distributed and heterogeneous in terms of the operation system. A robust knowledge management approach is required to boost the data integration level of the QbD in CTs. A distributed unauthorized data repository and management system may improve the knowledge management process which universally facilitates data sharing. Besides, the generated knowledge by QbD is personal sensitive data that their misuse might harm the agreements, corporation policies, and participant privacy. The knowledge of the QbD should be stored in a high level of security should not be manipulable by any authority. It is not surprising that, well data integration among the QbD activities and enablers will improve the quality. In this work, BCT is embedded in the QbD through the development of DT aiming to develop a distributed knowledge sharing and management system to overcome the above-mentioned shortcomings. The developed DT might enhance the quality of the CT by empowering the activities of the QbD with a secure network of sharing knowledge.

3. Proposed blockchain-based Quality-embedded digital twin in clinical trials

QbD in CTs is a systematic approach that aims to ensure quality and satisfy the regulators by applying risk management in entire processes of clinical studies. The quality of a CT significantly depends on the protection of participants and data integration. Therefore, based on this argumentation, the main elements of QbD are redefined as Quality Targets (QTs), Quality Attributes (QAs), Critical Safety Attributes (CSAs), and Safety Control Strategies (SCSs). QTs in clinical trials are defined as the reliability and credibility of information collected during the clinical research process. Reliability refers to all the aspects that affect the safety of the participants. Besides, QAs are aspects that affect the reliability and credibility of results. The QAs that are directly related to participants' safety in clinical trials are considered as the CSAs. Thus, the clinical processes with high levels of risks and probabilities of occurrence and impacts under CSAs should be highlighted for risk management. As a result, suitable execution domains to the selected processes would be defined in a way that the risks should be mitigated. These appropriate execution domains for the processes are called safety control strategies. The SCSs should be considered during the trials, analyzation of the risks, and impacts of the processes.
Figure 1 represents the developed DT that embeds QbD with the blockchain for CTs. The QbD phases in the architecture are represented as a rectangle, and the blockchain is represented by peers shown as computer symbols. In the DT, the QbD phases are named plan, do, check, and act. Initially in “Plan”; the Critical Safety Attribute Identifier (CSAI) unit highlights the factors that affect (s) on CSAs, the Measurement Procedure Determiner (MPD) unit, provides the measurement approach of the selected factors, and the Safe Domain Identifier (SDI) unit, provides the safe domains/states of the selected factors. The decisions made by these units are based on information available on the trial protocol. In the “Do”, the Pre-Trial (PrT) unit determines the amounts or states of the factors using the recommended measurement approaches. The trials will be conducted when the amounts/states of the factors provided by the PrT are in acceptable domains/states provided by the SDI. The post-Trial (PoT) unit identifies amounts/states of the factors after clinical trials conduction according to the measurement approaches. In “Check”, the Risk Assessor (RA) unit highlights the factors with high risk, and the Root-Cause Analyser (RCA) unit performs root-cause analyses based on the prior and later amounts/states of the factors. Finally, the “Act” Safety Control Strategy Developer (SCSD) unit generates safety control strategies that will be used in the next cycle of the plan.
Peers are a fundamental element of the blockchain-based DT since they host ledgers and smart contracts. Two types of ledgers are considered in the KMS namely Trial ledger (T-ledger) and Patient ledger (P-ledger). The safe domains/states of the factors are deposited in the T-ledger, and the amounts/states of the factors for each participant are stored on P-ledger. The peers in the blockchain would have different obligations namely “single”, “Double”, “ DC issuer”, “endorser”, “orderer”, and “committer”. The “single” peer host T-ledger or P-ledger and the “Double” peer host both the ledgers and their smart contracts. The “DC issuer” peer issues a digital certificate for the participants which is known as an identity certificate. “Endorser” peer simulates the transaction proposal and then validates or refuses the requested transaction. The “Orderer” accomplishes the mining process for the new blocks of transactions and the “committer” appends the validated transactions to the channel-specific ledger.
In the system, the QbD is in connection with blockchain through the peers. These interconnections realize knowledge exchange, sharing, and management among the phases and the entire architecture. Depends on the type of knowledge which is generated or necessary in/for a phase of QbD, “single” and/or “Double” peers are employed. These peers realize knowledge acquisition and/or repository from/to the ledgers. Commonly, the “endorser” peers in this blockchain might belong to independent third parties and the “orderer” and “committer” peers might be authorized by a national or international health authority. Any other involved organization (i.e., regulators, third parties) that should have access to the ledgers hold a “single” or “Double” peer.

3.1. The QbD phases activities and their interactions with blockchain

The blockchain-enabled quality improvement DT is an iterative four-phase quality improvement method embedded with KMS. Figure 2 represents the activities of the planning phase in the architecture and its interconnection with the blockchain. As shown in the figure, initially, the CSAI unit screens the trial protocol of the project received from the sponsor and identifies the factors that affect critical safety attributes. The MPD unit based on the trial protocol of the project outlines the measurement approaches of the factors. For each of the factors, SDI unit determines the safe domain/state that will be written on the T-ledger by the API located in the planning phase. In the next cycle, the SDI unit develops the next safe domains/states considering the safety control strategies and updates the T-ledger.
Figure 3 demonstrates the activities and interactions of the do phase in the DT. This phase which is directly linked with the Clinical Research Organizations (CROs) is responsible to conduct the trials. As soon as a list of patients received from a CRO, the patient DC issuer unit issues digital certificates for each of the participants and writes this information on the P-ledger. The PrT unit recalls the patients' list from the P-ledger and safe domains/states of the trials from the T-ledger and obtain the amount/state of the factors for each of the participants using the measurement procedure provided by the MPD. After trial conduction, the PoT unit gains the amounts /states of the factors for each of the participants and writes the post-trial results on the P-ledger.
Figure 4 shows the activities of the check and act phases and their interactions with blockchain. As shown in the figure RA unit extract safe domains/states of the factors from the T-ledger and the post amounts/states of the factors of the participants from the P-ledger and examines the safety concerns. The results of these investigations and the pre amounts/stats of the factors of the participants are used by the RCA unit to identify hazards with the potential to cause harm. This unit provides the root of the high risks factors to the act. In the “act” phase, the roots are used by the SCSD unit to develop safety control strategies to be used in the next plan phase.

3.2. Blockchain knowledge structure

The structure of the blockchain is represented in Figure 5. As shown in this figure, one single, double, orderer, and committer peers and three endorser peers are represented in the architecture. A single peer holds a copy of T-ledger and its smart contract (chain code) and a double peer preserves a copy of T-ledger and P-ledger and their smart contracts. A single peer is capable to read/write information on the T-ledger through an external interrelated Application Programming Interface (API), while double peers can realize the same activities on both ledgers. API is a computing interface that is in connection with a single or double peer in the blockchain to accomplish read and write activities from/into a ledger. In the architecture, the APIs are placed in SDI, PrT, PoT, RA, and RCA units. An endorser peer checks the details of a broadcasted proposal (by a single or a double peer) and certificate details of the requester to validate the transaction. The miner peer provides ordering and committing services to the network. It packages transactions into blocks and commits/saves the transaction in the ledgers to be delivered to peers on the channel. In the blockchain smart contracts are installed to the peers to perform related operations, such as instantiating, invoking, packaging, querying, and upgrading chaincode. Two channels are considered in which channel peers are sharing the same ledger and the smart contract. Channel1 is for data exchange amongst the peers that hold T-ledger and channel2 is for peers with P-ledger. In the network six Certificate Authority (CA) are considered to issue a digital certificate for the peers.

3.3. QbD interaction with blockchain

In total, four kinds of interaction can be accomplished by the APIs in the blockchain. The API of the SDI unit demands read and write actions from the T-ledger, the API of PrT unit requests read action from the T-ledger and write action to the P-ledger, the API of PoT unit write information to the P-ledger, the API of the RA unit request read action form the P-ledger and T-ledger, and the API of RCA unit read information from P-ledger. Figure 6 demonstrates the read and write procedure sequences that are requested by an API of a unit in the architecture. As shown in the figure an instance of an API, request to read information from a ledger. The smart contract of the API’s peer will be invoked and extract the requested information from the ledger and forward this information to the API. To write a transaction into a ledger, API develops a transaction proposal and forwards it to the API’s peer. This peer broadcasts the transaction proposal to the endorser peers. The endorsers simulate the transaction and provide the endorsed or refused transaction to the API through its peer for finalization. The final version of the endorsed transaction is forwarded to mining peers through the API’s peer. The mined transaction is written to the related ledger.

4. Case study

To verify the proposed blockchain-based digital twin embedded with QbD architecture, a simulation platform developed using Java Application Descriptor (JAD). The BCN is developed based on Hyperledger Fabric technology, using open-source Hyperledger composer. This tool is suitable for a private blockchain business environment and is based on JavaScript. The detailed fundamental implementation of BCT on a Hyperledger Fabric has been illustrated in (https://github.com/IBM/build-blockchain-insurance-app). The data of a completed pilot trial for the assessment of relative bioavailability in a generic drug product development is used for simulation.
The deployment diagram of the developed simulation platform is represented in Figure 7. In this platform, six servers are interconnected through TCP/IP and/or Ethernet links based on the blockchain-embedded QbD architecture. Four servers simulate plan, do, check, and act entities while two more act as endorsers, and orderer/committer peers of the blockchain networks.
In the simulation platform, each of the servers may host some artifacts, components, and APIs. The artifacts exhibit the characteristics of the entities and can interact with others to obtain output results. The components of the servers execute as a peer in the blockchain network. These peers maintain a copy of ledgers (P-ledger and/or T-ledger) and update the ledgers when an alteration happened in the network. There are three roles for a component – endorser, committer, and orderer. All the components have been designed such that a component is always a committer. The components of server1, and server 2 are designed to perform a role in the endorsing and ordering of transactions. API is an intermediary software with a set of definitions and protocols that allows artifacts to talk with components. Plan, do and check servers host APIs to interact with the blockchain network.
Figure 8 represents the illustration of the interactions in the DT. Matrix notation is used to simplify the presentation of interactions that are performed in the simulation platform. Three different types of matrixes are used in this illustration. Round brackets are for representing information that will be stored on local data repository systems. The information of the square brackets will be stored on P-ledger and the curly brackets are used to represent the information that will be stored on T-ledger in the blockchain. As shown in the figure, each artifact in the platform generates matrix(s) which are used as input for the interrelated artifacts and/or components.
Using feedback from experts, the artifacts of the plan server extract the factors of the trial, their measurement procedures as well as the safe domains/states. As shown in the Table 1, four subjective (F1-F4) and one objective (F5) factors are selected by the CSAI as the elements that affect the critical safety attributes of the trial, and the expected measurement approaches for each of the factors are provided by the MPD and the safe domains/states of the factors are identified by the SDI all according to the data available on the protocol. The MPD unit stores the extracted measurement approaches on the local database and this information is shared with the do server. It is SDI responsibility to develop a proposal to write the extracted safe domains/states of the factors to the T-ledger through the API-1. As soon as the developed proposal is endorsed the transactions are written to the ledger.
The do server started to work when the measurement approaches are provided by the plan and a list of the eligible participants are provided to the DCI unit. The DCI stores all the personal information of the participants on the local database and reads the required factors from T-ledger and develops a transaction for each of the participants. This transaction contains important demographic information of the participants, the value of the factors (considered as N/A), and the status of the conduction that is marked as “Initial”. The developed transactions are written to the P-ledger by the DCI unit. The artifact of the TO reads the transactions with “Initial” status from the P-ledger, the safe domains/states of the factors from the T-ledger, and the measurement approach of the factors from the local database to develop an order that enforces the CRO for conducting the trial according to the provided instructions. For each of the participants, the OC collects the outcomes of the factors before and after drug administration and develops transactions to be written in the P-ledger that holds the status of “Before” or “After”. Table 2 provides the information extracted from the P-ledger of the pilot trial.
In the check server, the RA unit read the safe domains/stats from the T-ledger and recall the after-drug administration transactions from the P-Ledger and highlight the participants that contain unsafe values. As shown in the table after drug administration F2 value of the P3, P8, P23, and P28 are on the unsafe borderline. RA forward the ID of the participants to the RCA unit for further considerations. The RCA reads all the transactions related to the selected participants from the P-ledger and their safe domains/states from the T-ledger to perform a root cause analysis for finding relations among the demographic information of the patricians, the value of the factors before administrations, and the unsafe values. Based on the P23 and P28 information, it is determined that the female participants' age between 60-65, low heart rate, and high fasting blood sugar may reveal an unsafe value of high heart rate after drug administration. It also committed that the male participants in the 35-40 age range (i.e., P3, P8), high heart rate, and high before meal blood sugar level may show the unsafe value of heart rate after drug administration. These roots of the unsafe values are forwarded to the SCSD unit in the act server where the unit develops a safety control strategy as a recommendation for the plan server. One possible safety control strategy is to exclude or limit the future participants holding the same specifications. This recommendation will be used by the artifacts in the plan in the next trials.

5. Discussion

The experience gained during the case study implementation allows us to draw some conclusions about the operation of the blockchain-based DT embedded with the QbD system. It was verified that the system works as specified in both the QbD and the blockchain sides, thus validating the robustness of the developed DT. Additionally, the artifacts, components, and APIs of the developed system were proven by their accurate reactions to the assigned responsibilities. The system identifies some unsafe borderlines values which are neglected during the real-life pilot trial. This system provides trustable data integration with clinical trial management and clinical data management systems with extracting sponsors, ethic committee, and regulators features to construct DT to be used by the CROs, based on participant safety as the main enabler of the quality in clinical research. However, some following questions remain for investigation:
  • How to realize the real development and application of the developed system?
  • How to persuade the sponsors, CROs, and regulating authorities to use current technologies in this field?
  • How do we ensure security during the process of collaboration?
To realize the gap between the clinical researches, the advanced technologies, and the proposed DT, it is mandatory to negotiate between all stakeholders in a drug life cycle to discuss how to implement this architecture in clinical trial studies to grasp its advantages and disadvantages. Because of the absence of clear standards of regulating authorities or little standardization, the stakeholder may have challenges. By growing the BCT and related infrastructures, the standards should be upgraded. Moreover, the quality concerns in the clinical trials may improve drastically by these approaches. Besides, the potential security risks of BCT can be mitigated by cybersecurity professionals as safely as possible.
This study has some limitations: Insufficient Sample size: the sample size of the study for proof of concept was based on a pilot clinical trial with 30 participants. This sample size is insufficient for obtaining precise and comprehensive results. By employing the data of different clinical trials with a larger population in three different phases of trials can be generalized the feasibility and practicability of the system. Lack of previous extensive research studies: currently, scant previous research and inadequately developed models fail to answer how and where QbD is effectively applicable and how these approaches can be employed in clinical trials efficiently. Further researches are needed to overcome this challenge. In future works, the architecture can be used in real-life clinical trials and the medical cyber-physical system can be used for real-time monitoring of the patients and equipment, artificial intelligence-based tools can be used to estimate the efficacy and accuracy of the architecture.

6. Conclusion:

In this paper, the blockchain-enabled digital twin embedded with quality by design for the clinical trials stage of drug development is developed and the applicability of the DT is tested using the data of a trial with 30 participants. The unified modeling language is used to present the architecture. The DT contains two main parts including QbD and KMS that are integrated through the internet. The plan, do, check, and act framework is used to develop the QbD while blockchain is employed to build distributed immutable ledger technology for storing the safety-related knowledge of the project. The DT is working in a way that, first the protocol of a trial is screened to highlight the parameters that might raise safety concerns for the participants, then the value of these subjective and objective concerns before and after drug administration are extracted, finally using root cause analysis, a control strategy is developed for the next trails. All the safety-related data of the participants are stored on a two-blockchain ledger. The reasons to employ blockchain instead of conventional knowledge repository technics are; the stored knowledge cannot be altered or deleted, a high level of data security including less risk of duplicate entry or fraud, time-stamped, transparency, scalability, and traceability.

References

  1. Hariry, R. E., & Barenji, R. V. (2023). Embracing Digital Technologies in the Pharmaceutical Industry. In Control Engineering in Mechatronics (pp. 141-165). Singapore: Springer Nature Singapore.
  2. Hariry, R. E., Barenji, R. V., & Azizi, A. (2022). Toward Pharma 4.0 in Drug Discovery. In Industry 4.0: Technologies, Applications, and Challenges (pp. 221-238). Singapore: Springer Nature Singapore.
  3. Yao, B., Zhu, L., Jiang, Q., & Xia, H. A. (2013). Safety monitoring in clinical trials. Pharmaceutics, 5(1), 94-106. [CrossRef]
  4. Little, R. J., D'Agostino, R., Cohen, M. L., Dickersin, K., Emerson, S. S., Farrar, J. T., ... & Stern, H. (2012). The prevention n and treatment of missing data in clinical trials. New England Journal of Medicine, 367(14), 1355-1360. [CrossRef]
  5. Bechtel, J., Chuck, T., Forrest, A., Hildebrand, C., Panhuis, J., Pattee, S. R., ... & Swezey, T. (2020). Improving the quality conduct and efficiency of Clinical Trials with Training: recommendations for preparedness and qualification of investigators and delegates. Contemporary clinical trials, 89, 105918. [CrossRef]
  6. Meeker-O’Connell, A., & Glessner, C. (2018). Clinical trial quality: From supervision to collaboration and beyond. Clinical Trials, 15(1_suppl), 23-26. [CrossRef]
  7. Brosteanu, O., Houben, P., Ihrig, K., Ohmann, C., Paulus, U., Pfistner, B., ... & Zettelmeyer, U. (2009). Risk analysis and risk adapted on-site monitoring in noncommercial clinical trials. Clinical trials, 6(6), 585-596. [CrossRef]
  8. Guideline, I. H. T. (2001). Guideline for good clinical practice. J Postgrad Med, 47(3), 199-203.
  9. Meeker-O’Connell, A., Glessner, C., Behm, M., Mulinde, J., Roach, N., Sweeney, F., ... & Landray, M. J. (2016). Enhancing clinical evidence by proactively building quality into clinical trials. Clinical trials, 13(4), 439-444. [CrossRef]
  10. Sangshetti, J. N., Deshpande, M., Zaheer, Z., Shinde, D. B., & Arote, R. (2017). Quality by design approach: Regulatory need. Arabian Journal of Chemistry, 10, S3412-S3425. [CrossRef]
  11. Rathore, A. S., & Winkle, H. (2009). Quality by design for biopharmaceuticals. Nature biotechnology, 27(1), 26-34. [CrossRef]
  12. Korakianiti, E., & Rekkas, D. (2011). Statistical thinking and knowledge management for quality-driven design and manufacturing in pharmaceuticals. Pharmaceutical research, 28(7), 1465-1479. [CrossRef]
  13. Lipa M. (2011, November 14-16). Knowledge Management: An Iterative Process. Pharmaceutical Quality System (ICH Q10) Conference. Brussels, Belgium https://www.fda.gov/media/84768/download.
  14. Rathore, A. S., Garcia-Aponte, O. F., Golabgir, A., Vallejo-Diaz, B. M., & Herwig, C. (2017). Role of knowledge management in development and lifecycle management of biopharmaceuticals. Pharmaceutical research, 34(2), 243-256. [CrossRef]
  15. Barenji, R. V. (2021). A blockchain technology based trust system for cloud manufacturing. Journal of Intelligent Manufacturing, 1-15. [CrossRef]
  16. Hasselgren, A., Kralevska, K., Gligoroski, D., Pedersen, S. A., & Faxvaag, A. (2020). Blockchain in healthcare and health sciences—A scoping review. International Journal of Medical Informatics, 134, 104040. [CrossRef]
  17. Saadati, Z., Zeki, C. P., & Vatankhah Barenji, R. (2021). On the development of blockchain-based learning management system as a metacognitive tool to support self-regulation learning in online higher education. Interactive Learning Environments, 1-24. [CrossRef]
  18. Kuo, T. T., Kim, H. E., & Ohno-Machado, L. (2017). Blockchain distributed ledger technologies for biomedical and health care applications. Journal of the American Medical Informatics Association, 24(6), 1211-1220. [CrossRef]
  19. Kuo, T. T., Zavaleta Rojas, H., & Ohno-Machado, L. (2019). Comparison of blockchain platforms: a systematic review and healthcare examples. Journal of the American Medical Informatics Association, 26(5), 462-478. [CrossRef]
  20. Agbo, C. C., Mahmoud, Q. H., & Eklund, J. M. (2019, June). Blockchain technology in healthcare: a systematic review. In Healthcare (Vol. 7, No. 2, p. 56). Multidisciplinary Digital Publishing Institute. [CrossRef]
  21. Bentley, C., Cressman, S., van der Hoek, K., Arts, K., Dancey, J., & Peacock, S. (2019). Conducting clinical trials—costs, impacts, and the value of clinical trials networks: a scoping review. Clinical Trials, 16(2), 183-193. [CrossRef]
  22. Sertkaya, A., Wong, H. H., Jessup, A., & Beleche, T. (2016). Key cost drivers of pharmaceutical clinical trials in the United States. Clinical Trials, 13(2), 117-126. [CrossRef]
  23. Benchoufi, M., & Ravaud, P. (2017). Blockchain technology for improving clinical research quality. Trials, 18(1), 1-5. [CrossRef]
  24. Evans, S. R. (2010). Common statistical concerns in clinical trials. Journal of experimental stroke & translational medicine, 3(1), 1.
  25. Nugent, T., Upton, D., & Cimpoesu, M. (2016). Improving data transparency in clinical trials using blockchain smart contracts. F1000Research, 5. [CrossRef]
  26. Kaur, S., & Singh, I. (2017). Artificial intelligence based clinical data management systems: a review. Informatics in Medicine Unlocked, 9, 219-229. [CrossRef]
  27. Hirano, T., Motohashi, T., Okumura, K., Takajo, K., Kuroki, T., Ichikawa, D., ... & Ueno, T. (2020). Data validation and verification using blockchain in a clinical trial for breast cancer: Regulatory sandbox. Journal of Medical Internet Research, 22(6), e18938. [CrossRef]
  28. Gordon, W. J., & Catalini, C. (2018). Blockchain technology for healthcare: facilitating the transition to patient-driven interoperability. Computational and structural biotechnology journal, 16, 224-230. [CrossRef]
  29. Santos, J. A., Inácio, P. R., & Silva, B. M. (2021). Towards the Use of Blockchain in Mobile Health Services and Applications. Journal of Medical Systems, 45(2), 1-10. [CrossRef]
  30. Cheng, X., Chen, F., Xie, D., Sun, H., & Huang, C. (2020). Design of a secure medical data sharing scheme based on blockchain. Journal of medical systems, 44(2), 1-11. [CrossRef]
  31. Alonso, S. G., Arambarri, J., López-Coronado, M., & de la Torre Díez, I. (2019). Proposing new blockchain challenges in ehealth. Journal of medical systems, 43(3), 64. [CrossRef]
  32. Kaur, H., Alam, M. A., Jameel, R., Mourya, A. K., & Chang, V. (2018). A proposed solution and future direction for blockchain-based heterogeneous medicare data in cloud environment. Journal of medical systems, 42(8), 1-11. [CrossRef]
  33. Wang, H., & Song, Y. (2018). Secure cloud-based EHR system using attribute-based cryptosystem and blockchain. Journal of medical systems, 42(8), 1-9. [CrossRef]
  34. Li, H., Zhu, L., Shen, M., Gao, F., Tao, X., & Liu, S. (2018). Blockchain-based data preservation system for medical data. Journal of medical systems, 42(8), 1-13. [CrossRef]
  35. Zhang, A., & Lin, X. (2018). Towards secure and privacy-preserving data sharing in e-health systems via consortium blockchain. Journal of medical systems, 42(8), 1-18. [CrossRef]
  36. Choudhury, O., Sarker, H., Rudolph, N., Foreman, M., Fay, N., Dhuliawala, M., ... & Das, A. K. (2018). Enforcing human subject regulations using blockchain and smart contracts. Blockchain in healthcare Today, 1, 1-14. [CrossRef]
  37. Choudhury, O., Fairoza, N., Sylla, I., & Das, A. (2019). A blockchain framework for managing and monitoring data in multi-site clinical trials. arXiv preprint arXiv:1902.03975.
  38. Maslove, D. M., Klein, J., Brohman, K., & Martin, P. (2018). Using blockchain technology to manage clinical trials data: a proof-of-concept study. JMIR medical informatics, 6(4), e11949. [CrossRef]
  39. Arabian, M., Ghadiri Nejad, M., & Barenji, R. V. (2022). Blockchain Technology in Supply Chain Management: Challenge and Future Perspectives. In Industry 4.0: Technologies, Applications, and Challenges (pp. 201-220). Singapore: Springer Nature Singapore.
  40. Albanese, G., Calbimonte, J. P., Schumacher, M., & Calvaresi, D. (2020). Dynamic consent management for clinical trials via private blockchain technology. Journal of ambient intelligence and humanized computing, 1-18. [CrossRef]
  41. EMA. https://www.ema.europa.eu/en/documents/other/report-ema-fda-qbd-pilot-program_en.pdf.
  42. EMA. https://www.ema.europa.eu/en/documents/scientific-guideline/international-conference-harmonisation-technical-requirements-registration-pharmaceuticals-human-use_en-11.pdf.
  43. Zurdo, J., Arnell, A., Obrezanova, O., Smith, N., Gómez de la Cuesta, R., Gallagher, T. R., ... & Höidén-Guthenberg, I. (2015). Early implementation of QbD in biopharmaceutical development: a practical example. BioMed research international, 2015.
  44. Rathore, A. S. (2014). QbD/PAT for bioprocessing: moving from theory to implementation. Current Opinion in Chemical Engineering, 6, 1-8. [CrossRef]
  45. Lionberger, R. A., Lee, S. L., Lee, L., Raw, A., & Lawrence, X. Y. (2008). Quality by design: concepts for ANDAs. The AAPS journal, 10(2), 268-276. [CrossRef]
  46. Xu, X., Khan, M. A., & Burgess, D. J. (2011). A quality by design (QbD) case study on liposomes containing hydrophilic API: I. Formulation, processing design and risk assessment. International journal of pharmaceutics, 419(1-2), 52-59. [CrossRef]
  47. Rapalli, V. K., Khosa, A., Singhvi, G., Girdhar, V., Jain, R., & Dubey, S. K. (2019). Application of QbD principles in nanocarrier-based drug delivery systems. In Pharmaceutical Quality by Design (pp. 255-296). Academic Press.
  48. Gupta, S., & Jhawat, V. (2017). Quality by design (QbD) approach of pharmacogenomics in drug designing and formulation development for optimization of drug delivery systems. Journal of Controlled Release, 245, 15-26. [CrossRef]
  49. Mishra, V., Thakur, S., Patil, A., & Shukla, A. (2018). Quality by design (QbD) approaches in current pharmaceutical set-up. Expert opinion on drug delivery, 15(8), 737-758. [CrossRef]
  50. Fukuda, I. M., Pinto, C. F. F., Moreira, C. D. S., Saviano, A. M., & Lourenço, F. R. (2018). Design of experiments (DoE) applied to pharmaceutical and analytical quality by design (QbD). Brazilian Journal of Pharmaceutical Sciences, 54(SPE). [CrossRef]
  51. Chatterjee, S. (2013, January). QbD considerations for analytical methods—FDA perspective. In US IFPAC annual meeting.
  52. Lloyd, D. K., Bergum, J., & Wang, Q. (2020). Application of quality by design to the development and validation of analytical methods. In Specification of Drug Substances and Products (pp. 59-99). Elsevier.
  53. Rathore, A. S. (2009). Roadmap for implementation of quality by design (QbD) for biotechnology products. Trends in biotechnology, 27(9), 546-553. [CrossRef]
  54. Zhang, L., & Mao, S. (2017). Application of quality by design in the current drug development. Asian journal of pharmaceutical sciences, 12(1), 1-8. [CrossRef]
  55. Javed, M. N., Alam, M. S., Waziri, A., Pottoo, F. H., Yadav, A. K., Hasnain, M. S., & Almalki, F. A. (2019). QbD applications for the development of nanopharmaceutical products. In Pharmaceutical quality by design (pp. 229-253). Academic Press.
  56. Beg, S., Rahman, M., & Kohli, K. (2019). Quality-by-design approach as a systematic tool for the development of nanopharmaceutical products. Drug discovery today, 24(3), 717-725. [CrossRef]
  57. Kappatou, C. D., Ehsani, A., Niedenführ, S., Mhamdi, A., Schuppert, A., & Mitsos, A. (2020). Quality-targeting dynamic optimization of monoclonal antibody production. Computers & Chemical Engineering, 142, 107004.
  58. Hariry, R. E., Barenji, R. V., & Paradkar, A. (2022). Towards Pharma 4.0 in clinical trials: a future-orientated perspective. Drug Discovery Today, 27(1), 315-325.
  59. Huang, G. D., Bull, J., McKee, K. J., Mahon, E., Harper, B., & Roberts, J. N. (2018). Clinical trials recruitment planning: a proposed framework from the clinical trials transformation initiative. Contemporary clinical trials, 66, 74-79. [CrossRef]
  60. Tenaerts, P., Madre, L., & Landray, M. (2018). A decade of the Clinical Trials Transformation Initiative: What have we accomplished? What have we learned?. Clinical Trials, 15(1_suppl), 5-12. [CrossRef]
  61. Yu, L. X., Amidon, G., Khan, M. A., Hoag, S. W., Polli, J., Raju, G. K., & Woodcock, J. (2014). Understanding pharmaceutical quality by design. The AAPS journal, 16(4), 771–783. [CrossRef]
  62. FDA. https://www.fda.gov/media/71012/download.
  63. EMA. https://www.ema.europa.eu/en/documents/presentation/quality-design-process-analytical-technology-risk-based-cmc-development-kowid-ho_en.pdf.
  64. EMA. https://www.ema.europa.eu/en/documents/scientific-guideline/international-conference-harmonisation-technical-requirements-registration-pharmaceuticals-human-use_en-3.pdf.
  65. FDA. https://www.fda.gov/media/71553/download.
  66. EMA. https://www.ema.europa.eu/en/documents/scientific-guideline/draft-ich-guideline-q11-development-manufacture-drug-substances-chemical-entities-biotechnological/biological-entities_en.pdf.
  67. EMA. https://www.ema.europa.eu/en/documents/scientific-guideline/ich-guideline-q12-technical-regulatory-considerations-pharmaceutical-product-lifecycle-management_en.pdf.
  68. Pramod, K., Tahir, M. A., Charoo, N. A., Ansari, S. H., & Ali, J. (2016). Pharmaceutical product development: A quality by design approach. International journal of pharmaceutical investigation, 6(3), 129–138. [CrossRef]
  69. EMA. https://www.ema.europa.eu/en/documents/scientific-guideline/ich-e-6-r2-guideline-good-clinical-practice-step-5_en.pdf.
  70. EMA. https://www.ema.europa.eu/en/documents/scientific-guideline/international-conference-harmonisation-technical-requirements-registration-pharmaceuticals-human-use_en-11.pdf.
  71. Sprenger, K., Nickerson, D., Meeker-O’Connell, A., & Morrison, B. W. (2013). Quality by design in clinical trials: a collaborative pilot with FDA. Therapeutic innovation & regulatory science, 47(2), 161-166. [CrossRef]
  72. Su, Q., Bommireddy, Y., Shah, Y., Ganesh, S., Moreno, M., Liu, J., ... & Nagy, Z. K. (2019). Data reconciliation in the Quality-by-Design (QbD) implementation of pharmaceutical continuous tablet manufacturing. International journal of pharmaceutics, 563, 259-272. [CrossRef]
  73. Barenji, R. V., Akdag, Y., Yet, B., & Oner, L. (2019). Cyber-physical-based PAT (CPbPAT) framework for Pharma 4.0. International journal of pharmaceutics, 567, 118445. [CrossRef]
  74. Zeinali, M., & Barenji, R. V. (2023). Communication Networks Characteristics Impact on Cyber-Physical Systems. In Control Engineering in Mechatronics (pp. 189-202). Singapore: Springer Nature Singapore.
  75. Saadati, Z., & Barenji, R. V. (2022). Toward Industry 5.0: Cognitive Cyber-Physical System. In Industry 4.0: Technologies, Applications, and Challenges (pp. 257-268). Singapore: Springer Nature Singapore.
  76. Herwig, C., Garcia-Aponte, O. F., Golabgir, A., & Rathore, A. S. (2015). Knowledge management in the QbD paradigm: manufacturing of biotech therapeutics. Trends in biotechnology, 33(7), 381-387. [CrossRef]
Preprints 74420 g001
Figure 1. Blockchain-embedded Quality improvement digital twin for clinical trials.
Figure 1. Blockchain-embedded Quality improvement digital twin for clinical trials.
Preprints 74420 g001
Preprints 74420 g002
Figure 2. “Plan” phase activities and interactions with the blockchain.
Figure 2. “Plan” phase activities and interactions with the blockchain.
Preprints 74420 g002
Preprints 74420 g003
Figure 3. “Do” phase activities and interactions with the blockchain.
Figure 3. “Do” phase activities and interactions with the blockchain.
Preprints 74420 g003
Preprints 74420 g004
Figure 4. “Check” and “Act” phase activities and interactions with blockchain.
Figure 4. “Check” and “Act” phase activities and interactions with blockchain.
Preprints 74420 g004
Preprints 74420 g005
Figure 5. Blockchain-based KMS Structure.
Figure 5. Blockchain-based KMS Structure.
Preprints 74420 g005
Preprints 74420 g006
Figure 6. Read and Write processes sequences of architecture.
Figure 6. Read and Write processes sequences of architecture.
Preprints 74420 g006
Preprints 74420 g007
Figure 7. Simulation platform deployment diagram.
Figure 7. Simulation platform deployment diagram.
Preprints 74420 g007
Preprints 74420 g008
Figure 8. Matrix notation of the QbD interactions in the architecture.
Figure 8. Matrix notation of the QbD interactions in the architecture.
Preprints 74420 g008
Table 1. Critical safety factors, their measurement methods, and safe domains/states.
Table 1. Critical safety factors, their measurement methods, and safe domains/states.
Factor Measurement approach Safe domain/state
F1: Blood Pressure (BP) Every 30min/24 hrs
   Breakfast: 8:00-80:30
   Lunch: 12:00-13:00
   Dinner: 18:00-18:40		 80-130 mmHg
F2: Heart Rate (HR) Every 15min/24hr
   Male: walking-Max
     60min /24 hrs
   Female: walking-Max
     50mins /24 hrs
   Watch TV: Max 30 min
     (no action)
   Use Phone: Max 30 min		 Age <40: 90-153 b/min
Age 41-45: 88-149 b/min
Age 46-50: 80-145 b/min
Age 51-55: 83-140 b/min
Age: 56-60: 80-136 b/min
Age: 61-65: 78-132 b/min
F3: Blood Sugar (BS) F3.1: Fasting
F3.2: 30min before meal
F3.3: 60 min after meal
F3.4: Bed time
   Breakfast < 1100 calorie
   Lunch< 1200 calorie
   Dinner < 800 calorie		 Fasting: less than 100 mg/dL
Before meal: 70-130 mg/dL
After meal: less than 180 mg/dL
Bed time: 100-140 mg/dL
F4: Stay on bed Before administration
After administration
Sleep style : Stargazer or Skydiver		 Before administration : 60min
After administration: 240 min
F5: Ambient Temperature Every 30mins/24 hrs 19-25 C
Table 2. P-ledger of the pilot trial.
Table 2. P-ledger of the pilot trial.
Participants Demographic Information Before Administration After Administration
Sex Age Weight (Kg) F1 F2 F3.1 F3.2 F3.3 F3.4 F4 F5 F1 F2 F3.1 F3.2 F3.3 F3.4 F4 F5
P1 M 55 70 129 122 90 88 167 103 57 23 117 101 90 129 163 103 231 22
P2 F 48 60 126 89 77 117 153 110 56 23 103 137 77 125 146 110 227 22
P3 M 39 72 86 140 93 127 144 102 58 23 94 152 93 77 172 102 234 22
P4 F 52 66 126 82 99 88 146 111 56 23 96 101 99 126 142 111 227 22
P5 M 48 80 103 126 85 79 133 107 56 23 83 89 85 97 166 107 227 22
P6 F 35 76 86 130 98 106 135 104 60 23 85 122 98 78 167 104 240 22
P7 F 25 58 98 105 82 114 155 121 52 23 105 133 82 91 155 121 215 22
P8 M 36 74 121 152 90 128 134 135 51 23 84 185 90 119 179 135 214 22
P9 M 28 78 103 152 98 107 166 130 58 23 116 124 98 98 179 130 234 22
P10 F 56 60 111 125 96 119 142 110 57 23 92 140 96 107 165 110 230 22
P11 F 47 65 103 82 87 117 155 135 55 23 105 136 87 97 142 135 225 22
P12 M 66 75 121 143 78 85 159 126 53 23 109 97 78 119 175 126 219 22
P13 M 26 68 97 92 97 108 163 107 57 23 113 126 97 91 147 107 232 22
P14 F 40 76 122 101 95 123 131 116 54 23 81 144 95 120 152 116 222 22
P15 M 49 79 81 109 92 83 176 136 54 23 126 94 92 71 157 136 223 22
P16 M 56 83 86 137 78 103 131 137 54 23 81 119 78 77 172 137 221 22
P17 F 60 75 88 140 92 72 177 139 53 23 127 80 92 80 173 139 218 22
P18 F 31 70 87 119 79 84 134 116 55 23 84 95 79 78 162 116 226 22
P19 M 42 83 86 118 75 92 154 101 52 23 104 106 75 77 161 101 217 22
P20 M 49 85 112 149 94 79 148 103 59 23 98 89 94 108 178 103 237 22
P21 F 35 71 104 117 83 106 144 127 57 23 94 123 83 99 161 127 230 22
P22 M 59 79 94 134 90 109 134 124 53 23 84 127 90 86 170 124 220 22
P23 F 61 67 82 78* 99* 83 142 134 59 23 92 152* 98 73 140 134 238 22
P24 F 50 79 109 99 86 108 161 134 51 23 111 126 86 105 151 134 212 22
P25 M 44 81 109 121 78 129 164 112 59 23 114 151 78 105 163 112 236 22
P26 F 39 66 104 122 86 113 179 130 53 23 129 132 86 99 163 130 219 22
P27 M 36 77 124 132 81 113 175 137 56 23 125 131 81 123 169 137 228 22
P28 F 64 72 97 80* 98* 95 143 102 53 23 93 146* 97 90 143 102 219 22
P29 M 24 72 89 105 84 104 176 114 56 23 126 121 84 81 154 114 227 22
P30 M 29 80 109 132 81 105 155 134 60 23 105 121 81 105 169 134 239 22
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.
Copyright: This open access article is published under a Creative Commons CC BY 4.0 license, which permit the free download, distribution, and reuse, provided that the author and preprint are cited in any reuse.
Prerpints.org logo

Preprints.org is a free preprint server supported by MDPI in Basel, Switzerland.

facebook logotwitter logolinkedin logo
MDPI Initiatives
Important Links
Subscribe

Disclaimer

Terms of Use

Privacy Policy

© 2024 MDPI (Basel, Switzerland) unless otherwise stated