China’s National Medical Products Administration (NMPA) has published a guideline for the registration review of artificial intelligence (AI) medical devices (issued by the Center for Medical Device Evaluation in 2022). This article summarises its key concepts.
Key concepts
Definition
An AI medical device is a device that uses AI technology to achieve its medical purpose based on “medical device data” – such as medical images, physiological parameters and in vitro diagnostic data. In special cases, data generated by general-purpose equipment for medical purposes are also included. In this context, AI and machine learning have essentially the same meaning.
Types of AI medical devices
- By software form: standalone software and software components.
- By intended use: decision-support devices that provide diagnostic or treatment suggestions, and non-decision-support devices that only provide reference information; both can be real-time or non-real-time.
- By function: pre-processing and post-processing functions, control functions and safety functions.
- By algorithm: several classification dimensions.
- By maturity: established and novel types.
Basic principles of the registration review
- The applicant selects suitable algorithms based on their characteristics and carries out verification and validation.
- The software safety level is determined based on the intended use and other factors, and risk management is carried out accordingly.
- A lifecycle process is established, with risk management applied throughout and algorithm performance evaluated continuously before and after market approval.
Algorithm research report
The report covers basic algorithm information, risk management, requirements specification, data quality control, algorithm training, algorithm performance evaluation, verification and validation, traceability analysis and conclusions. Any content that does not apply must be justified in detail.
Key considerations in the algorithm research report
Algorithm characteristics are the core focus of the safety and effectiveness evaluation, with quality control focused on the model and the data. Different algorithms, use scenarios and evaluation priorities differ, so applicants should choose suitable algorithms or combinations and carry out verification and validation based on algorithm characteristics and risk management.
- Risk management: the risk level can be expressed as the software safety level – the higher the level, the stricter the documentation requirements. It is determined by intended use, use scenario, core functions or risk class. Main risks include overfitting and underfitting of the algorithm and false-negative and false-positive results in use.
- Data quality control: requirements for data collection, curation, annotation and dataset construction. Data collection must define equipment, processes and de-identification; curation requires cleaning, pre-processing and controlled software tools; annotation is a critical step requiring defined rules; datasets must be split appropriately into training, tuning and test sets, and small datasets may be augmented.
- Algorithm training and performance evaluation: document the basic information and rationale for algorithm selection; train on the training and tuning sets with defined evaluation metrics; evaluate performance on the test set using multiple metrics, analyse influencing factors and perform stress testing and adversarial testing.
- Verification and validation: software verification confirms, based on requirements, that safety and effectiveness are ensured; software validation confirms that the software meets user needs and the intended purpose, including validation testing and clinical evaluation. Clinical evaluation depends on function and maturity, followed by an overall evaluation of algorithm performance.
How EVERBIZ can help
EVERBIZ shares its practical experience in quality systems and product registration with manufacturers and supports NMPA registration of software and AI medical devices. Contact us.
This article is for general information only and does not constitute legal advice. The current NMPA guideline is authoritative.