Traceability#

In Brief#

Traceability can be defined as the need to maintain a complete and clear documentation of the data, processes, artefacts and actors involved in the entire lifecycle of an AI model, starting from its design and ending with its production serving [2].

Abstract#

This entry introduces the motivations behind traceability and illustrates its core requirements, which encompass documenting the entire development cycle of an AI model and tracking its live functioning after the deployment in production.

Motivation and Background#

Developing an Artificial Intelligence (AI) model or an AI-powered system entails a considerable number of choices along the entire development process, which may result in diverse behaviours and functioning of the same model or system. This phenomenon is particularly relevant when learning-based approaches comes into play, due to the dependency of Machine Learning (ML) models on the data used for their training as well as the complexity and variety of the ML methods that might be used, especially when based on Deep Learning (DL). Furthermore, the development of such models relies often on large trial-&-error experimental processes, which are not commonly well documented (see the reproducibility entry).

This condition makes it evident the need for a comprehensive and clear documentation of the actions taken as well as the various processing steps performed when developing an AI or ML model, as, without this documentation, it might be difficult to reconstruct the reasons behind the outcomes and the functioning of an AI model. In consideration of this, the High-Level Expert Group on AI (AI HLEG) has included the traceability of an AI model as one of the main mean to enable the transparency principle for Trustworthy AI [1]. Overall, traceability aims to ensure the avoidance of any “grey” area about the AI model or system, thus guaranteeing the transparency of and the trust in the development, production functioning and usage of an AI system. The record-keeping activity entailed by traceability should regard the data used, the data pre-processing steps as well as the development settings, the development workflows and the actors involved [3]. This encompasses the detailed provision of information about the provenance and the usage of any data and artefacts involved in the development of the AI model or system. In this view, traceability incorporates the measures to ensure reproducibility and it can be understood as the technological mean for guaranteeing the auditability and accountability of AI models and systems [4].

The two souls of traceability#

AI models based on learning are data-inductive and dynamic systems, whose development relies on an initial set of data. This set, although large, might not necessarily span the whole variability range of real-world cases or conditions. This implies that, when used in practice, the AI model or system can encounter slightly different or novel data that differ from those it has been exposed to during training. This phenomenon calls for the monitoring of AI models after their deployment in production, in order to log their usage as well as to track over time their performance, vitality and conduct. Such an AI maintenance system is an important part of traceability, which can be then seen as one principle, two souls:

These two aspects will be further explained in the following entries.

Bibliography#

1

European Commission, Content Directorate-General for Communications Networks, and Technology. Ethics guidelines for trustworthy AI. Publications Office, 2019. URL: https://digital-strategy.ec.europa.eu/en/library/ethics-guidelines-trustworthy-ai.

2

European Commission, Content Directorate-General for Communications Networks, and Technology. The Assessment List for Trustworthy Artificial Intelligence (ALTAI) for self assessment. Publications Office, 2020. URL: https://doi.org/10.2759/002360, doi:10.2759/002360.

3

Michael Bücker, Gero Szepannek, Alicja Gosiewska, and Przemyslaw Biecek. Transparency, auditability, and explainability of machine learning models in credit scoring. Journal of the Operational Research Society, 73(1):70–90, 2022. doi:10.1080/01605682.2021.1922098.

4

Karim Lekadir, Richard Osuala, Catherine Gallin, Noussair Lazrak, Kaisar Kushibar, Gianna Tsakou, Susanna Aussó, Leonor Cerdá Alberich, Kostas Marias, Manolis Tsiknakis, Sara Colantonio, Nickolas Papanikolaou, Zohaib Salahuddin, Henry C Woodruff, Philippe Lambin, and Luis Martí-Bonmatí. Future-ai: guiding principles and consensus recommendations for trustworthy artificial intelligence in medical imaging. 2021. URL: https://arxiv.org/abs/2109.09658, doi:10.48550/ARXIV.2109.09658.

This entry was written by Sara Colantonio.