Annotating data

When it relates to personal data, the annotation must be done in compliance with the GDPR. It generally aims to the same purpose as the one defined for the processing and must comply with the principles laid down by the GDPR. In view of the risks for individuals in both the development and deployment phases, the CNIL wishes to draw the attention of stakeholders using annotation to the principles of minimisation, accuracy and loyalty.

The principle of minimisation

Minimisation consists of processing only data “adequate, relevant and limited to what is necessary in relation to the purposes for which they are processed” (Article 5.1.c GDPR). In practice, this means that the annotations, as much as the training data to be annotated, must be limited to what is necessary for the training of the model, as described in the sheet “Taking data protection into account in data collection and management”.

Annotations containing information that are not relevant to the intended purpose do not comply with the principle of minimisation. In some cases, information indirectly related to the purpose may be useful for improving the performance of the model (such as images of billboards for training a traffic sign identification model, as this addition should allow better avoidance of certain false positives). Information is relevant if its link to model performance is proven (theoretically, or empirically, notably in scientific publications) or sufficiently plausible.

Annotated datasets used by the developer of an AI system following a previous collection, purchase or download (from an open source or not) should contain only annotations relevant to the functionalities of the system. Where this is not technically possible, the controller must be able to justify it and try to use the most relevant annotated dataset. A triage of the data must then be carried out to limit the retained annotations to those that are relevant.

Where the activity of an organisation consists of setting up training datasets for the service of third parties, two situations must be distinguished:

• either, as recommended, the training set is created or configured specifically for the customer’s needs; the provider of the training dataset is then subcontracted to its customer for the application of the GDPR; the subcontractor must ensure that the dataset contains only relevant annotations;
   
• either the supplier makes available training sets already constituted; it must then have designed his product in such a way as to enable compliance with the principle of minimisation; a solution may be to provide for several categories of annotations, separable or cumulative, or to offer several distinct sets according to the types of annotation.

In any case, the training datasets must have been constituted and made available in compliance with the GDPR.

The CNIL’s recommendations on data minimisation made in the data sheets “Take data protection into account in the system design choices” and “Taking data protection into account in the collection and management of data” are applicable. 

Annotations may include contextual elements that are useful for measuring and correcting errors and biases. In the case of probabilistic systems such as the majority of AI systems, performance management relies on the ability to measure and correct errors and biases most likely to impact system efficiency. It may therefore be relevant to annotate training, testing or validation data with context elements (date and time, weather, etc.), in particular to measure possible performance deviations depending on the situation. On the other hand, particular attention must be paid to annotation with personal data such as a person’s name or sensitive data (religion, skin colour, etc. – see below).

The principle of accuracy

Accuracy requires for the data to be accurate and, where necessary, kept up to date. This principle implies that the annotation must contain only accurate information about the person to whom the data correspond. An incorrect annotation or one that is based on inappropriate or arbitrary criteria will not respect the principle of accuracy. In practice, this means that the developer will have to take appropriate steps to ensure that the annotation criteria are objective. This issue appears all the more important because the annotation is usually a single word, or a short expression, which is not enough to fully describe a person. This annotation has a risk of being perceived as degrading by people who should not be overlooked, especially since the system could reproduce the inaccuracies of the annotation later, during the deployment of the system and lead to inaccurate, even degrading or discriminatory exits.

Other principles that are less specific to annotation, such as loyalty, transparency, confidentiality or integrity, also apply.

The CNIL invites stakeholders to implement the following measures:

• Defining an annotation protocol, in accordance with the principles of accuracy and minimisation. The CNIL recommends following the following steps:
  • The choice of annotation labels. They should be adapted to the intended purpose for the deployment of the system, and limited to information relevant for training. Although this purpose is not always known precisely when designing the model, in particular for foundation models, the labels chosen should correspond to the expected functionalities at the end of the learning process. For more information on the purpose of foundation models and general purpose systems, please refer to the “Defining a purpose” sheet. In addition, they must allow an objective and unambiguous annotation. Since these labels serve to characterise a person’s data, their choice must be made in a fair manner towards persons whose data are annotated and in particular exclude any degrading, depreciative or term and value judgement that may damage the reputation of persons. Since annotations may serve as proxy values for other information about the person, such as special categories of data, particular attention must be paid to the fact that it does not lead to the unintended introduction of bias, and possible discriminations, into the system. In the event that the annotation results from an existing business process, a phase of triage or requalification of labels may be recommended in order to limit the annotations to what is necessary and relevant for the training of the AI model.
     
  • The definition of an annotation procedure. It should:
    • be documented;
       
    • provide for clear assignment of tasks, thus limiting access to data only to authorised persons;
       
    • allow persons performing the annotation to provide feedbacks on the annotation protocol, and in particular on labels and data, in order to identify when the protocol can be improved or is inappropriate;
       
    • include a validation phase to confirm the choice of labels and the functioning of the procedure, during which, for example, the inter-annotative agreement will be evaluated when several persons perform the annotation. An in-depth analysis of a random sample of annotated data will identify certain errors, inaccuracies, or recurrent inaccuracies during this phase;
       
    • be tracked by logging changes, or using a versioning tool;
       
    • rely on a reliable, robust and controlled annotation tool. Many annotation tools, often specific to certain types of data (images, text, sound, tabular data) exist; it is recommended to check their safety and relevance to the intended purpose, in particular when they incorporate a semi-automatic annotation feature. Etalab’s guide to preparing and conducting its annotation campaign proposes several criteria for selecting the most appropriate text annotation software for its situation (some of these criteria remain relevant for the annotation of other types of data).
      
      
      In the event that the annotation results from a business process, the latter must incorporate the previous recommendations and make the annotation of the data a fully-fledged objective.
       
    • The definition of a continuous verification procedure. This procedure to control the quality of the annotation should be implemented shortly after the start of the annotation phase and then continue through regular or continuous checks. It should be documented and may, for example, be based on:
      • discussion panels including the annotation team, the system development team, and system users when known;
         
      • the analysis of random samples of annotated data;
         
      • an internal or external audit;
         
      • an analysis of the relevance of annotations for each new use case requiring training data (a set of images created to train a vehicle recognition algorithm should be reviewed before being used for pedestrian detection, for example);
         
      • a procedure for taking into account feedback from users of the dataset or model trained on the quality of the annotation and on the corrections to be made;
         
      • the quality control procedure provided for in business processes, which will have to be adapted in order to include the quality of the annotation as an objective in its own right.
• Involve an ethics referent or committee, as a good practice, upstream and then throughout the annotation phase. The multidisciplinary and objective nature of this committee will enable to:
  • Choose the best option for data annotation, be it through in-house processing, subcontracting (and the choice of the subcontractor), or using a solution that does not require annotation (use of an existing set or synthetic data for instance);
     
  • Establish an annotation protocol, in particular to select and define the labels used for the annotation;
     
  • Verify the application of the annotation protocol;
     
  • Monitor the quality of the annotations and their suitability for the intended task in the deployment phase.
    
    In each of these tasks, the following detailed recommendations should be taken into account by the ethics committee. Best practices concerning the objectives and composition of this committee are to be found in the sheet “Taking data protection into account in the design choices of the system”. It should be noted that the setting up of an ethics committee must be adapted to the structural constraints of the organisation. For less resourced structures, an ethics referent will be able to play the role of the committee;

In the case of an annotation resulting from a business process, these measures must be integrated into that process (for example, if information collected as part of the business procedure, such as a medical diagnosis, is subsequently reused for training).

Persons must be informed of the annotation operations

The information of the individuals whose data are collected, whether individual or collective, must mention the annotation phase of the data. In addition to the information to be provided in accordance with the GDPR, it is recommended, as a good practice, to increase transparency by providing the following information:

• the purpose of annotation, such as identifying people in an image, or matching a patient’s medical diagnosis with their consultation notes.
   
• the organisation in charge of the annotation, whether it is a team formed by the controller, a processor, or a community of collaborators. In the case of the use of a subcontractor whose teams are located outside the European Union, the information must specify the existence of transfers outside the EU. The use of a processor must also be subject to contractual clauses such as those proposed on the CNIL's “Standard contractual clauses between controller and processor” web page.
   
• the criteria of corporate social responsibility met under the contract linking the persons responsible for the annotation to the controller, such as guarantees concerning working conditions, remuneration, or psychological support where the annotation relates to data that may contain shocking content.
   
• the security measures taken, and in particular those concerning the annotation phase.

Once the annotation has been completed, and where it is possible to inform the persons concerned a posteriori, they may be informed of the results of the annotation and in particular of the label which is assigned to their data with a view to transparency. This may be a good practice in some rare cases, in particular when:

• the annotation is likely to have consequences for people, which may be the case when its data represent the whole or a significant part of the training dataset. This may be the case when a person’s data is used to fine-tune the model for its particular use on the basis of an annotated sample.
  
  Example: For the parameterisation of a tool to help a household to reduce its gas consumption by automated analysis, allowing the users of the tool to confirm that the installations (such as heating, water heater or gas oven) detected are the right ones can allow a better analysis.
   
• the annotation is likely to have consequences for individuals, for example where unintentional disclosure of the data could damage the reputation of individuals.

Individuals must be able to exercise their rights over annotations

The rights may be exercised to labels associated with a person’s data when the derogations provided for in the texts (the GDPR and the law “Informatique et Libertés”) do not apply, as described in the sheet dedicated to the exercise of rights . Indeed, the annotation attributed to personal data may in many cases also be considered personal data. It follows that:

• the right of access applies to the annotation: the information provided following a request for a right of access must contain the annotations attributed to the person’s data;
   
• the rights to rectification, erasure (in particular following withdrawal of consent), opposition, and limitation apply to annotations. Where these rights are exercised, the same processing must be applied to the data concerned and their annotation.
   
• the right to portability applies to the annotation only where it has been provided by the person and the processing is based on the legal basis of the consent or on the basis of the contract;

The annotation can sometimes reveal special categories of data (ethnic origin, data on the health of the data subjects, political or trade union opinion, etc.) without the source data being itself a special category of data. The processing of this data is prohibited in principle by Article 9 GDPR; however, some exceptions exist. The organisation responsible for the annotation processing will have to identify one of these exceptions in order to be able to implement it legally.

This kind of annotation is therefore not impossible but is subject to special provisions which must be complied with. In the case of health research projects on data collected during care, for example, the exceptions provided for in Article 44-3 of the French Data Protection Act and 9-2-j of the GDPR may apply. The mobilisation of these exceptions and the completion of one of the formalities provided for in Article 66 of the French Data Protection Act, such as a commitment to comply with a reference methodology, or an application for authorisation granted by the CNIL, will allow the processing of annotated health data for the development of an AI system.

In view of the risks that the processing of these data entails for individuals, such as the risk of discrimination, the CNIL recommends using as much as possible other categories of data, such as synthetic data.

Where the applicable provisions are fulfilled and the treatment is lawful, special measures must nevertheless be taken in view of the increased risk to persons. In particular, the CNIL recommends the following measures:

• Annotate according to objective and factual criteria (such as the measurement of skin colour according to the RGB system rather than annotation of the ethnic origin of the person represented in a picture), which can be permitted by the use of technical annotation tools leaving no room for the annotator’s interpretation;
   
• Limit the annotation to the context of the data by avoiding drawing conclusions beyond the information present in the data;
   
• Strengthen the verification stage of annotations, in particular with regard to their regularity (e.g. by a higher frequency), their completeness (e.g. by analysing a larger volume of data), or checking their effectiveness (e.g. by logging the results of the checks, or by an external audit of the procedure). This verification step seems particularly crucial when automatic or semi-automatic annotation tools are used;
   
• Increase the security of annotated data by performing in-house annotation processing, processing data locally, and ensuring their security through encryption, logging, and stronger access restrictions;
   
• Study the risk of regurgitation and inference of special categories of data on models trained from them. Where the controller of the annotation of the dataset does not develop a model but merely makes the data available to other bodies, it should encourage them to conduct this analysis on the models they develop. The CNIL wishes to interrogate the organisms concerned by this matter on the cases where these risks are the most important and on the measures that help in reducing them through a dedicated questionnaire  (whether special categories of data are processed or not). The resulting recommendations will be published after the consultation phase.

The topic of using sensitive data for the management of discriminatory biases is a crucial issue in artificial intelligence and will be the subject of a dedicated practice sheet.