Confidence-based safety properties in CAISAR H/F

Job title:

Confidence-based safety properties in CAISAR H/F

Company:

Commissariat à l’Énergie Atomique

Job description

Entité de rattachement Le CEA est un acteur majeur de la recherche, au service des citoyens, de l’économie et de l’Etat.Il apporte des solutions concrètes à leurs besoins dans quatre domaines principaux : transition énergétique, transition numérique, technologies pour la médecine du futur, défense et sécurité sur un socle de recherche fondamentale. Le CEA s’engage depuis plus de 75 ans au service de la souveraineté scientifique, technologique et industrielle de la France et de l’Europe pour un présent et un avenir mieux maîtrisés et plus sûrs.Implanté au cœur des territoires équipés de très grandes infrastructures de recherche, le CEA dispose d’un large éventail de partenaires académiques et industriels en France, en Europe et à l’international.Les 20 000 collaboratrices et collaborateurs du CEA partagent trois valeurs fondamentales :

• La conscience des responsabilités
• La coopération
• La curiosité

Référence 2024-32918Description de l’unitéThe French Alternative Energies and Atomic Energy Commission (CEA) is a key player in
research, development, and innovation. Drawing on the widely acknowledged expertise
gained by its 16,000 staff spanned over 9 research centers with a budget of 4.1 billion
Euros, CEA actively participates in more than 400 European collaborative projects with
numerous academic (notably as a member of Paris-Saclay University) and industrial
partners. Within the CEA Technological Research Division, the CEA List institute addresses
the challenges coming from smart digital systems.
Among other activities, CEA List’s Software Safety and Security Laboratory (LSL) research
teams design and implement automated analysis in order to make software systems
more trustworthy, to exhaustively detect their vulnerabilities, to guarantee conformity
to their specifications, and to accelerate their certification. The lab recently extended
its activities on the topic of AI trustworthiness and gave birth to a new research group:
AISER (Artificial Intelligence Safety, Explainability and Robustness).Description du posteDomaineMathématiques, information scientifique, logicielContratStageIntitulé de l’offreConfidence-based safety properties in CAISAR H/FSujet de stageThe main objective of this internship will be to investigate, design, and implement a
support for confidence-based safety properties in the CAISAR, the AISER’s open-source
platform for characterizing Artificial Intelligence systems’ safety and robustness.Durée du contrat (en mois)5 – 6 moisDescription de l’offreThrough the recent developments of AI, the use of models produced by machine learning has become widespread, even in industrial settings. However, studies are flourishing
showing the dangers that such models can bring, in terms of safety, privacy or even
fairness. We can for example cite the adversarial attacks, small perturbations invisible
to naked eyes which can drastically change the output of our AI. To face these dangers
and improve trust in AI, research works are constantly emerging, resulting into several
tools like ERAN (Gehr et al. 2018) or Marabou (Katz et al. 2019).
In the past couple of years, the AISER team has developed CAISAR (Alberti et al. 2022), an open-source platform that focuses on the specification and verification of AI systems’
robustness and safety. In particular, CAISAR provides a language for formally specifying
AI properties, and integrates various state-of-the-art tools for establishing their validity
in a guaranteed way.The concept of a confidence-based safety property has been recently introduced by
Athavale et al. (Athavale et al. 2024) to recast robustness and fairness properties in
terms of the confidence score with which a neural network generates its outcomes.
The main objective of this internship will be to investigate, design, and implement a
support for confidence-based safety properties in the CAISAR, the AISER’s open-source
platform for characterizing AI systems’ safety and robustness.The broad internship goals are:

• familiarization with the state-of-the-art on formal approaches to properties for AI

safety (Casadio et al. 2022)

• familiarization with the work on confidence-based safety properties (Athavale et

al. 2024)

• getting started with the CAISAR platform
• design and implementation of confidence-based safety properties in CAISAR
• identification and evaluation against benchmarks

Moyens / Méthodes / LogicielsCAISARProfil du candidatThe candidate will work at the crossroads of formal verification and artificial intelligence.As it is not realistic to be expert in both fields, we encourage candidates that do not
meet the full qualification requirements to apply nonetheless. We strive to provide an
inclusive and enjoyable workplace. We are aware of discriminations based on gender
(especially prevalent on our fields), race or disability, we are doing our best to fight them.
One of our team member is formally trained against psychological harassment and sexual abuse.Minimal requirements

• master student or equivalent (2nd/3rd engineering school year) in computer science

• some knowledge of the OCaml programming language (or any other functional

language)

• some knowledge of the Python programming language

• ability to work in a team

Preferred requirements

• notions in formal methods (i.e. abstract interpretation, SAT/SMT solving, etc.)

• notions in machine learning and, in particular, neural networks

Localisation du posteSiteSaclayLocalisation du posteFrance, Ile-de-France, Essonne (91)VilleSaclayCritères candidatLangues

• Anglais (Courant)
• Français (Courant)

Diplôme préparéBac+5 – Master 2Formation recommandéeComputer sciencePossibilité de poursuite en thèseOuiDemandeurDisponibilité du poste03/03/2025

Expected salary

Location

Essonne

Job date

Sun, 15 Sep 2024 03:19:57 GMT

To help us track our recruitment effort, please indicate in your email/cover letter where (vacanciesineu.com) you saw this job posting.