As part of a young, dynamic start-up based in Paris, you will participate in the development of exoskeleton control using Reinforcement Learning and Deep Learning methods.

In recent years, impressive results have been obtained using Neural Networks as the basis for control algorithms. While these results were initially limited to simulated environments (vidéo), or quadrupeds in controlled environments (vidéo), more recent results have shown that these methods can be applied to more complex environments (vidéo 1, vidéo 2), and even to bipeds (vidéo). To achieve these results, a physical simulation environment is created, then researchers train controllers to perform a task within this simulator. Certain “transfer” techniques are applied during training (domain randomization, meta-learning) to ensure that the controller trained in simulation also works on the real robot, delivering similar performance.

Wandercraft played an active role in the development of the Jiminy open-source polyarticulated systems simulator. The latter reproduces the behavior of the exoskeleton in a convincing way, and enables us to train controllers in simulation in just a few hours. For instance, we were able to train a controller able to sustain a standing up position and do recovery steps in the event of moderately strong external disturbances, both in simulation and in reality. This work has been the subject of a scientific publication (video).

You will be integrated into the control team made up of PhD students and engineers, under the supervision of one of the team’s engineers. Depending on the candidate’s areas of expertise and skills, your main activities will be:

• Improving existing methods for simulating robot behavior,

• Development of innovative transfer techniques to ensure a smooth transition from simulation to reality,

• Training and optimization of controllers for bipedal walking,

• Evaluation of algorithms in simulation and estimation of performance (efficiency, robustness, etc.),

• Development or improvement of an experimental setup,

• Evaluation of algorithms on real robots and estimation of performance (efficiency, robustness, etc.),

• Determination of hardware/software requirements for on-board integration into the exoskeleton,

• Participation in robot testing with or without users.

Both proactive and rigorous, you have an excellent ability to work within a team and know how to be autonomous and proactive while following the instructions given. You enjoy working on various and varied tasks and have the well-being of your team at heart. You enjoy working in medium-sized structures (100 people) and you know how to adapt to rapid changes in the order of priority of tasks. You are resourceful, curious and very organized. Finally, you have excellent oral and written communication skills.

Qualifications:

• Diploma: final-year of a Master’s degree or a Grande Ecole d’Ingénieur in Applied Mathematics, Computer Science, Controls, Robotics or equivalent

• Fluency in Python

• Basic knowledge in Reinforcement Learning

• Experience with at least one relevant Machine Learning library (PyTorch, TensorFlow, Flax, …)

• Strong mathematics skills

• Good fundamentals in Rigid Body Dynamics

• Control theory: autonomous

• Robotic system controls: autonomous

• Basic knowledge in Git

  Bonus qualifications:

• Advanced knowledge in Reinforcement Learning

• Experience building and working with Neural Networks such as CNNs, RNNs, and Transformers

• Autonomous with Git and familiar with modern SW development practices

• Practical experience with distributed and GPU computing

  If you don’t tick all the boxes, don’t worry: your motivation and personal projects can also be powerful arguments

How to apply

1. Submit your application by clicking on the “Apply” button below

2. To validate your application, we would like you to demonstrate your interest in programming and problem solving by passing a “challenge”:

   • Go to https://adventofcode.com/

   • Solve the challenges to obtain 2 stars on the first 4 levels

   • Send us your solutions by replying to the “application received / candidature bien reçue” email that was sent to you after you applied. We expect the solutions found for each level (i.e. 8 integer numbers), as well as your source code (github link, or files directly attached to the email, uncompressed).

We look forward to meeting you!

This internship offer is also available in French language on request.