Senior Assurance Engineer – Machine Learning

Role
Our assurance team is responsible for reviewing high-assurance software in the tech, finance, and defense industries. Our evaluations allow our clients to make informed decisions about risk to their systems, including how to minimize or mitigate safety and security issues. In addition to dedicated teams focusing on application security, cryptography, blockchain security, and emerging platforms security, Trail of Bits has a machine learning (ML) practice that creates tools and techniques for the exploration of new attack surfaces and failures that can lead to the degradation of model performance, exploitation of ML system assets, and manipulation or lack of robustness of resulting ML outputs. Trail of Bits has also created and maintains more than 200 free and open-source tools (available in our GitHub repositories) and offers research and engineering services for the public and private sectors.
As a Senior Assurance Engineer – Machine Learning you will be a part of a new machine learning assurance team that aims to identify and taxonomize classes of failure modes which directly impact ML model performance, as well as novel hazards and threats that can compromise the ML operations pipeline for our clients. You will provide value to our clients through your expertise and experience within the ML development lifecycle as well performing risk assessments and threat modeling.
Responsibilities

 
Research and development of machine learning assurance frameworks and contributing to the shape of the new field of ML auditing and assurance.
 
Carrying out risk assessments and threat modeling on machine learning model capabilities and their failure modes.
 
Analysis of machine learning model degradation and failure from systematic defects.
 
Work directly with leading industry teams to review system code and architecture, and help assure their products through system analysis and modeling.
 
Develop and contribute to policy and regulatory frameworks to enable the assurance and auditing of machine learning and artificial intelligence to be deployed within mission critical applications.

Requirements

 
3+ years of experience with the machine learning data pipeline or machine learning development lifecycle.
 
3+ years of experience with systems, safety, or security engineering principles for mission critical applications.
 
3+ years of experience performing or evaluating analysis methods to assess machine learning performance, safety, or security from system-level requirements (risk assessments, system hazard analysis, STPA, etc. included).
 
Knowledge of ISO 38507, ISO 25012 or similar AI standards such as EU Commission Ethics Guidelines for Trustworthy AI.
 
Excellent verbal & written communication skills.
 
Proven success working in a geographically dispersed and remote work environment.
 
Can adapt and drive innovation and performance in a rapidly changing field. 
 
Technical background to be able to look at systems and analyze their behaviors and realize their failures.

The US estimated base salary for this full-time W2 employee role is $175,000-$225,000, not including bonus and benefits. Our salary ranges are determined by role, level, location, and employment type. The range displayed on each job posting reflects the minimum and maximum target for new hire salaries for the position across all US locations. Within the range, individual pay is determined by a variety of factors, include but not limited to work location, job-related skills, experience, and relevant education or training. Your recruiter can share more about the specific salary range for your preferred location during the hiring process.