[Remote] AI Engineer

Note: The job is a remote job and is open to candidates in USA. ETS is a global education and talent solutions organization dedicated to advancing fair and valid skill assessments. They are seeking an AI Model Development Engineer to design, develop, evaluate, and deploy machine learning models for assessment technologies, focusing on AI-driven scoring systems for various response types.

Responsibilities

• Develop, train, and optimize machine learning and deep learning models for applications such as automated scoring (including text, speech, or multimodel responses), item generation, content classification, anomaly detection, and personalization
• Implement feature engineering, representation learning, and model architectures appropriate for scoring and classification tasks
• Develop hybrid scoring approaches combining AI models with rules-based or human-in-the-loop workflows
• Use NLP, large language models, and multimodal modeling techniques to support assessment creation, delivery, and feedback
• Build model pipelines, evaluation frameworks, and testing approaches to ensure models are valid, reliable, fair, and aligned with ETS’s Responsible AI guidelines
• Collaborate with psychometric and validity teams to ensure AI-driven systems meet technical, fairness, and measurement standards
• Partner with engineering teams to deploy models into scalable, production-grade systems integrated with ETS operational platforms
• Conduct experiments to compare model architectures, datasets, training regimes, and performance tradeoffs
• Implement monitoring solutions to track model drift, robustness, and security risks over time
• Participate in cross-functional design sessions, helping translate assessment or business needs into implementable AI solutions
• Document model design, training data specifications, evaluation metrics, and deployment requirements
• Stay current with advancements in LLMs, generative AI, responsible AI, and educational technology, bringing forward ideas for innovation

Skills

• Bachelor's or Master's degree in Computer Science, Machine Learning, Data Science, Engineering, or related field
• 3+ years of experience developing machine learning models in production environments, preferably in large scale scoring environments
• Hands-on experience developing and deploying machine learning or deep learning models (TensorFlow, PyTorch, JAX, or equivalent)
• Experience with NLP techniques and models, including transformer architectures and LLM fine-tuning and/or speech processing (e.g., text classification, embeddings, ASR outputs)
• Strong proficiency in Python and familiarity with modern MLOps tools (MLflow, Kubeflow, KServe, or equivalent)
• Demonstrated experience evaluating model performance using appropriate metrics and validation strategies
• Understanding of model evaluation, bias and fairness assessment, and statistical validation
• Ability to work collaboratively with researchers, engineers, and product teams

Benefits

• ETS offers multiple Business Resource Groups (BRG) for you to learn and advance your career growth!
• At ETS you may qualify for up to an additional 8 hours of PTO for volunteer work on causes that are important to you!

Company Overview

Company H1B Sponsorship