Quality Engineer (AI experience)
As a Senior Quality Engineer, you will be responsible for ensuring the quality and reliability of our AI-powered chatbot systems and email drafting tools. You will design and execute both manual and automated tests, identify bugs, and collaborate closely with developers, product owners, and data scientists to deliver exceptional user experiences. Location: 100% remote. Working hours are based on US Central or Eastern Time Zone. With at least 6 hours of overlap. About Us: Abstra is a fast-growing, Nearshore Tech Talent services company, providing top Latin American tech talent to U.S. companies and beyond. Founded by U.S.-bred engineers with over 15 years of experience, Abstra specializes in sourcing skilled professionals across a wide range of technologies to meet our clients’ needs, driving innovation and efficiency. Job Description: Key Responsibilities:
- Design, develop, and execute manual and automated end-to-end test cases for conversational AI systems and generative email tools.
- Validate natural language processing (NLP) outputs for accuracy, tone, and contextual relevance.
- Build and maintain automated test frameworks for regression, integration, and performance testing.
- Collaborate with cross-functional agile teams to define test strategies, acceptance criteria, and quality benchmarks.
- Identify, document, and track bugs and inconsistencies in Azure DevOps Services and collaborate with the development team to prioritize and resolve issues.
- Perform exploratory testing to uncover edge cases and usability issues.
- Monitor and analyse test results and provide actionable feedback to the development team for rapid iteration and improvement.
- Create and maintain test documentation, including test cases, scripts, and reports.
- Work in an agile development environment, collaborating with the team to deliver quality software iteratively
- Contribute to the continuous improvement of QA processes and best practices.
- Responsible for protecting, securing, and proper handling of all confidential data held by Ascensus to ensure against unauthorized access, improper transmission, and/or unapproved disclosure of information that could result in harm to Ascensus or our clients.
- The I-Client philosophy and the Core Values of People Matter, Quality First and Integrity Always® should be visible in your actions on a day to day basis showing your support of our organizational culture.
Requirements:
- Strong experience as a Software Engineer, a Software Engineer in Test, Test Automation Engineer, or equivalent roles with a testing mindset.
- Experience testing AI/ML-based applications, especially chatbots or generative tools.
- Experience with LLM-based systems (e.g., GPT, Claude, Gemini).
- Knowledge of prompt engineering and evaluation techniques.
- Passion for AI and a keen eye for user experience and conversational quality.
- Familiarity with A/B testing, telemetry, and user feedback analysis.
- Strong experience developing test frameworks with tools such as Playwright, Selenium, Cucumber, or Leapwork.
- Familiarity with writing automation scripts with at least one Object Oriented Programming Language such as C# or Java.
- Strong experience with a scripting language such as Python, PowerShell, Perl, JavaScript.
- Strong experience working with Database testing techniques using Postgres, Sybase, SQL server, or Oracle with a strong understanding of database schemas.
- Experience working with performance Engineers or Performance tools such as BlazeMeter, JMeter and/or Taurus.
- Experience with testing REST API frameworks.
- Excellent verbal and written communication skills.
- Experience working in a fast-paced Agile or SCRUM development methodology.
What We Offer:
- Opportunities for professional growth and development.
- A collaborative and inclusive work environment.
- The chance to work on impactful projects with a talented team.
- Excellent compensation in USD.
- Hardware and software setup (mandatory)
Apply tot his job Apply To this Job