Share this job
QA & AI Evaluation Engineer
São Paulo, São Paulo, Brazil
Apply for this job

QA Engineer

Location: Brazil, São Paulo

Work Type: Hybrid 3 days/week in office

Employment Type: Full-Time

Experience: 5+ years

Compensation: $5,000 - $6,000/month

FitNext Exclusive: No


Company Overview

We’re working with a fast-growing Series A startup building an AI-powered platform for enterprise software delivery. The company is backed by leading investors and works with large global organizations on complex, high-impact projects.

You’ll join a small, highly technical team and work closely with Engineering, Product, and the founding team.

The Challenge

We’re looking for a QA Engineer to build and own the company’s quality function.

This role goes beyond traditional QA. You’ll be responsible for building automated test coverage while also developing ways to evaluate AI-generated and non-deterministic outputs.

You’ll have significant ownership over how quality is defined, tested, and maintained across the product.


Must Have

  • 5+ years of experience in QA, SDET, or Test Engineering
  • Strong experience building and owning automated test frameworks for production products
  • Strong experience with Python and/or TypeScript
  • Experience with modern E2E testing frameworks such as Playwright, Cypress, or equivalent
  • Strong API testing experience
  • Hands-on CI/CD experience, including building and maintaining test pipelines
  • Experience with regression, integration, and end-to-end testing
  • Experience identifying and managing flaky tests and false positives
  • Strong understanding of test prioritization, severity, and release quality gates
  • Experience testing systems with non-deterministic outputs
  • Professional-to-fluent spoken English


Nice to Have

  • Experience testing LLM-powered or agentic features
  • Experience designing evaluation frameworks, golden datasets, or scoring criteria
  • Experience with hallucination detection or human-in-the-loop workflows
  • Experience with SAP, ERP, CRM, or other enterprise software
  • Experience with SAP testing tools such as Tricentis or Worksoft
  • Experience with UAT and enterprise software testing cycles
  • Performance, load, or security testing experience
  • Ability to contribute code and fix bugs, not only identify them


Soft Skills

  • High ownership and autonomy
  • Strong judgment and prioritization
  • Comfortable working without an established playbook
  • Strong attention to quality and detail
  • Comfortable challenging decisions when quality is at risk
  • Strong communication and collaboration skills
  • Comfortable working in a fast-paced, early-stage environment


Benefits

  • Hybrid schedule in São Paulo — 3 days/week in office
  • Direct access to Engineering, Product, and the founding team
  • High ownership over the QA function and quality standards
  • Opportunity to work on challenging AI and enterprise software testing problems
  • Opportunity to build QA processes and infrastructure from the ground up


Hiring Process

  • Initial conversation with FitNext
  • Interview with the company
  • Technical assessment/interview
  • Final interview with the team


Apply for this job
Website Contact Us Linkedin
© 2026 FitNext Consulting. All rights reserved.