AI Test Automation Engineer

Hi, we’re Gravie. Our mission is to create health benefits that actually benefit small and midsize businesses and their employees. Our innovative benefit solutions and services are developed and delivered by a diverse group of unique people. We encourage you to be your authentic self - we like you that way.

A Little More About The role:

We’re looking for an AI Test Automation Engineer who will own the design and development of AI-driven test automation systems across our product portals. Rather than manually writing test cases, you will build the frameworks, feedback loops, and infrastructure that enable AI tools to generate, execute, and maintain automated tests at scale. You will work closely with our QA, Engineering, and Product teams to dramatically increase our automated regression coverage, accelerate delivery timelines, and reduce the risk of defects reaching production.

You will:

• Design and build AI-driven test automation workflows using tools like Claude Code, where AI agents generate, execute, and iterate on Playwright tests with minimal human intervention. Your primary job is building the system, not writing individual tests.

• Architect a Playwright test framework from scratch, establishing patterns, page-object models, fixtures, and conventions that are optimized for AI-assisted test generation. The framework should be structured so that AI tools can reliably produce correct, maintainable tests.

• Build tight feedback loops between local test execution, CI pipeline results, and merge request creation, so AI agents can run tests, interpret failures, fix issues, and resubmit with minimal manual intervention.

• Audit the existing Geb/Spock/Groovy test suites across three portals (Member, Employer/Broker, Admin) and classify each test into one of three categories: convert to Playwright, convert to an API or unit test, or delete (legacy functionality no longer in use).

• Lead the migration from Geb/Spock to Playwright, using AI agents to accelerate conversion at scale while maintaining coverage continuity.

• Integrate Playwright test execution into GitLab CI/CD pipelines with automated quality gates on every merge request and deployment. Optimize for fast feedback through parallelization and caching strategies.

• Instrument tests with reporting on pass/fail trends, flakiness rates, and coverage gaps. Use this data to prioritize AI-driven test generation toward the highest-value uncovered areas.

• Identify scenarios where end-to-end Playwright tests are not the right tool and implement API-layer automation or contract tests for microservice boundaries instead.

• Continuously improve the AI automation workflow by refining prompts, context structures, and validation patterns to increase the percentage of tests that AI generates correctly on the first pass.

You bring: 

• 5+ years of experience in test automation engineering, SDET, or software engineering with strong testing practices.

• Demonstrated experience using AI coding tools (Claude Code, Cursor, GitHub Copilot, or similar) to generate and maintain code at scale. This is not optional; we need someone who has already built workflows around AI-assisted development.

• Hands-on experience with Playwright or a comparable modern end-to-end testing framework. Experience building a framework from scratch is strongly preferred.

• Proficiency in TypeScript or JavaScript with the ability to write production-quality code.

• Experience with CI/CD pipelines (GitLab preferred), including writing pipeline configuration, test stages, and merge request workflows.

• Solid understanding of REST API testing and when to use API-level tests vs. UI-level tests.

• Experience migrating legacy test automation to a modern framework, including planning the transition and maintaining coverage during the migration.

• Strong communication skills: able to explain technical trade-offs to non-technical stakeholders and write clear documentation.

• A builder mindset. You think in terms of systems, feedback loops, and automation leverage, not individual test cases.

• Comfort working in ambiguity. Our AI-driven approach is new, and the right person will help define what good looks like.

Extra credit: 

• Experience with Groovy, Spock, or Geb (the legacy framework you will be migrating from).

• Experience with contract testing (Pact or equivalent).

• Background in health insurance, benefits administration, or regulated industries.

• Experience building or contributing to internal developer tooling or productivity platforms.

Gravie: 

In order to create a more equitable and sustainable future for employer-sponsored health insurance, we need talented people doing amazing work. In exchange, we offer a great overall employee experience with opportunities for career growth, meaningful mission-driven work, and an above average total rewards package.

The salary range for this position is $124,271 - $207,118 annually. Numerous factors including, but not limited to, education, skills, work experience, certifications, etc. will be considered when determining compensation.  I

Our unique benefits program is the gravy, i.e., the special sauce that sets our compensation package apart. In addition to standard health and wellness benefits, Gravie’s package includes alternative medicine coverage, generous PTO, up to 16 weeks paid parental leave, paid holidays, a 401k program, transportation perks, education reimbursement, and paid paw-ternity leave. 

A Little More About Us:

• We know healthcare. Our company was founded and is still led by industry veterans who have started and grown several market-leading companies in the space.

• We have raised money from top tier investors who share the same long-term vision as we do of building an industry defining company that will endure over the long run. We are well capitalized.

• Our clients love us. Customer satisfaction rates among employees using Gravie health plans consistently rank above 80% – nearly 40 points above the industry average.

• Our culture is unique. We tend to be non-hierarchical, merit-driven, opinionated but kind people who thrive working in a high-performance, fast-paced environment. People at Gravie care deeply about making a positive impact in the lives of the people we serve. We may not be the right place for everybody, but if you get energized by doing work every day that focuses on putting consumers at the front of the line, we could be a great place for you. It takes unique people and diverse perspectives to deliver our results. We encourage you to be your authentic self – we like you that way.