About the role
As an AI QA Engineer at TheWebVale, you will perform systematic quality assurance on AI-powered client platforms. You will test chatbot flows, RAG search pipelines, and automated text extractors for clinics, educational platforms, and commercial businesses.
You will partner with software engineers and our founder during our two-to-three week release cadences. You will stress-test AI outputs against unexpected inputs, test language variations, and ensure client software behaves reliably.
What you will do
- Develop detailed test plans for AI-driven features, chatbots, and document summarizers.
- Execute exploratory and adversarial testing to uncover hallucinations and model misbehavior.
- Verify that model outputs conform strictly to required JSON schemas and database types.
- Test application resilience against prompt injection attempts, toxic inputs, and spam.
- Document reproducible bug reports with input prompts, seed parameters, and system configs.
- Work alongside developers to re-test prompt adjustments and verify regression fixes.
What you will need
- Possess 2–4 years of software testing experience with exposure to AI and language models.
- Demonstrate practical skill writing test scripts in Python or modern JavaScript.
- Show understanding of how language models generate text, handle context, and fail.
- Possess experience testing web applications, API endpoints, and database records.
- Exhibit strong adversarial thinking to anticipate how users might break AI features.
- Communicate software defects and edge-case behaviors clearly to the team.
Nice to have
- Experience with automated API testing tools like Postman, Newman, or Pytest.
- Familiarity with AI safety guidelines and content moderation API standards.
- Knowledge of basic web security principles and input sanitization.