Singapore•Worldwide - Remote
Remote
Mid Level
Full Time
💰$6 - $65/hour
freelancecontractremoteAIQAmid-senior
Requirements
- •Bachelor’s, master’s, or PhD in computer science, data science, computational linguistics, statistics, or related field
- •Experience in QA for ML/AI systems
- •Safety/red-team experience
- •Experience with test automation frameworks (e.g., PyTest)
- •Hands-on work with LLM evaluation tooling (e.g., OpenAI Evals, RAG evaluators, W&B)
- •Skills in evaluation rubric design, adversarial testing/red-teaming, regression testing at scale
- •Bias/fairness auditing
- •Grounding verification
- •Prompt and system-prompt engineering
- •Test automation using Python/SQL
- •Strong communication skills with clear, metacognitive communication
What You'll Do
- •Model evaluation and testing
- •LLM safety and prompt robustness assessment
- •Data quality assurance
- •Multilingual and domain-specific testing
- •Grounding verification and compliance checks
- •Hallucination detection and factual consistency verification
- •Prompt-injection and jailbreak resistance testing
- •Bias and fairness audits
- •Chain-of-reasoning reliability testing
- •Tool-use correctness validation
- •Retrieval-augmentation fidelity testing
- •End-to-end workflow validation
- •Design and run test plans and regression suites
- •Build rubrics and pass/fail criteria
- •Capture error traces with root-cause hypotheses
- •Suggest improvements to prompt engineering and evaluation metrics
- •Partner on adversarial red-teaming, automation, and dashboarding
Benefits
- •Remote work
- •Flexible pay rate based on experience and location
About Invisible Agency
Clients give us complex business problems and we become their operations, automation, and AI partner to deliver operational breakthroughs.
San Francisco, CA, US
250 - 500
IT Services