Mindrift: Freelance Agent Evaluation Engineer

Related keywords: freelance remote jobremote job in indiaremote job with flexible hours

Overview

Mindrift is a company headquartered in India that specializes in connecting specialists with AI project-based opportunities for top tech companies. The position available is for a Freelance Agent Evaluation Engineer. This role involves participating in various projects aimed at testing, evaluating, and improving AI systems. As a freelance contributor, you have the flexibility to choose the projects you wish to partake in, allowing you to organize your work according to your own schedule provided that all deliverables meet project requirements and deadlines.

Job Responsibilities

The responsibilities for this freelance position include a variety of tasks that are unique to each project. In general, contributors may be required to:

• Create structured test cases that simulate complex human workflows.
• Define gold-standard behavior and establish scoring logic to evaluate how AI agents perform.
• Analyze agent logs, determine failure modes, and study the decision paths followed by AI systems.
• Collaborate with code repositories and test frameworks to ensure that all scenarios are validated.
• Continuously refine prompts, instructions, and test cases to enhance clarity and difficulty.
• Ensure that developed scenarios are production-ready, straightforward to run, and reusable for future tasks.

Required Skills

The opportunity is particularly well-suited for individuals with a background in software engineering who are looking for flexible, part-time, non-permanent projects. The ideal candidates should possess the following qualifications:

• At least 3 years of experience in software development, with a strong focus on Python.
• Familiarity with Git and experience with code repositories is important.
• Comfortable using structured formats such as JSON or YAML for descriptive scenarios.
• A robust understanding of core LLM limitations such as hallucinations, bias, and context limits, and how these factors impact evaluation design.
• Familiarity with Docker.
• Proficient in English, with at least B2 level proficiency.

Payment

Freelance contributions are paid, with payment rates being project-based and can go up to $80 per hour. However, it is essential to note that payment rates can vary based on a wide range of factors including expertise, skills assessment, location, and specific project needs.

• Some projects may also offer incentive payments.
• Rates may be lower during onboarding or in non-core project phases.
• Specific payment details will be provided for each project, ensuring transparency regarding compensation.

Project Time Expectations

For this role, it is estimated that contributors will need to dedicate around 6-10 hours of work per week during active project phases. This estimation is relative to the tasks chosen and does not create any form of minimum guaranteed hours. Candidates should be prepared for the flexibility that freelance projects offer while also adhering to the timelines set by the individual projects.

Application Process

To apply for this freelance position, candidates should submit their CV in English along with an indication of their English proficiency level. The typical application process consists of the following steps:

1. Submit application.
2. Pass requisite qualification assessments.
3. Choose a project to work on.
4. Complete tasks at your convenience within the project timelines.
5. Receive payment upon project completion.

Conclusion

This freelance opportunity at Mindrift represents an exciting chance for software engineers to participate in project-based roles focused on AI evaluation. With a flexible approach to work and competitive pay rates, Mindrift offers an ideal platform for individuals looking to enhance their careers in the tech industry while contributing to important AI advancements. Candidates with the requisite skills are encouraged to apply and enjoy the perks of remote flexible work.

This job offer was originally published on weworkremotely.com