Role in brief
Gauntlet is looking for a Research Engineer to enhance AI agent performance in financial systems. This role requires strong Python skills and experience with machine learning. Ideal candidates are those with a solid engineering background and a keen interest in AI.
About the role
Gauntlet builds innovative financial systems, focusing on comprehensive solutions rather than isolated products. They manage over $1.7B in client total value locked (TVL) across various sectors, including fintechs and traditional asset management.
The Research Engineer will develop evaluation frameworks for AI agents, design experiments, and collaborate across teams to improve agent performance. Success in this role involves turning qualitative feedback into actionable research and enhancing the overall capabilities of AI systems.
The salary for this role ranges from $175,000 to $195,000, reflecting the seniority and expertise required.
Skills that matter here
- Python: Strong Python skills are essential for writing production-quality code and building tools.
- machine learning: Experience with machine learning is crucial for evaluating and improving AI agents.
- data science: Data science skills will be used to analyze results and design experiments.
- software engineering: Solid software engineering fundamentals are necessary for maintaining code reliability and performance.
Who this role suits
- Candidates should have a hands-on approach to building and evaluating AI models.
- They need to be comfortable working independently and tackling open-ended problems.
- A collaborative spirit is important for working across different teams.
- An enthusiasm for AI and its applications in finance will help drive success.
From the employer
What you'll do
- Build evaluation frameworks for AI agents, including datasets, sandbox environments and metrics that measure performance on real-world tasks.
- Design experiments to compare models, agent harnesses, tools and execution environments, identifying regressions and understanding the trade-offs between performance, reliability and cost.
- Work with teams across Gauntlet to turn qualitative feedback about agent behaviour into concrete research questions, experiments and measurable outcomes.
- Investigate agent failures, understand whether problems come from the model, harness, tooling or prompting, and develop ways to improve performance.
- Build and improve agent harnesses, internal tools and integrations, including contributing to Aria and other agents used across the business.
- Help develop shared knowledge and reusable skills across agents, creating systems that can learn from previous work and improve over time.
- Work alongside Infrastructure and Security to ensure agents can operate reliably and safely within our production environment.
What we're looking for
- Hands-on experience building, evaluating or improving LLM models or agents, whether through professional work, applied research, open-source contributions or substantial personal projects.
- Strong Python skills and good software engineering fundamentals.
- You're comfortable writing production-quality code and understand the importance of testing, maintainability and reliability.
- Experience with applied research, machine learning or data science, including designing experiments, working with datasets and drawing conclusions from results.
- A good understanding of how LLM agents work beyond prompting, including tool use, harnesses, context management and common failure modes.
- The ability to take an open-ended problem, break it down into testable questions and work independently towards a solution.
- Experience building tools or systems that other people use, with an understanding of how to evaluate their effectiveness beyond technical benchmarks.
- Comfort working across research and engineering, collaborating with different teams and adapting as priorities change.
- Genuine interest in AI agents and how they can be made more capable, reliable and useful.
Benefits & Perks
- Remote first — work from anywhere in the United States
- Regular in-person company retreats and cross-country "office visit" perk
- 100% paid medical, dental, and vision premiums for employees
- $1,000 WFH stipend
- Monthly reimbursement for home internet, phone, and cellular data
- Unlimited vacation
- 100% paid parental leave of 12 weeks
- Fertility benefits
- Opportunity for incentive compensation
Questions about this role
Is this position remote?
Yes, this role is remote-first and can be performed from anywhere in the United States.
What is the salary range for this role?
The salary range for this position is between $175,000 and $195,000.
What skills are required for this position?
Candidates should have expertise in Python, machine learning, data science, and software engineering.
✓ Drop your CV once, then continue to the employer's application form. Your profile stays here for every recruiter hiring on web3vacancy.