Role in brief
Gauntlet is looking for a Research Engineer to enhance AI agents in financial markets. Ideal candidates will have strong Python and machine learning skills, with a focus on building evaluation frameworks and improving agent performance.
About the role
In this role, you will work at Gauntlet, a company that builds financial systems for both crypto and traditional asset management. Your primary responsibility will be to develop evaluation frameworks for AI agents, ensuring they perform effectively in real-world financial tasks.
You will collaborate with various teams to translate qualitative feedback into measurable outcomes, while also investigating agent failures to enhance their capabilities. The position requires a blend of applied research and engineering skills, aimed at improving the reliability and functionality of AI agents used across the business.
The salary range reflects the company's commitment to compensating talent based on skills and experience.
Skills that matter here
- AI: This role involves enhancing AI agents' performance and developing evaluation frameworks.
- Python: Strong Python skills are essential for writing production-quality code and building tools.
- machine learning: Experience in machine learning is necessary for designing experiments and analyzing data.
- data science: Knowledge in data science will help in working with datasets and drawing conclusions from experiments.
Who this role suits
- You have a background in AI research or engineering, with hands-on experience in developing LLM models or agents.
- You are comfortable working independently and can break down complex problems into manageable tasks.
- You thrive in collaborative environments, adapting to changing priorities while working across research and engineering teams.
- You have a genuine interest in AI agents and are motivated to improve their reliability and effectiveness.
From the employer
What you'll do:
- Build evaluation frameworks for AI agents, including datasets, sandbox environments and metrics that measure performance on real-world tasks.
- Design experiments to compare models, agent harnesses, tools and execution environments, identifying regressions and understanding the trade-offs between performance, reliability and cost.
- Work with teams across Gauntlet to turn qualitative feedback about agent behaviour into concrete research questions, experiments and measurable outcomes.
- Investigate agent failures, understand whether problems come from the model, harness, tooling or prompting, and develop ways to improve performance.
- Build and improve agent harnesses, internal tools and integrations, including contributing to Aria and other agents used across the business.
- Help develop shared knowledge and reusable skills across agents, creating systems that can learn from previous work and improve over time.
- Work alongside Infrastructure and Security to ensure agents can operate reliably and safely within our production environment.
What we're looking for:
- Hands-on experience building, evaluating or improving LLM models or agents, whether through professional work, applied research, open-source contributions or substantial personal projects.
- Strong Python skills and good software engineering fundamentals. You're comfortable writing production-quality code and understand the importance of testing, maintainability and reliability.
- Experience with applied research, machine learning or data science, including designing experiments, working with datasets and drawing conclusions from results.
- A good understanding of how LLM agents work beyond prompting, including tool use, harnesses, context management and common failure modes.
- The ability to take an open-ended problem, break it down into testable questions and work independently towards a solution.
- Experience building tools or systems that other people use, with an understanding of how to evaluate their effectiveness beyond technical benchmarks.
- Comfort working across research and engineering, collaborating with different teams and adapting as priorities change.
- Genuine interest in AI agents and how they can be made more capable, reliable and useful.
Nice to have:
- Experience building LLM evaluation pipelines, benchmarks, reinforcement learning environments or model testing infrastructure.
- Contributions to open-source agent frameworks, harnesses, evaluation datasets or benchmarks.
- Experience with agent frameworks such as Hermes, OpenClaw or similar tools.
- Experience in applied ML research, model training, fine-tuning or reinforcement learning.
- An advanced degree or postgraduate research experience in computer science, mathematics, statistics, machine learning or a related field.
- Experience in financial markets, quantitative research, DeFi or risk modelling.
- Familiarity with AI security considerations, including access controls, prompt injection and data protection.
Benefits & Perks:
- Remote first — work from anywhere in the United States
- Regular in-person company retreats and cross-country "office visit" perk
- 100% paid medical, dental, and vision premiums for employees
- $1,000 WFH stipend
- Monthly reimbursement for home internet, phone, and cellular data
- Unlimited vacation
- 100% paid parental leave of 12 weeks
- Fertility benefits
- Opportunity for incentive compensation
Questions about this role
Is this position remote?
Yes, this role is remote and can be performed from anywhere in the United States.
What skills are required for this role?
Candidates should have strong Python skills, experience in machine learning and data science, and a good understanding of LLM agents.
What is the salary range for this position?
The salary range for this role is between $105,000 and $175,000.
✓ Drop your CV once, then continue to the employer's application form. Your profile stays here for every recruiter hiring on web3vacancy.