Back to the Career Market

Senior Software Engineer, Model Evaluation

Clera

remote · Remote-friendly

About the role

About the Role Join an early-stage AI company building agents for manufacturing and supply chain operations. In this research-focused engineering role, you will work with procurement and manufacturing experts to create benchmarks, train models, and improve agents used in real operational workflows. What You'll Do - Design benchmarks and evaluation suites for supplier sourcing, qualification, compliance documents, purchase orders, freight, and order tracking. - Work with domain experts to scope tasks, write reference answers and rubrics, and build graders. - Train and fine-tune models hands-on, using evaluation results to guide experiments. - Identify agent failures, develop improvements, and validate those improvements with measured results. - Help establish research and evaluation practices as an early member of the team. What We're Looking For - At least 3 years of experience, with senior- or staff-level capability. - Experience designing and writing benchmarks or evaluations. - Experience authoring tasks, including defining scope, creating reference solutions or rubrics, and checking task quality. - Hands-on experience training and fine-tuning models, beyond using models through APIs. - Ability to work closely with non-ML domain experts and translate their knowledge into measurable tasks. - Supply chain, procurement, manufacturing, or early-stage startup experience is a plus. Compensation & Benefits Base salary is $180,000 to $250,000 annually, plus equity. Visa sponsorship policy was not provided. Location Fully remote in the United States.