Senior AI Engineer

This a Full Remote job, the offer is available from: Europe, EMEA, Ukraine

We are looking for a Senior AI Engineer to independently assess the quality of AI models and agents used in Finance. You will define how AI quality is measured, build evaluation frameworks and datasets and partner with Finance domain experts to ensure that AI outputs are accurate, reliable, explainable and relevant to the business. Responsibilities Independently validate AI models and agents for accuracy, reliability and explainability Develop evaluation frameworks, metrics, test datasets and quality standards for Finance AI products Conduct continuous evaluation of agents to identify risks, errors, regressions and improvement opportunities Work with Finance domain experts to validate outputs and ensure business relevance Design and maintain automated evaluation pipelines, including LLM-as-a-judge and human-in-the-loop review Analyze evaluation results statistically and communicate findings and recommendations to engineers and stakeholders Define acceptance criteria and release quality gates for AI agents Document model and agent behavior, limitations and risks to support governance and audit needs Requirements 3+ years of experience in AI/ML evaluation, model validation, data science or ML quality assurance Proficiency in Python for data analysis and building evaluation tooling Knowledge of statistics, including hypothesis testing, sampling, confidence intervals and inter-rater agreement Expertise in model validation methods and evaluation metrics for AI Understanding of LLM failure modes, including hallucinations, reasoning errors, bias and prompt sensitivity Background in Finance domain knowledge, such as financial reporting, accounting, FP&A or controlling Capability to design test datasets and annotation guidelines Strong analytical thinking and good communication skills in English Nice to have Familiarity with LLM evaluation tools and frameworks Experience evaluating RAG systems and multi-step AI agents Knowledge of model risk management practices and regulatory frameworks Working knowledge of SQL and Snowflake Experience with explainability techniques, and background in audit, risk or internal controls Experience with red-teaming or adversarial testing of AI systemsThis offer from "EPAM Systems" has been enriched by Jobgether.com and got a 0% flex score.

Back to blog

Other Jobs To Apply

Principal, CHQ Major Facilities Project Engineer 1 Remote / Telecommute Jobs

Automation & Controls Engineer - Critical Power Systems (Remote)

Lead Electrician – Petrochemical Plants

SNAP Policy Specialist

Apply Now: Remote FedEx Jobs: Work From Home Opportunities

Senior Rotating

Remote Director, Technical Services

Contract Negotiator: Lead Deal Support (TS/SCI CI Poly Crossover Remote / Telecommute Jobs

Remote Field Sales

Remote Contracts Director: AI

Remote Commercial Lines Client Service Manager

Technical Support Engineer (Remote/West Coast)

AI & Data Value Manager — Remote + Flexible WFH

Virtual Campus Tour Guide (Remote, Flexible Hours) | Vivaday | Handshake

Supply Chain Planning Specialist Remote / Telecommute Jobs

Palantir Forward Deployed Engineer – Full-Stack Software Enginee Jobs

Transpacific Trade Lead: FCL Growth

Entry-Level Contract Management Specialist Remote / Telecommute Jobs

Cyber Underwriter (VP) - Fully Remote / WeWork

Apple Business Solutions Engineer

Travel Advisor | Work From Anywhere

NYS Certified Special Education Teachers - Remote Opportunities

Virtual Hotel Reservationist

Data Entry Computer Job - Work from Home Part Time

Registered Nurse Multi Specialty (Remote Patient Monitoring-RPM)

Remote Membership Development Representative (Catholic/Christian Affiliated)

Telehealth Physical Therapist

Customer Service Representative (Galician-speaking) - Remote AV01

Manager - Digital Consulting (Life Sciences)

Software Engineer - Agent Team

Remote Customer Service Representative

Microsoft Active Directory Administrator, CDC Directory Services, Remote

Lead Product Manager - Agents (Image, Video & Formats)

Technical Support Specialist, Biology

Support Engineer III

Health Coach

Seasonal Sales Representative (Remote, select U.S. states)

Clinical Review Nurse - Prior Authorization

Partnerships Manager

Sr. Customer Success Manager

Environmental Science Tutor (m/f/d) – Remote | Flexible Hours

Group Head of Compliance

Job Posting Title Specialist Account Manager - Connected Communities

Digital Marketing Manager (Remote)

Patient Pathway Manager - East Germany

Sales Operations Analyst (Remote)

Consultant für Microsoft 365 (m/w/d) - Full-Remote - Deutschlandweit

Work From Home Hospitality Customer Service

Executive Assistant / Personal Assistant

Accounts Payable Specialist (Remote)

Common Interview Questions And Answers

1. HOW DO YOU PLAN YOUR DAY?

This is what this question poses: When do you focus and start working seriously? What are the hours you work optimally? Are you a night owl? A morning bird? Remote teams can be made up of people working on different shifts and around the world, so you won't necessarily be stuck in the 9-5 schedule if it's not for you...

2. HOW DO YOU USE THE DIFFERENT COMMUNICATION TOOLS IN DIFFERENT SITUATIONS?

When you're working on a remote team, there's no way to chat in the hallway between meetings or catch up on the latest project during an office carpool. Therefore, virtual communication will be absolutely essential to get your work done...

3. WHAT IS "WORKING REMOTE" REALLY FOR YOU?

Many people want to work remotely because of the flexibility it allows. You can work anywhere and at any time of the day...

4. WHAT DO YOU NEED IN YOUR PHYSICAL WORKSPACE TO SUCCEED IN YOUR WORK?

With this question, companies are looking to see what equipment they may need to provide you with and to verify how aware you are of what remote working could mean for you physically and logistically...

5. HOW DO YOU PROCESS INFORMATION?

Several years ago, I was working in a team to plan a big event. My supervisor made us all work as a team before the big day. One of our activities has been to find out how each of us processes information...

6. HOW DO YOU MANAGE THE CALENDAR AND THE PROGRAM? WHICH APPLICATIONS / SYSTEM DO YOU USE?

Or you may receive even more specific questions, such as: What's on your calendar? Do you plan blocks of time to do certain types of work? Do you have an open calendar that everyone can see?...

7. HOW DO YOU ORGANIZE FILES, LINKS, AND TABS ON YOUR COMPUTER?

Just like your schedule, how you track files and other information is very important. After all, everything is digital!...

8. HOW TO PRIORITIZE WORK?

The day I watched Marie Forleo's film separating the important from the urgent, my life changed. Not all remote jobs start fast, but most of them are...

9. HOW DO YOU PREPARE FOR A MEETING AND PREPARE A MEETING? WHAT DO YOU SEE HAPPENING DURING THE MEETING?

Just as communication is essential when working remotely, so is organization. Because you won't have those opportunities in the elevator or a casual conversation in the lunchroom, you should take advantage of the little time you have in a video or phone conference...

10. HOW DO YOU USE TECHNOLOGY ON A DAILY BASIS, IN YOUR WORK AND FOR YOUR PLEASURE?

This is a great question because it shows your comfort level with technology, which is very important for a remote worker because you will be working with technology over time...