17.09.2026 aktualisiert
Premiumkunde
60 % teilweise verfügbarAI Quality Engineer / SDET | LLM & RAG Evaluation Specialist
Wrocław / Breslau, Polen
Weltweit
Masterabschluss in Wirtschaftskibernetik, Nationale Taurische Wernadskyj-UniversitätÜber mich
AI Evaluierungs-Ingenieur mit 13+ Jahren Erfahrung in QA und Test-Automation. Spezialisiert auf LLM- und RAG-System-Evaluierung sowie End-to-End-Evaluation-Frameworks für KI-Produkte.
Skills
APIsKünstliche IntelligenzAtlassian JiraTest AutomationAutomatisiertes Test-FrameworkCloudtestingSoftware QualityLoadtestingSeleniumTest DesignTest ManagementTest-Management-ToolsTeststrategieTest-Driven DevelopmentChatbots
LLM & RAG Evaluation
Expertise in designing end-to-end evaluation pipelines for LLM and RAG systems, including LLM-as-a-judge frameworks, DeepEval-based Q&A generation, automated scoring workflows, and continuous quality measurement for AI chatbots.
Test Automation Engineering
Extensive experience with test automation using Selenium WebDriver, Playwright, Cypress, Cucumber, Appium, and SoapUI across web, API, and mobile platforms, with a strong focus on reducing manual effort and increasing regression coverage.
Performance & Load Testing
Proficiency in performance and load testing using JMeter, Locust, Artillery, and K6, including environment setup, test execution, and analysis for enterprise-scale applications.
AI-Driven Test Generation
Building AI-powered pipelines using Python, LLMs, and APIs to automatically generate human-readable test cases from existing artifacts, significantly reducing manual effort.
Cloud & DevOps
Hands-on experience with AWS, GCP, and Azure cloud platforms, as well as CI/CD tooling including Jenkins, Azure DevOps, Docker, Terraform, Git, and Perforce.
Database Testing
Practical knowledge of SQL, MongoDB, AWS Athena, Couchbase, SQLite, and Oracle RDS for database validation and data reconciliation tasks.
Programming Languages
Proficiency in Python, Java, JavaScript, and Groovy for building automation frameworks, data pipelines, and test tooling.
Test Management
Experience with test management tools including Jira, Xray, Allure, HP ALM, and Rally for planning, tracking, and reporting test activities.
Event-Driven & API Testing
Solid background in Kafka event-driven testing, REST/SOAP API testing, and web API end-to-end automation for enterprise platforms.
Expertise in designing end-to-end evaluation pipelines for LLM and RAG systems, including LLM-as-a-judge frameworks, DeepEval-based Q&A generation, automated scoring workflows, and continuous quality measurement for AI chatbots.
Test Automation Engineering
Extensive experience with test automation using Selenium WebDriver, Playwright, Cypress, Cucumber, Appium, and SoapUI across web, API, and mobile platforms, with a strong focus on reducing manual effort and increasing regression coverage.
Performance & Load Testing
Proficiency in performance and load testing using JMeter, Locust, Artillery, and K6, including environment setup, test execution, and analysis for enterprise-scale applications.
AI-Driven Test Generation
Building AI-powered pipelines using Python, LLMs, and APIs to automatically generate human-readable test cases from existing artifacts, significantly reducing manual effort.
Cloud & DevOps
Hands-on experience with AWS, GCP, and Azure cloud platforms, as well as CI/CD tooling including Jenkins, Azure DevOps, Docker, Terraform, Git, and Perforce.
Database Testing
Practical knowledge of SQL, MongoDB, AWS Athena, Couchbase, SQLite, and Oracle RDS for database validation and data reconciliation tasks.
Programming Languages
Proficiency in Python, Java, JavaScript, and Groovy for building automation frameworks, data pipelines, and test tooling.
Test Management
Experience with test management tools including Jira, Xray, Allure, HP ALM, and Rally for planning, tracking, and reporting test activities.
Event-Driven & API Testing
Solid background in Kafka event-driven testing, REST/SOAP API testing, and web API end-to-end automation for enterprise platforms.
Erfahrung
Berufserfahrungmehr als 10 Jahre
Freelancer seit2024
Sprachen
GermangutEnglishverhandlungssicherPolishverhandlungssicherRussianMutterspracheUkrainianMuttersprache
Projekthistorie
Designed an evaluation framework for an AI chatbot including quality metrics, evaluation prompts, and end-to-end workflows. Implemented an automated evaluation pipeline for LLM/RAG chatbot validation. Conducted ongoing evaluations to monitor quality, compare chatbot versions, and detect regressions. Collaborated with AI engineers to improve chatbot reliability and production readiness.
Built an AI-powered pipeline using Python, LLM, and Xray API to parse Allure/Jenkins artifacts and generate approximately 1,000 human-readable Xray test cases, cutting an estimated 8-month, 3-engineer effort to under one week. Implemented Kafka event-driven data flow test automation, database testing with AWS Athena and Couchbase, and performance testing with Locust.
Designed an end-to-end evaluation pipeline for an Azure-hosted RAG chatbot (GPT-based) using DeepEval to generate approximately 240 Q&A pairs from product docs, then ran daily Jenkins jobs with LLM-as-a-judge scoring (0-1 scale). Replaced inconsistent manual testing with continuous, measurable quality tracking across RAG accuracy and infrastructure health. Also performed LLM agent and RAG test automation, and AI chatbot functional and performance testing with Playwright and Artillery.