{"id":7667,"date":"2026-09-04T08:00:06","date_gmt":"2026-09-04T08:00:06","guid":{"rendered":"https:\/\/itdsportugal.com\/it-jobs\/mid-level-ai-automation-qa-engineer-ai-testing-validation\/"},"modified":"2026-09-04T10:47:27","modified_gmt":"2026-09-04T10:47:27","slug":"mid-level-ai-automation-qa-engineer-ai-driven-testing","status":"publish","type":"matador-job-listings","link":"https:\/\/itdsportugal.com\/en\/it-jobs\/mid-level-ai-automation-qa-engineer-ai-driven-testing\/","title":{"rendered":"Mid-Level AI &amp; Automation QA Engineer \u2013 AI-Driven Testing"},"content":{"rendered":"<p><strong>Unleash innovation in AI testing \u2014 drive the future of safe, reliable AI-powered solutions!<\/strong><\/p> <p>Porto-based opportunity with hybrid work model.<\/p> <p>As a <strong>Senior AI &amp; Automation QA Engineer<\/strong>, you will be working for our client, a leader in AI-powered features across conversational AI, productivity tools, and rating systems. You will play a pivotal role in building and maintaining the testing infrastructure that ensures the accuracy, safety, and regression-free performance of cutting-edge AI applications. This position offers a dynamic environment dedicated to advancing AI reliability and fostering professional growth.<\/p> <p><strong>Your main responsibilities:<\/strong><\/p> <ul> <li>Develop and maintain automated test frameworks using tools like Playwright, Cypress, or Selenium to ensure AI features perform flawlessly.<\/li> <li>Script custom testing solutions leveraging Python or JavaScript\/TypeScript to enhance automation and evaluation pipelines.<\/li> <li>Design and implement evaluation datasets and automated pipelines, including innovative LLM-based judgment approaches.<\/li> <li>Analyze AI outputs to identify hallucinations, errors, and biases, supporting continuous improvement.<\/li> <li>Integrate AI evaluation and functional tests into CI\/CD pipelines for seamless deployment.<\/li> <li>Utilize AI observability and tracing tools for debugging model behavior in production environments.<\/li> <li>Craft and execute test cases for conversational AI, including dialogue flows, edge cases, and adversarial prompts.<\/li> <li>Collaborate on ensuring AI outputs meet fairness, bias, and compliance standards through rigorous testing.<\/li> <\/ul> <p><strong>You&#8217;re ideal for this role if you have:<\/strong><\/p> <ul> <li>3+ years of experience in QA\/SDET roles, especially in automation testing.<\/li> <li>Hands-on experience building automated test frameworks with Playwright, Cypress, Selenium, or similar.<\/li> <li>Proficiency in scripting languages such as Python or JavaScript\/TypeScript.<\/li> <li>Understanding of AI-specific testing challenges, including prompt regression, hallucination detection, and output scoring.<\/li> <li>Experience designing evaluation datasets and using AI observability\/tracing tools.<\/li> <li>Knowledge of fairness, bias mitigation, and compliance testing for AI models.<\/li> <li>Familiarity with CI\/CD pipelines to automate testing processes.<\/li> <li>Strong root-cause analysis skills covering both code and probabilistic AI outputs.<\/li> <\/ul> <p><strong>It is a strong plus if you have:<\/strong><\/p> <ul> <li>Experience with AI-assisted test generation or self-healing scripts.<\/li> <li>Knowledge of LLM evaluation approaches and AI debugging tools.<\/li> <li>Certifications related to QA or AI safety.<\/li> <\/ul> <p><strong>Language Required for the role:<\/strong> Fluent English<\/p> <p><strong>Eligibility to work in Europe:<\/strong> Only candidates with an existing legal right to work in the European Union will be considered for this role.<\/p> <p>#MAKEYourCareerBETTER<\/p> <p>Interested? Apply now and include your CV (preferably in English) along with a statement confirming your consent to the processing and storage of your personal data.<\/p> <nav class=\"matador-job-navigation\"> <ul> <li> <a href=\"https:\/\/itdsportugal.com\/en\/?wpid=7667&#038;bhid=9992\" rel=\"button\" class=\"matador-button\" title=\"Apply Now\">Apply Now<\/a> <\/li> <li> <a href=\"https:\/\/itdsportugal.com\/en\/it-jobs\/mid-level-ai-automation-qa-engineer-ai-driven-testing\/\" rel=\"button\" class=\"matador-button matador-button-secondary\" title=\"More Info\">More Info<\/a> <\/li> <\/ul> <\/nav> ","protected":false},"excerpt":{"rendered":"<p>Unleash innovation in AI testing \u2014 drive the future of safe, reliable AI-powered solutions! Porto-based opportunity with hybrid work model. As a Senior AI &amp; Automation QA Engineer, you will be working for our client, a leader in AI-powered features across conversational AI, productivity tools, and rating systems. You will play a pivotal role in &hellip; <a href=\"https:\/\/itdsportugal.com\/en\/it-jobs\/mid-level-ai-automation-qa-engineer-ai-driven-testing\/\">Continued<\/a><\/p>\n","protected":false},"template":"","meta":{"_acf_changed":false,"_matador_source_id":"9992"},"matador-categories":[],"matador-locations":[],"matador-types":[14],"matador-remote-job-possible":[287],"matador-published-category":[],"matador-skills":[211],"matador-seniority":[1533],"matador-position":[],"matador-technology":[3234,1180,661,774,666,794,3235,662],"matador-contract-type":[343,369],"matador-priority":[135],"matador-language":[127],"matador-place":[371,370],"matador-visibility":[657],"matador-choose-website-to-publis":[399,400],"class_list":["post-7667","matador-job-listings","type-matador-job-listings","status-publish","hentry","matador-types-contract","matador-remote-job-possible-hybrid","matador-skills-automation-tester-qa","matador-seniority-mid-level","matador-technology-ai-observability","matador-technology-cypress","matador-technology-javascript","matador-technology-playwright","matador-technology-python","matador-technology-selenium","matador-technology-tracing-tools","matador-technology-typescript","matador-contract-type-b2b","matador-contract-type-permanent-contract","matador-priority-0-urgent","matador-language-english","matador-place-porto","matador-place-portugal","matador-visibility-online","matador-choose-website-to-publis-poland","matador-choose-website-to-publis-portugal"],"acf":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v25.2 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Mid-Level AI &amp; Automation QA Engineer \u2013 AI-Driven Testing - ITDS PORTUGAL<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/itdsportugal.com\/en\/it-jobs\/mid-level-ai-automation-qa-engineer-ai-driven-testing\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Mid-Level AI &amp; Automation QA Engineer \u2013 AI-Driven Testing - ITDS PORTUGAL\" \/>\n<meta property=\"og:description\" content=\"Unleash innovation in AI testing \u2014 drive the future of safe, reliable AI-powered solutions! Porto-based opportunity with hybrid work model. As a Senior AI &amp; Automation QA Engineer, you will be working for our client, a leader in AI-powered features across conversational AI, productivity tools, and rating systems. You will play a pivotal role in &hellip; Continued\" \/>\n<meta property=\"og:url\" content=\"https:\/\/itdsportugal.com\/en\/it-jobs\/mid-level-ai-automation-qa-engineer-ai-driven-testing\/\" \/>\n<meta property=\"og:site_name\" content=\"ITDS PORTUGAL\" \/>\n<meta property=\"article:modified_time\" content=\"2026-09-04T10:47:27+00:00\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"2 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/itdsportugal.com\/en\/it-jobs\/mid-level-ai-automation-qa-engineer-ai-driven-testing\/\",\"url\":\"https:\/\/itdsportugal.com\/en\/it-jobs\/mid-level-ai-automation-qa-engineer-ai-driven-testing\/\",\"name\":\"Mid-Level AI &amp; Automation QA Engineer \u2013 AI-Driven Testing - ITDS PORTUGAL\",\"isPartOf\":{\"@id\":\"https:\/\/itdsportugal.com\/en\/#website\"},\"datePublished\":\"2026-09-04T08:00:06+00:00\",\"dateModified\":\"2026-09-04T10:47:27+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/itdsportugal.com\/en\/it-jobs\/mid-level-ai-automation-qa-engineer-ai-driven-testing\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/itdsportugal.com\/en\/it-jobs\/mid-level-ai-automation-qa-engineer-ai-driven-testing\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/itdsportugal.com\/en\/it-jobs\/mid-level-ai-automation-qa-engineer-ai-driven-testing\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/itdsportugal.com\/en\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Mid-Level AI &amp; Automation QA Engineer \u2013 AI-Driven Testing\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/itdsportugal.com\/en\/#website\",\"url\":\"https:\/\/itdsportugal.com\/en\/\",\"name\":\"ITDS PORTUGAL\",\"description\":\"\",\"publisher\":{\"@id\":\"https:\/\/itdsportugal.com\/en\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/itdsportugal.com\/en\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/itdsportugal.com\/en\/#organization\",\"name\":\"ITDS\",\"url\":\"https:\/\/itdsportugal.com\/en\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/itdsportugal.com\/en\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/itds.pl\/app\/uploads\/sites\/3\/2026\/05\/ITDS_logo-1.png\",\"contentUrl\":\"https:\/\/itds.pl\/app\/uploads\/sites\/3\/2026\/05\/ITDS_logo-1.png\",\"width\":650,\"height\":400,\"caption\":\"ITDS\"},\"image\":{\"@id\":\"https:\/\/itdsportugal.com\/en\/#\/schema\/logo\/image\/\"}}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Mid-Level AI &amp; Automation QA Engineer \u2013 AI-Driven Testing - ITDS PORTUGAL","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/itdsportugal.com\/en\/it-jobs\/mid-level-ai-automation-qa-engineer-ai-driven-testing\/","og_locale":"en_US","og_type":"article","og_title":"Mid-Level AI &amp; Automation QA Engineer \u2013 AI-Driven Testing - ITDS PORTUGAL","og_description":"Unleash innovation in AI testing \u2014 drive the future of safe, reliable AI-powered solutions! Porto-based opportunity with hybrid work model. As a Senior AI &amp; Automation QA Engineer, you will be working for our client, a leader in AI-powered features across conversational AI, productivity tools, and rating systems. You will play a pivotal role in &hellip; Continued","og_url":"https:\/\/itdsportugal.com\/en\/it-jobs\/mid-level-ai-automation-qa-engineer-ai-driven-testing\/","og_site_name":"ITDS PORTUGAL","article_modified_time":"2026-09-04T10:47:27+00:00","twitter_card":"summary_large_image","twitter_misc":{"Est. reading time":"2 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/itdsportugal.com\/en\/it-jobs\/mid-level-ai-automation-qa-engineer-ai-driven-testing\/","url":"https:\/\/itdsportugal.com\/en\/it-jobs\/mid-level-ai-automation-qa-engineer-ai-driven-testing\/","name":"Mid-Level AI &amp; Automation QA Engineer \u2013 AI-Driven Testing - ITDS PORTUGAL","isPartOf":{"@id":"https:\/\/itdsportugal.com\/en\/#website"},"datePublished":"2026-09-04T08:00:06+00:00","dateModified":"2026-09-04T10:47:27+00:00","breadcrumb":{"@id":"https:\/\/itdsportugal.com\/en\/it-jobs\/mid-level-ai-automation-qa-engineer-ai-driven-testing\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/itdsportugal.com\/en\/it-jobs\/mid-level-ai-automation-qa-engineer-ai-driven-testing\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/itdsportugal.com\/en\/it-jobs\/mid-level-ai-automation-qa-engineer-ai-driven-testing\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/itdsportugal.com\/en\/"},{"@type":"ListItem","position":2,"name":"Mid-Level AI &amp; Automation QA Engineer \u2013 AI-Driven Testing"}]},{"@type":"WebSite","@id":"https:\/\/itdsportugal.com\/en\/#website","url":"https:\/\/itdsportugal.com\/en\/","name":"ITDS PORTUGAL","description":"","publisher":{"@id":"https:\/\/itdsportugal.com\/en\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/itdsportugal.com\/en\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/itdsportugal.com\/en\/#organization","name":"ITDS","url":"https:\/\/itdsportugal.com\/en\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/itdsportugal.com\/en\/#\/schema\/logo\/image\/","url":"https:\/\/itds.pl\/app\/uploads\/sites\/3\/2026\/05\/ITDS_logo-1.png","contentUrl":"https:\/\/itds.pl\/app\/uploads\/sites\/3\/2026\/05\/ITDS_logo-1.png","width":650,"height":400,"caption":"ITDS"},"image":{"@id":"https:\/\/itdsportugal.com\/en\/#\/schema\/logo\/image\/"}}]}},"_links":{"self":[{"href":"https:\/\/itdsportugal.com\/en\/wp-json\/wp\/v2\/matador-job-listings\/7667","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/itdsportugal.com\/en\/wp-json\/wp\/v2\/matador-job-listings"}],"about":[{"href":"https:\/\/itdsportugal.com\/en\/wp-json\/wp\/v2\/types\/matador-job-listings"}],"wp:attachment":[{"href":"https:\/\/itdsportugal.com\/en\/wp-json\/wp\/v2\/media?parent=7667"}],"wp:term":[{"taxonomy":"matador-categories","embeddable":true,"href":"https:\/\/itdsportugal.com\/en\/wp-json\/wp\/v2\/matador-categories?post=7667"},{"taxonomy":"matador-locations","embeddable":true,"href":"https:\/\/itdsportugal.com\/en\/wp-json\/wp\/v2\/matador-locations?post=7667"},{"taxonomy":"matador-types","embeddable":true,"href":"https:\/\/itdsportugal.com\/en\/wp-json\/wp\/v2\/matador-types?post=7667"},{"taxonomy":"matador-remote-job-possible","embeddable":true,"href":"https:\/\/itdsportugal.com\/en\/wp-json\/wp\/v2\/matador-remote-job-possible?post=7667"},{"taxonomy":"matador-published-category","embeddable":true,"href":"https:\/\/itdsportugal.com\/en\/wp-json\/wp\/v2\/matador-published-category?post=7667"},{"taxonomy":"matador-skills","embeddable":true,"href":"https:\/\/itdsportugal.com\/en\/wp-json\/wp\/v2\/matador-skills?post=7667"},{"taxonomy":"matador-seniority","embeddable":true,"href":"https:\/\/itdsportugal.com\/en\/wp-json\/wp\/v2\/matador-seniority?post=7667"},{"taxonomy":"matador-position","embeddable":true,"href":"https:\/\/itdsportugal.com\/en\/wp-json\/wp\/v2\/matador-position?post=7667"},{"taxonomy":"matador-technology","embeddable":true,"href":"https:\/\/itdsportugal.com\/en\/wp-json\/wp\/v2\/matador-technology?post=7667"},{"taxonomy":"matador-contract-type","embeddable":true,"href":"https:\/\/itdsportugal.com\/en\/wp-json\/wp\/v2\/matador-contract-type?post=7667"},{"taxonomy":"matador-priority","embeddable":true,"href":"https:\/\/itdsportugal.com\/en\/wp-json\/wp\/v2\/matador-priority?post=7667"},{"taxonomy":"matador-language","embeddable":true,"href":"https:\/\/itdsportugal.com\/en\/wp-json\/wp\/v2\/matador-language?post=7667"},{"taxonomy":"matador-place","embeddable":true,"href":"https:\/\/itdsportugal.com\/en\/wp-json\/wp\/v2\/matador-place?post=7667"},{"taxonomy":"matador-visibility","embeddable":true,"href":"https:\/\/itdsportugal.com\/en\/wp-json\/wp\/v2\/matador-visibility?post=7667"},{"taxonomy":"matador-choose-website-to-publis","embeddable":true,"href":"https:\/\/itdsportugal.com\/en\/wp-json\/wp\/v2\/matador-choose-website-to-publis?post=7667"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}