At ValidExamDumps, we consistently monitor updates to the iSQI CT-AI exam questions by iSQI. Whenever our team identifies changes in the exam questions,exam objectives, exam focus areas or in exam requirements, We immediately update our exam questions for both PDF and online practice exams. This commitment ensures our customers always have access to the most current and accurate questions. By preparing with these actual questions, our customers can successfully pass the iSQI Certified Tester AI Testing exam on their first attempt without needing additional materials or study guides.
Other certification materials providers often include outdated or removed questions by iSQI in their iSQI CT-AI exam. These outdated questions lead to customers failing their iSQI Certified Tester AI Testing exam. In contrast, we ensure our questions bank includes only precise and up-to-date questions, guaranteeing their presence in your actual exam. Our main priority is your success in the iSQI CT-AI exam, not profiting from selling obsolete exam questions in PDF or Online Practice Test.
Consider a natural language processing (NLP) algorithm that attempts to predict the next word that you would like to type in a text message. An update to the algorithm has been created that should increase the accuracy of the predictions based on user typing patterns. The old algorithm was rated for accuracy by the users. Then, after the new update was released, the users rated the updated algorithm. A statistical test was used to compare between the two versions of the algorithm to see whether or not the update should remain in place.
This is an example of what type of testing?
A/B testing is a statistical testing method that compares two different versions of a system to determine which one performs better. In this scenario, the old NLP algorithm was rated for accuracy, and after the update, the new algorithm was also rated by users. A statistical test was performed to compare the two versions, which is the fundamental approach of A/B testing.
A/B testing is commonly used in:
User experience testing (e.g., comparing different versions of a website).
ML model evaluation (e.g., comparing two AI-based classifiers).
Performance assessment (e.g., determining if a new recommendation algorithm is more effective).
This approach allows for data-driven decisions, ensuring that any changes to the system result in meaningful improvements.
Reference from ISTQB Certified Tester AI Testing Study Guide:
Section 9.4 - A/B Testing states that A/B testing is used to compare updates in AI-based systems to determine if the newer version is better.
Which ONE of the following tests is LEAST likely to be performed during the ML model testing phase?
SELECT ONE OPTION
The question asks which test is least likely to be performed during the ML model testing phase. Let's consider each option:
Testing the accuracy of the classification model (A): Accuracy testing is a fundamental part of the ML model testing phase. It ensures that the model correctly classifies the data as intended and meets the required performance metrics.
Testing the API of the service powered by the ML model (B): Testing the API is crucial, especially if the ML model is deployed as part of a service. This ensures that the service integrates well with other systems and that the API performs as expected.
Testing the speed of the training of the model (C): This is least likely to be part of the ML model testing phase. The speed of training is more relevant during the development phase when optimizing and tuning the model. During testing, the focus is more on the model's performance and behavior rather than how quickly it was trained.
Testing the speed of the prediction by the model (D): Testing the speed of prediction is important to ensure that the model meets performance requirements in a production environment, especially for real-time applications.
ISTQB CT-AI Syllabus Section 3.2 on ML Workflow and Section 5 on ML Functional Performance Metrics discuss the focus of testing during the model testing phase, which includes accuracy and prediction speed but not the training speed.
A bank wants to use an algorithm to determine which applicants should be given a loan. The bank hires a data scientist to construct a logistic regression model to predict whether the applicant will repay the loan or not. The bank has enough data on past customers to randomly split the data into a training data set and a test/validation data set. A logistic regression model is constructed on the training data set using the following independent variables:
Gender
Marital status
Number of dependents
Education
Income
Loan amount
Loan term
Credit score
The model reveals that those with higher credit scores and larger total incomes are more likely to repay their loans. The data scientist has suggested that there might be bias present in the model based on previous models created for other banks.
Given this information, what is the best test approach to check for potential bias in the model?
Bias in an AI system occurs when the training data contains inherent prejudices that cause the model to make unfair predictions. Experience-based testing, particularly Exploratory Data Analysis (EDA), helps uncover these biases by analyzing patterns, distributions, and potential discriminatory factors in the training data.
Analysis of the Answer Options:
Option A: ''Experience-based testing should be used to confirm that the training data set is operationally relevant. This can include applying exploratory data analysis (EDA) to check for bias within the training data set.''
This is the correct answer. EDA involves examining the dataset for bias, inconsistencies, or missing values, ensuring fairness in ML model predictions.
Option B: ''Back-to-back testing should be used to compare the model created using the training data set to another model created using the test data set. If the two models significantly differ, it will indicate there is bias in the original model.''
Back-to-back testing is used for regression testing and to compare versions of an AI system but is not primarily used to detect bias.
Option C: ''Acceptance testing should be used to make sure the algorithm is suitable for the customer. The team can re-work the acceptance criteria such that the algorithm is sure to correctly predict the remaining applicants that have been set aside for the validation data set ensuring no bias is present.''
Acceptance testing focuses on meeting predefined business requirements rather than detecting and mitigating bias.
Option D: ''A/B testing should be used to verify that the test data set does not detect any bias that might have been introduced by the original training data. If the two models significantly differ, it will indicate there is bias in the original model.''
A/B testing is used for evaluating variations of a model rather than for explicitly identifying bias.
ISTQB CT-AI Syllabus Reference:
Bias Testing Methods: 'AI-based systems should be tested for algorithmic bias, sample bias, and inappropriate bias. Experience-based testing and EDA are useful for detecting bias'.
Exploratory Data Analysis (EDA): 'EDA helps uncover potential bias in training data through visualization and statistical analysis'.
Thus, Option A is the best choice for detecting bias in the loan applicant model.
Written requirements are given in text documents, which ONE of the following options is the BEST way to generate test cases from these requirements?
SELECT ONE OPTION
When written requirements are given in text documents, the best way to generate test cases is by using Natural Language Processing (NLP). Here's why:
Natural Language Processing (NLP): NLP can analyze and understand human language. It can be used to process textual requirements to extract relevant information and generate test cases. This method is efficient in handling large volumes of textual data and identifying key elements necessary for testing.
Why Not Other Options:
Analyzing source code for generating test cases: This is more suitable for white-box testing where the code is available, but it doesn't apply to text-based requirements.
Machine learning on logs of execution: This approach is used for dynamic analysis based on system behavior during execution rather than static textual requirements.
GUI analysis by computer vision: This is used for testing graphical user interfaces and is not applicable to text-based requirements.
A tourist calls an airline to book a ticket and is connected with an automated system which is able to recognize speech, understand requests related to purchasing a ticket, and provide relevant travel options. When the tourist asks about the expected weather at the destination or potential impacts on operations because of the tight labor market the only response from the automated system is: "I don't understand your question."
This AI system should be categorized as?
Narrow AI (also known as Weak AI) is designed to perform specific tasks without possessing general intelligence or consciousness. The AI system in the question is capable of recognizing speech and responding to specific booking-related requests but fails when asked about unrelated topics (such as weather or labor markets).
Analysis of the Answer Options:
Option A: ''General AI''
Incorrect. General AI (AGI) refers to an AI system that can perform any intellectual task a human can. The described system is task-specific and does not exhibit general intelligence.
Option B: ''Narrow AI''
Correct. The AI system is limited to a predefined domain (ticket booking) and cannot process unrelated questions. This is characteristic of Narrow AI, which excels at specific tasks but lacks broader cognitive abilities.
Option C: ''Super AI''
Incorrect. Super AI surpasses human intelligence, exhibiting advanced reasoning and creativity. The AI in the scenario is far from this level.
Option D: ''Conventional AI''
Incorrect. Conventional AI is a broader term that may include rule-based systems. The described system relies on machine learning and natural language processing, making it more aligned with Narrow AI.
ISTQB CT-AI Syllabus Reference:
Definition of Narrow AI: 'Narrow AI refers to AI systems that are designed to perform a single task or a limited set of tasks, without general intelligence'.
General vs. Narrow AI: 'General AI remains an area of research, while most current AI applications fall into the category of Narrow AI'.
Thus, option B is the correct categorization for the AI-based ticket booking system.
