AI Testing (Testing of AI-based Systems)
Do you not only want to understand AI, but also want to go one step further and also ensure its quality and learn how to test ML models? This training first takes you through AI and machine learning basics as well as different specifications, data and metrics. You will then learn techniques and testing procedures to test AI systems.
ISTQB® certification possible Foundation
Goals
- Introduction to artificial intelligence and understanding of machine learning
- Understand what quality characteristics exist for AI systems
- Overview of different test systems and quality characteristics
- Methodologies for testing AI-based systems
- Which test environment does the use of AI require for test use cases?
Target Groups
Usability expert
AI Expert
Test manager
Tester
Test Automation Specialist
Test engineer
Product Owner
and everyone who wants to get involved in testing and using artificial intelligence
Content
01
1. Introduction to Artificial Intelligence (AI)
- Definition of AI and AI effects
- Narrow, General and Super AI
- AI-based and conventional systems
- AI technologies
- AI development frameworks
- Hardware for AI-based systems
- AI as a Service (AIaaS)
- Pre-trained models
- AI standards and regulations
02
2. Quality characteristics for AI-based systems
- Flexibility and adaptability
- autonomy
- evolution
- Bias
- ethics
- Side effects and reward hacking
- Transparency, interpretation and explainability
- Security and AI
03
3. Machine Learning (ML) – Overview
- Forms of machine learning
- ML workflow
- Select a form of ML
- Factors influencing the selection of ML algorithms
- Overfitting and underfitting
04
4. ML – Data
- Data preparation as part of the ML workflow
- Training, validation and testing datasets in the ML workflow
- Quality problems in the data set
- Data quality and its impact on the ML model
- Data labeling for supervised learning
05
5. ML performance metrics
- Confusion Matrix
- ML performance metrics for classification, regression, and clustering
- Limitations of ML performance metrics
- Select ML performance metrics
- Benchmark suites for ML performance metrics
06
6. Testing AI-based systems – overview
- Specification of AI-based systems
- Test levels for AI-based systems
- Test data for testing AI-based systems
- Testing automation bias in AI-based systems
- Document AI elements
- Test concept drift
- Choosing a testing approach for an ML system
07
7. Test AI-specific quality characteristics
- Challenges when testing self-learning systems
- Testing autonomous self-learning systems
- Test algorithmic, sampling, and inappropriate bias
- Challenges in testing probabilistic and non-deterministic AI-based systems
- Challenges in testing complex AI-based systems
- Testing the transparency, interpretation and explainability of AI-based systems
- Test oracle for AI-based systems
- Test objectives and acceptance criteria
08
8. Methods and techniques for testing AI-based systems
- Adversarial attacks and data poisoning
- Pairwise testing
- A/B testing
- Back-to-back testing
- Metamorphic Testing (MT)
- Experience-based testing of AI-based systems
- Selection of testing techniques for AI-based systems
09
9. Test environments for AI-based systems
- Test environments for AI-based systems
- Virtual test environments to test AI-based systems
10
10. Using AI for testing
- AI technologies for testing
- Using AI to analyze bug reports
- Use of AI to generate test cases
- Using AI to optimize regression testing
- Use of AI for error prediction
- Using AI for user interface testing
Certification
Certification according to ISTQB® is possible for this training.
After completion we recommend
Certified Trainings
Internationally recognized certifications for your career.
Experienced Trainers
Learn from competent experts with practical experience.
Flexible Formats
Webinars, video trainings or on-site – exactly as you need it.