(1) Testing Overview

Quality is TEAM work

• Requirement Analysis:

  3 Amigo session, Sprint grooming, Sprint planning

• Risk analysis
• CI/CD pipeline
• Testing Pyramid
• Early code delivery - early feedback

🪕 Ideal Testing Timeline in SPRINT

(2) Unit Testing

🪕 Functional vs non-functional testing

Functional testing refers to activities that verify a specific action or function of the code. These are usually found in the code requirements documentation, although some development methodologies work from use cases or user stories. Functional tests tend to answer the question of “can the user do this” or “does this particular feature work.”

Non-functional testing refers to aspects of the software that may not be related to a specific function or user action, such as scalability or other performance, behavior under certain constraints, or security. Testing will determine the breaking point, the point at which extremes of scalability or performance leads to unstable execution. Non-functional requirements tend to be those that reflect the quality of the product, particularly in the context of the suitability perspective of its users.

Functional Testing

• Approaches: Regression or Domain
• Levels: Smoke or Critical
• Types: Compatibility (兼容性) or Usability
• Disciplines: Manual or Automation

Non-functional Testing

• Baseline, Compliance (合规)
• Documentation
• Endurance (耐力)
• L10n (Localization) and I18n
• Performance and Load
• Recovery
• Resilience (弹性)
• Security
• Scalability
• Stress
• Usability
• Volume

Unit Testing

• Test functionality of small application elements
• Written by developers
• Easy to run in IDE
• Take a few minutes or seconds to run
• Easily integrated with CI

Unit Testing - Principles

• F: Fast
• I: Independent
• R: Repeatable
• S: Self-validating
• T: Timely

🪕 Unit Test Code Coverage

80%, 95, 70%?

• Code coverage is not the primary goal for unit testing. It doesn’t guarantee quality.
• Continuous integration can help to avoid decreases in code coverage.

(3) Test Automation

🪕 Development First Mode

🪕 Testing First Mode

TDD - Test-Driven Development: A technique where the unit test is designed first, then the feature is developed.

🪕 Test Automation’s Place In CI

• Delivery Team
• Versison Control
• Unit Test
• Integration Test
• Acceptance Test

1 most popular and important metrics in Test Automation:

ROI Return of investment

• xUnit

• TDD - Test-Driven Development

  this approach works when the test is developed first, then the feature goes.

• ATDD - Acceptance test-driven approach
• Keyword-Driven

• BDD - behavior-driven framework is 1 successor of TDD.

  Instead of defining the test itself, we focus on feature behavior.

• Data-driven

(4) Testing of Non-Functional Requirements

• Peformance
• Security
• Reliability
• Maintainability
• Usability
• Supportability

🪕 Performance

• Response time
• Capacity
• Stability
• Scalability

🪕 Engagement Planning

• Requirements analysis
• Scenarios creation + Defining load model
• Choosing load tools
• Developing test scripts

Manual Testing

🪕 Pros

• Lower short-term cost
• Each execution is slightly different - has certain randomness
• Easily adaptable to environment changes
• Flexible - allows flow changes
• Can be supported by tools
• Can detect sligth changes that tools cannot as not looking into them

🪕 Cons

• Time-consuming
• Certain task are hard to perform manually
• Can be repetitive and boring
• Less accurate than automated tests

Automation Testing

🪕 Pros

• Cost effective in long term
• Repeatable
• Quick execution - quick feedback
• Leaves time fro more creative testing activities
• Parallel execution

🪕 Cons

• Relies heavily on tools
• Tools have a cost even if they are open source
• Tools have limitations
• Automation is not suited for every testing type
• Effective implementation requires expertise

Suitable for: Regression, Load testing, Repeated execution

What is security tesing?

• Static Application Secuirty Testing (SAST)

Focus on analysis of the application source code for security vulnerabilities

• Dynamic Application Security Testing (DAST)

Focus on simulating attacks against running application

• Interactive Application Security Testing (IAST)

Combination of SAST and DAST

OWASP TOP 10 (2017)

The Open Web Application Security Project

Top 10 Critical Web Application Security Risks:

1. Injection
2. Broken Authentication (认证)
3. Sensitive Data Exposure (曝露)
4. XML External Entities (XXE)
5. Broken Access Control
6. Security Misconfiguration
7. Cross-Site Scripting (XSS)
8. Insecure Deserialization (反序列化)
9. Using Components with Known Vulnerabilities (漏洞)
10. Insufficient Logging & Monitoring

(4)

Frontend vs. Backend Testing

🪕 Frontend

• Visible, tangible (明白的, 明确的)
• Requires human perception to detect subtle errors
• Auto tests can be more fragile
• Compatibility testing with different Browsers, OS, Devices and Connection Speeds

Tests: Functional, GUI, Usability, Accessibility, etc

🪕 Backend

• Communication and data is less visible
• Auto tests are more stable
• May require customized tools
• Component interdependency

Tests: Functional, Integration, Performance, API, etc.

Mobile Testing

• Multiple Mobile Devices
• Device Fragmentation & Various S Platforms
• Different Mobile App Types
• Numerous Test Interfaces
• Variety of Testing Tools

🪕 Mobile Testing - Parameters

• Services
• Applications
• Application frameworks
• Operating systems
• Platforms
• Devices
• aggregators (聚合)
• Networks
• Operators

More…

• SonarQube

  SonarQube is an open-source platform developed by SonarSource for continuous inspection of code quality to perform automatic reviews with static analysis of code to detect bugs, code smells, and security vulnerabilities on 20+ programming languages.

• Fortify

  propriety tool to help do Static Code Analysis from a security point of view.

• JSHint

  a static code analysis tool for JS used at Mozilla, Facebook, wikipedia, Twitter etc.

Smoke Testing

Smoke Testing, also known as “Build Verification Testing”, is a type of software testing that comprises of a non-exhaustive set of tests that aim at ensuring that thte most important functions work. The result of this testing is used to decide if a build is stable enough to proceed with further testing.

Smoke tests can be functional tests or unit tests. Functional tests exercise the complete program with various inputs. Unit tests exercise individual functions, subroutines, or object methods.

In the DevOps paradigm, use of a BVT step is one hallmark (特点) of the continuous integration (CI) maturity stage.

Regression testing

Regression testing is the process of testing changes to computer programs to make sure that the older programming still works with the new changes. Regression testing is a normal part of the program development process and, in larger companies, is done by code testing specialists.

🪕 penetration (渗透) test

Vulnerability Assessment