White Box Testing - Part V

Loop Testing

• Loop testing is the testing of resource(s) multiple times under program control.
• The most important aspect of this test is to ensure that the control loop is executed multiple times and exited when a particular condition is satisfied.

# Typical bugs that arise are
• Variable not incremented, hence an infinite loop is established,
• Specifying the loop exit criteria incorrectly, hence the resultant output will be incorrect.

# Typical examples are

While not rs.EOF then
statement 1
statement 2
rs.MoveNext
Wend

Will generate an infinite loop, due to non-availability of the next incrementing record.


Important tip for Loop testing
• Note that unstructured loops are not to be tested.
• They need to be RE-DESIGNED for better performance.

Summary
• Ensuring an effective White Box test for your code, will generate an efficient, and
performance oriented code, will less memory leaks and unwanted memory
usage.
• Needs to be done with a lot of concentration and dedication.
• Usage of data flow graph is an effective tool for reducing unwanted data
declaration and usage.

Other Techniques/ Tools
# Profiling tool :
• Helps the tester to uncover bottlenecks as regards performance.
• Uncovers memory leaks and memory access errors.

# Code Based Fault Injection.
• Changes program states by injecting software source code to force changes into the state of the program as it executes.
• this technique forces non-normative behavior of the software, and the resulting
understanding can help determine whether a program has vulnerabilities that can lead to security violations.
• This technique can be used to force error conditions to exercise the error handling code, change execution paths, input unexpected (or abnormal) data, change return values, etc.

Abuse Cases
• Abuse cases help security testers view the software under test in the same light as
attackers do .
• With access to the source code, a tester is in a better position to quickly see where the weak spots are compared to an outside attacker.

• The simplest, most practical method for creating abuse cases is usually through a
process of informed brainstorming, involving security, reliability, and subject matter expertise.

Error Handling techniques
• Testing Exceptions and error-handling should be verified thoroughly by simulating partial and complete fails.
• Proper error recovery, notification and logging should be checked.

Transactions test
• This test is employed for Data test as regards database.
• Systems that employ transaction (either local or distributed) should be validated to ensure ACID (Atomicity, Consistency, Isolation,
Durability).

Thank you, Everybody

• You can use these techniques for those little programs you create, so that you are confident on creating efficient, performance oriented programs.
• Do get back for any clarification, whatsoever.

White Box Testing - Part IV

Data Flow Testing

• Data Flow testing targets the lifecycle of a particular piece of data (i.e. a variable) in an application.
• The technique to implement an effective Data Flow testing is to look for patterns of data usage based on which risky areas of the code can be found and more test
cases can be applied.

Let’s see some basics as regards DATA.

# DATA is used in 4 ways
• (D)efined,
• (U)sed in a predicate,
• (U)sed in a calculation, and
• (K)illed.

E.g.
• Usage of a piece of data in a calculation after
it has been killed, show an anomaly in the
code; Therefore a bug.

# Most Failures involve
• Execution of an incorrect definition
• Incorrect assignment or input statement
• Predicate is faulty
• Definition is missing

• Explore the sequences of events related to the data state and the unreasonable things that can happen to data.

• Explore the effect of using the value produced by each and every computation.
• Considerable effort needs to be given to data declaration statements, since most of
the code contains nearly half of the contents as data declaration.

Some examples are given below

• Sum = expression
• Read (var1, var2, var3)
• Write (total1, total2, total3)
• Method call : CompareValue(val1, val2)
• While : while Boolean do statements
• For statement : for (val1 = init to final)
• If then else : If Boolean then S1;

if Boolean then S1 else S2.
• case : Case var1:
Statement 1:
Statement 2:

# Use the techniques given below for an effective data flow test.
• DD -bug
• DK -bug
• DU – normal
• KD – normal
• KK – harmless, but a bug
• KU – bug
• UD – normal
• UK – normal
• UU -normal

where
D - Define / U - Use / K - Kill

White Box Testing - Part III

White Box testing types
• We have the following types for White box testing based on the technicalities
• Code Coverage Analysis (includes)
• Basis Path Testing
• Flow Graph Notation,
• Cyclomatic complexity,
• Control Structure Testing
• Conditions Testing,
• Data flow testing,
• Loop Testing.

Basis Path Testing
• This is a testing mechanism proposed by McCabe,
• The aim of this testing is to derive the logical complexity measure of a procedural design and use this information as a guide for defining a
basic set of execution paths.
• The resulting test cases will ensure that the basic set of paths will execute every statement at least once.

Flow Graph Notation
• It is a representation, using graph notation, of all paths that might be traversed
through a program during its execution,
• Each Node in the graph represents a basic block,
• Directed edges are used to represent jumps in the control flow,
• There should exist two blocks (Entry Block and Exit Block)
• Entry Block : through which control enters into the flow graph,
• Exit Block : through which all control flow leaves.

Flow Graph Notation : Example
• Consider the following code
0: (A) t0 = read_num
1: (A)ift0mod 2 ==0goto 4
2: (B) print t0 + " is odd."
3: (B) goto 5
4: (C) print t0 + " is even."
5: (D) end program



• In the given example, we have 4 basic blocks named A, B, C, D.
• A from line 0 to 1, B from line 2 to 3, C for line 4, and D for line 5.
• Here A is the entry block, and D is the Exit block.
• Lines 4 and 5 are JUMP TARGETS.


Cyclomatic Complexity
• It is a measurement concept developed by McCabe.
• It is used to measure the complexity of a program.
• It measures the number of linearly independent paths through a program.
• It is computed using a graph that describes the control flow of the program,
• The nodes of the graph corresponds to the commands of a program.
• A directed edge connects two nodes if the second command is executed immediately
after the first command.
• It is given by
M=E–N+P
• Where M = cyclomatic complexity
• E = number of edges of the graph
• N = number of nodes of the graph
• P = number of connected components.
• A simple way to determine the cyclomatic number is M = number of closed loops + 1
• It provides an upper bound for number of tests required to guarantee coverage of all program statements.
• It is denoted by V(G) and is given by
V(G) = (# Edges) – (#Nodes) + 1

Control Structure Testing
• In programming, statements are typically executed one after the other and in the
order they are written. However most programming languages have CONTROL
FLOW statements which allow variation in this sequential order.

# Control structures have the following characteristics.
• Statements may only be obeyed under certain conditions (choice statements),
• Statements may be obeyed repeatedly, (looping structures),
• A group of statements may be obeyed (subroutines).

• It consists of the following individual tests.
a) Condition testing,
b) Data Flow testing
c) Loop Testing.

Condition Testing
• The basis for carrying out CONDITION TESTING is the representation of the flow
of control within a program/ subprogram.
• The most important aspect is to ensure that the Boolean controlling expressions
are adequately tested.
• The process of CONDITION TESTING ensures that a controlling expression has
been adequately exercised by constructing a CONSTRAINT SET for every expression
and ensuring that every item in the CONSTRAINT SET is included to test the
expression.

E.g.
• If DateValid then
• While NOT DateValid then

The CONSTRAINT SET for both these expressions is {t,f} which indicates that to
adequately test these expressions they should be tested twice with DateValid
having the values True and False.

• Expanding the Constraint Set, we have the following set of values to be exercised.
{{t,t{ {t,f} {f,t} {f,f}}

• When the simple boolean expression is combined with a relational expression,
then the Constraint set will include more items.

• If only the left hand Boolean value is a relational expression the condition set would be
{{= ,t } {=,f }{> ,t}{< ,t } {>,f } {< ,f }}

• If both Boolean values are relational expressions the condition set would become {{=,=}{=,> }{=,<}{>,=}{<,=} {>,>}{>,< }{<,>} {< ,< }}

White Box Testing - Part II

White Box Testing
Abhilash Gopi
Part II

Types of Testing Techniques
# There are available the techniques
• Black Box Testing technique, and
• White Box Testing technique.

Black Box Testing
• A testing technique where testing is done based on previously understood
requirements or functionality without any knowledge of how the code is executed.

White Box Testing
• White Box Testing, or Clear Box Testing, or Glass Box Testing, or Structural Testing is used to check that the outputs of a program, based on certain defined inputs, conform to the structural specification of the program.

Execution Features of White Box Testing

• Needs to be done with a knowledge of the code.
• Most organizations involve the programmer along with the White Box Tester when the code is more complex.
• Some organizations go to the extent of involving number of programmers for complex codes to achieve a full level of White Box test. This is done since involving more programmers will enable them to gain more varying perspectives into the possible outcomes.

Generic Workflow for Application Development
Development Workstations
Integrated Env. Staging Area Production
Development Environment


Planning White Box testing
• Proper Planning is the key to a successful White Box Test.

• Planning involves the following
a) Decide the goals/ objective for performing the White Box Test,
b) Find out to what extent White Box Testing can be done (i.e. Find out the Scope),
c) Select the best person or team to perform the White Box test.

Planning White Box testing
• Assign Roles and Responsibilities for the people involved in the testing process,
• Select the best applicable methodology for carrying out the test,
• Confirm the deliverables during the course of carrying out the White Box test.

Deciding the Goal
• For most organizations who are involved in White Box testing, the goals are as
follows
• Initiating a strategic initiative to build quality throughout the life cycle of a software product or service,
• Accept White Box testing as a complementary function to Black Box testing,
• Take steps to perform and ensure complete
coverage at the component level,
• Optimizing code and improving performance.

Deciding the Scope of testing
• It indicates that aspect of testing as regards timing and coverage.
• Testing could be done throughout the various phases of the SDLC. The extent of
test and timing of white box test is very crucial for a successful application
development (with few performance and code related issues).

Selecting the Right team
• White Box testing requires a thorough understanding and knowledge of the code and
its functioning.
• A person who is conversant with the above technical skills and also with the software development process are selected for white box
testing.
• The team should ensure complete and thorough testing of all components involved in the application.

Assigning Roles and Responsibilities

• Typically during the course of an application development, a single point of contact is decided to facilitate the final maintenance and deployment.
• Each developer in various faculties of the application development process should
contribute their part to ensure that they have delivered thoroughly tested components towards the final application.

Methodology
• The organization should decide on the methodology to go about White Box
testing.
• It will include the following
• A well defined and concise Test Plan for carrying out the White box test,
• The testing to be done based on the set of test plans need to be tracked with the
milestones in the project schedule
• The Project Management should coordinate
regularly with the client to know any changes
forthcoming from the client.
• Plan accordingly based on these changes,
• Gather any client specific process/ methods
for testing these components,
• Also coordinate with the client to obtain client
data for a comprehensive testing.

Decide the Deliverables
• The Project Management should clearly identify the deliverables.
• These deliverables should clearly support the testing process carried out,
• Typical deliverables are
• Test Plan,
• Defect reports,
• Periodic status reports,
• Client requested deliverables.

Typical areas for White Box Testing
# White Box testing can be successfully implemented in the following areas
• Component functionality,
• Database testing,
• Network testing,

White Box Testing - Part I

White Box Testing


Part I



Types of Testing

# Two types of Testing
• White Box Testing
• Black Box Testing

# Black Box Testing
• Basically used and implemented to test for the functionality of a module/ program/
application, with no knowledge of internal design of the product/ application.
• Tests are primarily based on requirements and functionality.

# White Box testing

• Also called Glass Box, structural, Clear box or Open box testing,
• In white box testing, the internal logic is given due importance,
• Tests are based on coverage of code statements, branches, and conditions,

Black Box testing

# Advantages
• Efficient when used on larger systems,
• Tester and Developer work independently, hence test is balanced and unprejudiced,
• Testing and Tester can be non-technical,
• Tests are conducted from the end-users viewpoint,
• Identifies the vagueness and contradiction in functional aspects,
• Test cases can be designed as soon as functional specifications are complete.

Black Box Testing

# Disadvantages
• Test cases are tough and challenging to design, without having clear functional specifications,
• It is difficult to identify tricky inputs, if the test cases are not developed from functional specifications,
• Limited testing time, hence difficult to identify all possible inputs,
• Chances of having unidentified paths during testing.

White Box testing

# Advantages
• Test is accurate since the tester knows what individual programs are supposed to do,
• Deviation from the intended goals as regards the functioning of the program can be checked and verified accurately.

# Disadvantage
• Requires thorough knowledge of the programming code to examine the related inputs and outputs.




White Box Testing (Pre-requisites)

• The key to undertake any White box testing is to understand the System architecture.
• The System Architecture documents should form the basis for identification of Systems, Sub-systems and generation of test cases.

Identification of Test Items
• The Identification of Test Items should be done based on the specifications of the
product/ application

Specifications required
• The specifications required for identification of test items are
• Functions of the system (List of functions)
• Response criteria (bench marking and stress testing)
• Volume constraints (number of users, hits, stress testing)
• Database responses (flushing, cleaning, update rates)
• Network Criteria (network traffic, choking)
• Compatibility (environments, browsers)
• User Interface / Friendliness
• Modularity (ability to easily interface with other tools)
• Security

Criteria for Test Cases

# Each test case document/ item should have the following features
• It should be uniquely identifiable
• It should be unambiguous
• It should have well-defined test data or data patterns
• It should have well defined PASS/ FAIL criteria for each sub-item and Overall criteria for PASS/ FAIL of the entire test.
• It should be easy to record
• It should be easy to demonstrate repeatedly

What White Box testing contains?

• White Box testing basically strives to attain Code coverage.
• How can we obtain and achieve good code coverage?

Basic Path Testing

• It is a mechanism to derive a logical complexity measure of a procedural design
and use this as a guide for defining a basic set of execution paths.
• With these test cases, we can ensure that the BASIC SET will execute every statement at least once.

Flow Graph Notation

• A notation for representing control flow similar to flow charts.

Example (Flow graph notation)


# Eg.
X = ((b2 -4ac) ½ -b )/ (2a)

Can be represented using functional notation

div(sub(pow(sub(pow(b,2),mul(4,mul(a,c))), ½),b), mul(2,a))

The flow graph diagrammatic representation is given below.




Cyclomatic Complexity

• It gives a quantitative measure of the logical complexity.
• On obtaining this value, it is found to give the number of independent paths in the basis set, and an upper bound for the number of tests to ensure that each statement is executed at least once.
• Cyclomatic complexity essentially represents the number of paths through a particular section of code, which in object-oriented languages applies to methods.
• Cyclomatic complexity's formal academic equation from graph theory is as follows:

CC = E -N+ P

where E represents the number of edges on a graph, N the number of nodes, and P the number of connected components


public int getValue(int param1)
{
int value = 0;
if (param1 == 0)

{
value = 4;
}
else
{
value = 0;
}
return value;
}



# Note that there are two decision points:
an if and an else. (which accounts for 2 CC)

In addition the method's entry point automatically adds one, the final value equals 3.

Hence Cyclomatic Complexity is 3

Cyclomatic Complexity (Ignored facts)

• As the cyclomatic complexity of a method is fundamentally the number of paths contained in that method, a general rule of thumb states that in order to ensure a high level of test coverage, the number of test cases for a method should equal the method's cyclomatic complexity. Unfortunately, this bold statement
often is ignored.

Control Structure Testing

# Can be achieved by
• Conditions Testing
• Data Flow Testing
• Loop Testing

Conditions Testing

• Condition Testing aims to exercise all logical conditions in a program.
• The Conditions to be tested may be
- Relational Expressions (a operand b), where a and b are arithmetic expressions,
- Simple condition (Boolean variable or relational expression)
- Compound condition: made up of two or more simple conditions, boolean operators

Data Flow testing

• Data Flow testing is done based on the selection of test paths according to the
location of definitiions and use of variables.

Loop Testing

• Loop structures are fundamental to many algorithms.
• Loops can be defined as Simple, Concatenated, Nested, and unstructured.

Important tip for Loop testing

• Note that unstructured loops are not to be tested.
• They need to be RE-DESIGNED for better performance.

TEST AREAS

Unit Testing
• During the course of testing, We find that various constituents of the system are tested in increasing degrees of granularity. i.e. We test from a component (smallest) to the full system.
• A Component is the smallest unit for testing.

Component

• A Component is an independent, isolated and reusable unit of a program that performs a well-defined function.
• Most components have public interfaces that allow it to be used to perform its functions.

Module

• A module comprises of one or more components to achieve the business function.
• The module encapsulates the functionality of its components and appears as a “Black Box” to its users.

Sub-System

• Sub-Systems are defined as heterogeneous collections of modules to achieve a business function.
Eg. A banking application might interface to a signature reader, Accounting mechanism, and a database component to perform banking related operations.

System

• The full system uses multiple subsystems to implement the full functionality of the Application.

Functional Test areas

• Security
• Network resources
• Capacity
• Scalability
• Reliability
• Burn-in
• Clustering

Security

• To address Security issues, following aspects are tested.
• Authentication (The authentication mechanism is tested based on class of user and password validation)
• If third party products such as LDAP servers are used, those are tested as well.
• LDAP, Lightweight Directory Access Protocol, is an Internet protocol that email and other programs use to look up information from a server



Data Security

• Flow of information as well as firewall access issues are tested for all aspects of the system under test.
• If data encryption is being used, it is tested for compatibility with standards.

Network Resources

• The factors to be tested as regards Network resources are
• Bandwidth
• Application dependency

Bandwidth

• System behavior is tracked by varying the bandwidth available on the network from
1Kbps to 10Mbps.
• Parameters such as timeouts and breakdowns of operations are monitored.

Applications dependency

• If the System under test has dependencies on network applications like SNMP, these are tested
• SNMP are tested for failover and recovery.
• SNMP (Simple Network Management Protocol)

SNMP

• It is an application layer protocol that facilitates the exchange of management
information between network devices.
• It is part of the Transmission Control Protocol/Internet Protocol (TCP/IP) protocol
suite.
• SNMP enables network administrators to manage network performance, find and solve
network problems, and plan for network growth.

Capacity

• Capacity testing looks at resource demands for each of the test vectors in various parts of the system.
• The test vectors are designed to reflect normal usage in terms of workflows and
bandwidth requirements.

Capacity test

• The basis for a capacity test is a thorn for most organizations, because they lack test requirements.
• Factors that affect the applications capacity in terms of response time and performance are
- Inefficient SQL statements
- Hardware equipment
- LAN/ WAN
- S/W version
• Underlying database system and database design
• Customization carried out for the Application
• Adding end-users from a previous release in a
production environment

Scalability

• This aspect of testing recognizes the potential for increases in scalability from a
system architecture, design, development, and deployment viewpoint.
• The application should be tested for the ability to scale linearly by the balanced
addition of computing (N/w, CPU, Memory, Disk etc) resulting in a proportionate increase in capacity at the same performance level.

Reliability

• The purpose of this test is to ensure that the Application under test performs in a
consistent manner.

Burn-in

• In this testing, the system or subsystem under test is run at full and partial loads over an extended period of timeto test for consistency.

Clustering

• For systems, that use clustering support, one or more systems are removed from the
installation on an operational system to check for system recovery, and verify fail over to the backup system.
• The application is tested for damage or compromise of data.

Test Case Designing

Test Case Design
# The design of tests is based on the following
• Test Strategy
• Test Planning
• Test Specification
• Test Procedure

Designing Unit Test cases

# To ensure that Unit testing is done correctly, ensure the following steps
• Step 1 : Make it run
• Step 2 : Positive Testing
• Step 3 : Negative Testing
• Step 4 : Special considerations
• Step 5 : Coverage Tests
• Step 6 : Coverage Completion

# Step 1 : Make it run

• The purpose of the first test case in any unit test specification should be to execute the unit under test in the simplest way possible.
• Suitable techniques
• Specification derived tests
• Equivalence partitioning

# Step 2 : Positive Testing

• Test case should be designed to show that the unit under test does what it is supposed to do.
• Suitable techniques
• Specification derived tests
• Equivalence partitioning
• State-transition testing.(using Start, input, output, finish)

# Step 3 : Negative Testing
• Existing test cases should be enhanced and further test cases should be designed to
show that the software does not do anything that it is not specified to do so.
• This will mean incorporating Negative Testing to ensure the same.
• Suitable techniques
• Error guessing
• Boundary value analysis
• Internal boundary value testing
• State-transition testing

# Step 4: Special Considerations

• Test cases should be designed to address issues such as performance, safety
requirements and security requirements.
• Particularly in the cases of safety and security, it should be possible to give test cases special emphasis to facilitate security analysis or safety analysis and certification.
• Suitable techniques
• Specification derived tests

# Step 5 : Coverage Tests

• The test coverage likely to be achieved by the designed test cases should be visualized.
• If found wanting, further test cases can be added to achieve specific test coverage
objectives.
• Suitable techniques
• Branch testing
• Condition testing
• Data definition-use testing
• State transition testing

# Step 6 : Coverage Completion

• This step takes into consideration the organization standards for the specification of a unit.
• There are likely to have human errors made in the development of a test specification.
• There may be complex decision conditions, loops, branches within the code for which
coverage targets may not have been met on test execution.

# Step 6 : Coverage Completion

• When coverage objectives are not achieved, analysis must be conducted to determine,
Why?

Reasons (Coverage Completion)

• A coverage objective may not be achieved due to
• Infeasible paths or conditions : Provide justification why the path or condition was not tested.
• Unreachable or redundant code : correct the code by deleting the offending part of the code.
• Insufficient test cases : test cases should be refined and further test cases added.

Coverage Completion

• Suitable techniques
• Branch testing
• Condition testing
• Data definition use testing
• State transition testing

Test Case Design Techniques

• The techniques for designing test cases can be clubbed into two broad categories.
• Black box techniques
• White Box techniques




Thank you for your wonderful
patience