Software Testing - Basics

Software Quality Assurance & Testing - Abhilash Gopi

Definitions

Computer :
A Computer is a fast electronic device that accepts input data, processes it and produces the output information under the direction and control of a computer program.


Data :
raw facts (Eg. Marks of a student in individual subjects)
Processing :
Some Arithmetic or Logical operation carried out on the input data.

Information :
Processed Data (Eg. Total Marks, Percentage and Grade obtained by the student)

Computer Program :
A set of meaningful sequential instructions written in a computer language that will instruct the computer what to do, in order to solve the given problem.


Who is the Author of the Computer Program?
Programmer/ Software Engineer.

Who is the Programmer/ Software Engineer?
Basically a HUMAN, who is not perfect.
Every HUMAN makes mistakes, hence the reliability and stability of the Software is at stake

Verify Reliability and Stability?
How can we verify the Reliability and Stability of the concerned software?
Software Quality Assurance, and
Software Testing.

Why do Software have Defects/ Bugs?
To have a better understanding, we need to look into some related terms


Why do Software haveDefects/ Bugs?
- What is affected ?
- Software Work-Product
- Who is the constructor/ Author?
- Programmer/ Software Engineer
- Who is the Programmer/ Software Engineer?
- A Human Being.

Software Defects/ Bugs?
- Some related terms
- Mistake : Every human being makes mistakes. Hence a mistake is introduced in the product/ process.
- Fault : The manifestation of the MISTAKE is called a FAULT.
- Error : ERROR is the result of a FAULT. i.e a fault will introduce an Error.
- Defect : The degree of deviation with which the ERROR has occurred is termed as a DEFECT.

Why do Software haveDefects/ Bugs?



- Miscommunication or no communication.
- The Software product is produced because of correct requirements.
- If the requirements are not communicated to the Development team and other related teams, it will lead to a major defect during the course of the project.
- It is eliminated by having proper requirement documents like URS, SRS.




Why do Software haveDefects/ Bugs?
- Software Complexity.
- Complexity of the software to be built is one of the most common reasons for the failure.
- Windows type interfaces, client-server and distributed applications, data communications, enormous relational databases, and the large size of the application have all contributed to the complexity
- Programming Errors
- Error introduced due to human errors.
- Changing Requirements
- The Customer may or may not be aware of the changes done during the course of the project.
- It means Redesigning, recoding, throwing out what has been done and do a new part.
- Change in requirements can be controlled by having effective baselining and Change control measures. (PCR, SCR)
- Time Pressures
- Scheduling of projects have to be done correctly.
- If done incorrectly and deadlines are approaching, then it is a run for the finish, thereby introducing more bugs.
- Software Development Tools
- In some cases the development tools such as Visual tools, class libraries, compilers, scripting tools etc. often introduce their own defects thus adding to the overall defects.

Software Quality Assurance

What is Software Quality Assurance?
Software Quality Assurance looks into PREVENTION

How?
By Defining Global Organization Quality Policies and Processes.
These policies and processes are followed by the entire organization.

Software Testing
What is Software Testing?
Software Testing looks into DETECTION.

How?
By following different types of testing guidelines and techniques.

Testing Policy
- A Testing Policy is management’s definition of testing a department. A testing policy involves the following four criteria.
- Definition of Testing : A clear, brief and unambiguous definition of testing,
- Testing System : The method through which testing will be achieved and enforced,
- Evaluation : How testing team will measure and evaluate testing,
- Standards : The standards against which testing will be measured.

Quality Policy
- A Quality Policy is again a management definition of providing customer satisfaction for the first time and ever time.
- Understanding the definitions of excellence and quality is important because it is the starting point of any management team contemplating the implementation of a Quality Policy.

Testing Cost
- Cost of Quality
- Cost of Quality is a term used to quantify the total cost of failure, appraisal, and prevention costs associated with the production of software.

Testing Cost Curve
- Continuously testing the software is also not recommended, if cost of testing becomes high.
SDLC
- System Development Life Cycle.
- System Development Life Cycle (SDLC) is the overall process of developing information systems through a multistep process from investigation of initial requirements through analysis, design, implementation and maintenance


SDLC
- There are many different models and methodologies, but each generally consists of a series of defined steps or stages.
- Once upon a time, software development consisted of a programmer writing code to solve a problem or automate a procedure.



SDLC
- Nowadays, systems are so big and complex that teams of architects, analysts, programmers, testers and users must work together to create the millions of lines of custom-written code that drive our enterprises.
- To manage this, a number of system development life cycle (SDLC) models have been created
- waterfall, fountain, spiral, build and fix, rapid prototyping, incremental, and synchronize and stabilize.
- The oldest of these, and the best known, is the waterfall: a sequence of stages in which the output of each stage becomes the input for the next.



Stages of Waterfall model
- Feasibility Study
- Requirements Collection
- System Analysis
- System Design
- Coding
- Integration and Testing
- Acceptance
- Installation and Deployment
- Maintenance

Waterfall model
- Feasibility Study – Establishes a high-level view of the intended Project and determines its goals.
- Requirements Collection – Necessary to collect first hand data from the client site, to serve as the input for the Project.
- System Analysis – delves into the operations of the intended application.
- System Design – describes desired features and operations in detail. Features include Screen Layouts, Business Rules, Process Diagrams, Pseudocode, and other documentation.
- Coding – the actual code is written in this phase.
- Integration and Testing – Get all pieces together and then check for defects and interoperability.
- Acceptance – where the Client conveys the Acceptance of the s/w.
- Installation and Deployment – The software is put into production and runs the business.
- Maintenance – Changes, Additions, higher version changes goes on and on.


Drawbacks of Waterfall model
- Waterfall model assumes that the only role for users is in specifying requirements.
- All requirements are to be specified in advance.
- Unfortunately requirements grow and change very often throughout the process.
- This required considerable feedback at every stage and iterative consultation.

Fountain Model
- Similar to Waterfall model
- No strict definitions as regards phase.
- Hence some phases could overlap.

Spiral model
- Emphasized the need to go back and revisit earlier stages number of times as the project progresses.
- The spiral model is a series of short waterfall cycles producing an early prototype.
- This prototype is a part of the entire project.
- Was discarded due to its chaotic approach.

Build and Fix
- Is the crudest method of software development.
- Build the code- Evaluate it by customer- Fix it and this is done until the customer is happy.

Rapid Prototyping
- Called Rapid Application Development
- A prototype is created which looks and functions like the desired product in order to test its usefulness.
- Prototype is a part of requirements determination.
- Created using tools different than that used for the final product.
- Once prototype is approved, the Actual software is written.

Incremental model
- Functions like Build and Fix, but requires customer evaluation at every stage.
- Hence more likely to find defects in the earlier stage itself.
- Has become an important model, since most development is now done on Web.

Synchronize and Stabilize
- Method used by Microsoft and Netscape to build IE and Netscape Communicator respectively.
- Allows many teams to work efficiently in parallel.
- How is it done?
- A build of the entire project is created, bringing together all the current components of the project.
- Alpha test is carried out for internal testing.
- One or more BETA releases for wider testing outside the organization.
- Finally a release leading to a GOLD MASTER.
- The Gold Master is then released for manufacturing.


Project Management
- Effective Management of a Software Project is very crucial to the organization.
- Projects are run successfully by a mix of training, mentoring, and management support.
- This will lead to improved performance and achievement.
- Various aspects are involved in the Project Management activity.
- Technical Aspect
- Human Aspect
- A Project’s success has to be considered on different criteria.
- To help in understanding project management, 4 levels have been identified.
- The 4 levels for judging the success of a project are
- Level 1 (Management of the work to be done)
- Level 2 (Management of the management of the project)
- Level 3 (Did the Project deliver the business benefits that were claimed)
- Level 4 (Project selection and portfolio management)


Level 1 – Management of the work to be done
- Includes the following
- Clear scope of Project, and identification of deliverable,
- Thorough and continuous risk assessment,
- Planning to an appropriate level of detail,
- Good estimating practices,
- Appropriate scheduling and use of resources,
- Cost and Schedule Tracking.


Level 2 – Management of the management of the Project
- The Organization management should take the responsibility to scrutinize and manage the management of the project.
- Special Groups are formed to handle this aspect. (SEPG, SQA)


SEPG (Software Engineering Process Group)
- To improve the effectiveness of Software Engineering processes in an organization.
- The role of SEPG are
- Establish process standards,
- Maintain process metric and quality criteria,
- Serve as focal point for technology insertion,
- Provide key process education,
- Provide project consultation, and
- Make periodic assessment and status reports.


SQA
- SQA (Software Quality Assurance) group are the eyes and ears of the management.
- They police the project management activity.
- The members of SQA also assist the PM in carrying out project management activities.
- Every activity within the project is scrutinized, recorded and escalated to the top management when things seem to be heading in the wrong way.


Level 3- Did the Project deliver?
- The most asked question
“Did the Project DELIVER”
- The delivery is for the identified workproducts to the client, before the onset of the project.
- Is the Client satisfied with the end-product.


Level 4 – Project selection and Portfolio management
- Have we taken the optimum mix of projects that when taken together will most effectively contribute toward the growth of the organization.
- Only take up those projects, which the organization is capable of handling and delivering.

Why Projects fail?
- Basic facts about Projects
- Projects are not all same.
- Projects are different, hence have a different degree of failure.
- Projects are of 2 types, viz Type 1 projects and Type 2 projects.


Type 1 projects
- These are concerned with the delivery of some artefact / article.(eg. A construction project for a building, bridge etc).
- There is consensus at the beginning about the desired output and outcome.
- There is a tangible end product and has an intrinsic value.
- Type 1 projects may be complex, but complexity tends to be detail in nature.

Type 2 projects
- These projects are designed to bring about some form of organizational or societal changes. (eg a new performance related pay system, Information systems development etc).
- These projects have multiple powerful stakeholders, who have different viewpoints on the project, hence the output may be ambiguous.
- The output has no intrinsic value. Value is only attached when the output is put to use (which often depend on time and space).
- Ownership or sponsorship of the project normally change hands very often.
- Change in ownership reflects in new interpretation of the purpose of the project.
- These projects depend on entities beyond the bounds of the organization, in which they are conceived.
- Hence failure is HIGH in these type of projects.


Some startling facts
- A survey conducted in 1991 indicated the following.
- More than half of the projects carried out were failures.
- A mere 16% of projects were delivered on time and within cost.


What are the reasons?
- The projects were found to be either over-budget or critically delayed,
- Poor sponsorship,
- Original project selection inappropriate,
- Wrong Project Manager assigned,
- Un-supportive Upper Management,
- Inadequate defined tasks and WBS (Work breakdown structure), and
- Misused management technique.


Root cause for project failures
- Over optimism,
- Trying to chew a big piece of cake in one go,
- Risk contingencies not being considered appropriately,



How to evaluate project performance?
- We need to ask ourselves some basic questions.
a) Did the project produce the desired output?
- By proper management of Risk and having a proper business case monitoring, failure can be minimized,
- Identify the deliverables,
- Plan to appropriate level of detail,
- Use good estimating practices,
- Appropriate schedule and use of resources, and
- Cost and Schedule tracking.


How to evaluate project performance?
b) Do projects consistently produce the desired output.
- Consistency can be achieved by having organizational structures, to share and support best practices.
- This will ensure predictability of the outcome.
- Project management should be supported by standards and a mechanism for learning through experience and sharing these experiences.
c) Did the project outputs produces the desired outcomes.
- We need to have an assessment done to judge how well the project has delivered the business benefits that was claimed at the onset of the project,
- Improvements come at this level only through a thorough understanding of benefits logic and from initiatives designed to assign individual responsibilities to realize the benefits.
How to evaluate project performance?
d) Do the produced outcomes have the intended impact on the Business Strategy.
- Have we taken the optimum mix of projects that will contribute towards the realization of the stated Business Strategy,
- Measurement at this level requires a programmatic approach to organizational initiatives and a balanced approach to organizational performance as a whole,


How to evaluate project performance?

Improvements to Project Management Activity
- Improvements to Project Management activity has come about by setting goals, setting standards and carrying out the tasks with these goals in view.
- The tasks are then recorded, analyzed and measured to indicate the success of the project, thus evaluating the performance of the Project Management activity.
Analyzing and Evaluating Performance
- Performance of individual projects and thus of the entire organization is analyzed and evaluated by implementing certain basic techniques and methods.
- Analysis is done based on the techniques of Reviewing and Testing, whereas the evaluation is done based on collected metrics from the individual projects.
- In other words, the proper implementation of a Quality Management system has contributed constructively towards the realization of goals.