CS4472 151027 Interest Survey


By Tuesday Nov 3rd, we will have read XX papers on Specifications, Testing, and Quality Assurance. This has given us a broad overview of the field as well as a number of specific techniques (such as Combinatorial Testing, Mutation Testing, Random Testing, Equivalence Partitioning, Boundary Value Analysis and the use of Assertions). We have looked at the testing concerns of a number of large organizations including Google, IBM, Microsoft, Amazon, and Electronic Arts. We have also read some on testing in the areas of web applications and mobile applications. At this stage in the semester (post 3 Nov 2015), we have 10 more readings ahead and the question of what areas we should get deeper into. The purpose of this survey is to see what people are interested in pursuing in the course of the remaining class readings. For the purposes of this survey, figure that you have 10 readings to allocate across the topics. So, for any topic you can allocate any number from 0 (the default) to 10 (spend the rest of the semester on just this topic). In total, the non-negative integer values indicated by your survey should sum to no more than 10. If n people in the course hand in surveys, then a paper will be allocated for each multiple of n `votes'. The remaining votes on topics that were greater than 0 but less than n will be used for a weighted random draw (i.e., something with 2 unallocated votes is twice as likely to get picked as something with 1 unallocated vote). You may discuss among yourselves how you want to allocate your `votes' if you think that would be useful. Circle the first word in any title that looks of particular interest. The topics are:
  1. ____ Traditional quality assurance, covered by papers like: Software verification and validation: an overview (8 pages), Quantifying software validation: when to stop testing? (9 pages), Characterizing the software process: a maturity framework (7 pages), Using a defined and measured Personal Software Process (12 pages, looks at applying organizational principles to individual work habits), Engineering software under statistical quality control (10 pages), Software quality assurance-concepts and misconceptions (7 pages), Cleanroom process model (9 pages), The 10-Minute Test Plan (8 pages)
  2. ____ Code reviews, software inspection Software inspections: an effective verification process (6 pages), Expectations, outcomes, and challenges of modern code review (10 pages), A statistical approach to the inspection checklist formal synthesis and improvement (9 pages), Using dependence graphs to assist manual and automated object oriented software inspections (8 pages), Obstacles to Comprehension in Usage Based Reading (12 pages), Best practices in code inspection for safety-critical software (8 pages), Checklist Inspections and Modifications: Applying Bloom's Taxonomy to Categorise Developer Comprehension (6 pages)
  3. ____ Software reverse engineering, covered by papers like: Delocalized Plans and Program Comprehension (9 pages), Reverse engineering and design recovery: a taxonomy (5 pages), Structural redocumentation: a case study (9 pages), Theories, methods and tools in program comprehension: past, present and future (11 pages), Partial comprehension of complex programs (enough to perform maintenance) (8 pages)
  4. ____ Random testing, covered by papers like: When only random testing will do (9 pages), Adaptive random testing (10 pages), Adaptive Random Testing: The ART of test case diversity (7 pages) ARTOO: Adaptive Random Testing for Object-Oriented Software (10 pages),
  5. ____ Traditional testing techniques, covered by papers such as: The category-partition method for specifying and generating functional tests (11 pages), Selecting Software Test Data Using Data Flow Information (9 pages), A Data Flow Oriented Program Testing Strategy (8 pages), A data flow coverage testing tool for C (9 pages), The AETG system: an approach to testing based on combinatorial design (8 pages)
  6. ____ Unit testing, covered in papers like: Endo-Testing: Unit Testing with Mock Objects (9 pages) A Survey on Unit Testing Practices and Problems (11 pages), Unit test virtualization with VMVM (12 pages)
  7. ____ Regression testing (when a change is made, which tests should be run), covered by papers such as: Insights into regression testing (10 pages), Effectively prioritizing tests in development environment (10 pages), A study of effective regression testing in practice (11 pages)
  8. ____ Experiments evaluating testing techniques (problem definition, data collection, statistical analysis), covered by papers such as: Is mutation an appropriate tool for testing experiments? (10 pages), An Empirical Evaluation of Assertions as Oracles (10 pages), Coverage is not strongly correlated with test suite effectiveness (11 pages), Relationships between Test Suites, Faults, and Fault Detection in GUI Testing (10 pages), Studying the Characteristics of a "Good" GUI Test Suite, An Empirical Study of JUnit Test-Suite Reduction (10 pages), In Search of What We Experimentally Know about Unit Testing (9 pages), Further investigations into the development and evaluation of reading techniques for object-oriented code inspection (11 pages), Comparing the Effectiveness of Equivalence Partitioning, Branch Testing and Code Reading by Stepwise Abstraction Applied by Subjects (10 pages), What Do the Asserts in a Unit Test Tell Us about Code Quality? A Study on Open Source and Industrial Projects (10 pages), Have Agile Techniques been the Silver Bullet for Software Development at Microsoft? (10 pages), Pair programming: what's in it for me? (9 pages), How do professional developers comprehend software? (11 pages)
  9. ____ Performance testing (a bit less statistics than experiments, but similar), covered by papers like: Effectiveness of sampling based software profilers (5 pages) A prototype-driven approach to application-level performance testing: a case study of a large finance application (11 pages), A parallelism-based analytic approach to performance evaluation using application programs (10 pages), Assessing test-driven development at IBM (6 pages), The search for the laws of automatic random testing (6 pages)
  10. ____ Formal Methods overview, covered by such papers as Seven myths of formal methods (9 pages), Seven more myths of formal methods (8 pages), Ten commandments revisited: a ten-year perspective on the industrial application of formal methods (9 pages)
  11. ____ Assertions, covered by papers like Programming with assertions: a prospectus [software development] (7 pages), Design by Contract with JML (13 pages, special assertions system for Java programming called JML), Assertions in object oriented software maintenance: analysis and case study (10 pages), Assertion-Driven Development: Assessing the Quality of Contracts Using Meta-Mutations (10 pages), Assert use in GitHub projects (12 pages)
  12. ____ Meyer's Contract Driven Programming (uses his Eiffel language for examples), covered by papers like: What good are strong specifications? (10 pages), Applying `design by contract' (12 pages), Contract driven development = test driven development - writing test cases (10 pages) Programs That Test Themselves (10 pages), Uncovering hidden contracts: the .NET example (8 pages), Automated Fixing of Programs with Contracts (13 pages)
  13. ____ Program correctness proofs (verifying that a program meets a formal specification using logic), covered by papers such as: Programming pearls: Writing correct programs (6 pages), Program Specification Applied to a Text Formatter (9 pages), Specification techniques for data abstractions (13 pages)
  14. ____ Automatic test case generation involves constraint solving (like linear algebra with inequalities), covered in papers like: Function Substitution: Towards Constraint Solving in Software Testing (10 pages), Automated software test data generation (10 pages)
  15. ____ Testability, covered in papers like: Predicting class testability using object-oriented metrics (10 pages), Testability, fault size and the domain-to-range ratio: An eternal triangle (5 pages), Testability transformation (14 pages)
  16. ____ GUI testing technology, covered by papers like: GUI ripping: reverse engineering of graphical user interfaces for testing (10 pages), Black-Box Test Data Generation for GUI Testing (6 pages), Finite state testing and analysis of graphical user interfaces (10 pages)
  17. ____ Mobile application testing, covered by papers like: On the Use of Software Quality Standard ISO/IEC9126 in Mobile Environments (8 pages), Why application errors drain battery easily?: a study of memory leaks in smartphone apps (5 pages), Reverse Engineering iOS Mobile Applications (10 pages), Performance Testing of Mobile Applications at the Unit Test Level (10 pages), Automated Generation of Oracles for Testing User-Interaction Features of Mobile Apps (10 pages), Robot-assisted smartphone performance testing (6 pages)
  18. ____ Web applications, covered by papers like: Evaluating the reverse engineering capabilities of Web tools for understanding site content and structure: a case study (10 pages), Testing web applications with state objects (8 pages), Automated cross-browser compatibility testing (10 pages)
  19. ____ Cloud testing, covered by papers like: Framework for testing cloud platforms and infrastructures (7 pages, GoogleApp engine), Modeling and testing of cloud applications (8 pages), A Survey of Software Testing in the Cloud (6 pages),
  20. ____ Autonomic self-testing, covered by papers like: The vision of autonomic computing (10 pages) Migrating Autonomic Self-Testing to the Cloud (6 pages), A Comparative Case Study on the Engineering of Self-Testable Autonomic Software (10 pages)
  21. ____ Test driven development, covered by papers like: Test-driven development concepts, taxonomy, and future direction (8 pages), Mock roles, objects (11 pages), On the Influence of Test-Driven Development on Software Design (8 pages), Iterative and incremental developments. a brief history (10 pages), Combining Formal Specifications with Test Driven Development (12 pages)
  22. ____ Pair programming, covered in papers like: All I really need to know about pair programming I learned in kindergarten (7 pages), Micro-coordination: because we did not already learn everything we need to know about working with others in kindergarten (10 pages), Pair programming and the mysterious role of the navigator (11 pages), Strengthening the case for pair programming (7 pages), The Social Nature of Agile Teams (11 pages), Agile and Wellbeing -- Stress, Empowerment, and Performance in Scrum and Kanban Teams (10 pages)
  23. ____ Exploratory testing, covered by papers like: Exploratory Testing Explained (10 pages), Guidance for exploratory testing through problem frames (5 pages), Test Better by Exploring: Harnessing Human Skills and Knowledge (12 pages)