CS4472 Readings and Questions for Fall 2015 Q2

  1. first quarter material that was covered on the first quiz.
  2. For Tuesday 29 Sept 2015, read Too Darned Big to Test, Keith Stobie, Microsoft, Queue, Volume 3 Issue 1, February 2005, Pages 30-37. [[p30-stobie.pdf]] and answer the following reaction questions:
    1. In comparing this article with the Google Innovation Factory article, what would you say was the main testing consideration that both papers agreed on? Indicate where in each paper this consideration is mentioned. If you can't find the answer to this question, indicate that you couldn't find it and discuss your general impressions about what is common to both papers.
    2. In comparing this article with the Google Innovation Factory article, what would you say was the main testing consideration that the two papers disagreed on? Indicate where in each paper this consideration is mentioned. If you can't find the answer to this question, indicate that you couldn't find it and discuss your general impressions about what is most constrasting with regards to both papers.
    And now we add Microsoft to the companies whose approaches we have looked at.
  3. For Thursday 1 Oct 2015, read: Effective use of assertions in C++, Mike A. Marin, ACM SIGPLAN Notices, Volume 31 Issue 11, Nov. 1996, Pages 28-32. and answer the following reaction questions:
    1. What is the difference in effectiveness of class invariants if your class has public fields rather than keeping all fields private? Why?
    2. In mutation testing, we can generate mutants that go into an infinite loop. If we could keep the mutation from messing up the assert code, what type of invariant would be guarenteed to abort the program if it was headed into an infinite loop? Why?
    3. In the factorial example on page 30 (see page numbers at bottom of page), why is the expression n-i being used as the second parameter to the LoopVariant macro rather than the expression i?
    [note: ## is the C preprocessor's concatenation operator, see wikipedia entry on C preprocessor.]

    If you have forgotten about loop invariants or they weren't covered in your CS logic course, there is a reasonable wikipedia entry on loop invariants. Wikipedia also has an entry on the less commonly referred to loop variant. Also, CS logic course don't usually talk about class invariants, but there is a wikipedia entry on that too.

    Basically, they are all assertions. It is just useful to be able to distinguish different kinds of assertions. For loops, you want to remember to check each time around the loop to make sure it is doing the right thing (loop invariant). Also it is useful to check that the loop is making orderly progress toward coming to an end (loop variant). And, it is useful to check to make sure that their different parts of an object are consistant with the definition of the object (for example at the start and end of various methods), the class invariant. The point of the paper is to show how you can set up special macros to handle this in C and C++ and how you go about using such macros.

    Since Java doesn't have macros, the implementation of these types of invariants (as opposed to just using a bunch of asserts), would involve a non-standard package called JML ( yes it too has a wikipedia entry). This turns out to be more complicated that one would like because one needs a version of JML that matches the version of Java you are using. While you can't use C's preprocessor with Java as it tries to parse the input program, it turns out you can use the m4 preprocessor with Java (by first running the source code through m4 and then taking the result and handing it to javac) -- so it might be possible to get a set of macros that can sort of work with Java along the lines of the C/C++ macros described in thsi article.

  4. For Tuesday 6 Oct 2015, read How Amazon web services uses formal methods, Chris Newcombe, Tim Rath, Fan Zhang, Bogdan Munteanu, Marc Brooker, and Michael Deardeuff, Communications of the ACM, Volume 58 Issue 4, April 2015, Pages 66-73, and answer the following reaction questions:
    1. Formal specifications were discussed in the IBM article Software Debugging, Testing, and Verification that we read for 22 Sept 2015. Describe the difference between the view on formal specifications that the two articles take. Indicate which paragraph you are referring to when you mention that one of the articles says something (actual short quotes are also good).
    2. The combinatorial testing paper we read for 17 Sept 2015 also discusses introducing new methods to a company. Describe the difference between the view on formal specifications that the two articles take. Indicate which paragraph you are referring to when you mention that one of the articles says something (actual short quotes are also good).
    By `the two articles', I mean this Amazon article and the other one mentioned in the question.
  5. For Thursday 8 Oct 2015, read When to Test Less, Menzies, T. and Cukic, Bojan, Software, IEEE (Volume:17 , Issue: 5 ), Page(s): 107 - 112, and answer the following questions:
    1. Under assumption 1 on page 109, it says that the meaning of yes and no edges will be explained later. Which `later' paragraph contains the explanation of yes edges? Which `later' paragraph contains the explanation of no edges?
    2. To see if we can do a calculation based on the discussion in this paper, answer the following set of short questions:
      • a) How many inputs does the NAYO network in Figure 2b have? What are their names?
      • b) How many nodes does the NAYO network in Figure 2b have?
      • c) What is the probability of finding the fault demonstrated in the explanation tree shown in Figure 2b? Show how you calculated this? Indicate where in the calculation you weren't sure and what reasonable assumptions you made.
    3. The authors are talking about how much testing is needed in certain situations. What testing strategy are they assuming? Where (indicate paragraph(s)) in the paper do they say this?
    4. The authors say that the results in the article are applicable to both unit testing and complete product testing. But they don't say which would be more applicable? Which kind of testing do you think this result is more applicable to? Why?
  6. For Tuesday 13 Oct 2015 (the day after local Thanksgiving), read Automated QA (Quality Assurance) Testing at EA (Electronic Arts): Driven by Events, Terry Coatta, Michael Donat, and Jafar Husain, Queue, Volume 12 Issue 5, May 2014, 10 pages, and answer the following questions
    1. In the second sentence of the fourth paragraph from the bottom of page 2, the quote ``significant ROI to be realized''. ROI is an acronym. What does it stand for (not in article, you will have to look it up elsewhere)? Where did you find this information (or did you already know it)?
    2. There is a wikipedia entry for Model View ViewModel as well as one for Model View Controller (from which Model View ViewModel was derived). Given these two wikipedia pages and the discussion in this article, why is MVVM better for testing than MVC? If you can, quote the sentence (or indicate the paragraph) where this is answered.
    3. The paper started out talking about MVVM. Then suddenly it started talking about Rx. How do MVVM and Rx connect? (refer as much as possible to what the paper says citing paragraphs or quoting sentences where relevant)
  7. For Thursday 15 Oct 2015, Towards an automatic evaluation of web applications, Leandro Guarino de Vasconcelos and Laercio Augusto Baldochi, Jr., SAC '12 Proceedings of the 27th Annual ACM Symposium on Applied Computing, Pages 709-716 and answer the following questions NOTE: While we might not have the tool described in the paper, in describing how the tool works, the authors are also describing how a human doing the same thing might go about it.