Improving Crypto-Detectors: Inside the MASC Mutation Testing Framework

Table of Links

Abstract and 1 Introduction

2 Overview of MASC

3 Design Goals

4 Implementation of MASC

4.1 Mutation Operators

4.2 Mutation Scopes

5 Using MASC

6 Future Work and Conclusion, Acknowledgments, and References

2 OVERVIEW OF MASC

Overall, MASC works by (1) mutating a base crypto API misuse case to create mutated crypto-API instantiations or mutated misuse case, (2) seeding or injecting the mutated misuse case in source code, (3) analyzing both unmutated and mutated source code using a target crypto-detector, and (4) comparing the outputs of cryptodetector applied on both base misuse case and mutated misuse case to identify undetected (not killed) mutated misuse case. The overview of this process is shown in Figure 1.

Conceptually, MASC contextualizes the traditional mutation testing techniques of SE domain for the evaluation of crypto-detectors, while introducing crypto-API misuse mutation operators that instantiates variants or expressions of crypto-API misuse. To elaborate, while mutation operators from the traditional, SE mutation testing are used to describe operations that either add, modify, or remove existing source code statement(s), in the context of MASC, cryptoAPI mutation operators create expressive instances of crypto-API misuse independent of any source code or application. As shown in Listing 1, statement marked //1 is the base misuse case, whereas statements //2 – //5 are the mutated crypto-misuse cases instantiated by several mutation operators of MASC. We provide the design considerations and implementation details of MASC’s mutation operators in Sec. 4.1. These mutated misuse instances are then "injected" or "seeded" in source code, where the injection site depends on the mutation scopes of MASC, which we detail in Sec. 4.2.