Methods and tools for effective and efficient evaluation of source code quality based on operational quality models.
Motivation
Source code is a central artifact of a software project. There exist several techniques to find quality-related problems in source code, e.g., code reviews, inspections, or static code analysis. On the other hand, quality models like ISO 25010 try to structure the term quality to make it more tangible. These quality models typically do a good job in describing properties of code quality but lack to provide sufficient operationalization, i.e., how to measure the quality properties and how to aggregate individual findings to an overall code quality rating. There exist some general methods like ISO 25040 that provide guidance how to performe code analysis; nevertheless these approaches lack a level of detail that allows to carry out software quality evaluations in an effective way.
Goals
- An operational quality model to measure the code quality of Java and C++ source code.
- Invention of methods to effectively carry out code quality evaluations
- Software tools (Eclipse-based) that facilite the development of operational quality models and that support source code quality evaluations.
- Development of a benchmarking-based methods and tools that allow the evaluation of the source code quality in relation to other projects.
Results
- Operational quality model for ISO 25010 for Java and C++ containing more than 2500 measures. The measures were not developped in this project, but measures from various static code analysis tools (e.g., PC-Lint, PMD, Findbugs, Vera++) were integrated in the quality model. Besides the ISO 25010 view on quality, the quality model also provides a technical classification of all measures.
- EMISQ (Evaluation Method for Internal Software Quality) – a method that allows to effectively carry out source code quality evaluations. The method heavily relies on expert knowledge for deriving quality evaluations for a project.
- A benchmarking method and tool support for automatic evaluation of the source code quality of a project in relation to the quality of other projects contained in a reference benchmarking base.
- CQMM (Continuous Quality Monitoring Method) – a method to be used during project execution to properly evaluate the source code quality based on our benchmarking approach.
- SPQR (Software Product Quality Reporter) – an Eclipse-based tool that supports the EMISQ and CQMM methods to a large extent.
- Operational quality model that allows to judge the compliace of source code with the requirements imposed by the safety standard ISO 61508 (of course only for the source code related aspects of the standard). The evaluation method provides evaluations for the different safety integrity levels (SIL).
Partner: Siemens Corporate Technology
Contact: Reinhold Plösch
Code Quality Management (CQM)
Publications
R. Plösch, C. Neumüller: Does Static Analysis Help Software Engineering Students?, 9th International Conference on Educational and Information Technology (ICEIT 2020), Oxford, United Kingdom, February 11-13, 2020. Doi: 10.1145/3383923.3383957
Conference Article
R. Plösch: Software Quality Benchmarking, Published as Technical Report (Siemens Corporta Technology - CCT Software Systems and Processes), October 2018, Munich. The report is not available to the public.
Technical Report
R. Plösch: Design Quality Measurment, Published as Technical Report (Siemens Corporta Technology - CCT Software Systems and Processes), October 2018, Munich. The report is not available to the public.
Technical Report
T. Atzenhofer, R. Plösch: Automatically Adding Missing Libraries to Java Projects to Foster Better Results from Static Analysis, 17th IEEE International Working Conference on Source Code Analysis and Manipulation (SCAM 2017), Shanghai, China, September 17-18, 2017 Doi: 10.1109/SCAM.2017.10
Conference Article
J. Bräuer, R. Plösch, M. Windhager: Are CISQ Reliability Measures Practical? A Research Perspective, In Proceedings of the 10th IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW 2017), Tokyo, Japan, March 13-17, 2017, (Best Paper Award), doi:10.1109/ICSTW.2017.7.
Workshop Article
J. Bräuer, R. Plösch, M. Saft: Measuring Maintainability of OO-Software - Validating the IT-CISQ Quality Model, Proceedings of the 2015 Federated Conference on Software Development and Object Technologies, Zilina, Slovakia, November 19-20, 2015 doi:10.1007/978-3-319-46535-7_22.
Conference Article
J. Bräuer: Measuring Object-Oriented Design Principles, Proceedings of the Doctoral Symposium at the 30th ACM/IEEE International Conference on Automated Software Engineering (ASE 2015), Lincoln, Nebraska, USA, November 9-13, 2015, doi:10.1109/ASE.2015.17.
Conference Article
R. Plösch, S. Schürz, Ch. Körner: On the Validity of the IT-CISQ Quality Model for Automatic Measurement of Maintainability, Proceedings of the 39th Annual International Computers, Software & Applications Conference (COMPSAC 2015), IEEE Computer Society, Taichung, Taiwan, July 1-5, 2015, doi:10.1109/SEAA.2017.14
Conference Article
S. Wagner, A. Goeb, L. Heinemann, M. Kläs, C. Lampasona, K. Lochmann, A. Mayr, R. Plösch, A. Seidl, J. Streit, A. Trendowicz: Operationalised Product Quality Models and Assessment: The Quamoco Approach, Journal of Information and Software Technology (IST), Elsevier, Volume 62, pp 101-123, 2015, doi:10.1016/j.infsof.2015.02.009.
Journal Article
R. Plösch, A. Dautovic, M. Saft: The Value of Software Documentation Quality, 14th International Conference on Quality Software, Dallas, USA, October 2-3, 2014, IEEE Computer Society Press 2014, doi:10.1109/QSIC.2014.22
Conference Article
A. Mayr, R. Plösch, C. Körner: A Benchmarking-based Model for Technical Debt Calculation, 14th International Conference on Quality Software, Dallas, USA, October 2-3, 2014, IEEE Computer Society Press 2014, doi:10.1109/QSIC.2014.35
Conference Article
A. Mayr, R. Plösch, M. Saft: Objective Safety Compliance Checks for Source Code, Companion Proceedings of 36th International Conference on Software Engineering (ICSE 2014), Hyderabad, India, May 31 - June 7, 2014, doi:10.1145/2591062.2591178
Conference Article
A. Mayr, R. Plösch, M. Saft: Objective Measurement of Safety in the Context of IEC 61508-3, 39th Euromicro Conference on Software Engineering and Advanced Applications (SEAA 2013), Santander, Spain, September 4-6, 2013, IEEE Conference Publishing Services, 2013, doi:10.1109/SEAA.2013.32
Conference Article
A. Mayr, R. Plösch, M. Kläs, C. Lampasona, M. Saft: A Comprehensive Code-based Quality Model for Embedded Systems, Proceedings of the 23rd IEEE International Symposium on Software Reliability Engineering (ISSRE 2012), Dallas, USA, November 27-30, 2012, doi:10.1109/ISSRE.2012.4
Conference Article
H. Gruber, R. Plösch, S. Schiffer, A. Hentschel: Calculating Software Maintenance Risk - A Practical Approach, Proceedings of 11th IASTED International Conference on Software Engineering (IASTED SE 2012), June 18-20, Cretan, Greece, 2012, Fraunhofer IESE Report, no. 035.12/E, Kaiserslautern, Germany July, 2012, doi:10.2316/P.2012.780-005
Conference Article
S. Wagner, K. Lochmann, L. Heinemann, M. Kläs, A. Trendowicz, R. Plösch, A. Seidl, A. Göb, J. Streit: The Quamoco Product Quality Modelling and Assessment Approach, Proceedings of 34th International Conference on Software Engineering (ICSE), Zürich, June 2-9, 2012, doi:10.1109/ICSE.2012.6227106
Conference Article
A. Mayr, R. Plösch, M. Saft: Towards an Operational Safety Standard for Software, Proceedings of the 18th Annual IEEE International Conference and Workshops on the Engineering of Computer Based Systems (ECBS), Las Vegas, USA, April 27-29, 2011, doi:10.1109/ECBS.2011.8
Conference Article
H. Gruber, R. Plösch, M. Saft: On the Validity of Benchmarking for Evaluating Code Quality, Proceedings of the joined International Conferences on Software Measurement IWSM/MetriKon/Mensura 2010, Stuttgart, Germany, November 10-12, Shaker Verlag, Aachen, 2010, doi:10.13140/RG.2.1.1893.0961
Conference Article
R. Plösch, H. Gruber, C. Körner, M. Saft: A Method for Continuous Code Quality Management Using Static Analysis, Proceedings of 7th International Conference on the Quality of Information and Communications Technology (QUATIC), Porto, Portugal, September 29 - October 2, 2010, IEEE Computer Society CPS 2010, doi:10.1109/QUATIC.2010.68
Conference Article
Harald Gruber: Benchmarking-oriented Assessment of Source Code Quality - an Approach for Automatic Assessment using Static Code Analysis Tools, Doctoral thesis, Department of Business Informatics - Software Engineering, Johannes Kepler University Linz, March 2010.
PhD Thesis
R. Plösch, H. Gruber, C. Körner, G. Pomberger, S. Schiffer: Adapting Quality Models for Assessments - Concepts and Tool Support, SQMB 2010 Workshop, held in conjunction with SE 2010 conference, Paderborn, Germany, February 22nd 2010, Technical Report TUM-I1001 of the Technical University Munich, February 2010.
Workshop Article
R. Plösch, A. Mayr, G. Pomberger, M. Saft: An Approach for a Method and a Tool Supporting the Evaluation of the Quality of Static Code Analysis Tools, Proceedings of SQMB 2009 Workshop, held in conjunction with SE 2009 conference, March 3rd 2009, Kaiserslautern, Germany, published as Technical Report TUM-I0917 of the Technical University of Munich, July 2009.
Workshop Article
H. Gruber, A. Hentschel, R. Plösch: Experiment for Comparing the Automatically Assessed Source Code Quality with Experts' Opinions, Proceedings of SQMB 2009 Workshop, held in conjunction with SE 2009 conference, March 3rd 2009, Kaiserslautern, Germany, published as Technical Report TUM-I0917 of the Technical University of Munich, July 2009.
Workshop Article
H. Gruber, A. Hentschel, R. Plösch: Experiment for Comparing the Automatically Assessed Source Code Quality with Experts' Opinions, Proceedings of SQMB 2009 Workshop, held in conjunction with SE 2009 conference, March 3rd 2009, Kaiserslautern, Germany, published as Technical Report TUM-I0917 of the Technical University of Munich, July 2009.
Workshop Article
R. Plösch, H. Gruber, C. Körner, G. Pomberger, S. Schiffer: A Proposal for a Quality Model Based on a Technical Topic Classification, Proceedings of SQMB 2009 Workshop, held in conjunction with SE 2009 conference, March 3rd 2009, Kaiserslautern, Germany, published as Technical Report TUM-I0917 of the Technical University of Munich, July 2009.
Workshop Article
R. Plösch, H. Gruber, A. Hentschel, G. Pomberger, S. Schiffer: On the Relation between External Software Quality and Static Code Analysis, Proceedings of the 32nd Annual IEEE Software Engineering Workshop (SEW-32), Porto Sani Resort, Kassandra, Greece, October 15-16, IEEE Computer Society Press, pp 169-174, 2008, doi:10.1109/SEW.2008.17
Workshop Article
R. Plösch, H. Gruber, G. Pomberger, S. Schiffer, C. Körner: Tool Support for a Method to Evaluate Internal Software Product Quality by Static Code Analysis, Software Quality Professional Journal, American Society for Quality, Volume 10, Issue 4, Milwaukee, USA, September 2008.
Journal Article
R. Plösch, H. Gruber, G. Pomberger, M. Saft, S. Schiffer: Tool Support for Expert-Centred Code Assessments, Proceedings of the IEEE International Conference on Software Testing, Verification, and Validation (ICST 2008), April 9-11, 2008, Lillehammer, Norwegen, IEEE Computer Society Press, 2008, doi:10.1109/ICST.2008.43
Conference Article
R. Plösch, H. Gruber, A. Hentschel, Ch. Körner, G. Pomberger, S. Schiffer, M. Saft, S. Storck: The EMISQ Method and its Tool Support - Expert Based Evaluation of Internal Software Quality, Journal of Innovations in Systems and Software Engineering, Springer London, Volume 4 (1), March 2008, doi:10.1007/s11334-007-0039-7
Journal Article
H. Gruber, Ch. Körner, R. Plösch, G. Pomberger, S. Schiffer: Benchmarking-oriented Analysis of Source Code Quality - Experiences with the QBench Approach, Proceedings of the IASTED International Conference on Software Engineering (IASTED SE 2008), February 12-14, Innsbruck, Austria, IASTED, 2008.
Conference Article
R. Plösch, H. Gruber, A. Hentschel, Ch. Körner, G. Pomberger, S. Schiffer, M. Saft, S. Storck: The EMISQ Method - Expert Based Evaluation of Internal Software Quality, Proceedings of 3rd IEEE Systems and Software Week, March 3-8, 2007, Baltimore, IEEE Computer Society Press, 2007, doi:10.1109/SEW.2007.71
Conference Article
H. Gruber, Ch. Körner, R. Plösch, S. Schiffer: Tool Support for ISO 14598 based Code Quality Assessments, Proceedings of the 6th International Conference on the Quality of Information and Communications Technology, QUATIC, Lisbon, Portugal, September 12-14, 2007, IEEE Computer Society, 2007, doi:10.1109/QUATIC.2007.11
Conference Article