13th TAROT Summer School on Software Testing, Verification & Validation

    School Program

     

    School Program
       

    8.00 - 8.50

    8.50 - 9.00

    • Registration & Welcome
    • Welcome - Tarot General Chairs

    Monday,

    June 26th

      
     Session 1

    9.00 - 10.30

    10.30 - 10.50

    10.50 - 12.20

    • Intelligent Test Optimization -  Amaud Gotlieb
    • Coffee Break
    • Inject, Mutate, Improve, Slice - Jens Krinke.
      12.20 - 13.30  Lunch
     Session 2

    13.30 - 15.00

    15.00 - 15.30

    15.30 - 17.00

    • Advanced Coverage Criteria: Specification and Support in Automatic Testing Tools - Nikolai Kosmatov.
    • Coffee Break
    • Beyond Testing! - Franz Wotawa.

    Tuesday,

    June 27th

      
     Session 3

    9.00 - 10.30

    10.30 - 10.50

    10.50 - 12.20

     
    • Risk analysis - Sergey Zelenov
    • Coffee Break
    • Security SLAs in the Cloud -  Massimiliano Rak.
      12.20 - 13.30  Lunch
      Session 4  

    13.30 - 15.00

    15.00 - 15.30

    15.30 - 17.00

    • Mutation Testing Advances -  Michail Papadakis
    • Coffee Break
    • Model-based Validation, Verification and Analysis Approaches: Applications to Railways - Valeria Vittorini

    Wednesday,

    June 28th

      
    Session 5

    9.00 - 10.30

    10.30 - 10.50

    10.50 - 12.20

     
    • Challenges and opportunities for software testing in the cloud - Antonia Bertolino
    • Coffee Break
    • Automated Testing at the Graphical User Interface Level - Looking for
      the best TEST* -
      Tanja Vos
      12.20 - 13.30  Lunch
     Session 6  14.00 - 18.30  Tour in Herculaneum Archeological site

    Thursday,

    June 29th

      
    Session 7

    9.00 - 9.45

    9.45 - 10.30

    10.30 - 11.00

    11.00 - 11.45

    11.45 - 12.30

     

    Industrial and Research Project Presentations:

    • MUSA presentation & demo - Erkuden Rios.
    • ANASTASIA presentation & demo - Diego Rivera
    • Coffee Break
    • KONFIDO presentation - Luigi Coppolino
    • COMPACT presentation - Salvatore D'Antonio
      12.30 - 13.30  Lunch
     Session 8  14.00 - 17.30

     Students’ presentations:

    Coverage criteria for combination of static analysis and dynamic analysis -
    Viet Hoang LE

    Interactive Runtime Verification -
    Raphaël Jakse

    Search Based Path and Input Data Generation for Web Application Testing -
    Matteo Biagiola

    Automated Scenario-based Testing of Distributed and Heterogeneous Systems -
    Bruno Lima

    Development of a Cyber Range Platform for IoT and ICS domains - Luigi Scaglione

    Improving Traceability Management through Tool Integration: a case study for Software Testing Processes - Vincenzo De Simone

    A novel online functional testing methodology based on a fully distributed continuous monitoring approach applied to communicating systems - Jose Alfredo Alvarez Aldana

    A learning metric suggester tool for an automated accurate measurement process - Sarah Dahab

    Continuous integration in testing for aviation industry - LHS

       20.30 Social Dinner - Ristorante La Bersagliera

    Friday,

    June 30th

      
    Session 9

    9.00 - 10.30

    10.30 - 11.00

    11.00 - 12.30

     
    • Techniques and Tools for Mobile Testing Automation - Porfirio Tramontana
    • Coffee Break
    • Testing of Embedded Systems: Contemporary Aspects and Research Prospects - Daniel Sundmark
      12.30 - 13.30  Lunch
     Session 10

    13.30 - 14.30

    14.30 - 15.00

    15.00 - 15.45

    15.45 - 16.30


     
    • Runtime verification and enforcement - Yliès Falcone
    • Coffee Break
    • COSSMIC presentation & demo - Salvatore Venticinque
    • MMT - Montimage Monitoring Tool - Wissam Mallouli

    School Closure

     

     

     

     

     

     

     

     

     

     

     

     

     

     

     

     

     

     

     

     

     

     

     

     

     

     

     

     

     

     

     

     

     

     

     

     

     

     

     

     

     

     

     

     

     

     

     

     

     

     

    Speakers:

    Arnaud Gotlieb, SIMULA Research Laboratory, Norway

    Intelligent Test Optimization

    Abstract

    Test optimization is crucial when program testing is embedded into a continuous integration process. This seminar will review the various aspects of test optimization which includes at least test selection, reduction, prioritization and scheduling. It will also detail some advanced Artificial Intelligence techniques that have adressed these challenges, in particular Constraint-Based Testing.

    Bio

    Arnaud Gotlieb is a French research scientist in Computer Science, currently leader of the Certus centre which is the Norwegian research-based innovation centre on Software Verification & Validation hosted at Simula Research Laboratory. Dr. Gotlieb's expertise is on the application of Artificial Intelligence to Software Testing Challenges. He completed his PhD on automatic test data generation using constraint logic programming techniques in 2000 at the University of Nice-Sophia Antipolis and got habilitated (HDR) in Dec. 2011 from University of Rennes, France.
    Arnaud Gotlieb has co-authored more than eighty publications in international conferences and journals and he is the main architect of several constraint-based testing tools. He has served in the program committees of the IEEE Int. Conf. on Software Testing, Validation and Verification (ICST) from 2008 to 2015 and many other conferences such as ISSRE, ICSE-SEIP, TAP, QSIC and more recently CP and IJCAI. He has co-chaired the technical program of QSIC-13, the SEIP track of ICSE-14, the Testing and Verification track of CP-16 and CP-17. He has initiated the Constraints in Software Testing, Verification and Analysis (CSTVA) workshop series and was the main organizer during the first editions. He has successfully co-supervised ten PhD students and currently three in the domain of intelligent testing for robotic systems.

     

    Nikolai Kosmatov, CEA List

    Advanced Coverage Criteria: Specification and Support in Automatic Testing Tools.

    Abstract

    Automatic test data generation (ATG) is a major topic in software engineering. In this talk, we will try to bridge the gap between the coverage criteria supported by state-of-the-art white-box ATG technologies, especially Dynamic Symbolic Execution, and advanced coverage criteria found in the literature. First, we define a new testing criterion, called label coverage, and prove it to be both expressive and amenable to efficient automation. We propose several optimization techniques resulting in an effective black-box support for label coverage in ATG tools. Second, we show how a combination of static analysis techniques can be efficiently applied to detect infeasible test objectives. Finally, we present a new specification language, called HTOL, capable to further extend the expressive power of labels and encode even most advanced test coverage criteria (such as MCDC, dataflow criteria, non-interference properties, etc.).

    Bio

    Nikolai Kosmatov (http://nikolai.kosmatov.free.fr/) got a PhD in Mathematics in 2001 jointly from Saint-Petersburg State University and University of Besançon. Since 2006, he works as researcher in Computer Science at CEA List. Nikolai's research interests include software testing, constraint solving and combinations of various software verification techniques. Nikolai gives theoretical courses and exercise sessions on software verification and testing since 2009 at different universities.  He contributes to various verification tools and is the main author of the online testing service pathcrawler-online.com. Nikolai co-organized several tutorials at various international events, e.g. TAP, TAROT, ASE, QSIC, iFM, SAC, RV.

     

    Michail Papadakis

    Mutation Testing Advances, University of Luxembourg

    Abstract

    Much research on software testing and test techniques relies on experimental studies based on mutation testing. Mutation testing uses artificial faults to simulate the behavior of real faults. However, there is little evidence supporting this practice and little knowledge regarding its sensitivity to potential validity threats. Recent research has shown that mutation-based studies vulnerable to potential validity threats, leading to possible Type I errors; incorrectly rejecting the Null Hypothesis. In this talk, I will survey the recent advances in the area of mutation testing and will discuss the recent empirical studies demonstrating the advantages and potential threats to validity of mutation-based testing studies.

    Bio

    Dr. Papadakis is a research scientist at the Interdisciplinary Center for Security, Reliability and Trust (SnT) at the University of Luxembourg. He received a Ph.D. diploma in Computer Science from the Athens University of Economics and Business. His research interests include software testing, static analysis, prediction modelling, mutation analysis and search-based software engineering. 

     

    Franz Wotawa, Graz University of Technology   

    Beyond Testing!

    Abstract

    Test automation has become more and more in practical use in industry. This holds not only for test execution but also for test suite generation. Although, there are still stumbling blocks around preventing testing from being completely automated, which require additional research work, there is the question what happens next, when dealing with automation in software engineering? One answer to this question is the automation of fault localization and program repair, both having a high potential for getting rid of intricate, time consuming, but necessary activities to be still carried out manually. In my talk I will introduce several methods for fault localization that have being developed over the past decades, e.g., spectrum-based fault localization and model-based debugging. In addition, I will discuss the potential of these methods to be applied in practice, and outline future research directions of automated debugging.

    Bio

    Franz Wotawa received a M.Sc. in Computer Science (1994) and a PhD in 1996 both from the Vienna University of Technology. He is currently professor of software engineering at the Graz University of Technology. Since the founding of the Institute for Software Technology in 2003 to the year 2009 Franz Wotawa had been the head of the institute. His research interests include model-based and qualitative reasoning, theorem proving, mobile robots, verification and validation, and software testing and debugging. Beside theoretical foundations he has always been interested in closing the gap between research and practice. For this purposes he founded Softnet Austria in 2006, which is a non-profit organization carrying out applied research projects together with companies. During his career Franz Wotawa has written more than 310 papers for journals, books, conferences, and workshops. He has supervised 79 master and 30 PhD students. For his work on diagnosis he received the Lifetime Achievement Award of the Intl. Diagnosis Community in 2016. Franz Wotawa has been member of numerous program committees and has organized several conferences, workshops and special issues of journals. He is a member of the Academia Europaea, the IEEE Computer Society, ACM, the Austrian Computer Society (OCG), and the Austrian Society for Artificial Intelligence and a Senior Member of the AAAI.

     

    Antonia Bertolino, ISTI-CNR, Italy

    Challenges and opportunities for software testing in the cloud

    Abstract
    Nowadays, cloud computing gained significant attention as an emerging paradigm for developing and delivering computing applications and services. Cloud computing impacts all software development phases and opens new issues and challenges in software testing bringing the  opportunity to offer testing as a service (TaaS).
    This talk will provide an introductory overview of the state-of-the-art on testing in the cloud, trying to evaluate  and compare the most innovative testing methods and tools. We will explore basic concepts of cloud-based testing, covering its main objectives, requirements and benefits and will investigate differences between conventional software testing and cloud-based software testing. In  addition we will analyze the current practice and market trends on cloud-based testing and hint at questions and opportunities for research inside software testing community.


    Bio
    Antonia Bertolino is a research director of the Italian National Research Council (CNR), in Pisa. Her research covers software and services engineering, with particular interest in testing approaches. On these topics she has worked in several national and European projects, including the recent FP7 projects Learn PAd, CHOReOS, CONNECT and NESSOS, and the just started H2020 ElasTest that will develop an elastic platform for testing in the cloud of complex distributed software systems. Currently she serves as the area editor for Software Testing for the Elsevier Journal of Systems and Software, and as an associate editor of ACM Transactions on Software Engineering and Methodology, and of Springer Empirical Software Engineering. She serves regularly in the Program Committee of the most renowned conferences in the field of Software Engineering, such as ESEC-FSE and ICSE, and in software testing and analysis, as ISSTA and ICST. She has been the General Chair of the ACM/IEEE Conference ICSE 2015, May 2015, Florence (Italy). She has (co)authored over 150 papers in international journals and conferences.

     

    ​Daniel Sundmark​, Malardalens University, Sweden

    Testing of Embedded Systems: Contemporary Aspects and Research Prospects

    Abstract

    Engineering of embedded systems is a discipline in rapid change. In many sectors dealing with embedded systems (e.g., the vehicular industry, power control systems, or industrial automation), software growth has been immense in the past few decades. The resulting complexity has made testing of such systems a challenging task. This situation is further complicated by issues like limited system observability or controllability, and the potential severity of system failure. This talk covers the state of the practice in embedded system testing including contemporary and foreseeable challenges, as well as current and future research prospects addressing these challenges.

    Bio

    Daniel Sundmark received a MSc degree in Information Technology from Uppsala University (Sweden) in 2003 and a PhD degree from Mälardalen University (Sweden) in 2008. Since then, he has undertaken industrial research visits at ABB Corporate Research, Scania CV and Volvo Construction Equipment. He has also spent four years as a senior researcher at the Swedish Institute of Computer Science. Since December 2015, Daniel serves a Professor at Mälardalen University in Sweden, where he leads the Software Testing Laboratory research group. Daniel's research focuses on engineering of embedded software and systems, particularly targeting system architecture and software and system testing.

     

    Massimiliano Rak, University of Campania Luigi Vanvitelli

    Security SLAs in the Cloud

    Abstract

    Can security be provided as-a-Service? Is it possible to guarantee a security service by a proper Service Level Agreement? This talk tries to reply to these questions by discussing open issues and available techniques to quantify security. In particular, the state of the art on security metrics and security evaluation techniques will be presented as foundation to move from standard control frameworks and security controls towards the formal definition of Security Service Level Agreements that need to be negotiated, enforced and continuously monitored.

    Bio

    Prof. Massimiliano Rak is an Associate professor at University of Campania Luigi Vanvitelli (SUN). He got his degree in Computer Science Engineering at the University of Napoli Federico II in 1999. In November 2002 he got a PhD in Electronic Engineering at Second University of Naples. His scientific activity is mainly focused on the analysis and design of High Performance System Architectures and on methodologies and techniques for Distributed Software development, Security Evaluation and Enforcement in Distributed Systems, Cloud Computing.  He coordinated the FP7 SPECS project and led the CERICT operational unit in the MUSA H2020 project. Moreover, he acted as principal investigator of several regional and national projects.

     

    Porfirio Tramontana, University of Napoli Federico II

    Techniques and Tools for Mobile Testing Automation

    Abstract

    Development and diffusion of Mobile applications go forward at a tremendous rhythm, due to the always increasing impact of smartphones and other mobile devices on people’s habitudes but many applications are uninstalled and discarded by users when they experience the presence of bugs. Functional testing represents a crucial activity in the context of mobile applications, and there is a large request in both industry and scientific community for mobile testing methodologies, techniques and tools. In particular, since these activities are usually repetitive, expensive and time consuming, there is a remarkable request for automated techniques and tools supporting them.

    This talk will provide a view of the state-of-the-art on techniques and tools supporting the automation of functional testing of mobile applications, including model based, model learning, search based, user session based and random testing techniques.

    Bio

    Porfirio Tramontana is an Assistant Professor at the University of Naples Federico II since 2006. His research interest are focused on Software Engineering applied to Mobile and Web applications. His research fields include reverse engineering, testing, maintenance, comprehension, migration of legacy systems, software quality. He teaches Testing Automation both in the context of the Software Engineering courses and in the context of the PhD course in Information Technology and Electrical Engineering at the University of Naples Federico II.

     

    Sergey Zelenov, ISPRAS - Russia

    Modeling and Risk Analysis of Hardware/Software Systems

    Abstract

    Modern safety-critical systems grow very extremely. Thus, in order to design and analyze such a system, one have to use automated tools. Nowadays the most useful practice is model-based design and early verification and validation.
    We present methods and tools for design and automated analysis of safety-critical systems. Formal model of an architecture of a system under design is specified using Architecture Analysis & Design Language (AADL). In order to describe safety-related requirements, an AADL model is augmented with error specification in AADL Error Model Annex notation. We focus on main techniques of risk analysis, such as Functional Hazard Assessment, System Safety Assessment, Fault Tree Analysis, Failure Modes and Effects Analysis.​

    Bio

     

    Valeria Vittorini, University of Napoli Federico II

    Model-based Validation, Verification and Analysis Approaches: Applications to Railways


    Abstract
    The usage of model-based approaches in critical systems development is widely recommended
    to cope with several challenges, such as capturing requirements, specifying behaviours, evaluating design choices, performing RAMS analysis and deriving test cases. An upward trend is to automate the V&V and evaluation processes by providing toolchains in support of effective model-based approaches able to interoperate with well-known and assessed tools for quality and architecture management. This talk will present a concrete realization of a model-based approach for automating the functional system-level testing of a modern railway control system as well as the application of model-driven techniques for deriving availability and vulnerability models of railway services and infrastructures.

    Bio
    Valeria Vittorini is Associate Professor of Computer Engineering at the University of Naples Federico II since November 2005. She teaches computer programming, formal modeling, workflow and process automation. Her research interests include dependability and performance evaluation of computer systems, validation and verification of critical systems, critical infrastructures protection and model-driven approaches applied to the automatic generation of formal models.
    This research work is performed in cooperation with academic institutions and national enterprises such as Ansaldo STS. She participated in several European and national research projects.

     

    Tanja Vos, Open University of Netherlands

    Automated Testing at the Graphical User Interface Level - Looking for the best TEST*

    Abstract

    Automated testing at the user interface level is challenging but rewarding. It difficult to define the expected behaviour of the graphical user interface (GUI) in a level of detail required for automated testing, but there are a lot of bugs to be found at user interface level, especially when there are multiple platforms and devices that are supported. We will present TESTAR (http://www.testar.org/), an open source tool for automated testing at the GUI level. With TESTAR it is possible to detect if the GUI crashes or hangs with very little human effort. No definition of test cases or oracles, no implementing or maintaining scripts required. Only time for automated test execution. Of course, the type of bugs found is limited without “proper” test oracles. By putting more effort in guiding TESTAR defining test oracles and possibly using search-based techniques to select actions, it is possible to find more and different kinds of bugs.

    Bio

    Prof. Dr. Tanja Vos is a full professor at the Open University (Netherlands) and an associate professor at the Universitat Politècnica de València (Spain). For over 20 years she has been teaching and researching in the area of software testing. She has worked with many companies on automated testing projects in an industrial setting. Tanja has successfully coordinated various EU-funded project (EvoTEST and FitTEST) related to search based software testing and has been involved in various Erasmus and Leonardo initiatives that try to help business understand academia an vice versa. She is currently project lead for the TESTAR.org approach for automated testing at the Graphical User Interface level. She has organized many events amongst which tool competitions on search-based testing and a series of eight A-TEST workshops. She started the Software Testing Innovation Alliance in Spain and is now involved in the first European edition in Amsterdam at the testdag.

    Erkuden Rios, ICT Division, TECNALIA

    MUSA: The solution to SecDevOps in multi-cloud.

    Abstract
    SecDevOps refers to addressing security since the very beginning of DevOps practices. In this session we will explain the MUSA solution for SecDevOps of applications which components are distributed over heterogeneous cloud services (the so-called multi-cloud applications). The MUSA solution is a framework that integrates a number of tools that tackle security aspects in the application life-cycle: from application architecture and security requirements modelling to continuous monitoring at operation.
     
     
    Bio
    Erkuden Rios is R&D project manager of Cybersecurity and Safety team within the ICT Division of Tecnalia. She is currently the coordinator of the H2020 MUSA project on multi-cloud security (www.musa-project.eu), and the coordinator of the Data Protection, Security and Privacy (DPSP) in Cloud Cluster of EU-funded research projects, launched by DG-CNECT in April 2015 (https://eucloudclusters.wordpress.com/data-protection-security-and-privacy-in-the-cloud ). She is specialized in trust and security engineering technologies and has worked in a number of large European and Spanish national projects on the subject such as ANIKETOS, SWEPT, TACIT, RISC, CIPHER and SHIELDS. Erkuden collaborates with technology platforms and forums such as ECSO, AIOTI and the Spanish Technology Platform on Trust & Security – eSEC. After obtaining her MSc in Telecommunication Engineering at the University of Basque Country (Spain), she worked for Ericsson Spain for 6 years before joining Tecnalia in 2003.

     

    Jens Krinke, University College London

    Inject, Mutate, Improve, Slice

    Abstract

    Mutation Testing, Genetic Improvement, and Program Slicing seem to be separate research areas and not necessarily all connected to Software Testing. However, they are all based on the principle of drastic changes to the underlying program.  This talk will present recent research based on the three areas and how they are connected and which role mutation plays as the core element of the research areas.

    Bio

    Jens Krinke is Senior Lecturer in the Software Systems Engineering Group at the University College London, where he is Director of CREST,
    the Centre for Research on Evolution, Search, and Testing.  His main focus is software analysis for sofware engineering purposes. His
    current research interests include mutation testing, software similarity, and modern code review.  He is well known for his work on
    program slicing and clone detection.

     

    Prof. Salvatore D’Antonio, University Parthenope

    Methods and Methodologies to protect local Public Administration from Cyber security Threats

    Abstract

    While the cybersecurity landscape is changing, and Local Public Administrations (LPAs) are rapidly becoming an attractive target for cybercriminals, the budgetary constraints often precludes them from putting in place highly granular organisational structures, retaining dedicated information security personnel and making significant investments in cybersecurity products or services. The talk will introduce the H2020 funded project COMPACT and will focus on PDCA methodology to increase LPAs awareness, skills, and protection against cyber threats; and to favour the information exchange both among LPAs and with the EU-level entities. Techniques such as Risk assessment, game-based education, monitoring and knowledge sharing services will be introduced.

    Bio

    Salvatore D'Antonio is currently an assistant professor at the University of Naples ”Parthenope”, Italy. He is an expert in network monitoring, intrusion detection systems, and critical infrastructure protection. He was the technical coordinator of the FP7 EU INTERSECTION project and the project coordinator of the INSPIRE and the INSPIRE-INCO projects. He actively participates to IETF standardization activities.

     

    Luigi Coppolino, University Parthenope

    Secure and Trusted Paradigm for Interoperable eHealth Services

    Abstract

    Security is considered the most important barrier for the adoption of new health applications as a major tool of care. In addition, during the last decade, we witness a considerable increase of citizen’s mobility in Europe for education, training, working and touristic purposes. Increasing mobility of patients is an important feature for independent living along the lines of active and healthy aging promoted by EIP-AHA (European Innovation Partnership on Active and Healthy Ageing) but it also adds additional issues while preserving health data security. This talk will introduce the work being done in the framework of the KONFIDO project which favours and especially contributes to the development and improvement of the prerequisites for intra- and cross-border patient mobility by allowing to widespread eHealth deployment and the secure exchange of personal health data among cross-border health systems.

    Bio

    Luigi Coppolino is an assistant professor at the University Parthenope, Italy. His research activity mainly focuses on dependability of computing systems, critical infrastructure protection, and information security. He is author of more than 50 scientific publications in the field and has been consultant for many companies and public administrations including the National Department for Information Security. He has been involved as principal investigator in many European Commission funded research projects.

     

    Yliès Falcone, University of Grenoble

    Runtime verification and enforcement 

    Abstract

    In this tutorial Prof. Falcone will present the fundamentals of runtime verification and enforcement. We will cover the basic concepts, classical algorithms, and current trending topics. Finally, he will show how to implement basic runtime verification and enforcement monitors for properties for good programming practices of the Java programming languages.

    Bio

    Yliès Falcone received the Master degree (2006) and Ph.D. (2009) in computer science from the University of Grenoble at Verimag Laboratory. His research interests concern formal runtime validation techniques for various application domains, i.e. techniques aiming at evaluating whether a system meets a set of desired properties during its execution. Since September 2011, he is associate professor at University of Grenoble Alpes and a researcher at Inria and Laboratory of Informatics Grenoble.