Experimental evaluation of test-driven development with interns working on a real industrial project: Replication Package
Experimental evaluation of test-driven development with interns working on a real industrial project:
- raw experimental data: unit test, integration tests results and coverage,
- R statistical analysis script,
- processing scripts.
calculate_conformance.py - Command line: `python calculate_conformance.py [REPONAME]` (e.g. `python calculate_conformance.py tdd1`). Calculate processconformance for the given repository. Uses different classification algorithmdepending on type of the respository - tdd or tld. For nut always returns100. Uses data from `classified_commits` directory.calculate_participant_conformance.py - Command line: `pythoncalculate_participant_conformance.py [PARTICIPANT ID]` (e.g. `pythoncalculate_participant_conformance P1`). Calculates average process conformancefor a participant, averaging over TDD and TLD. Also exports functions forcalculating conformance for TDD and TLD separately. Uses data from`classified_commits` directory.conformance_report.py - Command line: `python conformance_report.py`. Generatesa table with process conformance for each repository for each iteration. Eachcell contains process conformance for commits only for the appropriate iterationfor TDD or TLD according to the technique that should have been used in eachrepository. Uses data from `classified_commits` directory.bugcount_report.py - Command line: `python bugcount_report.py`. Generates atable with number of bugs in each repository in each iteration. Requiresunzipped `test_results` directory with all test results.calculate_coverage.py - Command line: `python calculate_coverage.py [XML TESTRESULTS FILE NAME]` (e.g. `python calculate_coverage.pytest_results\2016-08-07_unit_cover.xml`). Extracts the code coverage data fromNCover test results XML report. Requires unzipped `test_results` directory withtest results.coverage_report.py - Command line: `python coverage_report.py`. Generates atable with code coverage in each repository in each iteration. Requires unzipped`test_results` directory with all test results.assignment_plan.py - Plan of assignment of participants to repositories for eachiteration in the experiment. Also contains assignment of participants tocompetence groups.date_str.py - Utility for managing custom tuple date format.get_commit_author.py - Exposes a function that returns ID of a participant whohad been working in a given repository at a given date.get_failed_categories.py - Utility exposing a function to parse NUnit testresult TRX reports. Exposes function to calculate the number of failures foreach test category. Each test category is mapped to each task the participantshad to implement.get_iteration_idx_from_date.py - Utility exposing a function to find an index ofan iteration for a given date.issue_dates.py - Utility exposing a dictionary mapping each task (issue) to astart date of an iteration in which the task had been introduced to theparticipants.iteration_finish_dates.py - Utility exposing a list mapping indexes ofiterations to their end dates.repo_list.py - Utility exposing lists of repository names in different orders.classified_commits/categorize_commits.py - Command line: `pythoncategorize_commits.py [REPO NAME]` (e.g. `python categorize_commits.pytdd1`). Classifies commits into "good", "bad" and "ignored", according to thetechnique assigned to the given repository and produces a report text filecontainins a section for each category, with short explanation for eachcommit. Requires the repository to be cloned locally. Depends on `git` and`msbuilder` Python modules and the `NUGet.exe` tool with its configuration, tobe able to recompile and retest the code in the repository after each commit.