People and Management Related Debt in ML-Integrated Software Development Projects: Structuring Insights from Industry
- Citation Author(s):
-
Pelin Dayan Akman
(Graduate School of Informatics, Middle East Technical University, Universiteler mah, Cankaya, Ankara, Türkiye)
Özden Özcan Top
(Graduate School of Informatics, Middle East Technical University, Universiteler mah, Cankaya, Ankara, Türkiye)
Tuğba Taşkaya Temizel (Graduate School of Informatics, Middle East Technical University, Universiteler mah, Cankaya, Ankara, Türkiye) - Submitted by:
- Pelin Dayan Akman
- Last updated:
- DOI:
- 10.21227/0dvz-j489
- Data Format:
9 views
- Categories:
- Keywords:
Abstract
The accelerated development of Machine Learning (ML) tools, combined with broader access to frameworks and infrastructures, has driven the rapid adoption of ML-based solutions in industry. However, their integration into software systems introduces unique challenges, particularly for managing technical debt (TD). While existing frameworks/standards such as Cross-Industry Standard Process for Data Mining (CRISP-DM) and ISO/IEC 5338 provide guidance for ML development, they fail to address the complex interplay of technical and nontechnical factors contributing to TD. Traditional TD research focuses primarily on technical issues, but in ML systems, people, and management factors, referred to as nontechnical debt (NTD), play a critical role in TD accumulation and persistence. In this study, we investigate the underexplored dimension of NTD in ML-integrated software systems, focusing on people and management-related factors. Using Design Science Research (DSR) methodology, we developed an artifact that categorizes NTD issues in ML systems. As part of this process, we conducted semi-structured interviews with 18 professionals from 15 companies, examining 22 ML projects. Through thematic analysis, we identified 15 NTD categories, each associated with underlying causes, short-term fixes, and potential solutions. Our findings indicate that NTD in ML projects often arise from decision-making practices, team dynamics, and communication barriers, all of which substantially affect project outcomes. While temporary fixes (i.e., band-aid solutions) may provide short-term relief, they frequently contribute to the accumulation over time. To support practitioners and researchers, our study complements the proposed artifact with actionable recommendations informed by expert perspectives and literature.
Instructions:
| How to interpret data table: |
| There are eight distinct types of information extracted by coding analysis: The identified issue subject to nontechnical debt (NTD) The root cause of the issue The impact or consequences of the issue The suboptimal solutions causing recursive NTD The root cause of the suboptimal solution The impact of the suboptimal solution The suggested solutions eventually implemented The rationale behind selecting the suggested solution |
| After thematic analysis, categorized data was restructured into an Excel structure: Interview Participant ID → Unique Project ID → Unique Company ID → Project Business Domain → ML Application Domain Assigned Parent Category of NTD Case → Assigned NTD Category → Case Subject to NTD → Result of NTD Assigned Parent Category of NTD Root Cause → Assigned NTD Root Cause Category → Root Cause of NTD Case Assigned Parent Category of Band-aid Solution → Assigned Band-aid Solution Category → Band-aid Solution → Root Cause of Band-aid Solution → Result of Band-aid Solution Suggested Solution → Reason of Suggested Solution |
| NTD cases were assigned unique IDs and categorized under the corresponding parent (People and Management) categories and NTD categories under parent categories. When a TD case is mentioned by multiple interview participants, separate data entries are made for each but under the same NTD ID, recognizing that despite the similarity in the case, the band-aid solutions or suggested recovery strategies applied by different individuals can vary. Each TD case is thoroughly examined within its flow, encompassing the root cause, sub-optimal solutions, and the suggested recovery approach, ensuring that no detail is overlooked and providing a comprehensive understanding of each case. This structured approach facilitated domain-specific evaluations either, allowing contextual assessment of each case with relevant metadata (e.g., project domain, business domain, interviewee details). Root causes of the NTD cases are also categorized. When their categories are different than NTD cases, they are written in another row and color-coded as green and so their assigned categories are also placed in H and I columns. |
| Definitions for each column are provided below for clarity: 1. NTD ID (Column A): Each entry starts with a unique NTD Case ID, ensuring easy identification and reference. 2. Problem, Band-aid Solution, and Suggested Solution (Column B): This column indicates whether the NTD case includes a problem, a band-aid solution, or a suggested (better) solution. It shows which of these elements the interview participant has addressed for the specific case, allowing you to identify if the NTD discussion in the row includes the description of a problem, a short-term fix, or the suggested long-term solution. 3. Interview Participant, Project, and Company Details (Columns between C-G): Each case links to project IDs, anonymized participant and company IDs, providing context for where the technical debt arose. 4. TD Categorization (Columns H and I): The structure categorizes NTD cases using ‘Parent NTD Category’ and ‘NTD Category’, allowing for hierarchical classification, making it easier to analyze high-level and granular issues. 5. Root Cause Analysis (Columns K and P): The structure incorporates columns for the root causes from participant perspectives, both for the original NTD and the resulting band-aid solutions. This ensures that the depth of each issue is captured. 6. Applied Band-Aid Solutions and Suggested Solutions (Columns O, Q and R): The structure encourages including both sub-optimal fixes and the suggested solutions, along with reasons provided by participants. 7. Potential Consequences (Columns S and T): These columns reflects insights from the researchers’ perspective. It includes potential issues that may arise if the band-aid solution continues, as well as possible consequences of NTD in terms of project cost, effort, project quality and duration, discussed by academic experts during the iterative refinement of the artifact. |
