Discussion¶
18 Limitations¶
Description
Discuss any limitations of the study (such as nonrepresentative sample, few events per predictor, missing data).
Limitations should be placed in perspective (e.g. current research directions and practices) and each that is problem identified should include a critical discussion of how the problem may lead to potentially unintended social impacts, including any associated uncertainty.
Reporting limitations offers researchers the opportunity to reflect on and deliberate over the possible impacts of their study (e.g. on scientific or social practices), including the degree of uncertainty associated with these impacts. Doing so offers researchers an opportunity to exercise responsibility, to engage in reflexive practices, and to strengthen their research by connecting with subsequent research efforts that may be able to address the identified impacts.
19a Interpretation¶
Description
For validation, discuss the results with reference to performance in the development data, and any other validation data.
Highlighting and interpreting the main results of a validation study (e.g. how a validated model is similar to or different from the original model) provides an opportunity to contextualise the discussion of the model’s performance alongside additional studies.
A contextual discussion of the models performance provides an opportunity to critically evaluate any biases that may have impacted the comparison (e.g. ‘confirmation bias’) and to exercise a responsible attitude towards the reporting of research findings. An accessible interpretation also supports the clear communication of findings to both a technical and a non-technical readership.
19b Interpretation¶
Description
Give an overall interpretation of the results, considering objectives, limitations, results from similar studies, and other relevant evidence.
Interpreting the results of the study enables researchers to situate their findings within the context of additional evidence (e.g. different models with the same or different outcomes).
Interpreting the results of a study is a reflexive process, which can provide an opportunity to report on findings in a responsible and context-aware manner. For instance, a critical evaluation of the study’s findings, with reference to the original objectives and similar studies, allows subsequent research to build upon the limitations of the study. Over-interpretation of the results can obscure this process and may create harms if the model is implemented in an inappropriate context.
20 Implications¶
Description
Discuss the potential clinical use of the model and implications for future research.
Prediction models can have a variety of uses, some of which are likely to be inappropriate given the limitations of the study. For example, newly developed models may be inappropriate to use in safety critical domains without extensive validation. When discussing potential uses, reference can be made to suitable domains for application (e.g. specific patient groups, geographic locations) and appropriate use cases (e.g. to confirm presence of a specific disease, to support human-in-the-loop decision making). Domains or uses that are not suitable for the model can also be discussed, and should include explanatory reasons (e.g. limitations of study based on internal validation only).
Discussing the potential uses of a model, and its implications for future research is a core component of responsible research and innovation (e.g. exercising reflexive discussion to identify and anticipate potential impacts of the study or model).
The discussion section is also a natural place for researchers to include possible recommendations for being responsive to any unintended consequences that could arise from misuse of the model or study findings. This is additionally a good place to include discussion of the possibility of concept drift and to set out recommendations for future re-evaluation and re-validation.