US Navy health settings may benefit from using external civilian-based risk models for suicide prevention.
A new cohort study has assessed the performance of a civilian-based suicide risk model in US Navy primary care settings, revealing its potential to be an effective tool in suicide prevention.
Key Findings:
- Generalization to Military Health Settings: The study involved 260,583 active-duty US Navy service members and found that the civilian-based suicide risk model performed well in US Navy primary care, suggesting that risk models developed in civilian settings might generalize effectively to military health settings.
- External Validation: The civilian-based model was externally validated by applying it to every visit at Naval Medical Center Portsmouth (NMCP), NMCP Naval Branch Health Clinics (NBHCs), and TRICARE Prime Clinics (TPCs). It showed an area under the receiver operating curve (AUROC) of 0.77 and an area under the precision-recall curve (AUPRC) of 0.004 at NBHCs. However, this external model exhibited poor calibration.
- Retraining and Updating: Retraining the model with US Navy data improved the AUROC to 0.92 and the AUPRC to 0.66, resulting in better performance. Model updating with Department of Defense (DoD)-specific factors added minimal improvement.
- Numbers Needed to Screen (NNS): The NNS for the top risk tiers was 366 for the external model and 200 for the retrained model, indicating better performance for the retrained model.
- Temporal and Domain Validation: Temporal and domain validation results showed that external validation performance was similar to retrained and updated models, suggesting that implementing an external model in US Navy primary care clinics could expedite suicide prevention efforts.
Implications:
The study’s results suggest that civilian-based risk models can be effective in military health settings, saving time and resources. These models can serve as valuable tools for identifying at-risk individuals, prompting further screening and discussions to prevent covert distress and expedite suicide prevention efforts in the US Navy primary care clinics.
Limitations:
While the study found success in generalizing the civilian-based model to military settings, it acknowledged limitations, including the potential for overfitting, the need for further refinement in retraining, and the importance of exploring additional DoD-specific features for model updating.
Conclusion:
The study’s findings offer promising insights into the use of civilian-based risk models in military health settings, providing a potentially efficient and cost-effective way to enhance suicide prevention efforts among US Navy service members. Further research is needed to refine and optimize these models for military-specific applications.