If you don't remember your password, you can reset it by entering your email address and clicking the Reset Password button. You will then receive an email that contains a secure link for resetting your password
If the address matches a valid account an email will be sent to __email__ with instructions for resetting your password
The clinical performance and cost-effectiveness of two psychosocial assessment models in maternity care: The Perinatal Integrated Psychosocial Assessment study
National Perinatal Epidemiology and Statistics Unit (NPESU), Centre for Big Data Research in Health and School of Women’s and Children’s Health, University of New South Wales, Sydney, NSW, 2052 Australia
National Perinatal Epidemiology and Statistics Unit (NPESU), Centre for Big Data Research in Health and School of Women’s and Children’s Health, University of New South Wales, Sydney, NSW, 2052 Australia
Centre for Health Service Development, Australian Health Services Research Institute, University of Wollongong, NSW 2522, AustraliaPerinatal and Women’s Mental Health Unit, St John of God Health Care, 23 Grantham St, Burwood NSW 2134, AustraliaSchool of Psychiatry, University of New South Wales, Sydney, NSW 2052, Australia
Drug and Alcohol Services, South Eastern Sydney Local Health District, 591 South Dowling Street, Surry Hills, NSW 2010, AustraliaDiscipline of Addiction Medicine, University of Sydney, Camperdown, NSW 2006, AustraliaSchool of Public Health and Community Medicine, Faculty of Medicine, University of NSW, Sydney 2052, Australia
Corresponding author at: Perinatal and Women’s Mental Health Unit, St John of God Health Care and University of New South Wales, PO Box 261, Burwood, NSW 1805, Australia.
Perinatal and Women’s Mental Health Unit, St John of God Health Care, 23 Grantham St, Burwood NSW 2134, AustraliaSchool of Psychiatry, University of New South Wales, Sydney, NSW 2052, AustraliaRoyal Hospital for Women, Sydney, Barker Street, Randwick, NSW 2031, Australia
Although perinatal universal depression and psychosocial assessment is recommended in Australia, its clinical performance and cost-effectiveness remain uncertain.
Aim
To compare the performance and cost-effectiveness of two models of psychosocial assessment: Usual-Care and Perinatal Integrated Psychosocial Assessment (PIPA).
Methods
Women attending their first antenatal visit were prospectively recruited to this cohort study. Endorsement of significant depressive symptoms or psychosocial risk generated an ‘at-risk’ flag identifying those needing referral to the Triage Committee. Based on its detailed algorithm, a higher threshold of risk was required to trigger the ‘at-risk’ flag for PIPA than for Usual-Care. Each model’s performance was evaluated using the midwife’s agreement with the ‘at-risk’ flag as the reference standard. Cost-effectiveness was limited to the identification of True Positive and False Positive cases. Staffing costs associated with administering each screening model were quantified using a bottom-up time-in-motion approach.
Findings
Both models performed well at identifying ‘at-risk’ women (sensitivity: Usual-Care 0.82 versus PIPA 0.78). However, the PIPA model was more effective at eliminating False Positives and correctly identifying ‘at-risk’ women (Positive Predictive Value: PIPA 0.69 versus Usual Care 0.41). PIPA was associated with small incremental savings for both True Positives detected and False Positives averted.
Discussion
Overall PIPA performed better than Usual-Care as a psychosocial screening model and was a cost-saving and relatively effective approach for detecting True Positives and averting False Positives. These initial findings warrant evaluation of longer-term costs and outcomes of women identified by the models as ‘at-risk’ and ‘not at-risk’ of perinatal psychosocial morbidity.
Depressive and anxiety disorders in the perinatal period are associated with a range of salient risk factors, which may lead to poorer outcomes for both mother and infant. The most effective approach to identifying risk, initiating referral, and the economic costs associated with psychosocial and depression screening, remain uncertain.
What is already known
Routine psychosocial assessment is becoming more widespread in the Australian maternity setting and psychosocial assessment in pregnancy is associated with greater referral to psychosocial and mental health services.
What this paper adds
This paper provides initial evidence for the effectiveness of the Perinatal Integrated Psychosocial Assessment (PIPA) model of care compared to Usual-Care for psychosocial screening in hospital maternity settings. Assuming the slightly lower sensitivity of PIPA is clinically acceptable, this study indicates that the PIPA model would be cost-saving and a relatively effective approach for detecting the True Positive cases and averting False Positive cases of women requiring further assessment of psychosocial risk.
1. Introduction
The perinatal period – pregnancy through to the first postnatal year - is a time of increased risk for recurrent and new onset mental health morbidity [
]. Depression and anxiety disorders arising within the perinatal period are recognised as an international public health problem, affecting around 15% women [
] and up to 40% of women with symptoms of postnatal depression also experienced these in pregnancy, highlighting the importance of beginning psychosocial and depression assessment during pregnancy [
The national perinatal depression initiative: an evaluation of access to general practitioners, psychologists and psychiatrists through the Medicare Benefits schedule.
Mental health consultations in the perinatal period: a cost-analysis of Medicare services provided to women during a period of intense mental health reform in Australia.
]. Recent Australian data indicates that the total healthcare costs to mothers alone of untreated perinatal depression and anxiety to be $70 million in the first postnatal year, increasing to over $500 million after additional healthcare and productivity costs for fathers and children are taken into consideration [
Over the last two decades, routine integrated psychosocial assessment (inclusive of depression screening) has been examined in terms of its possible value in the early identification of women at greater risk of mental health morbidity or poor adjustment to parenting [
Psychosocial assessment and depression screening in the perinatal period: benefits, challenges and implementation.
in: Sutter-Dallay A.L. Glangeaud-Freudenthal N. Guedeney A. Riecher-Rössler A. Joint Care of Parents and Infants in Perinatal Psychiatry. Springer International Publishing,
Cham2016: 167-195
]. Integrated psychosocial assessment is a broader approach to assessment that also enquires about issues such as domestic violence, substance misuse, social support and stressors, mental health history and history of childhood trauma, in addition to current symptoms. Current Australian national clinical guidelines recommend the routine use of the Edinburgh Depression Scale (EPDS) [
] to flag women at significant psychosocial risk needing further monitoring, referral or support. This approach is also reflected in a number of state-based initiatives and is embedded into routine maternity care in a range of maternity hospitals in Australia. In New South Wales, for example, the SAFE START policy directive [
] have underpinned midwife-led antenatal psychosocial care at the study site (Royal Hospital for Women [RHW]) since 2010.
A review of the effectiveness of integrated psychosocial assessment in perinatal women concluded that ongoing research is critical if the ideal model of routine depression screening in conjunction with integrated psychosocial assessment is to be established, and that availability of integrated, on-site assessment and intervention services improves accessibility to those identifed as being ‘at-risk’ [
The use of the antenatal psychosocial health assessment (ALPHA) tool in the detection of psychosocial risk factors for postpartum depression: a randomized controlled trial.
] than those who were not. Two Australian survey-based studies have shown that the various perinatal mental health initiatives of the last two decades have been associated with a steady increase in perinatal integrated psychosocial assessment [
However, there remains considerable controversy and uncertainty around the clinical and cost-effectiveness of routine depression screening in the perinatal period [
]. For example, a study which used modelled data to examine the value of routine perinatal depression screening using validated tools such as the Edinburgh Depression Scale (EPDS) [
] (but without broader psychosocial assessment), coupled with best practice depression treatment in a primary care setting, concluded that formal identification methods for postnatal depression did not represent value for money [
]. However, the authors noted that a key determinant of cost-effectiveness was the likely additional costs associated with ‘False Positive’ cases, that is managing women incorrectly diagnosed as depressed, highlighting the importance of minimising false positives if integrated psychosocial assessment and depression screening in pregnant women is to be cost-effective. Similarly, an internal review of the SAFESTART policy implementation at RHW found that a significant number of women were unnecessarily referred for psychosocial services leading to an undue burden on the hospital’s psychosocial services (A. Datta, personal communication, 1st January–31st October, 2012). These initial findings, and the need to examine the performance and cost-effectiveness of an integrated psychosocial assessment in the perinatal period as articulated in the Australian national guideline [
The aim of this study was to compare the clinical and cost-effectiveness of two models of routine integrated psychosocial assessment and referral – Usual-Care and PIPA – in a real-world maternity setting using contemporary data, with the key outcome and reference standard being midwife agreement with the ‘at-risk’ flag generated in each model.
2. Methods
2.1 Study design, participants and data sources
This was a prospective cohort study that consecutively recruited women attending their first antenatal visit at RHW. The data collection period for Usual-Care was from January to December 2015, and for the PIPA model was from August 2017 to July 2018. For this study, administrative data were extracted from the hospital administrative database for all eligible women [
] who completed a routine integrated psychosocial assessment as part of their first midwife visit: data was drawn from ObstetriX for Usual-Care and from eMaternity for PIPA (as eMaternity was implemented across NSW from 2017).
Sociodemographic and clinical information extracted included maternal age, gestation, country of birth, interpreter required, parity, partner status (available for PIPA only), EPDS scores and responses to the psychosocial assessment questions.
The EPDS is a 10-item measure and is arguably the most widely used depression screening tool in perinatal care worldwide [
]. An EPDS score of 13 or more has been shown to have moderate sensitivity (0.66; 95% CI 0.58–0.74) and high specificity (0.95; 95% CI 0.92–0.96) for detecting possible major depression in pregnant and postpartum women [
Accuracy of the Edinburgh Postnatal Depression Scale (EPDS) for screening to detect major depression among pregnant and postpartum women: systematic review and meta-analysis of individual participant data.
The psychosocial questions used in Usual-Care were those provided in the SAFE START policy and guideline, whereas the PIPA model used the Antenatal Risk Questionnaire-Revised (ANRQ-R; see below). The ANRQ-R has been shown to have an acceptable area under the curve (AUC = 0.79−0.80) and to correctly classify 72–76% of current ‘cases’ and ‘non-cases’ when used to detect current depression or anxiety in the perinatal period (sensitivity = 0.70−0.74, specificity = 0.72−0.76) [
Supporting routine psychosocial assessment in the perinatal period: the concurrent and predictive validity of the Antenatal Risk Questionnaire-Revised.
]. No corresponding psychometric information is available for the SAFE START psychosocial questions.
Information relating to whether a woman was flagged as ‘at risk’ was also extracted. Key differences in how this ‘at risk’ flag is defined in the Usual-Care and PIPA models is described in more detail in the following section.
Resource consumption and unit resource cost estimates were collected by survey and time in motion observational data collected during the study period. These data included time required for the midwife to complete the psychosocial assessment, administrative tasks associated with the initiation of referrals and discussion at the Triage meeting.
The study protocol, including more detailed information relating to data collection procedures, is available elsewhere [
The project was approved by South Eastern Sydney Local Health District Human Research Ethics Committee (SESLHD HREC; Ref. 14/117).
2.3 Key differences between Usual-Care and PIPA models of integrated psychosocial assessment and referral
Women completed a series of questions during their first antenatal visit with a midwife in both the Usual-Care and PIPA models. Unlike Usual-Care, the PIPA model used a scored psychosocial questionnaire (the ANRQ-R, a revised version of ANRQ) which measures cumulative risk and generates both item level and total scores; auto-scores both the EPDS and ANRQ-R (eliminating manual scoring errors); includes structured questions to explore self-harm on the EPDS; and enables the midwife to document specific clinical concerns not elicited by the questionnaires (e.g., the woman’s presentation at interview). In addition, PIPA generates clear psychosocial risk levels and referral pathways tailored to the woman’s psychosocial risk level (see Table 1 for detail). While each model included an ‘at-risk’ flag, indicating that referral to the hospital Triage Committee was appropriate, a critical difference between the Usual-Care and PIPA model flags was that just one risk factor could trigger referral in Usual-Care, while PIPA required a higher threshold of risk for referral to the Triage Committee.
Table 1Comparison of key features of the Usual-Care and PIPA models of integrated psychosocial assessment.
Three levels of psychosocial risk, defined as: Level 1: no specific vulnerabilities or risk Level 2: one or more risk factors of variable severity and significance including, but not limited to, low supports, multiple birth, financial stress, isolation, ‘mild-moderate’ depression or anxiety, history of mental health problem, young age. Level 3: one or more of four complex risk factors (domestic violence, involvement with child protection services, substance misuse, severe mental illness)
Six levels of psychosocial risk, defined as: No risk: ANRQ-R = 0; EPDS < 13 (Q10 = 0); no clinician concerns. No risk on ANRQ-R (ANRQ-R = 0) but clinician concerns and/or EPDS=13 or 14. Low risk: ANRQ-R = 1–24 (excluding e.g., significant mental health history
However, based on her opinion during the antenatal visit, the midwife undertaking the assessment, acting as the ‘reference standard’, could either override or agree with a positive or negative ‘at-risk’ flag generated by the Usual-Care or PIPA models resulting in the four possible outcomes for each woman: True Positive, False Positive, True Negative or False Negative. This approach acknowledges that unlike symptom-based screening, there is no ‘gold standard’ comparison measure available for the more comprehensive psychosocial assessment central to this study. That is, use of traditional gold standards instruments (such as the SCID [
]) is inappropriate in the context of a study that focuses on the clinical and cost-effectiveness of broader psychosocial assessment rather than the effectiveness of symptom-based mental health screening per se. Instead, the approach we have applied is in line with similar studies that have effectively used midwives as a ‘reference standard’ when determining the true psychosocial risk status of patients relative to women being identified as such through an electronic algorithm [
]. We hypothesised that the more detailed and precise PIPA risk and referral flag might be expected to reduce False Positives, and in turn reduce unnecessary referral to the Triage Committee meeting and costs associated with this.
Midwives have been undertaking routine integrated psychosocial assessment at the study site for many years and have regular in services on integrated psychosocial assessment, while new midwives are trained in integrated psychosocial assessment when they commence working in the antenatal clinics [
]. There were no differences in the two timeframes in terms of midwife training.
2.4 Descriptive statistics and clinical performance analysis
Descriptive statistics were used to compare the demographic characteristics of women who received care under the Usual-Care and PIPA models. The outcome measure for the performance of each model of care was whether the midwife undertaking the assessment (henceforth referred to as the screening midwife) agreed with the ‘at-risk’ flag or not. Measures of model performance included sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), likelihood ratios and Youden’s Index [
], using midwife agreement as the reference standard.
2.5 Cost-effectiveness analysis
The cost analysis was performed from a healthcare perspective, and included the time commitment of the screening midwives, administrative staff and Triage Committee clinicians. Costs were quantified using a bottom-up approach using the staff time in minutes for the antenatal booking-in visits, administrative tasks and the Triage Committee meetings [
]. Overhead costs, such as office space, depreciation and consumables, were not included as they were assumed to be the same for both models. All salaries were costed at 2017 pay rates in Australian dollars (AUD).
A cost-effectiveness analysis was performed for the cases flagged as ‘at-risk’ (according to the PIPA or Usual-Care flag) in each model plus whether the screening midwife agreed with the ‘at- risk’ flag or not. Effectiveness was measured in terms of True Positives (that is, the screening midwife agreed with the positive ‘at-risk’ flag and the woman was referred to the Triage Committee), and False Positives (that is, the screening midwife did not agree with the positive ‘at-risk flag and the woman was not referred to the Triage Committee). These specific measures of performance were chosen because of the perceived high False Positive rate under the Usual-Care model and in other psychosocial screening tools [
] reflecting the screening, assessment and referral for psychosocial care during pregnancy for the Usual-Care and PIPA models.
Overall uncertainty of costs and effectiveness was quantified using the percentile method which estimated the 95% confidence intervals from 5,000 Monte Carlo simulations of a probabilistic sensitivity analysis (PSA) for costs and a non-parametric bootstrapping the case outcomes. A Gamma probability distribution was assigned to the cost parameters computed from the variation of individual costs activities. The 5,000 replications were plotted on a cost-effectiveness plane to visualise the uncertainty around the parameter estimates.
Analyses of the study were performed using SPSS v24.0 [
Between January and December 2015, 3673 women were recruited to the Usual-Care model, and between August 2017 and July 2018, 3132 women were recruited to the PIPA model. Only 4.9% of women birthed twice or more in the total study period (January 2015 to September 2018). While there were statistically significant differences between the cohorts on key sociodemographic and clinical characteristics, including age (mean 32.2 years vs 32.6 years, p < .001) and gestational age at booking-in visit (15.3 weeks vs 15.6 weeks, p < .001), the effect size differences for all variables compared were small or not clinically significant (Table 2). The first antenatal visit occurred at 15.4 weeks gestation on average.
Table 2Sociodemographic and clinical characteristics, Usual-Care and PIPA.
3.1 Clinical performance of Usual-Care and PIPA models of integrated psychosocial assessment
Fig. 1 outlines the results of the cohorts of women who received care under the Usual-Care model and the PIPA model. The decision tree was used to track women based on the results of the model’s integrated psychosocial assessment at their first antenatal visit (positive or negative ‘at-risk’ flag indicating whether a woman should be referred to the Triage Committee), and whether the screening midwife agreed with the ‘at-risk’ flag and sent the women to the Triage Committee or not. Of the 3673 women who were assessed under the Usual-Care model, 1343 (37%) were identified as being at psychosocial risk by the Usual-Care ‘at-risk’ flag. By contrast, of the 3132 women assessed under the PIPA model, 850 (27%) were identified as being at psychosocial risk by the PIPA ‘at-risk’ flag (Table 3).
Fig. 1Schematic diagram of study results Usual-Care versus PIPA models of care.
The probability of detecting a True Positive case (sensitivity) was marginally higher in the Usual-Care model than the PIPA model (0.82 and 0.78, respectively). That is, the PIPA model identified a slightly smaller proportion of women who the midwife thought needed to be referred to the Triage Committee. Conversely, the probability of detecting a True Negative case (specificity) was lower in the Usual-Care model than the PIPA model (0.78 and 0.89, respectively). The Positive Predictive Value (PPV), that is, the probability of the midwife agreeing that a woman was at psychosocial risk in those with a positive ‘at-risk’ flag, was lower in the Usual-Care model than the PIPA model (0.41 and 0.69, respectively). The PPV for the PIPA model was in the ‘moderate’ range (0.69) and significantly superior to that seen for the Usual-Care model (which was in the ‘low’ range at 0.41). The Negative Predictive Values (NPV), that is the probability of the midwife agreeing that a woman was not at risk, in those with a negative ‘at-risk’ flag, were very similar (0.95 and 0.93, respectively).
The PIPA model was also better at excluding women who the midwife did not agree were ‘at’ risk’. That is, it was over twice as effective at eliminating False Positive cases (False Positive Ratio of 26% and 11%).
The Youden index provides an overall measure of test performance, by balancing sensitivity (detecting women who the midwives agree are at psychosocial risk) and specificity (detecting women who the midwives agree are not at psychosocial risk). Values range from 0 to 1, with a value of one representing no False Positives. Generally, a cut-off point of above 0.5 represents an acceptable clinical test. The PIPA model performed better than Usual-Care (0.67 and 0.56, respectively).
3.2 Cost-effectiveness of Usual-Care and PIPA
The mean staff time to screen and refer women as needed, along with the salary rates and resulting costs are presented in Table 4. The mean cost of assessment and screening was marginally lower in the PIPA model ($13.63) than Usual-Care ($14.38), with almost identical costs for the Triage Committee meeting under both models ($11.32 and $11.34 respectively).
Table 4Staff times and costs associated with Usual-Care vs PIPA, AUD 2017.
Staff salary rates sourced from New South Wales public health system awards. Available at: https://www.health.nsw.gov.au/careers/conditions/Pages/awards.aspx. Hourly salary rates; Level 3 Admin officer: $32.34, Midwife - Junior: $37.40, Senior Midwife: $44.99, Data Manager: $44.99, Senior Social Worker: $51.84, Clinical Midwife Consultant - Level 2: $64.93.
SD
Distribution
Usual-Care
3673
20.6
10
$14.38
$1.19
Gamma
PIPA
3132
20.2
9.2
$13.63
$0.79
Gamma
Triage Committee
Usual-Care
673
3.7
17.3
$11.34
$2.98
Gamma
PIPA
744
3.7
16.2
$11.32
$2.38
Gamma
All women
Usual-Care
3673
–
–
$16.45
$1.31
Gamma
PIPA
3132
–
–
$16.32
$0.98
Gamma
a Staff salary rates sourced from New South Wales public health system awards. Available at: https://www.health.nsw.gov.au/careers/conditions/Pages/awards.aspx. Hourly salary rates; Level 3 Admin officer: $32.34, Midwife - Junior: $37.40, Senior Midwife: $44.99, Data Manager: $44.99, Senior Social Worker: $51.84, Clinical Midwife Consultant - Level 2: $64.93.
For True Positives (that is the screening midwife agreed with the ‘at-risk’ flag and referred the women to the Triage Committee), the mean cost for the PIPA model was $16.32 (95% CI: $14.53–$18.30), while the mean cost of the Usual-Care model was $16.46 (95% CI: $14.04–$19.12). See Table 5.
Table 5Expected outcomes and mean costs PIPA versus Usual-Care for women identified as ‘at risk’.
The overall proportion of women that were identified as True Positives was 18.6% (582/3132) for PIPA and 15.1% (554/3678) for the Usual-Care model. Thus, the incremental cost-effectiveness ratio (ICER) of per True Positive women detected as a proportion of all women screen was −$3.97.
For False Positives (that is where the screening midwife over-rode the positive ‘at-risk’ flag and did not refer the case to the Triage Committee), the mean cost (the cost of assessment) for the PIPA model was $13.63 (95% CI: $12.11–$15.23) while that of the Usual-Care model was $14.38 (95% CI: $12.15–$16.93). The overall proportion of screened women that were identified as False Positives was 8.6% (268/3132) for PIPA and 21.4% (789/3678) for Usual-Care, while the overall cost per woman assessed was $16.32 for PIPA and $16.45 for Usual-Care. Thus, the ICER results showed that the PIPA model was more effective at averting a False Positive case detected at a cost saving of $5.80/woman.
Overall, the results indicated that these two measures of model performance are cost-saving using the PIPA model to assess for psychosocial risk. However, the wide confidence intervals indicate a high degree of uncertainty. This is demonstrated by the Monte Carlo simulation plotted on the cost-effectiveness planes for the two models of care in Fig. 2. The screening effectiveness of the PIPA model was better than Usual-Care for both True Positive and False Positives with all simulations in the favourable direction (in the right-hand quadrants for True Positives cases and the left-hand quadrants for False Positives cases). However, a significant number of simulations fall above and below the upper and lower quadrants indicating a great degree of uncertainty around the cost-effectiveness.
Fig. 2Cost-effectiveness plane for True Positive (‘correct’ referrals) and False Positives (‘incorrect’ referrals avoided), Usual-Care versus PIPA.
True Positive ICER replication in blue (upper right quadrant). False Positive replication in grey (lower left quadrant).
Note: The screening effectiveness of the PIPA model was better than Usual-Care for both True Positives and False Positives with all simulations in the favourable direction (in the right-hand quadrants for True Positive cases and the left-hand quadrants for False Positive cases).
The effectiveness of the PIPA model at identifying women as being at psychosocial risk was better than Usual-Care for both True Positives and False Positives with all simulations in the favourable direction (in the right-hand quadrants for True Positive cases and the left-hand quadrants for False Positive cases).
4. Discussion
This study showed that both the Usual-Care and PIPA models performed well in terms of appropriately identifying those women ‘at-risk’ (in terms of True Positives and False Positives) and requiring referral to the Triage Committee, with the Usual-Care model having a slightly better sensitivity (82%) compared to PIPA (78%). The study also found that the PIPA model had a higher probability than Usual Care of correctly identifying women who the midwife agreed were at sufficient psychosocial risk to refer to Triage (PPV: 69% and 41%, respectively). This better PPV for the PIPA model likely reflects the ability of the PIPA model algorithm, underpinned by the cumulative risk factor score calculation generated by the ANRQ-R, to set a more refined referral threshold and thus increase the proportion of True Positive cases. The PIPA model was also over twice as effective at eliminating False Positive cases at excluding women who the midwife did not agree were at sufficient psychosocial risk to refer to the Triage Committee.
The lower threshold that is set for the Usual-Care for women to be flagged as ‘at-risk’ places a greater demand on midwives to decide whether psychosocial referral is appropriate and as such relies more heavily on the presence of staff who are highly skilled in integrated psychosocial assessment. The midwives at the participating site were trained and highly confident in undertaking routine psychosocial assessment, resulting at least in part from workplace culture that has prioritise routine perinatal mental health care consistently over the last 20 years. However, we acknowledge that this is not necessarily the case in all maternity settings nationally and internationally, thus the more directive referral prompts that are central to the PIPA model may be useful for less well-trained midwives or in less resourced maternity settings. Furthermore, the impact of clinical competencies on the effectiveness of the model remains to be determined [
The mean cost of integrated psychosocial assessment and depression screening was only marginally lower in the PIPA models ($13.63) than the Usual-Care model ($14.38), but the PIPA model resulted in more correct referrals (True Positive cases) than the Usual-Care model and less incorrect referrals (False Positive cases) than the Usual-Care model. The cost savings per correct referral under the PIPA model compared to the Usual-Care model was $3.97, and per incorrect referral avoided was $5.80. Future studies that examine whether these results are an underestimate of the true cost savings are required, as this study only captured costs from the initial antenatal booking-in appointment to the Triage Committee meeting and did not capture costs associated with any subsequent or unnecessary engagements with mental health and social work clinicians.
The Usual-Care and PIPA models flagged 37% and 27% of women as ‘at-risk’, respectively, which largely reflect the range of values reported in other perinatal universal integrated psychosocial assessment studies. Most studies, like ours, have used a combined depression screener and structured psychosocial questionnaire to identify which women warrant a referral for further assessment. In a relatively disadvantaged Australian sample, Matthey et al. [
The use of the antenatal psychosocial health assessment (ALPHA) tool in the detection of psychosocial risk factors for postpartum depression: a randomized controlled trial.
] in Canada reported 26% and 36% respectively of their pregnant sample had significant psychosocial risk depending on how psychosocial risk was identified by maternity staff: at routine consult or using a structured assessment tool. Honikman et al. [
], in a South African sample referred 32% of their sample for counselling after routine integrated psychosocial assessment. Clearly variation of prevalence rates for ‘at-risk’ women assessed in this way, is to be expected depending on sample characteristics, how psychosocial risk is defined and the detection approach used (e.g., structured questionnaire versus clinical interview, and whether an additional ‘filter’ such as the clinical judgement of the midwife, is included in protocols for generating an ‘at-risk’ flag). While the rates of psychosocial risk found in our study appear to be in line with other populations, the performance of clinical tests is partially dependent on prevalence of the condition in question, and thus the generalisability of our findings to other antenatal populations should be confirmed before adoption of this approach more broadly, is planned.
There have been calls for studies which establish an optimal model of administering routine perinatal depression screening in conjunction with broader psychosocial assessment and which rigorously examine the effectiveness of electronic algorithms to support the early identification of possible depression or elevated psychosocial risk [
]. This study responds to these evidence gaps through its dual examination of clinical performance and cost-effectiveness of two alternative models of integrated psychosocial care. Its use of ‘real world’ psychosocial assessment, electronic decision aids and referral data overcomes some of the limitations associated with the use of simulated data in previous studies [
This study has a number of limitations. It was a cohort study rather than a randomised controlled trial. While randomised trials may represent a superior design, the risk of contamination between groups in the real world setting central to this study was high. There is also the possibility that the sequential nature of the data collection (2015 and 2017) may have resulted in differences in midwives’ experience with integrated psychosocial assessment. Midwives were used as the reference standard against which the performance of the ‘at-risk’ flag was assessed. The study was conducted at one site only, restricting the generalisability of our findings to other settings.
In this study, only an intermediate measure of cost-effectiveness of the PIPA and Usual-Care models were performed. That is, an assessment of the performance of the models according to midwives’ opinion, and the cost-effectiveness of False Positive and True Positives results. This was in accordance with the published protocol [
]. It should be also noted that both models included clear care pathways for women with subclinical issues, including attendance at antenatal psychoeducational groups, midwife monitoring and repeat administration of the EPDS for women with elevated but subthreshold scores at the booking-visit. To perform a comprehensive cost-effectiveness analysis, quantification of health-related quality of life, generally measured by Quality Adjusted Life Years, would need to be estimated for the complete cohort of women who completed an integrated psychosocial assessment, either through a series of multi attribute utility instrument surveys at different time points, or using estimates from the literature which are few and not directly relevant to this study.
Finally the cost-effectiveness analysis focused solely on screening and triage activities and did not account for other costs associated with the two models. In particular False Negatives have a cost in terms of untreated morbidity, and that cost could not be accounted for in this study. Ideally a replication study that includes longer term follow-up data would allow additional examination of performance of the PIPA model in terms of clinical outcomes.
5. Conclusion
This study has extended the existing evidence base by examining both the clinical and cost-effectiveness of two integrated models of integrated psychosocial assessment and depression screening embedded within a primary care maternity setting. Assuming the slightly lower sensitivity of PIPA is clinically acceptable, the PIPA model of care would be a cost-saving and effective approach for detecting the True Positive cases and averting False Positive cases for integrated psychosocial assessments and depression screening by midwives in hospital maternity settings. However, because the midwives’ assessment of risk during screening was used as the reference standard, meaning that True and False Negatives could not be determined, the overall cost-effectiveness and health outcomes of women screening by the PIPA model and Usual-Care could not be calculated.
Studies which address overall cost-effectiveness of perinatal integrated psychosocial assessment programs, that incorporate longer-term health outcomes, are warranted.
Conflict of interest
The authors have no financial or other conflict of interest to declare.
Ethical statement
The research on which this manuscript is based involved human research. We used administrative data routinely collected by midwives when women first present for antenatal care for this study and thus individual consent was not required. Permission to use this administrative data for our study was granted by the South Eastern Sydney Local Health District Human Research Ethics Committee and Research Governance Office (SESLHD HREC Ref: 14/117; SSA Ref: 14/336) on the 23 October 2014. The scientific value, methodological value and safety of the research has been reviewed and approved by the SESLHD HREC, Scientific Review Sub-Committee.
Funding
No direct research funding was received for this study. MPA, NR and EB thank St John of God Health Care for infrastructure funding. NR thanks Australian Rotary Health and the University of Newcastle (2018–2020) and the University of Wollongong (current) for postdoctoral fellowship funding support. DK holds the Lois Hole Hospital for Women Cross-Provincial Chair in Perinatal Mental Health and a Canadian Institutes of Health Research New Investigator Award. The funding bodies had no involvement in the design of this study, analysis or interpretation of data, writing of the manuscript or decision to submit the article for publication.
We gratefully acknowledge the contributions of Virginia Spear in the weekly extraction of routinely collected administrative and clinical data and for her ongoing support in the implementation of the PIPA Project. We sincerely thank Catriona Andronicus and Dinaz Kaithakulathil for preparing the scripts and contributing to the testing of the PIPA algorithm. We also thank Christine Marsh and the team at Meridian Health Informatics for their support in embedding the PIPA model in the NSW Health eMaternity database. Sincere thanks to RHW management, in particular Helen Jarman, Vanessa Medunic and Alyce Finch, and Nursing Unit Managers Jo Arkwright, Michele Pezzuti and Rebecca Moore, for their support of the PIPA Project. We thank the midwives and other clinicians and administrative staff who have generously given their time and expertise to this project. We also extend thanks to Drs Matthew Holt and Rohini Vasudevan for their contribution to data collection for the economic component, and Victoria Mule for her thorough editing of this manuscript. Finally, we gratefully acknowledge the co- investigators on the larger PIPA project (Professor Virginia Schmied, A/Professor Stephen Matthey and Dr Andrew Bisits).
References
Howard L.M.
Molyneaux E.
Dennis C.-L.
Rochat T.
Stein A.
Milgrom J.
Non-psychotic mental disorders in the perinatal period.
The national perinatal depression initiative: an evaluation of access to general practitioners, psychologists and psychiatrists through the Medicare Benefits schedule.
Mental health consultations in the perinatal period: a cost-analysis of Medicare services provided to women during a period of intense mental health reform in Australia.
Psychosocial assessment and depression screening in the perinatal period: benefits, challenges and implementation.
in: Sutter-Dallay A.L. Glangeaud-Freudenthal N. Guedeney A. Riecher-Rössler A. Joint Care of Parents and Infants in Perinatal Psychiatry. Springer International Publishing,
Cham2016: 167-195
The use of the antenatal psychosocial health assessment (ALPHA) tool in the detection of psychosocial risk factors for postpartum depression: a randomized controlled trial.
Accuracy of the Edinburgh Postnatal Depression Scale (EPDS) for screening to detect major depression among pregnant and postpartum women: systematic review and meta-analysis of individual participant data.
Supporting routine psychosocial assessment in the perinatal period: the concurrent and predictive validity of the Antenatal Risk Questionnaire-Revised.
00086-X&id=gr1.jpg){kind=link}
00086-X&id=gr2.jpg){kind=link}