Skip to main content
Download PDF
Download ePub

The Refugee-Mental Health Assessment Package (R-MHAP); rationale, development and first-stage testing amongst West Papuan refugees

International Journal of Mental Health Systems volume 9, Article number: 29 (2015) Cite this article

Background

Researchers and practitioners are concerned with a range of inter-related factors that impact on the mental health of refugees, including the traumatic events (TEs) of conflict, conditions of post-migration adversity, and the broader psychosocial disruptions that these societies have experienced. In addition, there is growing recognition that refugees are at risk of a wide range of common mental disorders that include but extend beyond the commonly studied category of post-traumatic stress disorder (PTSD). There is a need therefore to establish an integrated and comprehensive package of measures to assess all elements of relevance to the mental health of refugees. This report describes the rationale, design, development and first stage testing of the four modules (TEs, ongoing adversity, psychosocial impacts, and mental disorder categories/reactions) of the Refugee Mental Health Assessment Package (R-MHAP), drawing on data from a survey of West Papuan refugees (n = 230) residing in Port Moresby, Papua New Guinea.

Methods

We detail the sequence of qualitative (based on informant interviews, focus groups) and psychometric tests undertaken in the development of the four modules.

Results

Preliminary modules were adapted according to our qualitative findings. Psychometric testing of the relevant modules yielded coherent factorial structures and demonstrated sound convergent and discriminant validity. For TEs, the factors comprised conflict-related experiences, traumatic losses, witnessing murder, childhood adversities, and health stress; for ongoing adversity, factors included deprivation of basic needs, difficulties accessing health care/health problems, displacement, and conditions of safety in the community; and for psychosocial impacts, factors included threats to safety; ruptured relationships; injustice related to past human rights violations; challenges to identity; and existential meaning (related to expression of political aspirations). There was strong agreement in assignment of cases of mental disorder (as a composite grouping) between the R-MHAP administered by field workers and structured clinical interview conducted blind by a psychologist. Symptom means for the majority of diagnostic categories remained stable over a 6-month period.

Conclusions

Our first-stage assessment suggests that the R-MHAP holds promise as a comprehensive assessment package for use in the refugee mental health field.

Background

The research field of refugee and post-conflict mental health spans diverse areas of interest, including exposure to the traumatic events (TEs) of war, conditions of post-migration adversity, the broader psychosocial changes following conflict and forced migration, and a range of common mental disorders. There is a need for a comprehensive package of measures that incorporates key indices of relevance to researchers and practitioners working in the field worldwide. We report the design, development, and first-stage testing of the four modules comprising the Refugee-Mental Health Assessment Package (R-MHAP), drawing on data gathered from West Papuan refugees living in Papua New Guinea (PNG).

The establishment of a common package of assessment modules will facilitate closer evaluation of key questions that remain to be answered in the field, for example, regarding the relative roles of trauma exposure and ongoing adversity in determining mental health outcomes [1], the contributions of disruptions to the broader psychosocial context to mental health [2], and the range and salience of common mental disorder categories that arise from these experiences [3]. An important challenge in devising measures is to integrate the universal experiences of refugees across contexts and the specific historical and cultural factors that are particular to each society. In devising a set of measures, it is important to provide guidance to future researchers concerning the broad procedures to follow in order to adapt the package to other cultures and contexts.

In devising our trauma event (TE) module, we draw on the legacy of research in the traumatology literature to include events commonly experienced by refugees worldwide, such as exposure to human rights violations (torture and other forms of interpersonal violence), witnessing murder and intentional harm to others, lack of access to emergency medical care, gross deprivations (shelter, food, water), and in many countries, natural disasters [4, 5]. In addition, we draw on qualitative data to supplement this list with the context-specific experiences relevant to the West Papuan community.

In relation to ongoing forms of adversity in the post-migration environment, past studies have identified a relatively consistent set of stressors across refugee populations [1, 2, 6–9], although the specific context influences the nature and salience of particular types of hardship [10]. As with TEs, we therefore include both universal as well as context-specific stressors in our measure.

A novel aspect of our package is the inclusion of an index of the broader psychosocial changes resulting from conflict and displacement. In devising this module, we draw on the Adaptation and Development after Persecution and Trauma (ADAPT) model, a conceptual framework that identifies five interrelated psychosocial “pillars” or domains that are disrupted by mass conflict and displacement: safety and security; interpersonal bonds and networks; justice (in relation to past and ongoing human rights violations); roles and identities; and existential beliefs and systems of meaning [11, 12]. The ADAPT model assesses the longer-term changes in the eco-social context over time and place. In contrast, measures of TE exposure and adversity represent specific, proximate experiences that pose an immediate source of threat or stress.

Based on a growing consensus, we include the mental disorders and reactions that are relevant to the refugee experience, including but extending beyond post-traumatic stress disorder (PTSD) and depression [3, 13]. Constraints in time, resources and skills generally preclude the use of lengthy and complex diagnostic interviews in field studies amongst refugee populations. A further challenge is the extent to which measures reflect international diagnostic constructs and criteria as opposed to indigenous idioms of distress. We explicitly based our measure on contemporary international classification systems to ensure broad comparability with studies undertaken in other settings. The provisional taxonomy and diagnostic criteria are drawn from the Diagnostic and Statistical Manual for Mental Disorder, fourth (DSM-IV) and fifth (DSM-5) editions of the American Psychiatric Association [14, 15] and the International Classification of Diseases (ICD), both the tenth edition and forthcoming eleventh edition [16, 17]. We use qualitative techniques to ensure that the diagnostic categories applied are appropriate to the context and culture, drawing on feedback from psychiatrists working in the setting (PNG). In addition, we undertake individual interviews and focus groups to ensure that symptoms are meaningful and salient to West Papuans residing in the specific context. In future developments of the R-MHAP, we intend to focus on developing modules that assess indigenous or emic constructs of mental distress.

In designing the R-MHAP, we give particular attention to the overall structure of the package, particularly the sequencing and formatting of modules, the aim being to ensure ease of administration in a transcultural context where field workers have received limited training and where respondents may have low levels of literacy and education [18]. In addition, we adopt a similar format across modules to maximize accuracy of administration, comprehensibility, and fidelity of responses amongst participants.

In summary, the objective of this report is to outline the sequence of qualitative and quantitative steps applied in the development and validation of the four modules (TEs, ongoing stressors, psychosocial impacts of conflict represented by the ADAPT index, and mental disorders/reactions) comprising the Refugee-Mental Health Assessment Package (R-MHAP), testing the properties of the modules amongst a sample of West Papuan refugees residing in Port Moresby, PNG.

Methods

Preparatory literature review

We used a rational expert-driven approach informed by existing reviews [19] and our extensive knowledge of the field to search for assessment packages that included at least two of the four intended modules. Based on existing reviews, we extended our search using the terms (alone and in combination) “mental health”, “mental disorder”, “psychiatric/psychological disorder”; “post-traumatic stress”, “depression”, “anxiety”, “panic”, “complicated and/or prolonged grief”, “adult separation anxiety”, “somatization and/or somatic symptoms”, “intermittent explosive disorder and/or explosive anger”, and “psychosis”, limiting results to adults (aged over 18 years), studies based on refugees and/or post-conflict populations, and articles published in English up to the present. Search results obtained from PsycInfo, PubMed, Medline and Scopus revealed only one widely used and cited package comprising the Harvard Trauma Questionnaire [5] and its companion measure, the Hopkins Symptoms Checklist [20]. Our search therefore confirmed the need to develop an expanded package of measures.

Baseline sample

Our sample (n = 230) of West Papuan refugees resided in Port Moresby, PNG. Because there are no general population census data identifying West Papuans in Port Moresby and this minority is dispersed within a larger population of PNG nationals, we applied a key informant, targeted sampling approach which involved two phases. First, we conducted an extensive process of location mapping and qualitative data gathering over three preparatory visits by the Australian-based team, each lasting approximately 3 weeks (May–December 2010).

Six settlements in Port Moresby, Hohola, Rainbow, Six-Mile, Eight Mile, Nine-Mile, and Tokarara/Waigani, were identified based on a desk top review of the relevant literature concerning West Papuans living in PNG and informant data provided by local government officials (the PNG Department of Health and Foreign Affairs) and the United Nations High Commissioner for Refugee (UNHCR) office which had undertaken an extensive mapping exercise of this community. Triangulation of available information indicated that the identified six settlements housed an estimated 90% of West Papuan refugees residing in the city. The settlements are geographically defined communities characterised by high density, makeshift housing, and few facilities. Based on all sources, we established a conservative estimate of 250 adult West Papuans (men and women) at the time of the survey.

In the next step, we mapped the locations of the refugees in these settlements based on information provided by key informants and supplemented by door-to-door inquiries covering the entire resident population in these locations. We were not able to contact 20 of the 250 eligible respondents, the most common reason being that the person had relocated to other areas of Port Moresby or further afield and could not be contacted. The final sample therefore comprised 230 adults (response rate 92% of the identified pool). Characteristics of the study sample are reported in Table 1.

Table 1 Sociodemographic characteristics of the study sample (n = 230)
Full size table

Follow-up sample

A follow-up study was conducted based on a subsample selected from the baseline study comprising roughly equal numbers of persons who met criteria for at least one of the mental disorders (n = 42) assessed and the remainder who had no mental disorder (non-cases) (n = 57). Of the 120 participants in this stratified sample we approached at follow-up, 102 agreed to participate (85% response). The main reason for non-response was that we were not able to make contact with past participants because they were away or had relocated. The follow-up study was completed between September and October, 2012. The average time from baseline to follow-up interview was 6.5 months.

Ethics statement

Ethical permission for the study was provided by the University of New South Wales Human Research Ethics Committee (HREC11273) and the Medical Research Council of PNG Ethics Committee (MRAC12.27). All respondents gave written or oral consent to participate in both the baseline and follow-up studies. Interviews were conducted in a private location or within the home of the respondent, depending on their preference.

Field personnel training

The in-country expatriate team operating in PNG consisted of two male researchers (one West Papuan research assistant, MK, who has extensive contacts with the community in Port Moresby, the other, AT, a bilingual Bahasa-speaking psychologist experienced in transcultural and refugee research). A team of five West Papuans employed as field workers received 3 weeks’ of interviewer training conducted by the expatriate team prior to the commencement of the community survey (January–May, 2013). The field interviewers included two female and three male West Papuans (mean age = 35) who were selected based on consultations with community leaders to ensure we included a broad representation of competent workers from different settlements with a spread of gender and ages. The interviewers worked in pairs over the course of the study. Prior to the follow-up study, we conducted an additional 1-week training with field workers to ensure their competence in applying the revised study protocol.

Translation and back-translation

Provisional modules were translated and back-translated from English to Bahasa Indonesian by three bilingual Bahasa-Indonesian speaking researchers and a psychologist following established guidelines [21, 22]. Our research personnel were proficient in the relevant local languages (Bahasa Indonesian, Pinyin), conducting interviews in the respondent’s preferred language throughout the study period.

Development of R-MHAP

The development of the package of measures was undertaken in consecutive phases commencing with desk-based and team work to prepare the preliminary modules, followed by qualitative and then quantitative studies. Other established measures used to supplement the R-MHAP modules are listed in Table 2.

Table 2 The Refugee-Mental Health Assessment Package (R-MHAP) and recommended supplementary measures
Full size table

Desk and team-based development of preliminary modules of TEs, stressors, ADAPT index, and mental disorders/reactions

Our preliminary list of TEs and ongoing stressors were based on the general literature, our own prior research in the region and provisional “blueprint” lists derived from existing measures such as the Harvard Trauma Questionnaire (HTQ) [5] and the HESPER [23]. Universal dimensions identified for TEs were conflict-related experiences, witnessing murder, traumatic losses, access to emergency medical care; and for ongoing stressors, extreme poverty, access to basic needs, health care, displacement, safety in the community [4, 8, 27, 28]. These lists were adapted and refined based on our prior knowledge of the history and conditions that prevailed amongst the West Papuan community and our qualitative work with community members.

The list of items measuring psychosocial adaptive changes was generated by consensus within the team based on our long familiarity with the five “pillars” that are considered at the core of the ADAPT model.

The mental disorder categories/reactions (defined by DSM-IV, DSM-5, ICD-10, and the proposed ICD-11 criteria) supported by the literature included PTSD [29–31], depression [31], generalized anxiety disorder [31], panic disorder [32, 33], somatic symptom disorder [34], complicated grief [35], separation anxiety disorder [36], psychotic symptoms [30, 31], and intermittent explosive disorder [37]. We also included persistent complex bereavement disorder, a newly proposed but provisional DSM-5 diagnosis. Participants were asked to respond to all items irrespective of whether or not they screened positive on the initial screening items.

Qualitative procedure

Contextual adaptation We consulted four experienced local psychiatrists (including the chief psychiatrist of PNG, the Mental Health Advisor at the Department of Health, and two academics at the Department of Psychiatry, Port Moresby General Hospital) over four formal consultation sessions to assess the face validity and practical utility of the nine selected mental health categories/reactions. The majority of expert informants were Melanesian in background, the broad cultural/ethnic grouping to which West Papuans belong. The focus of interviews was to assess whether the diagnostic categories we intended to include in our module had face validity and practical utility within the culture and context.

Contextual adaption of the modules was based on information obtained from five individual interviews and two focus groups (10 participants in each, mean age = 37 years) involving members of the West Papuan community. We purposively selected participants to ensure inclusion of persons including leaders and general members of the community with a range of ages and gender (five men and five women in each focus group).

Interviews and focus groups were conducted by a clinical psychologist fluent in Bahasa Indonesian, the lingua franca of the community, and a West Papuan research assistant. Facilitators of focus groups adopted an explicitly non-judgmental approach in which past experiences and current symptoms were contextualized according to the culture, history and current living circumstances of the community. Facilitators explicitly aimed to reduce any sense of stigma or shame related to revealing symptoms of distress by avoiding any inference that disclosures by participants indicated the presence of mentally illness, an approach that proved successful in encouraging sharing of individual experiences. Lack of familiarity with western diagnoses of mental disorder is likely to have reduced further any sense of stigma when West Papuans discussed their own experiences. However, where necessary, feedback sessions were undertaken in private with individual members to discuss and address personal issues of concern that may have arisen from information revealed in focus groups.

Qualitative data were gathered, analysed and integrated with quantitative data following established guidelines [22, 38].

Quantitative data and statistical analysis

Psychometric and other quantitative analyses were conducted on data from the full sample at baseline and on the stratified sample at follow-up. TE and stressor items were analysed as dichotomous variables. Items of the ADAPT module were dichotomized by assigning a summary score of 1 for any item rated 2 (very strongly) or 3 (extremely) (see hereunder for details of scoring). Preliminary latent structures underlying the provisional lists of items comprising TEs, stressors, and the ADAPT module were specified according to theoretical considerations and the findings of prior research. The preliminary structures were then subjected to psychometric testing using Confirmatory Factor Analysis (CFA).

The Mean-adjusted Weighted Least Square method (WLSMV) was used in all analyses [39, 40], as an established statistical procedure in applying CFA to dichotomous variables, an approach applied extensively in past studies [41, 42]. Standardized factor loadings (based on probit regression coefficients) and R2 (the proportion of variance explained in the latent response variables underlying each CFA model) were calculated. Where we were able to demonstrate a good fit for a first-order factorial structure, we examined further for a second-order factor as a test of the unitary nature of the overall construct. Cross-loading items (with a factor loading ≥3.0) and items with overlapping content were eliminated sequentially until we achieved an adequate model fit. We judged model fit based on a suite of indicators including a non-significant Chi square test, the Root Mean Square Error of Approximation (RMSEA) (<0.05 indicates a good fit), the Tucker Lewis Index (TLI), and Comparative Fit Index (CFI) (>0.9 indicates a good fit for the latter two indicators) [43–45]. Internal reliability of relevant item pools was examined using the Kuder-Richardson (KR20) reliability index recommended for dichotomous items.

Convergent and discriminant validity were examined in a series of correlated uniqueness (CU) multitrait multimethod (MTMM) models. The correlated residuals are assumed to represent the effects of the measurement method [46, 47]. Convergent validity was assessed by estimated trait (factor) loadings and discriminant validity by cross-factorial correlations [46].

Descriptive statistics were calculated for mental disorder symptom scores (summation of all items for each disorder) to allow comparisons between baseline and follow-up amongst the structured subsample drawn from the baseline sample. Our analysis of concordance data comparing mental disorder diagnoses on the SCID with the field measure (described in full hereunder) applied the Area under the Curve (AUC), sensitivity, specificity, and positive and negative predictive values based on a Receiver Operating Characteristic (ROC) analysis [48, 49].

Quantitative analyses were performed using STATA version 13 [50] and Mplus version 7 [51].

Results

Qualitative findings examining transcultural validity of modules

Face validity of nine selected mental health categories/reactions

All psychiatrists endorsed the alignment between the categories of mental disorder selected and locally recognized mental health responses within the Melanesian context indicating that they applied all diagnoses in their routine practices (further details are provided in Additional file 1). The consensus achieved amongst these expert informants offered some evidence in support of the face and content validity as well as the clinical utility of these constructs within the context.

Content relevance

Our observations derived from individual interviews and focus groups supported the cultural acceptability of the R-MHAP modules amongst respondents. The focus group responses indicated that the translated modules demonstrated strong contextual validity. For example, our provisional list of conflict-related TEs, derived from both universal (torture, witnessing murders) and context-specific (displacement, sexual violence) were readily recognized as major events to which many West Papuans had been exposed. Symptoms of key mental disorder also were readily recognized and endorsed.

Linguistic equivalence (correct interpretation of items)

Respondents were asked whether translated items “made sense” and would be readily understood by the community. Items that participants found challenging in their meaning or in ease of interpretation were removed or reworded based on feedback from the focus groups.

Technical equivalence (appropriateness of response scale)

Respondents were asked to judge the ease of response to the scoring format for items. Based on recommendations from the focus groups, we changed the provisional frequency-based response scale to one of severity (for the ADAPT index: 0 = none, 1 = moderately, 2 = very strongly, 3 = extremely), which was uniformly regarded as easier to respond to compared to the previous format.

Completeness (equivalence of construct)

Completeness was examined by the extent to which items in each module reflected the underlying constructs we sought to study. This was achieved by iteratively inquiring whether participants could think of additional items or experiences that offered to expand the modules. Where there was a consensus on the addition, removal or alteration of items, modifications were made to the relevant modules.

Quantitative results

Traumatic events

Table 3 shows endorsement rates and factor loadings for the 23 TE items (derived following our qualitative techniques from the original pool of 48 items). 129 (56%) participants reported experiencing at least one listed TE. Both first- and second-order CFA models yielded a good fit (first-order: χ2 [220] = 241.87, P = 0.149, CFI = 1, TLI = 1, RMSEA = 0.021; second-order: χ2 [199] = 227.20, P = 0.083, CFI = 0.998, TLI = 0.998, RMSEA = 0.025). The five dimensions yielded by CFA included conflict-related experiences, traumatic losses, witnessing murders, childhood-related adversities, and health stress. The subscale items demonstrated good internal consistency as indicated by KR20 coefficients: 0.95 (conflict-related experiences); 0.93 (traumatic losses), 0.80 (witnessing murders); 0.74 (childhood adversities) to 0.92 (health-related stress). The CU model demonstrated a good fit (χ2 [163] = 162.85, P = 0.48, CFI = 1, TLI = 1, RMSEA = 0.00). Convergent validity was supported by a high level of factor loadings (0.87–1.00) and low discriminant validity (0.60–0.94), indicating that the trauma domains were correlated with one another.

Table 3 Confirmatory Factor Analysis of the constituent items of the module of common Traumatic Events (TEs)
Full size table

Ongoing stressors

Table 4 shows the prevalence and standardized loadings for the pool of 13 stressor items. CFA based on the item pool yielded four dimensions: basic survival needs, access to health care; separation from families and displacement from home, and safety issues in the community (χ2 [59] = 64.21, P = 0.29, CFI = 1, TLI = 1, RMSEA = 0.020.) The second-order model also demonstrated a good fit with the data (χ2 [61] = 63.42, P = 0.39, CFI = 1, TLI = 1, RMSEA = 0.013). The subscale items demonstrated sound internal consistency indicated by the KR20 coefficients: 0.93 (basic survival needs); 0.85 (access to health care/health problems), 0.71 (separation from families and displacement) to 0.81 (safety in the community). The CU model exhibited a good fit (χ2 [48] = 47.07, P = 0.309, CFI = 1, TLI = 1, RMSEA = 0.02) supported by sound convergent (factor loadings = 0.94–1.00) and discriminant validity (cross-factorial correlations = 0.61–0.77).

Table 4 Confirmatory Factor Analysis of the constituent items of post-traumatic ongoing stressors
Full size table

The ADAPT index of psychosocial impacts of conflict and displacement

Table 5 presents endorsement rates and factor loadings for the 22 ADAPT items. 163 (69.5%) respondents reported at least one item from each ADAPT domain.

Table 5 Confirmatory Factor Analysis of the constituent items of the ADAPT index of psychosocial impacts of conflict based on the Adaptation and Development After Persecution and Trauma (ADAPT) model
Full size table

CFA yielded five dimensions corresponding to the ADAPT domains (χ2 [199] = 227.20, P = 0.083, CFI = 0.998, TLI = 0.998, RMSEA = 0.025). Items constituting each domain demonstrated sound internal consistency as indicated by the KR20 coefficient: 0.88 (safety/security); 0.94 (bonds/networks), 0.92 (access to justice); 0.89 (roles/identities) to 0.84 (existential meaning/political aspirations). The CU model indicated a good fit (χ2 [165] = 161.65, P = 0.56, CFI = 1, TLI = 1, RMSEA = 0.00) supported by sound convergent (factor loadings = 0.88–1.00) and discriminant validity (cross-factorial correlations = 0.56–0.78).

Diagnostic concordance in psychiatric case assignment by trained field workers

We tested diagnostic concordance between the lay-administered mental disorder assessment and the Structured Clinical Interview for DSM-IV (SCID) applied in a blinded manner by an expatriate psychologist (AT) [52]. Since the SCID had not yet been modified for DSM-5, we added relevant items to allow diagnoses to be made according to that system as well as DSM-IV. As indicated, we drew a stratified subsample from the baseline study of 102 persons comprising 59 men (57.8%) and 43 women (42.1%) with a mean age of 42.51 (SD = 1.63), including approximately equal numbers with or without a mental disorder based on the baseline field module. No significant differences in gender, age, or length of settlement were detected between participants with and without a mental disorder.

The SCID and the field module concurred in assigning 42 (41.1%) participants as cases (presence of one or more current disorder), and 57 (55.8%) as non-cases. Of the 3 discordant case assignments, 2 were identified as cases by the field measure and 1 by the SCID respectively. The ROC analysis (cases versus non-cases) yielded an AUC of 0.93 (95% CI 0.87–0.98); a sensitivity of 0.98 (CI 0.88–1.00); a specificity of 0.97 (CI 0.88–1.00); a positive predictive power of 0.95 (CI 0.85–0.99) and negative predictive power of 0.98 (CI 0.91–1.00). The overall correct classification was 0.97 (CI 0.92–0.99), the intra-class correlation being 0.94 when comparing case assignments across the two methods.

The follow-up study also allowed examination of the predictive validity of the field module for mental disorder/distress. We found that 69% (n = 42) of 61 cases identified at baseline remained cases at follow-up. In comparison, only 13.6% (n = 8) of 59 baseline non-cases converted to cases at follow-up. We computed summary symptom scores for each diagnostic category based on the addition of all symptom items (1 for endorsed) for each diagnosis (noting that all participants completed the full diagnostic modules irrespective of whether or not they met the screening criteria for a category). In all instances, scores for cases were significantly higher than those for non-cases at both baseline and follow-up. Amongst cases, pairwise t-tests show no statistically significant change in mean symptom scores for each of the mental disorder categories from baseline to follow-up, with the exception of panic disorder where there was improvement over time (Table 6).

Table 6 Follow-up analysis of change in symptom scores of mental disorder categories/reactions between cases and non-cases at baseline and follow-up
Full size table

Discussion

To our knowledge, the R-MHAP is the only contemporary integrated package of measures in the field of refugee and post-conflict mental health that is specifically designed to incorporate key concepts and indices of relevance to researchers and practitioners worldwide. The conceptualization and development of the constituent modules was based on a thorough consideration of the relevant literature, drawing on the fields of psychiatric traumatology, clinical psychology, human rights, social sciences, and transcultural studies. Other established measures recommended to complement the R-MHAP are indicated in Table 2. Our qualitative and quantitative findings offer substantial support for the contextual appropriateness (in relation to West Papuan refugees) and psychometric properties of four core modules of the R-MHAP, including TEs, ongoing stressors, psychosocial impacts of conflict, and a range of mental disorders. Our findings provide support for the procedural validity of the R-MHAP in that trained lay interviewers with no background knowledge of international mental health constructs were able to administer the package effectively with a high level of adherence and completion by respondents.

Strengths of the study include the sound psychometric properties identified for the constituent modules, findings that set the stage for further testing of the R-MHAP amongst other populations of refugees. The design of the R-MHAP gives emphasis to the principle of coherence in the sequencing of topics, as well as ensuring procedural simplicity, consistency and ease of administration [53]. We have outlined the steps that can be followed by future researchers to adapt key indices (particularly TEs and stressors) to each context.

Limitations of our inquiry include the sample size, a natural constraint that required us to aggregate mental disorders/disturbances in testing for convergence of case assignment against the SCID. The sample was heterogeneous, including refugees born in West Papua and offspring born in PNG. The advantage, however, was that we obtained a wide variation in responses which facilitated our statistical analyses. Future studies with larger samples will be able to investigate more closely the influence of demographic characteristics such as age, gender, and place of birth, by allowing application of more sophisticated statistical techniques such as multi-group factor analysis. Our testing of the modules is limited to West Papuan refugees, cautioning against generalizing our results to other refugee populations. Future studies are needed to test the contextual and psychometric properties of our modules across other refugee groups worldwide. The development of the module for mental disorders was based on international criteria, although qualitative data from our focus groups provided some support for the face validity of the mental disorder categories under investigation. Further in-depth interviews involving a diverse range of informants is needed for to validate the diagnostic module within this and other transcultural populations. In future developments of the R-MHAP, inclusion of modules for indigenous constructs of mental disorder will provide a more comprehensive account of the mental reactions relevant to each society under study [54].

The present study focused primarily on the core modules (covering trauma events, adversity, psychosocial effects of conflict, and common forms of mental disorder) of general importance to refugee mental health. The key steps we outlined in this study may be valuable in guiding field researchers and workers in the process of devising and adapting their own assessment tools.

We note that there invariably is a trade-off between cultural specificity/validity and comparability/reliability in developing mental health measures, particularly in transcultural contexts. Applying a strictly emic approach in which idioms of distress are derived de novo from the culture under study has the advantage of maximizing contextual validity but that approach inevitably forfeits comparability in relation to studies amongst other populations and settings. An approach limited to the translation and minimal adaptation of international measures such as the Composite International Diagnostic Interview (CIDI) has the advantage of increasing consistency and reliability of findings across cultures but at the potential expense of reducing the local validity of responses [55]. We elected to take an intermediary approach in which the original structure and provisional symptom pool of our diagnostic measure was derived from international classification systems prior to being subjected to extensive qualitative testing and adaptation to ensure the relevance and comprehensibility of underlying constructs and constituent items. In so doing, we attempted to achieve a balance between local validity and the potential for the package to be adapted and used in other cultural settings. We further note that it may not always be feasible to apply the package as a whole especially in settings of mass conflict or disasters. Limiting the scope of assessment to the core indices (e.g., traumatic events, ongoing adversities, the psychosocial effects of conflict, and common forms of mental distress) outlined in our study may prove valuable in medium-term planning and development for psychosocial and mental health services following complex emergencies [13].

In relation to our substantive findings, our TE module identified five domains (conflict-related experiences, traumatic losses, witnessing murders, childhood adversities, and health stress) that match closely the known history of West Papuan refugees, offering indirect support for the validity of the module. For example, consistent with the history, direct exposure to political conflict and experiencing homes being burnt down yielded the highest factor loadings on “conflict-related experiences” (factor 1) [56]. Our results are consistent with findings from previous studies amongst conflict-affected populations in the region, particularly neighbouring Timor-Leste, a territory also invaded and occupied by Indonesia [4, 8]. The five derived factors were predicted by a higher-order factor, supporting the use of either a composite score of TEs or individual scales in future studies.

Four dimensions of ongoing stressors were identified by our second-order CFA, comprising basic needs, access to health services, stress from prolonged displacement, and safety concerns in the community. Again, these domains assess experiences that are consistent with the contextual and historical background of the population, as identified in our preceding qualitative study [57]. There was substantial endorsement of most stressor items, which was consistent with observations of the extreme conditions of deprivation in the settlements in Port Moresby where poverty, lack of services and endemic violence represent major challenges [58, 59].

In relation to our index of psychosocial impacts, CFA identified five dimensions consistent with the ADAPT model, namely safety and security; interpersonal bonds and networks; justice; the integrity of roles and identities; and existential meaning (related to expression of political aspirations). The establishment of this module offers the potential to test the ADAPT model in future studies to assess the impact of broader psychosocial disruptions on mental health outcomes in addition to the effects of TEs and ongoing stresses experienced by refugees.

In testing our mental health assessment module, there was a high level of convergence between the field measure and the SCID in identifying aggregated cases. The few discordant cases were equally divided across the two measures, indicating that the field interview was not over-sensitive in identifying persons with mental disorder compared to the SCID, a common problem encountered in past field studies using screening measures [60]. The most likely reason for this finding is that our field measure followed a structured format that is consistent with a diagnostic interview conducted by clinicians. The predictive validity of the diagnostic module was supported by our findings that symptoms tended to persist over a 6 month period, suggesting that the mental disturbances identified were not simply manifestations of transient distress. Further support for the discriminant validity of our diagnostic module is that only a modest number of non-cases transitioned into becoming cases over six months, consistent with expectations.

Conclusions

The R-MHAP offers a comprehensive and integrated assessment package that covers a wide range of mental health and psychosocial indices of relevance to modern refugee research and practice. Our data offer support for the cultural validity and psychometric properties of the four modules that comprise the R-MHAP, as tested amongst West Papuans residing in Port Moresby. These outcomes were achieved by field workers previously unfamiliar with western mental health concepts who received limited training prior to the survey. Further testing of the R-MHAP is needed to support the validity, capacity for transcultural adaptation and general utility of the measure as a multidisciplinary research tool in the refugee and related fields. Electronic adaptation of the R-MHAP is in the early development phase and will be available for use in the near future.

References

  1. Miller KE, Kulkarni M, Kushner H. Beyond trauma-focused psychiatric epidemiology: Bridging research and practice with war-affected populations. Am J Orthopsychiatry. 2006;76(4):409–22.

    Article  PubMed  Google Scholar 

  2. Miller KE, Rasmussen A. War exposure, daily stressors, and mental health in conflict and post-conflict settings: bridging the divide between trauma-focused and psychosocial frameworks. Soc Sci Med. 2010;70(1):7–16.

    Article  PubMed  Google Scholar 

  3. Tol WA, Rees SJ, Silove DM. Broadening the scope of epidemiology in conflict-affected settings: opportunities for mental health prevention and promotion. Epidemiol Psychiatr Sci. 2013;22(3):197–203.

    Article  CAS  PubMed  Google Scholar 

  4. Silove D, Brooks R, Bateman CS, Steel Z, Amaral ZF, Rodger J, et al. Social and trauma-related pathways leading to psychological distress and functional limitations 4 years after the humanitarian emergency in Timor-Leste. J Trauma Stress. 2010;23(1):151–60.

    CAS  PubMed  Google Scholar 

  5. Mollica RF, Caspi-Yavin Y, Bollini P, Truong T, Tor S, Lavelle J. The Harvard Trauma Questionnaire: validating a cross-cultural instrument for measuring torture, trauma, and posttraumatic stress disorder in Indochinese refugees. J Nerv Ment Dis. 1992;180(2):111–6.

    Article  CAS  PubMed  Google Scholar 

  6. Silove D, Steel Z, Susljik I, Frommer N, Loneragan C, Chey T, et al. The impact of the refugee decision on the trajectory of PTSD, anxiety, and depressive symptoms among asylum seekers: a longitudinal study. Am J Disaster Med. 2007;2(6):321–9.

    PubMed  Google Scholar 

  7. Steel Z, Silove D, Bird K, McGorry P, Mohan P. Pathways from war trauma to posttraumatic stress symptoms among Tamil asylum seekers, refugees, and immigrants. J Trauma Stress. 1999;12(3):421–35.

    Article  CAS  PubMed  Google Scholar 

  8. Brooks R, Silove D, Steel Z, Steel CB, Rees S. Explosive anger in postconflict Timor Leste: Interaction of socio-economic disadvantage and past human rights-related trauma. J Affect Disord. 2011;131(1–3):268–76.

    Article  PubMed  Google Scholar 

  9. Jordans MJ, Semrau M, Thornicroft G, van Ommeren M. Role of current perceived needs in explaining the association between past trauma exposure and distress in humanitarian settings in Jordan and Nepal. Br J Psychiatry. 2012;201(4):276–81.

    Article  PubMed  Google Scholar 

  10. Silove D, Sinnerbrink I, Field A, Manicavasagar V, Steel Z. Anxiety, depression and PTSD in asylum-seekers: Associations with pre-migration trauma and post-migration stressors. Br J Psychiatry. 1997;170(APR):351–7.

    Article  CAS  PubMed  Google Scholar 

  11. Silove D. The psychosocial effects of torture, mass human rights violations, and refugee trauma: Toward an integrated conceptual framework. J Nerv Ment Dis. 1999;187(4):200–7.

    Article  CAS  PubMed  Google Scholar 

  12. Silove D. The ADAPT model: a conceptual framework for mental health and psychosocial programming in post conflict settings Intervention 2013, 11.

  13. Silove D, Bryant R. Rapid assessments of mental health needs after disasters. JAMA. 2006;296(5):576–8.

    Article  CAS  PubMed  Google Scholar 

  14. American Psychiatric Association. Diagnostic and statistical manual of mental disorders. Forth ed. Washington, DC: American Psychiatric Association Press; 1994.

    Google Scholar 

  15. American Psychiatric Association. Diagnostic and statistical manual of mental disorders. Fifth ed. Washington, DC: American Psychiatric Association Press; 2013.

    Google Scholar 

  16. The ICD-10 Classification of Mental and Behavioural Disorders. Diagnostic Criteria for Research. The ICD-10 Classification of Mental and Behavioural Disorders 1992.

  17. Maercker A, Brewin CR, Bryant RA, Cloitre M, van Ommeren M, Jones LM, et al. Diagnosis and classification of disorders specifically associated with stress: proposals for ICD-11. World Psychiatry. 2013;12(3):198–206.

    Article  PubMed Central  PubMed  Google Scholar 

  18. Mollica RF. The trauma story: a phenomenological approach to the traumatic life experiences of refugee survivors. Psychiatry. 2001;64(1):60–3.

    CAS  PubMed  Google Scholar 

  19. Hollifield M, Warner TD, Lian N, Krakow B, Jenkins JH, Kesler J, et al. Measuring trauma and health status in refugees: a critical review. JAMA. 2002;288(5):611–21.

    Article  PubMed  Google Scholar 

  20. Mollica RF, Wyshak G, de Marneffe D, Khuon F, Lavelle J. Indochinese versions of the Hopkins Symptom Checklist-25: a screening instrument for the psychiatric care of refugees. Am J Psychiatry. 1987;144(4):497–500.

    Article  CAS  PubMed  Google Scholar 

  21. Malterud K. Qualitative research: standards, challenges, and guidelines. Lancet. 2001;358(9280):483–8.

    Article  CAS  PubMed  Google Scholar 

  22. Tong A, Sainsbury P, Craig J. Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups. Int J Qual Health Care. 2007;19(6):349–57.

    Article  PubMed  Google Scholar 

  23. Semrau M, van Ommeren M, Blagescu M, Griekspoor A, Howard LM, Jordans M, et al. The development and psychometric properties of the Humanitarian Emergency Settings Perceived Needs (HESPER) Scale. Am J Public Health. 2012;102(10):e55–63.

    Article  PubMed Central  PubMed  Google Scholar 

  24. Babor TF, De la Fuente JR, Saunders J, Grant M: AUDIT—the alcohol use disorders identification test: guidelines for use in primary health care. AUDIT: The Alcohol Use Disorders Identification Test: Guidelines for Use in Primary Health Care 1989.

  25. Ustun TB, Chatterji S, Kostanjsek N, Rehm J, Kennedy C, Epping-Jordan J, et al. Developing the World Health Organization disability assessment schedule 2.0. Bull World Health Organ. 2010;88(11):815–23.

    Article  PubMed Central  PubMed  Google Scholar 

  26. Garcia-Moreno C, Watts C, Jansen H, Ellsberg M, Heise L. Responding to violence against women: WHO’s multicountry study on women’s health and domestic violence. Health Hum Rights. 2003;6(2):112–27.

    Article  Google Scholar 

  27. Rees S, Silove D, Tay AK, Moses K. Human rights trauma and the mental health of West Papuan refugees resettled in Australia. Med J Aust. 2013;199(4):280.

    Article  PubMed  Google Scholar 

  28. Rees S, Silove D, Verdial T, Tam N, Savio E, Fonseca Z, et al. Intermittent explosive disorder amongst women in conflict affected Timor-Leste: associations with human rights trauma, ongoing violence, poverty, and injustice. PLoS One. 2013;8(8):e69207.

    Article  CAS  PubMed Central  PubMed  Google Scholar 

  29. Steel Z, Chey T, Silove D, Marnane C, Bryant RA, Van Ommeren M. Association of torture and other potentially traumatic events with mental health outcomes among populations exposed to mass conflict and displacement: A systematic review and meta-analysis. JAMA J Am Med Assoc. 2009;302(5):537–49.

    Article  CAS  Google Scholar 

  30. Fazel M, Wheeler J, Danesh J. Prevalence of serious mental disorder in 7000 refugees resettled in western countries: a systematic review. Lancet. 2005;365(9467):1309–14.

    Article  PubMed  Google Scholar 

  31. Jones L, Asare JB, El Masri M, Mohanraj A, Sherief H, van Ommeren M. Severe mental disorders in complex emergencies. Lancet. 2009;374(9690):654–61.

    Article  PubMed  Google Scholar 

  32. Hinton D, Chau H, Nguyen L, Nguyen M, Pham T, Quinn S, et al. Panic disorder among Vietnamese refugees attending a psychiatric clinic: prevalence and subtypes. Gen Hosp Psychiatry. 2001;23(6):337–44.

    Article  CAS  PubMed Central  PubMed  Google Scholar 

  33. Beiser M, Fleming J. Meauring psychiatric disorder among Southeast Asian refugees. Psychol Med. 1986;16(3):627–39.

    Article  CAS  PubMed  Google Scholar 

  34. Van Ommeren M, Sharma B, Sharma GK, Komproe I, Cardeña E, De Jong JTVM. The relationship between somatic and PTSD symptoms among Bhutanese refugee torture survivors: examination of comorbidity with anxiety and depression. J Trauma Stress. 2002;15(5):415–21.

    Article  PubMed  Google Scholar 

  35. Momartin S, Silove D, Manicavasagar V, Steel Z. Complicated grief in Bosnian refugees: associations with posttraumatic stress disorder and depression. Compr Psychiatry. 2004;45(6):475–82.

    Article  PubMed  Google Scholar 

  36. Silove D, Momartin S, Marnane C, Steel Z, Manicavasagar V. Adult separation anxiety disorder among war-affected Bosnian refugees: comorbidity with PTSD and associations with dimensions of trauma. J Trauma Stress. 2010;23(1):169–72.

    PubMed  Google Scholar 

  37. Fincham D, Grimsrud A, Corrigall J, Williams DR, Seedat S, Stein DJ, et al. Intermittent explosive disorder in South Africa: prevalence, correlates and the role of traumatic exposures. Psychopathology. 2009;42(2):92–8.

    Article  PubMed Central  PubMed  Google Scholar 

  38. Fetters MD, Curry LA, Creswell JW. Achieving integration in mixed methods designs-principles and practices. Health Serv Res. 2013;48:2134.

    Article  PubMed Central  PubMed  Google Scholar 

  39. Muthen B. Contributions to factor analysis of dichotomous variables. Psychometrika. 1978;43:551–60.

    Article  Google Scholar 

  40. Muthen B, Kaplan AS. A comparison of some methodologies for the factor analysis of non-normal Likert variables. Br J Math Stat Psychol. 1985;38:171–89.

    Article  Google Scholar 

  41. Palmieri PA, Weathers FW, Difede J, King DW. Confirmatory factor analysis of the PTSD Checklist and the Clinician-Administered PTSD Scale in disaster workers exposed to the World Trade Center Ground Zero. J Abnorm Psychol. 2007;116(2):329–41.

    Article  PubMed  Google Scholar 

  42. Stein DJ, Rothbaum BO, Baldwin DS, Szumski A, Pedersen R, Davidson JR. A factor analysis of posttraumatic stress disorder symptoms using data pooled from two venlafaxine extended-release clinical trials. Brain Behav. 2013;3(6):738–46.

    Article  PubMed Central  PubMed  Google Scholar 

  43. Kline RB. Principles and practice of structural equation modeling. New York/London: The Guilford Press; 1998.

    Google Scholar 

  44. Browne MW, Cudeck R. Alternative ways of assessing model fit. Soc Methods Res. 1993;21(2):230–58.

    Article  Google Scholar 

  45. Barrett P. Structural equation modelling: Adjudging model fit. Pers Individ Dif. 2007;42:815–24.

    Article  Google Scholar 

  46. Prigerson HG, Bridge J, Maciejewski PK, Beery LC, Rosenheck RA, Jacobs SC, et al. Influence of traumatic grief on suicidal ideation among young adults. Am J Psychiatry. 1999;156(12):1994–5.

    CAS  PubMed  Google Scholar 

  47. Prigerson HG, Shear MK, Jacobs SC, Reynolds Iii CF, Maciejewski PK, Davidson JRT. Consensus criteria for traumatic grief. A preliminary empirical test. Br J Psychiatry. 1999;174(JAN):67–73.

    Article  CAS  PubMed  Google Scholar 

  48. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33(1):159–74.

    Article  CAS  PubMed  Google Scholar 

  49. Shrout PE, Spitzer RL, Fleiss JL. Quantification of agreement in psychiatric diagnosis revisited. Arch Gen Psychiatry. 1987;44(2):172–7.

    Article  CAS  PubMed  Google Scholar 

  50. StataCorp. Stata Statistical Software: Release 13. College station: StataCorp LP; 2013.

    Google Scholar 

  51. Muthén LK, Muthén BO. Mplus User’s Guide 1998. Sixth Edition. Los Angeles, CA: Muthén & Muthén

  52. Spitzer RL, Williams JBW, Gibbon M, First MB. The structured clinical interview for DSM-III-R (SCID): I: history, rationale, and description. Arch Gen Psychiatry. 1992;49(8):624–9.

    Article  CAS  PubMed  Google Scholar 

  53. DiCicco-Bloom B, Crabtree B. The qualitative research interview. Med Educ. 2006;40:314–21.

    Article  PubMed  Google Scholar 

  54. Rees S, Silove D. Sakit Hati: A state of chronic mental distress related to resentment and anger amongst West Papuan refugees exposed to persecution. Soc Sci Med. 2011;73(1):103–10.

    Article  PubMed  Google Scholar 

  55. Steel Z, Marnane C, Iranpour C, Chey T, Jackson JW, Patel V, et al. The global prevalence of common mental disorders: a systematic review and meta-analysis 1980–2013. Int J Epidemiol. 2014;43(2):476–93.

    Article  PubMed Central  PubMed  Google Scholar 

  56. Brundige E, King W, Vahali P, Vladek S, Yuan X. Indonesian Human Rights Abuses in West Papua: application of the Law of Genocide to the History of Indonesian Control. New Haven: Allard K. Lowenstein International Human Rights Clinic, Yale Law School; 2004.

    Google Scholar 

  57. Rees S, Silove D, Kareth M. Dua sakit (double sick): Trauma and the settlement experiences of West Papuan refugees living in North Queensland. Australasian Psychiatry. 2009;17(SUPPL. 1):S128–32.

    Article  PubMed  Google Scholar 

  58. Griva M: Border Refugees in PNG. In.; 2009.

  59. Merdeka FWPP: Villages of West Papuan refugees burn on PNG border-humanitarian crsis developing. 2011.

  60. Silove D, Manicavasagar V, Mollica R, Thai M, Khiek D, Lavelle J, et al. Screening for depression and PTSD in a Cambodian population unaffected by war: comparing the Hopkins Symptom Checklist and Harvard Trauma Questionnaire with the structured clinical interview. J Nerv Ment Dis. 2007;195(2):152–7.

    Article  PubMed  Google Scholar 

Download references

Authors’ contribution

AKT, SR, DS conceived study. AKT, MK performed the research. AKT performed and JC supervised the analysis. AKT, SR, DS, JC drafted and revised the manuscript. All authors read and approved the manuscript.

Acknowledgements

We thank the following contributors to this project: Mr Michael Kareth, Mr Paul Wandik, Mr Martinus Anari, Mr Freddy Waromi, Ms Dolly Songona, and Ms Olvianna Fonataba, Drs Goiba Tieneng (Chief Psychiatrist of PNG) and Uma Ambi (Principal Mental Health Advisor, Department of Health, PNG).

Compliance with ethical guidelines

Competing interests The authors declare that they have no competing interests.

Funding support

NHMRC program number: RM08333; South Western Clinical School Faculty of Medicine Scholarship.

Author information

Authors and Affiliations

  1. Centre for Population Mental Health Research, Liverpool Hospital and Psychiatry Research and Teaching Unit, School of Psychiatry, University of New South Wales, Cnr Forbes and Campbell Streets, Liverpool, NSW, 2170, Australia

    Alvin Kuowei Tay, Susan Rees, Moses Kareth, Mohammed Mohsin & Derrick Silove

  2. Psychiatry Research and Teaching Unit, School of Psychiatry, Liverpool Hospital, Ingham Institute, University of New South Wales, Sydney, NSW, 2052, Australia

    Alvin Kuowei Tay, Susan Rees, Moses Kareth, Mohammed Mohsin & Derrick Silove

  3. Simpson Centre for Health Services Research, University of New South Wales, Sydney, Australia

    Jack Chen

  4. South Western Sydney Clinical School, Liverpool Hospital, University of New South Wales, Sydney, Australia

    Alvin Kuowei Tay & Jack Chen

Authors
  1. Alvin Kuowei Tay
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Susan Rees
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jack Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Moses Kareth
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Mohammed Mohsin
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Derrick Silove
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Alvin Kuowei Tay.

Additional file

Additional file 1:

Qualitative data.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Tay, A.K., Rees, S., Chen, J. et al. The Refugee-Mental Health Assessment Package (R-MHAP); rationale, development and first-stage testing amongst West Papuan refugees. Int J Ment Health Syst 9, 29 (2015). https://doi.org/10.1186/s13033-015-0018-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s13033-015-0018-6

Keywords

International Journal of Mental Health Systems

ISSN: 1752-4458