Skip to Navigation | Skip to Content

PUBLIC HEALTH IN ACTION

Monitoring & evaluation in the real world:
Evaluating a psychosocial program for
Darfur refugees in eastern Chad

Andrew Rasmussen, Ph.D.
New York University School of Medicine
(See also Bellevue/NYU Program for Survivors of Torture)

 

Click here for PDF version.

BACKGROUND

Over 230,000 refugees from Darfur currently reside in twelve refugee camps in eastern Chad. (Click here for a Nicholas Kristoff review of books on “what to do about” Darfur. For an alternate perspective, read an interview with Mahmood Mamdani on his book Saviors and Survivors: Darfur, Politics, and the War on Terror.) These individuals come from throughout Darfur, an area of Sudan larger than France that covers three climate zones and includes multiple ethnic groups. Most in the camps traveled there from small agrarian settlements, where they lived for generations through seasonal farming and selling their crops in local markets. Ethnic competition for resources has been endemic to this region for years, and stories of strife and rebellion are common features of regional lore. What changed this pattern into a crisis was the Sudanese government’s active support of ethnically Arab tribes’ violence against other, so-called “Black” tribes. With encouragement and material support from the Sudanese military, the Arab militias, known as janjaweed, gained an edge in the struggle and stepped up their campaign. Satellite imagery of Darfur shows thousands of burned-out villages. First hand accounts of the destruction describe systematic killing of men, boys, and the elderly; a mass campaign of raping women and girls; and the killing of livestock, theft of material goods, and destruction of property. Those who survived these attacks first hid, then settled in internal displacement camps, and then walked west across the border into Chad.

The United Nations High Commissioner for Refugees (UNHCR) set up camps in the Wadi Fira, Ouaddai, and Dar Sila provinces of Chad (just across the border from Darfur, Sudan) to receive these refugees. These camps vary in size, the largest being Bredjing, with a population of 30,000, and most others being about half that size. It is hard to imagine a more inhospitable environment in which to introduce a quarter of a million new residents. Most camps are located in the Sahel and have limited water and agricultural resources. Stressors reported by refugees include conflicts with local Chadians over access to water and firewood. To begin to address refugee needs, several humanitarian aid agencies began providing support to the people in these camps, including a relatively new type of aid, “psychosocial aid.”

International Psychosocial Interventions (IPI; IPI is not the organization’s actual name but is a pseudonym) was a well-established refugee services organization, originally founded to help Jews fleeing persecution in eastern Europe. IPI’s psychosocial program in Chad was funded by a grant from a United States government agency, which required IPI to evaluate their program after one year. At the end of their first year, IPI asked our organization to undertake that evaluation.

What follows is a description of our efforts to perform an effective evaluation of IPI’s psychosocial programs in Chad, including an overview of our results. However, the bulk of this presentation walks readers through the evolutionary process of program evaluations in less than ideal settings – the most common setting for evaluations. To this end, we highlight our “decision points” – points in the process during which we had to make decisions (or had decisions made for us) that changed the nature of the evaluation.

(Click here for discussion questions on this section.)

Return to top

PSYCHOSOCIAL AID

The term “psychosocial” is an umbrella term intended to capture the notion that mental well-being is a function of both cognitive and emotional well-being (the “psycho” in psychosocial) and interpersonal support (the “social”). Although the concept has been applied to refugees for some time (e.g., Boothby, 1992), it has only recently been made a priority in humanitarian aid for displaced populations (IASC, 2007).

The idea that mental health should be a concern in humanitarian aid to refugees grew out of the convergence of two forces: (1) the identification of refugees as a population at risk for posttraumatic stress disorder (PTSD), and (2) the recognition that pervasive emotional distress can be a source of danger to the aid workers and refugees themselves. Recognition that aid workers were at risk for PTSD came through the voice of the aid workers. Identification of PTSD among refugee populations in grounded in Western psychiatric theories of trauma response and treatment and came about largely through the work of American psychiatrists who used surveys to identify PTSD symptoms in refugee camps and among resettled refugees living in the United States. (The pioneering work of Richard Mollica and Mark Kinzie with Southeast Asian refugees in the 1980s are examples of this work; Kinzie, Fredrickson, Ben, Fleck, & Karls, 1984.)

The Darfur refugees living in the camps in Chad are characteristic of those at risk for posttraumatic stress. They are individuals from small farming hamlets, having lost family and friends and fled for their lives, who converge upon a large settlement of 15-30,000 people where they know only a few other residents. Basic food aid is distributed monthly, with the amounts based upon the number of individuals in families, and water is limited. Scarce resources and unfamiliarity breed suspicion. Combine suspicion with posttraumatic reactions and overwhelming grief and loss, and it become obvious as to why mental well-being should be a very practical concern for humanitarian aid.

But what does psychosocial aid look like? In theory, it would involve some aspects of both psychological treatment and social organization, although there is considerable debate in the literature about the right balance between the two (see Miller & Rasmussen, 2010). In practice, psychosocial programming runs the gamut from creating opportunities for sports for children to distributing psychopharmacological medication to refugees with schizophrenia. The broad variety in the field makes it very difficult to identify the common elements of such services, and therefore, difficult to compare the effectiveness of any one to another.

(Click here for discussion questions on this section.)

Description of IPI’s psychosocial programs

IPI was established in three camps: Bredjing, Treguine, and Gaga. These camps were close to one another in proximity, and residents were primarily members of the Massalit ethnic group.

IPI’s psychosocial programming involved two key components:

  • Identification of refugees suffering from severe distress and managing their cases
  • Group counseling

The first component was carried out by “community mobilizers,” refugees hired from within the camps, and by “psychosocial assistants,” Chadian staff with basic training in identifying the signs and effects of stress and in case management. The second component was performed only by the psychosocial assistants. All staff were supervised by expatriate counseling professionals who held professional degrees in psychology, social work, or a related field.

Community mobilizers (CMs)

Refugee camps were divided into administrative blocks. Each bloc consisted of approximately 500 people and was led by a “Chief of Block.” IPI initially requested that Chiefs of Blocks select individuals – one man and one woman – as community mobilizers (hereafter, CMs). These CMs were then hired to serve as community-based advocates and referral agents for refugees. CMs were trained to:

  • Visit refugees (“home visits”) in which they check for signs of stress
  • Engage fellow refugees in basic problem-solving techniques
  • Make referrals to IPI’s group counseling, as needed
  • Make referrals to other services in the camp (e.g., medical or non-food aid), as needed

Empowering refugees to solve problems for themselves was the core value behind this approach.

Psychosocial assistants (PAs)

Supporting the network of block-based CMs were the Chadian psychosocial assistants, or PAs. PAs were trained and instructed to

  • Attend home visits of more serious cases
  • Conduct case conferences, during which records of home visits are reviewed and interventions proposed (case management)
  • Lead group counseling sessions

Psychosocial counseling professionals

Expatriate psychosocial counseling professionals were responsible for supervising CMs and PAs, ensuring the quality of record-keeping and maintaining good relations with other NGOs and UNHCR staff. Prerequisites for this role included a higher education degree in psychology, or a related field, and facility in French or Arabic.

(Click here for discussion questions on this section.)

Return to top

PROPOSED INDICATORS AND REDESIGNING THE EVALUATION FRAME

IPI’s original proposal to the funding agency included several “indicators” that if met, would signify success. Several of these were straightforward – e.g., “Reach 1,500 refugees through the support of group psychosocial services” – which could (in theory) be easily measured by reviewing IPI’s records. Others appeared straightforward but were not. For instance, the indicator “Reduce cases of those identified as manifesting aggressive behavior and being physically restrained in the camp by 90%” provided concrete goals but ignored the fact that there were no available baseline data on the prevalence of these cases. This meant that the percent reduction could not be measured. “Identify and refer 70% of refugees who could not participate in income-generating activities” had the same problem – the assumption that 100% of these cases had been identified – but no one had. This made it impossible to determine how many people the program would have refer in order to reach the 70%. Operationalizing indicators (i.e., making them measureable) was the first major challenge of the evaluation.

DECISION POINT: WHAT TO DO WITH THE PROPOSED INDICATORS?

Because many indicators in the original proposal were not “operationalizable,” we chose instead to identify core elements of IPI’s programs and to evaluate these. Where indicators were able to be operationalized – e.g., “Provide four or more sessions of individual counseling to 80% of refugees identified by the community mobilizers” – we made every attempt to evaluate them. More often than not, however, we framed our evaluation plan to define the extent of IPI’s program and to critique its core elements: home visits, case management, and counseling.

Ideally, we would have started with some version of a pre-/post-test design, and perhaps even used randomized control group trials. (Also see St. Germain’s “Experimental Research Designs” and Marshall, 2007.) However, we were entering the evaluation after IPI’s programs had been up and running for a year, making pre-tests and random assignment to an intervention group impossible. We looked instead for some proxy for these approaches. At this point in the process, IPI was also beginning to offer psychosocial services in two camps to the south, and we thought we might do pre-tests in these camps and use them as comparison groups – i.e., do some assessment of mental health in the Bredjing, Treguine, and Gaga and treat the assessment of this group as a post-test, comparing it to the pre-test groups in the new camps, and thus obtaining a rough estimate of change over the course of the intervention.

DECISION POINT: PROPOSING A NEW EVALUATION DESIGN, AND THEN ANOTHER

We met with IPI’s central administration and explained the dilemma regarding the indicators. They then conferred with the funding agency’s officials. Both parties were in agreement that as written, most indicators were unrealistic. Both parties were also in agreement that the money to fund the evaluation could not be used to fund any activity in the new camps, thus negating our proposal of an approximate pre-/post-test (or cohort control group) design. Moreover, the funding agency’s officials made it explicit that they were mostly interested in whether or not any evaluation could be carried out in the harsh environment and were thus amenable to more general, limited approaches.

After some arguing over the pre-/post-test design – arguments that we lost – we redesigned our evaluation to target particular pieces of IPI’s program using observation and program records and a small comparison group design. Our work was thus primarily aimed at process evaluation, as opposed to outcome evaluation.
We conceptualized our design as this:

Home visits and case management

  • Observation of home visits and case conferences
  • Review of records
  • Interviews with refugees identified in records

Group counseling

  • Observation of counseling sessions
  • Comparison group trials to compare outcomes of intervention

We took this design to the NYU Institutional Review Board (IRB), and following its approval, to the leadership council of the camps where we would be working. They also approved the study. (For more on IRB’s in the United States, see the Department of Health and Human Services Office of Human Research Protection.)

A note on evaluation results

The results presented in the sections below are overviews only. They are presented to flesh out the narrative of our evaluation, not to show the value of IPI’s program. Note how preliminary results led to the subsequent methodological steps. While reading, think about the choices we made along the way, how they might have been made differently, and how these choices shaped the results of the evaluation.

(Click here for discussion questions on this section.)

Return to top

HOME VISITS AND CASE MANAGEMENT

We viewed home visits as the backbone of IPI’s program – the point at which the organization had initial contact with the problems of camp residents. Because of home visits’ centrality, we wanted to spend some time on them and ended up conducting three separate studies. The first consisted of observations of home visits and case conferences. The second was a review of home visit records, which lead us to add a third study: interviews with beneficiaries of home visits.

Observations of home visits

In April of 2007 as part of our evaluation, we accompanied PAs and CMs on their home visits and reviewed processes surrounding these activities. The home visit process was this:

  1. CM identifies the individual in need and alerts the PA (CMs may visit an individual repeatedly).
  2. The alert leads to a visit by the PA.
  3. PA makes contact with the identified client and other family members.
  4. The problem is clarified and identified/diagnosed (e.g., marital conflict, “mental trouble,” medical complications).
  5. Interventions are made in the form of suggestions (e.g., self-care, familial support) and/or referrals to further services (health center, group counseling).
  6. Follow-up visits, usually two or three, are scheduled.

Observed methods of intervention

CMs and PAs approached the head of the families for permission to proceed with home visits. Diagnostic assessment of the problem was done through direct interaction with clients, observations and collateral information from family members. PAs then undertook their intervention. These ranged from basic psychoeducational suggestions to intervening in family conflicts to specialized referrals. PAs conferred with all involved and scheduled further visits. Following home visits, PAs met with their supervisors to conceptualize and record the cases.

Results

Strengths: We found that PAs had received basic training in psychoeducation and diagnosis of posttraumatic and adjustment issues, paid attention to psychological impairments while remaining sensitive to normative cultural/familial factors, and used an attentive, proactive, pro-client approach. We also found that each visit/session reflected comprehensive approach. The successful community outreach achieved by the CMs created a sense of credibility and trust of IPI staff, which facilitated the participation of the refugees in the referral process.

Need for improvement: There was little clinical feedback to the CMs and PAs and few opportunities for them to receive advanced training. Although supervisors often insisted that chart notes and reports be clear, supervision was largely clerical and was not accompanied by clinical supervision. During our observations of home visits we found a high rate of turnover among PAs. (High turnover rates decrease service delivery productivity and effectiveness.) Clinical supervision might have mitigated this. Having feedback from clinical supervisors could have increased both the effectiveness of the staff and their sense of the value of their work. It could also have minimized counter-productive behaviors such as mistakes, burn-out, and blind spots.

Observation provided us with the basic structure of home visits. Observation could not inform us on the consistency with which the basic structure was applied. For that we needed to review records.

Review of Home Visit Records

After performing observations of the home visits but while still in Chad, we scanned the home visit records for January to September 2006 and then took these with us back to the United States to continue the review. We wanted to do two things: (1) examine the scope of problems addressed in these records and (2) follow cases over time. We first reviewed all records to identify the particular types of problems noted in home visit forms. Following the suggestion of one IPI supervisor, types of problems were categorized into material, physical, and mental health. Referrals were categorized into referral for IPI’s counseling and referral to other UNHCR partner organizations. These records were then coded for camp, block, number of types of problems recorded in each home visit record, number and types of referrals made, and follow-up scheduling. While there were many types of problems and types of referrals within each category, we believed that a simple summary of the number of types of problems would be a reliable, general approximation that would permit statistical analysis.

Types of problems recorded in home visit records

Between 1 January and 30 September 2006, 446 home visits were recorded. Almost 800 problems were logged by IPI staff during their home visits. A summary is presented in Table 1. Physical needs were the most prevalent in this population, with an average of one and one-half problems reported for each visit made.

Table 1: Types and numbers of problems recorded in home visit records


Problem


Number


Percent

Average #
per visit

Material

52

6.5

0.16

Physical

469

59.1

1.49

Mental Health

273

34.4

0.86

Total

794

100.0

2.52

Review of two consecutive months of home visits and follow-up visits

Instead of trying to follow all cases throughout the mass of paper records, we chose to limit our review to a time period of two months. There were 315 home visit records dated between 1 August and 30 September 2006. These records represented home visits to 213 individuals in Bredjing (n=63, 29.6%), Treguine (n=86, 40.4%), and Gaga (n=64, 30.0%).

Home visit records indicated that 123 visits (out of 315, or 39.0%) resulted in a request for referral to counseling sessions conducted by IPI and 171 visits (54.3%) in a request for referral to other agencies (primarily for medical and material support). This seemed a substantial number for two months: 171 referrals over this time period amounted to 2.8 referrals per day. This means that on paper IPI’s home visit program contributed about three referrals a day to other aid agencies working in the camps.

Records indicated that about three-quarters of the 213 individuals (73.2%) were visited only once. Another 15% had two visits only, 7% had three visits only, and about 5% had four or more visits. Percentages were similar between men and women and did not differ by age.

Records indicated that a total of 210 follow-up home visits were scheduled during this time. Evidence that these were completed was present in only 74 cases (35.2%; no differences between men and women or across age). It is not clear why this rate was so low. We wondered if records were not kept on follow-up visits that did not involve lengthy discussions, or it may have been that many follow-ups were not in fact completed. Lack of actual follow through did not to match with our impressions during observation.

We made attempts to use information within the records to judge whether home visits resulted in a decrease in the number of problems between visits. We used t-tests to examine whether the number of problems mentioned at first and second visits were different. We found that decreases in the number of problems were marginally statistically significant only for material problems (t(df=34)=1.712, p=.09). In other words, the only problems that showed a decrease between visit 1 and 2 were material ones. The number of physical and mental health problems reported did not change. This was the case for men as well as women and was not related to age. (Comparisons were not made for later visits (i.e., 3rd, 4th, or 5th visits) as there were not enough of these records to make meaningful comparisons.)

While examining cases with both initial and follow-up records, we noted that many follow-up records lacked information about the continued presence of the problems noted in initial visits. It was not clear whether these initial problems had resolved or were just not included in the second visit’s record because they had been mentioned in the first. We were therefore unable to examine whether the minimal improvement in the number of problems between first and second visits were the result of original problems not being solved or were the result of new problems arising.

DECISION POINT: ADDRESSING WHAT APPEARED TO BE INCOMPLETE RECORDS

In our review of home visit records, we found evidence that IPI CMs visited substantial numbers of individuals with many types of problems and made many referrals to partner organizations that would otherwise likely not be made. However, we still felt that home visit records were incomplete and a more direct approach was needed to accurately judge the effectiveness of IPI’s home visit program. We decided to go to the source: the refugees themselves.

Home Visit Interviews

In July 2007, we surveyed randomly selected beneficiaries one month or more after their initial home visits. We interviewed these individuals about their contact with CMs, the referrals made as a result of this contact, and their problems at the time of interviews.

We designed a structured interview focused on three constructs:

  • problem identification
  • home visits and referral/recommendation
  • improvement and resolution of problems.

Problem identification concerned how CMs became aware of the problem targeted by the home visit. Home visits and referral/recommendation concerned the actual visit, whether or not CMs presented solutions to clients, the content of those suggestions, and what CMs did to facilitate them (e.g., accompanied the beneficiaries to the referral or arranged transportation). Improvement and resolution concerned outcomes (whether or not the problem had been resolved or improved) and the client’s opinion of the home visit’s role in addressing the problem. Two interviewers worked with Massalit interpreters. Interviews took place in respondents’ homes, to which we were directed by CMs. With respondents’ consent, interviews were recorded and later transcribed.

To create a representative sample of home visit beneficiaries, we selected every sixth case in IPI home visit records for April and May of 2007. As the records were arranged in date order only, this systematic random sampling technique ensured an unbiased selection of cases, but of course the records may have been incomplete.

Our final sample consisted of 46 (of 305) case records for April and May of 2007 (or, 15% of all home visits during this period). Of the 46 cases identified, we were able to complete interviews with 41 (89% response rate). The most common reason for incomplete interviews was that we could not find either the beneficiary or knowledgeable others (i.e., family members). We suspect this is because beneficiaries had moved to another camp or because identifying information in home visit records was insufficient. In 16 of the cases in which we did make contact (39%), the identified beneficiary was not able to speak with us. Reasons included age, incapacity because of physical or psychological limitations, and not being at home when we visited. In these cases, we decided to speak to family members.

Results

Figure 1 presents the results of our interviews in a flow chart. Organized to reflect the home visit process, the chart begins with Problem Identification, followed by the Home Visit and Referral/Recommendation, and Resolution.

Figure 1: IPI Home Visits

Problem Identification

Problems that were addressed in home visits are presented in Table 2. Most respondents reported more than one problem, so the total number of problems exceeds the number of respondents. Problems are those reported by respondents themselves (not in IPI records). Terms that are out of the ordinary for Western health workers or have an alternate meaning in Sudanese culture are in quotation marks.

Table 2: Problems addressed in IPI home visits, as reported by respondents (n = 41)

Physical problems

 

Frequency

Mental health problems

 

Frequency

headaches

10

“pain in heart”

4

muscle/joint pain

7

stress/grief

3

stomach problems

4

“possession by devils”

3

vision problems

4

nonspecific mental problem

3

“epilepsy”

3

palpitations

3

sore throat

3

dizziness/weakness

2

paralysis

2

Clearly physical problems were reported more frequently than mental health problems, although some physical complaints may have been related to stress reactions (this seems likely for many reporting “epilepsy,” which was often not consistent with biomedical definitions of the disorder), and some mental health problems have physical attributes (e.g., “pain in heart”).

Home Visit and Referrals/Recommendations

Almost all cases were addressed with a referral or recommendation, most frequently a referral to the hospital. The actual referral process was usually somewhat arduous, which meant that CM recommendations and referrals were followed by recipients in 33 cases out of the 41. Nonadherence was due to a number of factors, including the refusal of family members to permit follow through and the need for IPI staff to facilitate the process. In 17 cases, CMs accompanied clients to their referrals (usually the hospital) in order to assure that services were delivered. In twice that number of cases, CMs followed up with clients either at home or at the agency to which they were referred. There was a sense among many respondents that IPI was able to force the hospital to treat them. For 29 individuals, some form of treatment was delivered by referral agencies. All these individuals received medication of some kind. Other interventions included IPI counseling and obtaining new UNHCR distribution cards.

Improvement and Resolution

One fifth of the sample reported that the problems that brought them to the attention of IPI had been completely resolved. Counter to what one might expect, complete resolution of problems was not statistically associated with referrals (i.e., whether or not recommendations were followed, whether treatment was delivered, whether CMs accompanied clients to referrals, or whether CMs checked to see that referrals had taken place). Problems persisted to a certain extent despite attempts to address them.

Although complete resolution was found in a minority of cases, two thirds of respondents reported that their particular problems had improved (this includes those for whom problems had been resolved). As presented in Figure 2, three-quarters of those who followed recommendations reported that their problems improved; only a quarter of those who did not follow recommendations reported that their problems improved.

Figure 2: Following IPI recommendations was associated with reported improvement


Figure 3: Treatment delivery was associated with reported improvement

Figure 3 shows that 82% of those who received treatment as a result of referrals and recommendations reported that their problems had improved, whereas only 14% of those who did not receive treatment reported that their problems had improved. None of those who received treatment reported that things had gotten worse.

In short, three fifths reported some improvement and one fifth reported complete problem resolution, which means that four fifths of home visit clients reported that home visits were helped them in some way.

Nine of the 12 (75%) individuals who reported that they were referred to IPI counseling reported that they attended counseling. The average number of sessions reported was 3.36 (range 1 – 10). Notably, IPI home visit records had a note for only three of these referrals. Six who received counseling reported that it helped them, and three reported that it did not help. Reports of what happened in IPI counseling sessions and why they were helpful centered on advice to forget what happened in Darfur. In addition to clients attending counseling, several respondents reported that other members of their family had been referred to IPI counseling as well. Although not a representative sample of “family member referrals” (this information was not queried systematically but arose in conversation), it illustrates how IPI CMs approached cases as families rather than simply as individuals.

Methodology

Because of the sampling methodology we employed (systematic random sampling), we were confident that the sample of home visit cases we selected was representative of IPI home visits in general.

Process evaluation

Interviewing home visit beneficiaries had indeed allowed us to move beyond archival records to get a richer view of IPI’s home visit program. CMs were seen as frequent participants in the lives of home visit clients, visiting them far more often than home visit records seemed to indicate. Concerns about a low follow-up rate displayed in the previous study were allayed by clients’ reports in the current one.

Outcomes

A clear message to emerge out of these data was that complete resolution of most problems was rare, but improvement was possible. That improvement was associated with following IPI’s recommendations and delivery of treatment may suggest that IPI’s referrals are, on the whole, appropriate. That said, over a third of respondents reported that their problems had not improved (i.e., had gotten worse or stayed the same). While a few of these cases fit into categories outlined above (e.g., those who did not follow recommendations), most were cases in which recommendations were followed.

Limitations of the findings

The major limitation to our findings is that IPI CMs lead us to clients’ homes. In some cases, the CMs were present during the interviews. This undoubtedly introduced a bias in favor of IPI. The data presented here are therefore likely a rosier picture than the truth. While this is the case, we were surprised when several clients expressed negative views of IPI even with CMs present. Because of cases like these, we do not feel that the bias skewed results so much as to make them spurious.

Conclusions

Our summary finding of this portion of the review is that although the IPI home visit program was intended to be the backbone of a psychosocial program, it served primarily to refer to and to facilitate access to medical care – by far, most referrals were to the hospital. CMs followed referrals or accompanied clients in most cases. Most problems did improve, but only one fifth were actually resolved, which may reflect the chronic nature of the stressors and challenges this population was facing. In addition to facilitating outcomes, home visits provided day-to-day social support for many marginalized camp residents.

(Click here for discussion questions on this section.)

Return to top

GROUP COUNSELING

We initially planned to run some form of comparison group trials to evaluate the effectiveness of IPI’s group counseling program. We intended to randomly assign clients to intervention and wait-list control groups for the length of the counseling intervention. If this was not feasible, we would make use of the delay inherent in running structured interventions and compare those waiting for services with those in the intervention. (This would create a comparison group but one that may not be equally as representative as one created through random assignment.) In preparation for these, we observed group counseling sessions over two weeks in early April of 2007.

IPI’s approach and intervention

Group counseling sessions generally proceeded thusly:

  • Counselor prompted the refugees to talk about their traumatic experiences and difficulties
  • Participants disclosed past trauma, each in turn
  • Counselor provided basic psychoeducation regarding post-traumatic stress reactions
  • Group discussed adaptive and maladaptive coping strategies

The number of sessions in group counseling was not set, with sessions occurring weekly and clients returning if they wished; most only attended one session

Staff strengths

The staff showed signs of basic training in recognizing symptoms of posttraumatic stress and identifying coping strategies. They were also reliable in following up with particular problems of individual members in the group.

Refugees’ responses

When asked about their troubles, refugees in all observed group sessions spontaneously brought up the difficulties of camp life (worries about safety, feeling stuck and powerless, struggling with inadequate provisions). Current life stressors were the most prominent concerns and the most immediately accessible topics of discussion. During the course of sessions, they also brought up interpersonal problems that seemed derived from the loss of economic stability and the disintegration of the family structure, as well as from posttraumatic stress. Finally, under the prompting of counselors, they brought up posttraumatic psychological problems such as intrusive thoughts, preoccupation with the past, “pain in heart,” anxiety, etc.

Refugees were compliant with the structure of the counseling sessions, and they seemed receptive to psychoeducation and support for posttraumatic stress symptoms. They also seemed to appreciate the concrete referrals. However, the number of participants and the “turn-taking” model of testimony seemed to encourage a passive, withdrawn stance among participants. When participants did engage, it was always with the counselor, not with one another. We observed little intra-group communication. Consequently, individual members’ problems were addressed within the group setting, but the health and strength of the group (as an entity) was not facilitated. Group counseling sessions at times appeared more like several individual counseling sessions run in sequence, not a group session per se.

DECISION POINT: WHAT TO DO ABOUT THE INCOMPLETE TREATMENT MODEL FOR THE COUNSELING INTERVENTIONS?

It became obvious during observation that the intervention model was incomplete. It was without clearly-defined guidelines and activities. Nor did the counseling have time-related, identified goals. The “counseling” was essentially a one-time meeting where people were encouraged to relate the terrible things that happened to them and to listen to others doing the same. As a result most participants did not return. This led us to question, Exactly what is it we are trying to evaluate?

Cancelling the comparison group design

Comparison group evaluation designs are typically difficult to carry out, in particular because they require systematic assessments at two time points, and IPI had not performed baseline assessments. These designs also require measures to assess adherence to a structured intervention. Yet the group counseling sessions we observed lacked a clear treatment model. It could not be conceptualized and evaluated as a formal, operationalizeable intervention. To properly evaluate program outcomes, we would have had to first structure the counseling sessions to look more like theoretically meaningful models – thereby changing the nature of the intervention. This led us to cancel our plan to use a comparison group evaluation design and instead to make recommendations on how to improve counseling, which is what we did.

(Click here for discussion questions on this section.)

Return to top

CONCLUSIONS

Our overall findings from the evaluation of IPI’s psychosocial programs in three refugee camps in eastern Chad can be summed up thusly: IPI’s psychosocial programming seemed to do “social” well and “psycho” not so well. Home visits provided a solid backbone for the program: it served as an effective referral source for basic needs and resolved simple interpersonal problems. IPI’s counseling program was theoretically unsophisticated, and in terms of processes, minimally structured, and we felt that it needed improvement before its outcomes could begin to be evaluated.

Lessons learned

Evaluations on paper may seem rather straightforward. In the field, whether or not they are simple depends on time, resources, and institutional willingness to be flexible. We encountered challenges in all three domains. We feel we found indicators of IPI’s successes and failures, but are well aware that our findings are not without significant threats to reliability. In retrospect, we may have relied too heavily on IPI’s records, which often seemed to be incomplete, and on the self-reports of IPI’s beneficiaries. Supplementing self-reports with a measure of functional well-being in the interviews we did with home visit cases may have provided a broader picture of the strengths (or weaknesses) of the program. However, given the level of theoretical sophistication of the program (particularly the counseling services) and the fact that comparison methods were not available to us, we feel we addressed the core elements of IPI’s psychosocial program appropriately.

(Click here for discussion questions on this section.)

 

FACILITATOR DISCUSSION NOTES

We propose questions for group discussion and further investigation

Return to top

REFERENCES AND FURTHER READING

Bolton, P., Bass, J., Neugebauer, R., Verdeli, H., Cougherty, K. F., Wickramaratne, P. Speelman, L., Ndogoni, L., & Weissman, M. (2003). Group interpersonal psychotherapy for depression in rural Uganda. Journal of the American Medical Association, 289, 3117-3124.

Boothby, N. (1992). Displaced children: Psychological theory and practice from the field. Journal of Refugee Studies, 5(2), 106-122.

Inter-Agency Standing Committee (IASC). (2007). IASC Guidelines on Mental Health and Psychosocial Support in Emergency Settings. Geneva: IASC.

Kinzie, J. D., Fredrickson, R. H., Ben, R. Fleck, J. & Karls, W. (1984). Posttraumatic stress disorder among survivors of Cambodian concentration camps. American Journal of Psychiatry, 141, 645-650.

Marshall, P.A. (2007) Ethical challenges in study design and informed consent for health research in resource-poor settings. Social, Economic and Behavioural Research. Special Topics No.5. TDR/SDR/SEB/ST/07.1. Geneva: World Health Organization on behalf of the Special Programme for Research and Training in Tropical Diseases

Miller, K. & Rasmussen, A. (2010). War exposure, daily stressors, and mental health in conflict and post-conflict settings: Bridging the divide between trauma-focused and psychosocial frameworks. Social Science & Medicine, 70, 7-16.

Miller, K., & Rasco, L. (2004). The mental health of refugees: Ecological approaches to healing and adaptation. Lawrence Erlbaum: Mahwah, NJ. (Also see book review)

Neuner, F., Onyut, P. L., Ertl, V., Odenwald, M., Schauer, E., & Elbert, T. (2008). Treatment of posttraumatic stress disorder by trained lay counselors in an African refugee settlement: A randomized clinical trial. Journal of Consulting and Clinical Psychology, 76, 686-694.

Neuner, F., Schauer, M., Klaschik, C., Karunakara, U., & Elbert, T. (2004). A comparison of Narrative Exposure Therapy, supportive counseling, and psychoeducation for treating posttraumatic stress disorder in an African refugee settlement. Journal of Consulting and Clinical Psychology, 72, 579-587.

Return to top


ABOUT THIS SITE

Want to learn more about public health? We recommend that you start here.

The site is for people new to the public health field as well as those who have experience and knowledge but who want to learn more. Each section serves a different but complimentary purpose, and we invite you to explore.

More about this site

What's in the site
Notes on using this site
Acknowledgements

PUBLIC HEALTH NEWS
Loading...

Our news is from a Google alert. We do not review its content.


Sponsored by the New York University Master of Public Health Program and
made possible through the generous support of the Josiah Macy, Jr. Foundation.