External Validity of Studies on Aggressive Behavior in Patients with Schizophrenia: Systematic Review

INTRODUCTION

There is good evidence from epidemiological studies that the diagnosis of schizophrenia is associated with an increased risk of violence in the community [1], which persists with increasing age [2], is found in both males and females [3] and is strongly associated with co-morbid substance abuse [4, 5]. This poses a serious burden for relatives [6], staff, society as a whole and, last but not least, patients with psychoses themselves due to the subsequent stigmatisation [7]. Thus there is a considerable interest in conducting clinical studies on the origin and therapy of violent behaviour in these patients. However, clinical studies require written informed consent due to ethical reasons. In hospital settings, violent behaviour of patients with psychoses is associated with involuntary treatment [8], lack of insight into their illness and treatment [9, 10], cognitive distortion [11], disorganisation [12] and positive symptoms [13]. According to the nature of the disorder and the severely disturbed behaviour, violent psychotic patients are mostly uncooperative and distrustful, and lack the ability to understand and appropriately process new information. Therefore, these patients are typically neither willing nor able to give informed consent to study participation and as such it is obviously a significant challenge to conduct clinical studies with patients possessing these characteristics. Patients difficult to treat, e.g. with antisocial behaviour, show higher refusal rates than patients without antisocial behaviour [14]. Epidemiological studies find significantly divergent prevalence rates of aggression depending on how many sources of information the authors relied on [15] indicating that one single data source may cause bias. This indicates that studies on aggression in schizophrenia have to take into account risks of selection bias and underestimation of prevalence rates. More than a decade ago, Volavka and Citrome highlighted methodological concerns in research on atypical antipsychotics in the treatment of persistently aggressive psychotic patients, including expectations of doctors and patients in non-randomised trials, influence of environmental factors, use of proxy measures as outcomes, insufficient definition of aggression, consent selection bias, inappropriate observation periods, and unrevealed diagnostic confounders such as substance abuse [16].

Nevertheless, numerous randomised controlled trials on the treatment of aggressive patients with schizophrenia and meta-analyses of these trials have been conducted [17] and recommendations in treatment guidelines are based on these results [18, 19]. This poses the question of whether or not patients included in these studies are representative of those violent patients who are described in epidemiological studies and in descriptive studies of in-patient violence, i.e., whether these studies have sufficient external validity to allow for generalisability of the obtained results.

We conducted a systematic literature review on clinical studies of violent patients with schizophrenia. The primary objective was to analyse whether the published data provides information about a reference population and representativeness of the study sample. As a secondary objective, we wanted to obtain information on the average proportion of patients who are reluctant to participate or drop out after inclusion into the studies. This is important for the planning of future studies on such difficult-to-recruit populations.

METHODOLOGY

RESULTS

The literature search yielded 1419 hits (PubMed 1096, PsycINFO 323). The abstracts of all studies were screened with respect to the inclusion and exclusion criteria. In case of doubt, if no abstracts were available, the full texts were analysed by one of the authors (K.H.). In total, 33 studies fulfilled the defined inclusion criteria. There were several studies that based on the same data sets but dealt with different topics. In total, there were 20 independent data sets.

11 out of 33 studies investigated drug effects, 13 symptoms, prediction and course, 6 neurobiological issues, 2 subjective views and 1 a psychotherapeutic intervention. 19 of the studies included were published prior to the year 2000, 14 studies were published since then.

DISCUSSION

The primary objective of this systematic literature review was to investigate the external validity of published clinical studies on aggressive and violent behaviour in people with schizophrenia. Thus we focused our inclusion criteria on studies which required informed consent, required patient’s active participation and required aggressive behavior as an inclusion criterion. Though there is plenty of literature on violence in people with schizophrenia, only a limited number of 33 studies, based on 20 independent data sets, fulfilled the criteria.

To assess the external validity, we analysed whether criteria for study inclusion were clearly defined and whether appropriate information was given on drop-outs during the respective study, as well as on patients who had been screened for participation but were reluctant to participate. We found that only 15% of the studies that fulfilled our inclusion criteria for the review used inclusion criteria defined a priori by a minimum score on an aggression scale such as the Modified Overt Aggression Scale [52]. This finding is in line with a recent overview on mood stabilisers in aggressive patients with schizophrenia where the authors criticised a lack of transparency regarding the inclusion criteria and definitions of aggression [16]. The difficulty in recruiting patients for such studies is shown by the study description of Citrome et al. [46], where the inclusion threshold had to be lowered during the study because after 15 months only 8 patients had been recruited, then also allowing for the inclusion of milder forms of aggression. However, the inclusion of very mild or not clearly defined forms of aggressive behaviour makes the external validity of the respective studies questionable. This does not only concern the severity, but also the causes of aggression. Only 8 out of 33 studies included made some differentiation between reactive, psychopathology-related and other forms of violence, e.g., violence related to antisocial personality traits or intoxication.

An analysis and comparison of non-participants and participants was reported in only one study, and characteristics of drop-outs were reported only in 4 studies, though their percentage (where it was reported) was about 25% of the study population. Even if detailed information on non-participants may not be accessible due to ethical limits, it would be helpful to give some basic information on these patients. For futher investigations it is essential to know which subgroups of patients are likely to participate or not. Altogether, the external validity of most studies on aggressive behaviour in schizophrenia seems questionable, and sample selection bias is rather probable in the sense that patients with severe violent behaviour and very little cooperation – the most difficult to treat and manage – were not included. This is well in line with previous studies on non-participants in studies on patients with mental illness. Non-participants tend to be male, unmarried, and to have a more severe course of illness. They also typically tend to have a low socioeconomic status, co-morbid substance abuse problems and a history of violence [53-55]. Patients with schizophrenia are more frequently reluctant to participate in studies than patients with bipolar disorder [55, 56]. Among patients with schizophrenia, non-participation is associated with the disorganised subtype [54], severe positive symptoms and more frequent hospitalisation [56]. Additionaly, we compared the reporting quality in pharmacological and non-phamacological studies. Although pharmacological studies overall showed a better reporting quality that may be due to stricter study protocols, there is even a need to improve the reporting quality. Comparing studies published prior to the year 2000 and more recent publications did not yield a remarkable increase in the quality of reporting.

Our second objective was to elucidate difficulties in recruiting appropriate study samples among aggressive and violent patients with schizophrenia, which is important for the planning of future studies. Unfortunately, most of the studies did not report many of the relevant data, such as percentages of non-participants. Incomplete reporting of such essential study data is common in clinical studies and randomised controlled trials (RCT) with regard to patients with schizophrenia [57, 58]. Where reported, only about 12% of eligible patients had refused to participate. This seems to be a small proportion; however, in many studies, the requirement for patients was only participation in an interview, participation in an RCT on medication, or consent to taking blood specimens for genetic tests. There is a remarkable discrepancy between the huge body of literature on predictors and associated factors of violence in schizophrenia and the nearly complete lack of evaluated intervention programmes, with only one exception [48], which might indicate the difficulty of conducting such studies. Not only might it be difficult to recruit patients, but bias emerges as the result of a considerable percentage of drop-outs during the study, which was 22% in the studies detected here (where information was reported at all). This was still less than the 40% in RCTs Gilbody et al. [58] reported in an overview of schizophrenia studies. If all types of studies on schizophrenia were included, this percentage would be much smaller (12%) [59].

The mean sample size was 93, more than the reported average of studies in schizophrenia in general [59], but the well-known tendency of studies to be underpowered is found here, too. The vast majority of studies used in-patients as samples. This can be understood as an indirect hint at the difficulties of conducting clinical studies outside the hospital setting with patients who have displayed violent behaviour. However, this can also be a further source of bias, because the factors associated with in-patient violence are rather different from those associated with community violence [3]. The recruitment period, if reported at all, was about 3 years on average, which may be a further hint at the difficulities in realising such studies. If recruitment of patients for studies is difficult and, in addition, it is difficult to keep patients to study protocols, financial incentives for participants are helpful and frequently used [60] but rarely reported [61]. Unfortunately, none of the detected studies reported on incentives for participants. A better reporting quality would not only give helpful hints for recruitment in future studies, but is part of scientific ethics. Motivated by one of the reviewers of this paper, we tried to contact the principal investigators of the reviewed studies via e-mail, as indexed in the SCOPUS database, concerning the question of incentives for participating patients. However, a considerable proportion of the studies is more than ten years old and some of the authors had moved to other institutions or retired, from others we did not receive an answer. The resulting picture is incomplete but suggests that incentives of different kind may have been given in most cases. Financial compensation was described as small amounts of about 10 $ or €. Other incentives could be transferral to a special research unit with better staffing and more convenient conditions of hospitalization.

Given the difficulties in recruiting aggressive patients mentioned above, it is questionable that studies based on informed consent of patients are representative for the nature and severity of aggression in schizophrenia. It has to be assumed that the most aggressive and ill patients are mostly reluctant to participate in studies. Therefore, findings from clinical studies, particularly RCTs, have to be supplemented by data from other sources, e.g. observational and epidemiological studies, to allow generalisability to clinical practice and to improve external validity.

A limitation of this literature review is that we confined our search to MeSH terms and to studies published in English and available in PubMed or PsycINFO. However, there is little reason to assume that other studies not fulfilling these criteria are better reported in terms of study designs and outcomes.

CONCLUSION

Most studies on violence in patients with schizophrenia are poorly reported in terms of external validity, and are questionable with regard to generalisability. For future studies, it is essential to report the following methods and outcomes: inclusion criteria, especially a clear definition of aggression or violence on an appropriate scale; number of patients screened for inclusion; number and characteristics of participating and non-participating patients, and comparison of these groups; number and characteristics of finishers and drop-outs, and a comparison of these groups, especially in terms of violent behaviour at inclusion; and, intention-to-treat analyses. In addition, information on consent procedures and incentives should be reported. Table 3 summarizes the recommendations for improving reporting quality.

SUPPLEMENTARY MATERIAL

CONFLICT OF INTEREST

The authors confirm that this article content has no conflicts of interest.

REFERENCES