Over the last 25 years a community care model for patients with mental disorders has been set up. This psychiatric care model is centred on a community-integrated network of mental health facilities (Community Mental Health Centres (CMHC), Psychiatric Wards in General Hospitals, Residential Facilities and Day-care Facilities) located in the neighbourhood, and coordinated by the Mental Health Department (MHD); this model has nothing to do with mental hospitals.

Despite this model's dissemination throughout Italy and other European and non-European countries, its effectiveness has never been properly assessed. Indeed, there have been few analyses of the system taken as a whole, although there are many assessments of the individual facilities and activities. Comparisons have been made of innovative and routine services, e.g. the PRISM "psychosis study" 1, and local area studies such as the South Verona Outcome project 2, but data to analyse the effectiveness of the daily practice of community mental health services on a large geographic scale are still scarce. An exception to this can be seen in the Australian Mental Health National Outcomes and Casemix Collection (MH-NOCC), in which Burgess et al. 3 describe care episodes provided for adults in public mental health services across Australia. Score changes and effect size on Health of the Nation Outcome Scales (HoNOS) 4 were calculated for 14,659 acute inpatient episodes and 23,692 community episodes. The results showed a general improvement for people in contact with the public sector of mental health services, though the level of change depended on the setting and type of episode. In the present analysis the episodes of care effectiveness were analyzed separately according to episode type.

In the year 2000, the psychiatric network in Lombardy consisted of 63 public Departments of Mental Health, 16 officially licensed private community residential facilities, 4 officially licensed private Day Centres and 3 private psychiatric clinics. Each public Department of Mental Health had, on average, a Psychiatric Ward with 15 beds in a General Hospital, 2 Community Mental Health Centres, 2 Community Residential Facilities and 1 Day Care Centre.

Let us look a moment at the present situation for a population of 10,000 people over the age of 15: The psychiatric facilities network consists of 96 Community Mental Health Centres, 68 Day Care Centres (1.3 daily attendance at each), 58 Psychiatric Wards in General Hospitals (1 bed for each) and 176 Community Residential Facilities (2.3 beds for each). In addition there are 3 active private Psychiatric Clinics (0.2 bed in each).

Today Mental Hospitals, as such, no longer exist in Italy; the last ones, which, during the '80s and '90s had wards for long term patients, were closed in 1999. At that time the Lombardy population aged over 15 years was 7,926,581.

In any case, difficulties can arise when trying to make an adequate evaluation of the effectiveness of mental health services, and these concern:

1. The evaluation of a patient's clinical and psycho-social problems during daily living. Several routine outcome assessment tools are now available, but to be acceptable such tools must be effective – this means they must be reproducible, easy to use (by involved professionals), sensitive to change, able to cover clinical and psychosocial issues and, finally, usable in different settings 5.

2. Description of treatment. Community Mental Health Centres (CMHCs) dispense different treatments, and this information must be summarized in order to evaluate the overall effectiveness of the services. Indeed, the treatment package is an interesting analysis model as it overrules artificial separations in the analysis of mental health activities, and focuses on the complexity of the mental health system. According to the UK NHS definition 6, a package of care is "a cluster of services provided to an individual based on carefully constructed components". Thus, such a package includes the characteristics of the patient, the type of treatment and the intensity of the care provided. Packages of care can be summed up as 'the mix of treatments provided to an individual patient within a specific timeframe involving different settings' e.g., CMHCs, day-care facilities, general hospital wards, community residential facilities) 7.

3. Statistical analysis of data. The most promising instruments for the evaluation of clinical services effectiveness are longitudinal models, and this is due to the recent improvements that have been made to such models 8. The data that lead to the inclusion of a patient in a specific group are derived by individual decisions, not acquired through random patient allocation, thus the selection process itself can be the source of outcome differences. In order to limit such bias, any statistical analysis of effectiveness must meet the "coeteris paribus" rule with respect to the multiple treatment-provider principle; this means that the efficacy indicator must be independent of other operator-dependent confounding factors like differences in patient severity at the beginning of treatment. Therefore, comparisons need to be adjusted for severity, or case-mix. Risk adjustment is a statistical control process of the characteristics of patients that participate in studies concerning treatment quality, costs and outcome 9. It has been proven that comparisons of severity-adjusted data and unadjusted data produce results different from those made of treatment and/or setting performance 10.

The aim of the present study was to evaluate the effectiveness of community Mental Health Departments in Lombardy, applying the above described methodological innovations, and particularly by analysing:

1) care package appropriateness;

2) MHD overall effectiveness

3) effectiveness of the different care packages.

This study at MHDs in Lombardy has enabled a first-time evaluation of mental health service effectiveness in the region. However the results must still be considered preliminary, not only because the percentage of treated patients in the sample was small (less than 1/20), but also because the study was subject to some limitations at the methodological level. Thus, a different study design is needed: a cohort of patients should be followed and assessed, not only at six month intervals but also when settings change e.g., when a care episode concludes at one facility and treatment begins at another. Assessing patients after there has been a change in their settings allows a more comprehensive understanding of any severity changes that might occur. Further improvement to the model could be achieved by including intensity of care as a variable. Although this analysis did not evaluate care intensity, i.e. the number of community contacts, daycare centre attendance and days spent in hospitals and residential facilities provided within the packages, the splitting of each package into terms of high-, medium- and low- resource sub-packages, according to intensity, could refine the model.

Despite these limitations, the present study suggests that the network of community Mental Health Departments in Lombardy affects patient outcome positively; moreover, the organization of the study treatment blocks is appropriate as the most serious patients receive the most complex treatments.

Some packages are more effective than others in improving patient outcomes. Indeed, the package involving both hospital and community facilities gives the best results, while the package involving hospital, community and day-care activities turns out to be less effective. The limited or nil effect shown by the more complex packages, which involve several facilities (like the package with several residential facilities and the one including hospital, community and day-care activities), may be explained by the many difficulties encountered in the coordination and clinical governance of such excessively complex cases (in terms of facilities). Moreover this package is more likely to be reserved for more serious cases, patients who are "resistant to treatment" and not responsive to other, previously provided, less complex packages.

The reduced effectiveness of the clinical package is probably related to patient selection, according to the design of this study. A cross-sectional sample with repeated measurements such as this one tends to recruit chronic cases rather than onsets. Thus this selection may have produced a bias toward more complex packages rather than toward "lighter" treatments that, in fact, are often limited to a short ambulatory treatment; clearly the outcome of these treatments is not highlighted by this study design.

The results related to the effectiveness of community care, for patients with different diagnoses, are contradictory. On the one hand, the model is able to discriminate between the effectiveness of the different packages used for the same diagnosis just as the diagnosis can suggest the care package to be used, on the other hand, for example, the lack of CLIN package efficacy in neuroses is questionable. Thus it is quite clear that for a more conclusive evaluation, further research is needed into the relationship between care packages and diagnoses or, better still, between care packages and specific patient problems.

This survey represents the first evaluation of the effectiveness of community care in Italy involving a relatively large sample of patients. It confirms that it is possible to evaluate the effectiveness of community Mental Health Departments and that such evaluation can be conducted at reasonable costs, provided appropriate tools are used.

The feasibility of HoNOS is a crucial requirement for routine outcome assessments: indeed, it is unthinkable to have the use of more tools for a survey involving 10 Departments of Mental Health and 10,000 patients for a period of a year. Not only has HoNOS been proved useable, it has also been revealed to be "sensitive to change", and able to give a detailed picture of outcomes.

The information system that collects psychiatric information in Lombardy enabled both the sample recruitment and the complete elaboration of the data collection. Thanks to this system, it has been possible to accurately describe the care packages.

The used statistical method proved to be a suitable methodological instrument for an effective evaluation of growing and changing data, as highlighted by Gilbody 26 in his systematic review of mental health services outcomes. In fact, the multilevel growth models 14 not only protected the principle of "severity adjustment" but also brought several benefits, compared to traditional longitudinal models based on uni- or multivariate analysis of variance, contrast analysis and fixed effect models. By clearing the outcome of the effects due to individual differences and the different resources used, the models allow an analysis of context and case-mix variables at different levels of hierarchy, even in observational studies. Such models are flexible and applicable to normally distributed and non-normally distributed continuous outcome categorical variables. They are robust also in the case of irregular, dispersed, and missing data, and can contain time-invariant or time-variant covariates. Such variables often occur in administrative registries. The flexibility of these statistical methods allows the use of otherwise useless data, and enables the recruitment of larger samples.

We firmly believe that the combination of appropriate instruments for the evaluation of routine outcomes, patient-oriented informative systems and appropriate statistical methods will, in the future, lead to useful results that can be used to evaluate the effectiveness of community mental health care.

The authors declare that they have no competing interests.

All authors participated in the design of the study. EM, AL and PL wrote the manuscript. AE and GV reviewed the manuscript. PL and GV performed the statistical analysis. All authors read and approved the final manuscript.