Posted
November, 2012
Multiple
Facets of Cross-Informant Agreement
Meta-analyses
of correlations between ratings of child, adolescent, and
adult psychopathology by different informants have shown low
to moderate levels of agreement between different informants.
This means that no one informant is likely to provide precisely
the same information as other informants. Instead, comprehensive
assessment requires data from multiple informants, as well
as other kinds of data, to take account of the fact that people's
functioning varies from one context to another and that informants
shape information according to their own perspectives. Although
numerous studies have replicated the modest levels of agreement
between different sources of data regarding psychopathology,
much remains to be learned about factors affecting agreement
levels and, ultimately, how to utilize multi-source data most
effectively.
Three
new studies shed light on different facets of these important
issues. In one study, an Erasmus University team compared
cross-informant agreements for a Dutch general population
sample of 1,875 individuals assessed on seven occasions spanning
24 years from when the youngest were 4 years old to when the
oldest were 40 years old (Van der Ende, Verhulst, & Tiemeier,
2012). Data were analyzed for 12,059 pairs of parent, teacher,
spouse/partner, and self-reports obtained with ASEBA instruments.
It was found that correlations among informants' ratings of
Internalizing and Externalizing problems depended more on
the particular kinds of informant pairs than on the problem
type or age group. However, differences between informants'
ratings of Internalizing problems increased as the participants
grew older. Furthermore, self-ratings generally yielded higher
problem scores than parent, teacher, or spouse/partner ratings.
To determine
whether self-ratings produced higher problem scores than ratings
by others in other societies, Rescorla et al. (2012) compared
YSR ratings by 27,861 11-18-year-olds with CBCL ratings by
their parents in population samples from 25 societies. Mean
scores on all problem scales were higher on the YSR than the
CBCL in all 25 societies, significantly so in all societies
except Puerto Rico. Averaged across societies, significantly
more youths than parents endorsed each of the 98 problem items
common to both forms. Despite the consistently higher YSR
than CBCL problem scores, Q correlations between mean
item ratings by youths and parents for the 98 items ranged
from .72 (Japan) to .94 (Romania), with a mean = .85. This
indicated high agreement between youths and parents with respect
to the items they rated low, medium, or high in each society.
Because youths and parents tended to rate the same items as
low, medium, or high, the higher scale scores on the YSR than
on the CBCL were not due to youths endorsing different
problems than their parents did. Furthermore, bi-society Q
correlations between the mean rating received by each item
in each society averaged .72 across all pairs of societies
for the YSR and .73 for the CBCL. These large Q correlations
indicated considerable consistency in the problems that received
low, medium, or high ratings across the different societies.
The Van
der Ende et al. (2012) and Rescorla et al. (2012) studies
showed that self-reports typically yield higher problem scores
than collateral reports across broad spans of ages and societies
in general population samples. To determine whether such findings
apply to youths assessed in forensic contexts, Penney and
Skilling (2012) compared YSR and CBCL scores for 373 Canadian
12-19-year-olds who had been arrested and were undergoing
court-ordered assessments. As found in the general population
samples studied by Van der Ende et al. and Rescorla et al.,
YSR scores were significantly higher than CBCL scores on the
Withdrawn/Depressed and Somatic Complaints syndromes. However,
unlike the general population samples, YSR scores were significantly
lower than CBCL scores on the Aggressive Behavior syndrome
and did not differ significantly on the other syndromes. Cross-informant
rs of .34 for Internalizing and .47 for Externalizing
were in the same general range as for most other samples.
Other measures indicated that YSR-CBCL discrepancies were
higher in families experiencing high levels of stress and
conflict than in other families. Furthermore, youths who scored
high on measures of arrogance and deceitfulness provided more
discrepantly low ratings on the YSR Attention Problems, Rule-Breaking
Behavior, and Aggressive Behavior syndromes than other youths.
Measures of psychopathic features and self-deceptive enhancement
were also associated with more discrepantly low ratings on
some YSR scales. The authors concluded that the discrepantly
low YSR ratings could reflect youths' intentional efforts
to misrepresent themselves and/or impaired insight into their
problems.
References:
Penney,
S.R., & Skilling, T.A. (2012). Moderators of informant
agreement in the assessment of adolescent psychopathology:
Extension to a forensic sample. Psychological Assessment,
24, 386-401.
Rescorla, L.A. et al. (2012). Cross-informant agreement between
parent-reported and adolescent self-reported problems in 25
societies. Journal of Clinical Child & Adolescent Psychology,
41, in press.
Van der Ende, J., Verhulst, F., & Tiemeier, H. (2012).
Agreement of informants on emotional and behavioral problems
from childhood to adulthood. Psychological Assessment,
24, 293-300.
