 |
||||||
 |
Home >> IGDI Measures >> ECI (Communication) >> Early Communication Indicator -- Technical SoundnessThe Early Communication Indicator (ECI) was developed in a program of research designed to test its technical soundness as a measure of early communication (see Luze et al., 2001). Some of the important features of soundness, expected of any competent measure, are the attributes of reliability and validity. A measure is reliable when two observers simultaneously recording a child’s early communication return the same, or nearly the same score. A measure is also reliable when a child’s Total Communication score on the ECI on one occasion is comparable to that obtained on another occasion separated by only a very brief period of time, e.g., several days. A measure is valid when it is shown to measure what it is intended to measure, in this case, early communication (criterion validity). One proof of validity is a significant correlation between the ECI and a recognized standardized measure of early communication such as the Preschool Language Scale (PLS - 3: Zimmerman, Steiner, & Pond, 1992). A measure is also valid when it behaves as it is conceptualized to behave (construct validity). A second proof would be finding a significant difference in the communication proficiency of older children compared to younger children – because in general, we expect older children to use spoken words and to be more proficient communicators than younger children in the birth to 3 years age range. Sample Description Sample 1 (Original Sample, N = 50 Children) Fifty children who varied in race, SES, and with and disability status were purposively sampled from five childcare centers to capture a wide range of individual differences in communication. Centers served children birth to three years of age. Any child in the target age range at each center was eligible and all children whose parents provided written permission were included. A university-affiliated center located in one city served both typically developing children and children with disabilities. Four other centers were located in a second city: two serving urban and two serving suburban populations. Two of these centers had as a primary mission serving the children of teen parents attending special high school programs. All five centers served children from a number of cultural and linguistic groups, two centers had a majority of non-white families. The number of participants from each center ranged from 2 to 25. The attained education of the participating children’s mothers ranged from less than high school (19%), to high school diploma/GED (8%), to some college or college degree (42%), to graduate school (31%). Sixteen, 20, and 14 children comprised three age cohorts, (0-12, 13-24, and 25-36 months of age). At the start of the study, children in the sample ranged in age from 3 to 34 months (M = 8.5 months). Forty-eight percent of the children were female, 52% were male. Thirty-seven (74%) of the children were European-American/White, 8 (16%) were African-American/Black, 3 (6%) were Hispanic, and 2 (4%) were of mixed race. Six children had identified disabilities and Individual Family Service Plans (IFSPs) based on information provided by childcare center staff. Parents were not asked to specify the type of disability the children had, however, the majority of children identified at this age are considered to have a general developmental delay. The mean ages of the children with (n = 6) and without IFSP’s (n = 44) were not significantly different at 20.1 months versus 17.1 months at the start of the study. Sample 2 (Composite 2005 Sample, N = 1,653) In order to improve the representativeness of ECI norms, data from three new samples and studies were combined with those of the original sample. Like the original sample (n = 50), two of the new samples were from research studies using the ECI and one from a statewide Early Head Start effort. Collectively, this aggregate sample of children represents children drawn from 22 center and 14 home-based programs in Kansas participating during 1999 to 2004 ranging from community- to university-based programs, and including Part C, and Early Head Start programs. It also represented children who were racially diverse, with Spanish and English as home languages, and children with and without identified delays/disabilities. And, although the combined sample was predominantly from children reared in low-income families, middle- to high-SES children and families were also represented. Sample Characteristics of the 2005 Early Head Start sample are described below:
Technical Measurement Results Reliability - Interobserver Agreement (the following are based on the Original Sample unless otherwise indicated) Interobserver agreement assesses the extent to which two observers produce the same score. Agreement assessments tap the extent that two observers record the key skills elements displayed by the same child being observed by both observers at the same time. High percentage agreement indicates that observers are well trained because they understand and apply the key skill element definitions in the same way in the recording process. Percentage Agreement Findings
By Key Skill Elements, the scores between two observers of the same child were in agreement to the following extent:
Prior research and practice confirms that relatively high levels of Interobserver agreement are attainable with the ECI. Use of words, single and multiple, in the 70% or higher range are the lowest and clearly most challenging for observers to agree on their occurrence. Consequently observer training should focus strongly on these definitions and factors that affect hearing and understanding children’s spoken language during recording. Reliability - Split-half (Odd vs. Even) This form of reliability tests the comparability of ECI Total Communication scores when formed using odd versus even observation occasions. Split-half reliability correlations were:
Both the child’s mean level (average score based on all of an individual’s scores) and slope (rate of growth) have reasonable reliability levels – mean level at .89 particularly so. Reliability - Alternate Forms This test of reliability compares Total Communication scores formed when observations were made using alternate toys, in this case the Barn versus the House.
This reflects a moderate to high level of reliability – highly similar scores result regardless of the toy used. Criterion Validity Criterion validity is designed to test whether or not the ECI Total Communication score correlates to other measures of early communication. Moderate to high levels of correlation indicate that the ECI does measure early communication. Two criterion validity measures were selected to assess expressive communication and to provide a multi-method, multi-informant dimension to the validity study.
Does the ECI measure early communication?
Is the ECI sensitive to age differences in early communication?
Is the ECI sensitive to changes in Key Skill Elements (Gestures, Vocalizations, Single and Multiple Words)?
Is the ECI sensitive to changes over time (Composite Sample Data)? Because of the small numbers of children younger than 6 months of age with data, results for these children were not reported.
Is the ECI sensitive to growth over time (Composite Sample Data)?
|
|||||
|
|
|
|||||
|
||||||
|
||||||
