Home >> IGDI Measures >> EMI (Movement) >>  

Early Movement -- Technical Soundness

The Early Movement Indicator (EMI) was developed in a program of research designed to test its soundness as a measure of early movement (Greenwood, Luze, Cline, Kuntz, & Leitschuh, 2002). Some of the important features of soundness, or the technical adequacy expected of any sound measure, are reliability and validity.

A measure is reliable when two observers simultaneously recording a child’s performance return the same, or nearly the same score. A measure is also reliable when a child’s score on one occasion is comparable to that obtained on another occasion separated by only a very brief period of time (e.g., several days).

A measure is valid when it is shown to measure what it is supposed to measure, in this case, early movement.

• One proof of validity is a significant correlation between the EMI and a recognized standardized measure of early movement such as the Peabody Developmental Motor Scales (PDMS-2: Folio & Fewell, 2000).
• A second proof would be finding a significant difference in the movement proficiency of older children compared to younger children – because in general, we expect older children to be more proficient than younger children in the birth to 3 age range.

Sample Description

Children were recruited at two child care centers serving infants and toddlers located in the inner city neighborhoods of metropolitan Kansas City. The centers served children of varied racial and socioeconomic backgrounds. Both centers were affiliated with neighboring high schools serving adolescent mothers. Any child in the center who was in the 0 to 36 month age range was eligible to participate in the study. Each eligible child’s parent received a packet of information that included an informed consent flyer and demographic questionnaire. Any child whose parents returned a signed informed consent was included in the study. Thirty-nine informed consent forms (77%) were returned with parental permission.

From this recruited sample, 34 children completed some aspects of measurement and 29 met a minimum criterion of three repeated EMI data points in the analysis sample. The mean age of these children at the start of the project was 15.3 months, with a range of 1 to 34 months (SD = 9.6 months). Fifteen (52%) of these children were male, 14 (48%) were female. For analytical purposes, these children were assigned to three age cohorts: birth to 12 months, 12 to 24 months, and 24 to 36 months based on their age at the first measurement. The mean ages for these cohorts were 6.4 (n = 13), 18.4 (n = 10), and 29.5 (n = 6) months, respectively.

The racial breakdown of the sample was African American (84.6%), Hispanic/Latino (5.1%), European-American (5.1%), and other or mixed races (5.1%). Five parents (17%) reported that their children were involved in Part C programs serving children with disabilities and had Individualized Family Service Plans (IFSP). One of the children had Down Syndrome, the others had general developmental delays. Three children were at risk for a delay in movement as evidenced by their scores on the Peabody Developmental Motor Scales (Folio & Fewell, 2000) movement developmental quotient score, at or below 0.78 (-1.5 SD). One of these children was described by teachers as having hydrocephaly.

Demographic information obtained from parents revealed that annual family income levels ranged from $0 to over $50,000, the modal income bracket was $10,000 to $17,000. Twenty-three percent of families had very low annual incomes in the $0 - $9,999 range. The mothers’ highest educational level ranged from eighth grade to six years of post high school education. The modal level of attained education was 11th grade with 44% of mothers indicating that this was the highest level of completed schooling.

Technical Measurement Results

Reliability - Interobserver Agreement
Interobserver agreement assesses the extent to which two observers produce the same score. Agreement assessments tap the extent that two observers record the key skills elements displayed by the same child being observed by both observers at the same time. High percentage agreement indicates that observers are well trained because they understand and apply the key skill element definitions in the same way in the recording process.

Percentage Agreement Findings

• 93% (SD = 0.07) Overall (for a randomly selected 78 [38%] of 206 assessments made)

By Key Skill Element, percentage agreement scores were:

• 93%, transitional movement
• 88%, grounded locomotion
• 89%, vertical locomotion
• 89%, throwing/rolling
• 86%, catching/trapping

Pearson r also was used to calculate the similarity between observers’ scores.

• .98 for total movement Overall

By Key Skill Element, correlations were:

• .97 transitional movement
• .96 grounded locomotion
• .95 vertical locomotion
• .95 throwing/rolling
• .92 catching/trapping

A final analysis compared the observers’ counts of each of these behaviors. In all cases, observers' mean values were highly similar and not significantly different with the exception of catching/trapping (M = 0.54 responses per minute vs. M = 0.65 responses per minute, t(77) = -2.001, p = 0.05).

Reliability - Split-half (Odd vs. Even)
This form of reliability tests the comparability of EMI scores when scores are based on odd versus even observation occasions and compared.

Split-half reliability findings were:

• Pearson r = .88 for mean Total Movement.

Reliability - Alternate Toy Forms
This test of reliability compares movement scores formed when observations were made using alternate toys, in this case the Blocks/Balls (BB), Shopping Cart (SC), and Window House (WH).

Findings

• Correlations were strong and positive at 0.85 (BB versus WH), 0.84 (WH versus SC), and 0.91(BB versus SC) and each statistically significant (p < 0.001).
• Total movement mean estimates were 9.2 versus 9.4 (BB versus WH), 10.1 versus 6.8 (WH versus SC), and 9.9 versus 7.0 (BB vs. SC).
• Paired t-tests indicated no differences in total movement for BB compared to WH. However, SC produced estimates that were on the order of 3 responses per minute lower than that of either BB (t [24] = 4.73, p < 0.001) or WH (t[25] = 5.22, p < 0.001).

Thus, the Shopping Cart’s use is not recommended in favor of BB and WH because it will increase the variability in children’s movement scores when used.

Criterion Validity
Tests of validity were conducted to test whether or not EMI scores correlated with other measures of movement. Two measures differing in informant and method of assessment were used. Together, these measures provided information from both trained examiners and parent-caregivers. These measures were administered both at the beginning and 8 months later separated by EMI measurements conducted every 3 weeks.

The first measure was a standardized test, the Peabody Developmental Motor Scales 2 (PDMS-2: Folio & Fewell, 2000). The PDMS-2 is comprised of 6 subscales: reflexes, stationary, locomotion, object manipulation, grasping, and visual-motor integration. Appropriate to the 0-3 age range, however, only four of the six gross motor subscales were used. Thus, the stationary and locomotion subscales were administered to all children, whereas the reflexes subscale was used only with children 12 months or younger, and the object manipulation subscale (e.g., kicking, throwing, etc.) was used only with children 12 months and older.

The developers created the second criterion measure to tap parents' perceptions. The Caregiver Assessment of Movement Skills-Gross Motor (CAMS-GM – Kuntz, 2001) is a 40-item scale (39 objective items, 1 open-ended item) with items modeled after the Bayley Scales of Infant Development - II (Bayley, 1993), the Denver Developmental Screening Test - II (Frankenburg & Dodds, 1990), and the Alberta Infant Motor Scale (Piper & Darrah, 1993, 1994).

Does the EMI measure early movement?

As shown below, strong correlations were in evidence between total movement and the individual validity measures:

• r = 0.90 and r = 0.86 at time 1 and time 2 between total movement and the PDMS-2 locomotion scale.
• r = 0.80 and r = 0.77 at time 1 and time 2 between total movement and the PDMS-2 stationary scale.
• r = 0.85 and r = 0.87 for time 1 and time 2 between total movement rate and the CAMS-GM total movement score.

Is the EMI sensitive to age differences in early movement?

As shown below, children were increasingly more proficient in total movement by year of age:

• 0 to 12 Months: M = 5.26 total movements per minute (mean level)
• 13 to 24 Months: M = 9.95 total movements per minute (mean level)
• 25 to 36 Months: M = 14.65 total movements per minute (mean level)

• 0 to 12 Months: M = .65 total movements per minute per occasion (slope)
• 13 to 24 Months: M = .35 total movements per minute per occasion (slope)
• 25 to 36 Months: M = .05 total movements per minute per occasion (slope)

As children aged, the rate of growth (slope) declined suggesting an accelerating then decelerating curve (quadratic) rather than a linear growth curve over the entire age range.

Is the EMI sensitive to changes in Key Skill Elements (transition in position, grounded locomotion, vertical locomotion, throwing/rolling, catching/trapping)?

Summary (View a .pdf chart of the summary data described below):

• The major movement activities for children between 3 and 12 months (Cohort 1) measured by the EMI were emergence of and growth in transitional and vertical locomotion, followed in order by grounded locomotion, throwing and rolling, and catching and trapping.
• For children between 13 and 24 months (Cohort 2), these skills all occurred more often and continued to grow (transitional movement; throwing/rolling) or level off (vertical and grounded locomotion) with catching/trapping still emerging.
• For children between 25 and 36 months (Cohort 3), these skills again occurred more frequently but trends over time were declining (vertical locomotion) or leveling off (transitional movement; grounded locomotion). Throwing/rolling and catching/trapping continued to grow.

Is the EMI sensitive to growth over time?

• The mean rate of growth was 0.38 total movements per minute per month of age

When translated to real time, the average number of movements across all children grew by 0.51 movements per minute per month of age . (View total movement growth chart)