Home >> IGDI Measures >> ECI (Communication) >>  

Early Communication -- Checking Quality: Interobserver Agreement on ECI Recording

To evaluate Interobserver Agreement two observers should plan to record the same child and ECI session each using the ECI Recording Sheet. To ensure independence in the recording of each observer, sit apart and face recording sheets so as not to influence one another.

Both observers must start and stop recording as described above and one may signal the other with a gesture when to start and to stop. Achieving nearly equivalent scores will depend on viewing the same child and play partner, starting and stopping at the same time, knowledge of the key skill definitions, and experience using the timer and the recording sheet.

1. Each observer uses a recording sheet, recording the child’s communicative as previously described.
2. After recording is completed, sum the frequencies for each skill element on the Recording Sheet as usual.
3. Begin the Agreement Check Calculation by entering these totals on the first row of the Calculation Sheet for the Primary Observer (row 1); second row of the Calculation Sheet for the Secondary Observer.

If you are using the Calculation Sheet, follow the directions on the bottom of the sheet. They are as follows:

• Record Primary coder scores in first row
• Record Reliability coder scores in second row
• Record the number on which they agreed on the third row
• Record the number on which they disagreed on the fourth row
• Calculate Percent Agreement for each Key Skill Element (by column) using the formula below dividing agreements by agreements + disagreements x 100.
• Sum the Key Skill Element Agreements (ATotal) and Disagreements (DTotal) to find row totals for each (use the third and fourth rows)
• Calculate the Overall Percent Agreement using Agreements (ATotal) and Disagreements (DTotal) this formula: Percent Agreement = ((Number of Agreements/Number of Agreements + Disagreements) x 100)