Scoring Individual Community Readiness Assessment (CRA) Interviews*
With the final implementation of the Community Needs Assessment (CRA) “just-around-the corner” (the CRA process for grantees will begin in February 2022), the NLCC T/TA team thought it would be helpful to highlight two (2) key points for scoring individual key informant interviews. Scoring of individual interviews is a labor intensive and important process for producing valid CRA results. For grantees using the same staff as scorers for the CRA process, this brief article serves as a refresher for scoring individual interviews. For those grantees with new staff to the CRA process, this article well help prepare them for scoring individual interviews. Key points are presented below.
Key Point #1: In order to get a general feeling and impression from each individual interview, read through each individual transcribed interview in its entirety (or if applicable, watch the entirety of the video recording of the interview) before scoring any of the dimensions. Take notes, highlighting key responses that may give insight into scoring the dimensions.
Why Reviewing Interviews Before Scoring is Important. Although questions in the CRA are arranged in the interview to pertain to each of the six (6) specific dimensions (e.g., community efforts, community climate), other interview sections may have some responses that will help provide richer information and insights that may be helpful in scoring other dimensions.
Key Point #2: Each of the six (6) dimensions have an anchored rating scale from one (1) to nine (9). Scorers should read the anchored rating scale for each of the dimensions being scored before beginning the scoring process. Always start with the first anchored rating statement. Using the dimension of Community Knowledge of Efforts for an example, if the community exceeds the first statement (community has NO knowledge of the need for efforts addressing language acquisition/language retention), proceed to the next statement. In order to receive a score at each stage, all previous levels must have been met up to and including the statement which the scorer believes best reflects what is stated in the interview. In other words, a community cannot be at stage 7 and not have achieved what is reflected in the statements for stages 1 through 6.
Anchor rating scales are intended to capture more accurate ratings of community attitudes/ behaviors by providing specific examples associated with a rating, instead of generic descriptors commonly found with traditional Likert-type scales (e.g., agreement levels).
Tips for Scoring. After scoring a dimension, move to scoring the next dimension. Consider any dimension score during the scoring of the individual interview as preliminarily, as what is learned in scoring subsequent dimensions may either validate a score for a previous dimension or provide cause for modifying the score. Once completing scoring for each of the six (6) dimensions, then each of the dimension scores can be considered final.
Anchored Rating Scale for Scoring
You may assign scores in intervals of .25 to accurately reflect a score. The hyphens (“-“) under each of the levels of readiness (i.e., 1 through 9) for each dimension indicates intervals of .25 (e.g., 1.00, 1.25, 1.50, 1.75, 2.00).
Dimension B. Community Knowledge of Efforts
1 = Community has NO knowledge of the need for efforts addressing language acquisition/language retention.
2 = Community has NO knowledge about efforts addressing language acquisition/language retention.
3 = A few members of the community have heard about efforts, but the extent of their knowledge is limited.
4 = Some members of the community know about local efforts.
5 = Members of the community have basic knowledge about local efforts (e.g., purpose).
6 = An increasing number of community members have knowledge of local efforts and are trying to increase the knowledge of the general community about these efforts.
7 = There is evidence that the community has specific knowledge of local efforts including contact persons, training of staff, tribal members involved, etc.
8 = There is considerable community knowledge about different community efforts, as well as the level of project effectiveness.
9 = Community has knowledge of program evaluation data on how well the different local efforts are working and their benefits and limitations.
Download this scoring scale as an image at this link.
*Key points are based upon the recommendations from The Community Tool Box developed by the Center for Community Health and Development at the University of Kansas regarding the Community Readiness Assessment process.