• Set is the collection of items being compared.
• Script is the item being evaluated, and comes from early research literature on comparative judgment.
• Test is the open-ended task that prompted student work and the standardized conditions or parameters within which a student freely responds.
• Score is the estimated parameter value of item after comparison.
• Judge is the person doing pairwise comparisons.
• Student ID is the anonymous ID code that links an item to the person who created it.

The reliability of results is expressed as confidence intervals above and below an estimated score. Simplicitest provides a 95% confidence interval, which may be interpreted as the range within which 95% of score estimates would fall with repeated sampling judgments from the same judges. This is useful as for evaluating the precision of the measurements and for determining the significance of differences between scores.

As a rule of thumb, when lower confidence interval limits of one estimated score align with upper confidence interval limits of another estimated score, this indicates a 1% likellihood that there isn't an actual difference between the two scores. Overlap of half the length of one arm indicates a 5% likelihood that there really is no difference between the two scores. Both of these thresholds may be interpreted as statistically significant differences between the scores (p = .01 and p = .05 respectively). (See Cumming (2008), Inference by Eye: Reading the Overlap of Independent Confidence Intervals.)

The resulting measurements include ranks, percentile ranks, and scale scores of the scripts in a set.

Scores are valid measures of the independent, holistic judgment of educators with appropriate experience to serve as assessment experts in their field.

The validity of the scores depends entirely on the context represented by those assessing the student work.

For more discussion on interpreting validity, contact us.

Thigmo Limited Co. (Thigmo) operates www.simplicitest.com (Simplicitest), a website and service provided to teachers in the U.S. for reciprocal, holistic assessment of their students’ work using the adaptive comparative judgment (ACJ) method.

Thigmo is operated by Steven Heil, a second-year doctoral student at Indiana University School of Education, Department of Curriculum and Instruction, specializing in Art Education with a minor in Quantitative Inquiry and Research Methodologies. His resumé includes papers and presentations on relevant topics at the Art Education Research Institute Annual Symposium, National Association for Gifted Children convention, National Art Education Association convention. He was awarded a 2021 Research Grant from the National Art Education Foundation for research to establish the software and methodology to be used by Thigmo.

Contact us: info@simplicitest.com

During the pilot year (2024), users will be art educators exchanging reciprocal assessment services for results reports. Users will be limited to assessment of Clark's Drawing Abilities Test (CDAT) only. A summary of literature about the CDAT may be found at the CDAT Project.

1. Understand what CDAT measures

CDAT prompts students to draw pictures from their imagination with a pencil on plain paper and with just ten minutes for each picture. There is no single correct response to the open-ended prompts. A typical response shows the level of the student’s acquisition of conventions of pictorial representation.

Conventions of pictorial representation are the shared expectations that help makers of the pictures communicate with viewers. Conventions may include expectations for representing space, time, movement, light, atmosphere, and familiar objects.

The assessment is holistic, meaning that judges use their own implicit criteria when comparing student work, but the constraints of the CDAT isolate of pictorial representation skill and disposition as the most prominent construct assessed.

Judges may not all share the same expectations for pictorial representation. Their differences are examined and accounted for in the score calculations.

The constraints of the task—the ten-minute response time and the materials of pencil and paper—limit the likelihood of creating a deeply thoughtful and complete work of art. Therefore the drawings are not judged with the same level of subjectivity as works of art. This is to be expected. Evaluating pitcorial skill with a drawing test without assessing artistic quality is like evaluating skill with writing for a language test without concern for its poetic quality. In both situations, the ideas expressed and the conventions of communication would be the target for assessment.

2. Understand who the CDAT is designed to test

CDAT student form and directions were designed, are suitable, and have been used for decades for testing school-age children with or without disabilities from age 5 through 17. An updated form of the test includes a side-by-side Spanish translation appropriate for Spanish first-language English learners and a rewording of the fourth prompt to use the word “choice” rather than “fantasy.” The test is designed to be administered in large groups, small groups, or individually. When tested in groups, individuals should be separated enough to prevent excessive sharing of ideas for responses to the prompts, though some sharing would not invalidate the tests. If a student can match the skill of their more advanced neighbor within the time constraints of the test, their quick mastery of conventions of pictorial representation may contribute to their CDAT score.

3. Understand how CDAT is assessed on Simplicitest

The assessment system combines the holistic judgments of at least three art educators who substantially agree about what makes one student’s response more advanced than another. Although most assessors are primarily looking for evidence of skill and knowledge of drawing from imagination, other constructs such as originality of ideas, emotional effect, or formal design qualities may contribute to their judgments to a lesser extent.  

4. Understand the value of the constructs being assessed

Children who spend more of their free time drawing from their imagination or from observation with a pencil on paper may develop more skill and knowledge of the conventions of pictorial representation. They would practice skills that have been taught to them or that hey have learned on their own by studying whatever illustrations, images, photos, or videos are accessible.

The validity of drawing as evidence of art ability relies also on the fundamental relationship between drawing and other media as a tool for planning and ideation. An argument for validity further relies on the meaningfulness drawing has for children picturing what they imagine in play. Unlike other art mediums, drawing from imagination with a pencil on paper is accessible to nearly all children, with or without instruction. Drawing with pencil on plain paper provides nearly universally valid evidence of drawing ability.

Assumptions that student responses will vary based on ability, speed of learning in art, and motivation to spend time making art further establishes the value of a test based on drawing.

Adolescents who continue to make art with other media in either formal or informal settings may still use drawing as a tool to begin to realize ideas, even if drawing is not their preferred medium. Their skill, knowledge, and disposition to draw pictures from imagination will have been bolstered by early practice as a younger child and current explorations in drawing for ideation in other mediums as a youth. Facility with pictorial representation, if not abandoned in intermediate years, may be useful also for adults when applied to play, projects, hobbies, or work. Skill, knowledge, and disposition for pictorial representation, as the primary construct assessed by CDAT, may have lasting value throughout a lifetime.

5. Match your purpose for testing with your choice of test

During the pilot year on Simplicitest, as appropriate age and grade norms for various populations are being established, only whole classes or grade levels may be enrolled, not isolated individuals or small groups. Administering the test to all students of a grade or class ensures that all students’ results are locally normed, meaning that their reported rankings and percentile rankings will be relevant in the context of other students with similar opportunities to learn.

Reports for the whole class will include recommended interventions for advanced, average, and below-average performance on the CDAT compared with others in a norm set. Recommended interventions include learning objectives for skill, knowledge, and dispositions for pictorial representation as well as goals to develop appropriate strategies for ideation and formal design.

The results will not include a recommended cut score for advanced or gifted and talented programs. For high-stakes educational decision-making, a single scaled score, rank, or percentile rank is not sufficient. A student’s performance on one test may contribute, however, to a body of quantitative and qualitative evidence used for identification of enduring educational need. Reporting results to students and their parents or guardians should always accompany recommendations for individually appropriate next steps in learning.

6. Communicate appropriately with students and parents or guardians

Simplicitest users must communicate appropriately with parents or guardians to comply with legal requirements protecting student and family privacy and to align with principles of respect, beneficence, and justice that protect individuals involved in research.

7. Testing as normal practice

Administering any test may function as part of educators’ normal educational practice or as a diagnostic instrument requiring informed consent. Follow your school or school district’s policies and procedures for communicating with parents about testing. When enrolling students on Siplicitest, you may anonymize the student data, that is, leave the name fields in the form blank, and instead associate the tests and results by a locally meaningful, unique identification number .

8. Understand the rationale for validity, reliability, and precision of the Simplicitest results

Validity. In addition to the validity rationale detailed above, the results of CDAT assessment on Simplicitest come from the independent agreement of at least three experienced art educators. Their criteria for what makes one drawing more advanced than another are the shared knowledge of experts in the field.

Reliability and precision. The reliability of each score is estimated based on the amount of assessment information collected about the student response. Reliability is expressed as 95% confidence intervals. Confidence intervals are computed from the SEM to show the range of likely scores if the same student response were assessed again under similar conditions. Largely overlapping confidence intervals show that the placement of two student responses on the continuum is imprecise, and the two responses may be reversed in rank under alternate, similar conditions. Minimally overlapping confidence intervals show that the placement of two items on the continuum is more precise, and the two ranks are unlikely to trade places when assessed under alternate, similar conditions.

9. Be prepared to administer the test under standardized conditions

Follow the administration protocol for CDAT, follow the directions, and maintain the same testing conditions as everyone else who administers the test. This will ensure that the only thing that is measured is the construct of interest for individuals in the testing. Varying the conditions of the test may result in scores that correspond more to the conditions than the individuals. If a CDAT session is interrupted, shortened, or extended, or if an individual shows insufficient effort, attention, or energy, note the instance and discuss implications with Simplicitest. For special accommodations, note the accommodations required for students to show what they know and can do related to the construct independent of disabilities.

10. Review the Simplicitest Privacy Policy and Terms of Use

Make sure you are comfortable with our Privacy Policy and Terms of Use and agree before enrolling students. To safeguard the integrity, validity, and fairness of the testing process, educators have a duty to administer a test only with the particular considerations:

• Clarifying, as much as possible, the reasons a test taker will be assessed;
• Selecting a test to align with the purpose of the assessment;
• Selecting a test appropriate for the demographic subgroup being examined;
• Being familiar with the evidence for validity for the intended uses of scores from the selected test;
• Being familiar with the evidence of reliability/precision of scores from the selected test;
• Being familiar with the meaning of the construct being assessed by the selected test;
• Establishing testing conditions that are standardized;
• Establishing testing conditions that are appropriate to the test taker’s needs and abilities;
• Describing and justifying any non-standardized test administration conditions, such as accommodations, differences in time limitations, materials, environment, grouping, or effort;
• Understanding who will have access to the test results and reports and how the test results will be shared with the test taker;

Users understand the Privacy Policy and agree to the Terms of Use.