In the first stage of a youth’s involvement in WIA program activities, an objective assessment is mandated to identify important baseline information about the youth:

• Academic levels, including basic skill levels
• Skill levels, including occupational skills
• Supportive service needs
• Prior work experience, employability, interests, and aptitudes, including interests and aptitudes for nontraditional jobs
• Developmental needs

In addition, the results of objective assessment are a baseline against which the youth’s progress can be measured; they serve as a pre-test. As the youth receives the services listed in the ISS, ongoing assessment identifies gains and improvements in academic levels, skill levels, and developmental needs. If there are gains and improvements, the ISS should evolve to meet the youth’s evolving needs, ensuring that ongoing services remain appropriate to the youth.

• Younger Youth Skill Attainment Rate, under statutory Performance Measures, identifies and documents gains in basic skills, occupational skills, and work readiness skills by all in-school youth and by out-of-school youth assessed to be in need of those skills.
• Literacy/Numeracy Gains, under Common Measures, identifies and documents literacy/numeracy skill gains by out-of-school youth who are basic skills deficient.

Initial Assessment and Objective Assessment

Objective assessment, on the other hand, does initiate participation in WIA and involves formal assessment. There are many types of formal assessments used in WIA youth objective assessment. The most familiar type of formal assessment is the achievement test. This type of assessment measures what a youth already knows or can do. Examples include a math test or a basic skills test.

Another type of achievement test is a skill or performance test, which requires test-takers to demonstrate a skill or procedure, rather than just demonstrate knowledge about it. An occupational skills test is one example.

Other formal assessments measure traits, characteristics, interests, or aptitudes related to work and careers, like a career interest survey, a vocational aptitude battery, a life skills inventory, a learning style assessment, a measure of depression, or an intelligence scale.

The Ohio Department of Job and Family Services (ODJFS) mandates the use of specific basic skills assessments for objective assessment of WIA out-of-school youth; ODJFS also strongly recommends a specific instrument, TABE 9/10, for the required basic skills assessment for in-school youth. Local Workforce Investment Boards (WIBs) are free to select appropriate assessments for traits, characteristics, interests, or aptitudes related to work and careers. Assessments given by schools or other agencies up to six months prior to date of participation may be used for literacy/numeracy measures for both older and younger youth.

Fortunately, assessment developers and publishers provide a great deal of information and guidance for users of assessments such as WIA staff. The following are some key questions that should be addressed for each assessment to be conducted:

• Is this an appropriate assessment?
• How should this assessment be administered?
• How should this assessment be interpreted and used?
• How do I ensure that this assessment is conducted ethically?

• To measure a youth’s current skill level. A youth’s academic skills, basic skills, occupational skills, and work readiness skills should be measured:
  • At the beginning of program participation as the basis for planning services
  • Periodically during program participation to identify skill gains
  • After program participation to determine final outcomes

• To determine a youth’s employment-related interests and aptitudes. What kinds of work tasks appeal to a youth? What fields of work interest the youth? How about specific occupations? What is the youth good at? What characteristics and traits does the youth have, and what occupations or careers are suited to those? Answers to questions like those help youth select realistic educational and employment goals.

• To diagnose a youth’s disabilities or developmental characteristics and needs. Physical, cognitive, and emotional/behavioral disabilities must be carefully accommodated in planning and providing WIA youth services. Other developmental characteristics or needs (e.g., life skills, maturity, self-confidence) can indicate a need for particular program services.

Although WIA youth staff may administer a pre-screening assessment for learning disabilities (with appropriate training), only qualified specialists should make a formal diagnosis of disabilities. In other cases that call for formal assessment, WIA youth staff can use the following guidelines to select an appropriate assessment:

• Use a criterion-referenced assessment, if possible.
• Use a standardized assessment, if possible.
• Use an assessment with evidence of reliability.
• Use an assessment with evidence of validity.
• Use an assessment designed for the appropriate target population.
• Use the same assessment for pre-test and post-test.

Assessments are usually designed to be scored in one of two ways.

If possible, WIA youth programs should use criterion-referenced assessments to identify youth’s academic levels, including basic skills levels, and skill levels, including occupational skills. Criterion-referenced assessments provide detailed information on which specific skills youth have and which they don’t have. Norm-referenced assessments, on the other hand, only tell you how one youth compares to other youth – they don’t tell you which skills the youth has and which skills the youth lacks. So, norm-referenced assessments don’t provide the detailed information necessary to identify specific services needed or to measure skill gains over time.

An assessment is standardized when every test-taker has an equal opportunity to accurately demonstrate his or her knowledge or other characteristic being measured. Developers achieve this, in part, by examining individual items and the assessment as a whole to eliminate any possible racial, ethnic, cultural, linguistic, or socioeconomic bias. The assessment is normed against a representative sample of the target population to verify that items are not biased and to provide statistical evidence of quality.

Developers also provide standard instructions for administering the assessment. Instructions are often elaborate, including scripts that must be read verbatim. This ensures that all test-takers are receiving the same directions – if directions are ambiguous, at least they are ambiguous for everyone!

Instructions may also recommend adjustments in assessment procedures for youth with specific disabilities or for youth who speak English as a second language (ESL), along with statistical evidence of the quality of such adjustments. Finally, specific and detailed instructions are provided on scoring the assessment and interpreting results.

Reliability is a critical technical characteristic of a good assessment; an assessment is reliable if it gives consistent results – over time, across different administrations, or across different raters. Positive correlations (+0.6 or greater) provide statistical evidence of reliability.

Expert judgment. The assessment should be a representative sample of the domain of knowledge and skills being assessed, and that is a matter of expert judgment. So, look for an assessment developed or endorsed by authoritative, acknowledged content experts in the domain.

On the other hand, a negative correlation, between -0.6 and -1.0, would occur if one factor increases (for instance, a score on a coping skills assessment) as another decreases (for instance, aggressive behavior). Such a negative correlation can also be evidence for validity.

Logical and intuitive appeal. Youth sometimes may not see the connection between the assessment items and the knowledge or skills being assessed. If the items don’t have a logical, intuitive appeal to youth, youth may:

• Be suspicious of the assessment, deny its importance, and refuse to accept its results.
• Be frightened and experience anxiety – “test anxiety” – that can artificially lower their results.
• Not take the assessment seriously and not try to do well, leading to lower results.

Use an Assessment Designed for the Appropriate Target Population

Any assessment selected should be designed specifically to be used by youth like the WIA youth population – ages 14 -21, from a variety of racial, ethnic, cultural, and socioeconomic backgrounds, and with barriers to success.

For example, consider a basic skills reading assessment. If the assessment is written for adults, many items on the assessment may involve aspects of independent adult living – voting, purchasing a house, personal finances, parenting, and so on. A 16-year-old high school dropout might score low on the assessment because the content of the items is unfamiliar – not because he or she doesn’t have the required reading skills.

Finally, some assessments are available in both a short form and a long form; using the short form can help youth perform better. With a long form, youth can lose concentration and become fatigued – or even become so discouraged they exit the program prematurely, especially if they have had bad testing experiences in the past.

Use the Same Criterion-Referenced Assessment for Pre-Test and Post-Test

Youth should have an opportunity to familiarize themselves with the format of the assessment and its items and any materials or equipment being used (calculator, scientific apparatus, computer). Computers are increasingly being used for assessment and may be an attractive option for youth.

Most youth are comfortable with computers, but keyboarding, mouse use, and reading lengthy material from a screen may all detract from the knowledge and skills being assessed. If computers are used, policies and procedures should be in place to deal with software, hardware, or power failures.

Environmental factors can interfere with test-takers’ ability to demonstrate their true knowledge or skills so the physical environment should be controlled as much as possible:

• Set the thermostat at a comfortable level – not too hot, not too cold.
• Eliminate noise or other distractions.
• A clock should be visible during a timed assessment.
• Make snacks and water available during a long assessment.

Assessments should be scored and results should be interpreted and reported as prescribed in the developer’s instructions.

Percentage score. In order to make judgments about performance, raw scores are converted into percentages for criterion-referenced assessments. A percentage score is calculated using the number of correctly answered items divided by the total number of items; for instance answering 18 of 20 items correctly achieves a 90 percent score. The percentage shows how closely an individual approaches the criterion or standard.

Cut score. Cut scores, or passing scores, specify the score that is considered proficient for the assessment – a “passing grade,” so to speak. Cut scores may be set using raw scores, percentages, or percentiles.

Percentile. The percentage differs from a percentile, or percentile rank, used with norm-referenced assessments. A percentile is calculated by determining the percent of scores within a group that fall below a given raw score; for instance, a score in the 90th percentile indicates that 90 percent of other test-takers received the same or lower score. Percentile ranks are not always evenly distributed and do not necessarily indicate quality of response. A test-taker could be in the 99th percentile and have correctly answered only 50 percent of the items, if everyone else in the norm group answered 50 percent or fewer of the items correctly.

Grade equivalent. Another way that scores are expressed is the grade equivalent – for example, a 7.6 grade equivalent score on a math test. However, a 7.6 grade equivalent score does not mean that the youth has math skills appropriate to the 6th month of grade 7. Rather, it means that the youth’s raw score was average for youth in the norm group who were in the 6th month of grade 7 – whether those youth had math skills appropriate to that grade level or not.

Educational Functioning Levels

Be wary of both perfect scores and scores that approach chance (scores that could be achieved by random guessing); neither of those scores provides an accurate result. A youth who receives a perfect score may be taking an assessment that is too easy; a youth whose scores hover around 50 percent may be taking an assessment that is too difficult. Reassessment with the appropriate level will provide better information about the youth’s specific needs. Some assessments, such as the TABE 9/10, offer brief Locator Tests that help determine the appropriate level to administer.

Using Assessment Results

Use both formal and informal assessment results to aid in developing an ISS and to help youth make educational and career plans. Informal assessment can be incorporated into all staff interactions with youth. Basic skills assessments must be given at least annually; other objective assessments may be given when youth reach an appropriate goal or milestone.

Remember that an assessment, no matter how rigorously developed, administered, and interpreted, only provides limited information about a youth. Never make important decisions solely on the basis of the results of a single assessment; use other methods and results to makes important decisions about a youth’s capabilities, interests, and needs.

Assessment results have consequences for WIA youth programs. Local area youth programs are held to performance standards, under both original WIA Performance Measures and new Common Measures. Two of those performance standards, the Younger Youth Skill Attainment Rate and Literacy/Numeracy Gains, involve specific skill gains that are identified through assessment. Since local areas can be sanctioned for poor performance or rewarded for good performance, temptations can occur.

Remember that the ultimate purpose of assessment is to accurately demonstrate what youth know and can do. Information from assessment results can help improve instruction offered to youth and other programming. The best way to ensure good assessment results is to provide an effective program.

• American Educational Research Association. (1999). Standards for educational and psychological testing. Washington, DC: author.

• Baker, E.L., Linn, R.L., Herman, J.L., & Koretz, D. (2002). Standards for educational accountability systems. Policy Brief 5. Los Angeles, CA: National Center for Research on Evaluation, Standards, and Student Testing. Retrieved March 15, 2006, from http://www.cse.ucla.edu/products/newsletters/polbrf54.pdf

• CTB/McGraw-Hill. (2003). Beyond the numbers: A guide to interpreting and using the results of standardized achievement tests. Monterey, CA: Author. Retrieved February 20, 2007, from http://www.ctb.com/media/articles/pdfs/ResearchArticles/BeyondtheNumbers.pdf
  ?FOLDER%3C%3Efolder_id=2534374302134983&bmUID=1171993798059.

• Ellis, B.B., & Raju, N.S. (2003, August). Test and item bias: What they are, what they aren’t, and how to detect them. Chapter 7 in J.E. Wall & G.R. Walz, Eds., Measuring up: assessment issues for teachers, counselors, and administrators. Greensboro, NC: ERIC Counseling and Student Services Clearinghouse. Retrieved April 17, 2006, from http://www.eric.ed.gov/ERICDocs/data/ericdocs2/content_storage_01/0000000b/80/23/2d/c4.pdff

• Gronlund, N.E. (2006). Assessment of student achievement. (8th ed.) Boston, MA: Pearson. Harcourt Assessment, Inc. (2005). Some things parents should know about testing: A series of questions and answers. San Antonio, TX: Author. Retrieved November 17, 2005, from http://harcourtassessment.com/haiweb/Cultures/en-US/Resources/
  Additional+Resources/Some+Things+Parents+Should+Know.htm

• McMillan, J. H. (2000, November). Basic assessment concepts for teachers and school administrators. ERIC/AE Digest. College Park, MD: ERIC Clearinghouse on Assessment and Evaluation. Retrieved April 20, 2006, from http://edweb.sdsu.edu/Courses/Ed791BCF02/ED447201.pdf.

• Friedman, M.I., Hatch, C.W., Jacobs, J.E., Lau-Dickinson, A.C., Nickerson, A.B., & Schnepel, K.C. (2003). Educators’ handbook on effective testing. Columbus, SC: Institute for Evidence-Based Decision-Making in Education, Inc.

• Hammill, D.D., Brown, L., & Bryant, B.R. (1992). A consumer’s guide to tests in print (2nd ed.). Austin, TX: PRO-Ed, Inc.

• Joint Committee on Testing Practices. (2004). Code of fair testing practices in education. Washington, DC: author. Retrieved March 21, 2006, from http://www.apa.org/science/fairtestcode.html

• Keith, T.Z., & Reynolds, C.R. (2003). Measurement and design issues in child assessment research. Chapter 4 in C.R. Reynolds & R.W. Kamphaus, Eds., Handbook of psychological and educational assessment of children: Intelligence, aptitude and achievement (2nd ed.). New York: Guilford Press.

• Power, M.A., ed. (2001). Ethical standards in testing: Test preparation and administration (Rev. ed.). WERA Professional Publications Volume 1. University Place, WA: Washington Educational Research Association. Retrieved March 23, 2006, from http://www.wera-web.org/pages/publications/WERA_Test_Ethics.pdf

• Rotherham, A. J. (2006, July). Making the cut: How states set passing scores on standardized tests. Washington, DC: Education Sector. Retrieved July 28, 2006, from http://www.educationsector.org/analysis/analysis_show.htm?doc_id=385844

• Rudner, L.M., & Schafer, W.D. (2001, April). Reliability. ERIC Digest. College Park, MD: ERIC Clearinghouse on Assessment and Evaluation. Retrieved March 24, 2006, from http://www.ericdigests.org/20022/reliability.htm

• Rumsey, D. (2003). Statistics for dummies. Hoboken, NJ: Wiley Publishing.

• Salkind, N.J. (2006). Tests and measurement for people who (think they) hate tests and measurement. Thousand Oaks, CA: Sage Publications.

• Stuart-Hamilton, I. (1995). Dictionary of psychological testing, assessment, and treatment. London: Jessica Kingsley Publishers.

• Walsh, W.B., & Betz, N.E. (2001). Tests and assessment (4th ed.) Upper Saddle River, NJ: Prentice-Hall.

• Zenisky, A.L., & Sireci, S.G. (2005, April 15). No adult left behind either: Creating large-scale computer-based tests for adult basic education students. Center for Educational Assessment Report No. 563. Paper presented at the annual meeting of the American Educational Research Association. Amherst, MA: Center for Educational Assessment. Retrieved March 21, 2006, from http://people.umass.edu/azenisky/AERA_2005_Z&S_final.pdf

LearningWork Connection is an initiative of the Center for Learning Excellence at The Ohio State University.