FOUN 8350 Advanced
Measurement Theory
Fall, 2003
Instructor: Margaret E. Ross Meeting Time: Mondays
Phone: 844-3084 Room:
HC 3307
e-mail: rossma1@auburn.edu
(1 = one)
Office Hours: 4 – 5 Mondays and
Tuesdays,
3 – 4 Wednesdays,
by appointment
________________________________________________________________________
Text:
Crocker, L, & Algina, J. (1986). Introduction to classical and
modern test theory.
Pedhazur, E. J.
& Schmelkin, L. P. (1991). Measurement,
design, and analysis: An integrated
approach.
AERA, APA, & NCME. (1985). Standards
for educational and psychological testing.
Course Description:
This course is designed to provide
an overview of classical test theory with an introduction to item response
theory and generalizability theory. Validity and reliability, as concepts, will
be addressed and techniques for evaluating validity and reliability will be
applied through statistical analyses. We
will, also, evaluate existing instruments that might be used in research,
counseling settings, or educational settings.
Other possible topics include scale construction, norming,
standard setting, test bias, test equating and item analysis.
Course Objectives:
1. Students will gain an
in-depth understanding of classical test theory and a basic understanding of
item response theory and generalizability theory.
2. Students will understand and
apply techniques for establishing validity and reliability, including
correlation and regression, factor analytic, and analysis of variance
procedures.
3. Students will understand and
apply item analysis procedures such as item difficulty and discrimination as
well as differential item functioning.
4. Students will evaluate
existing instruments they might use in their research, in educational settings,
or in counseling setting in relation to integrity of norming,
and standard setting.
5. Students will evaluate
instruments they might use in relation to the establishment of validity and
reliability.
6. Students will understand norming and scale development procedures.
Artifacts
will reflect application or understanding of concepts and skills taught in the
course. The assignments will include your
opinions or evaluations of topics covered in the course. You will meet with me
to discuss the exact content of your portfolio. Portfolio content will be
assessed on an on-going basis with the full portfolio, which includes the
project, due the last class period.
1. In-depth conceptual
treatment of validity and reliability (required)
2. Instrument Evaluations (#1
or #2 required)
3. Assignments/readings (#1 or
#2 required)
4. Analyses and Interpretations
(required)
5. Project (required…described
below)
The
project will provide for an in-depth application of skills learned in the
course. It may require that the
student(s) take responsibility for acquiring knowledge and skill in areas not
covered in classes.
Option
1:
Test/Instrument
Development
The individual or small group will construct an
instrument (test, attitude, questionnaire, etc.) and pilot the instrument. To pilot the instrument, it must be
administered to a “sample” of appropriate individuals. Evidence of the validity and reliability of
inferences made based on the instrument must be provided.
The
Report should contain the following:
group/individual scores
descriptive statistics
results of reliability analyses
results of validity analyses
etc.
***
Note: You will want to read over
appropriate chapters in Crocker & Algnia.
Option
2
Research
Project/Proposal
Identify
a question of interest related to measurement theory and design a project to
study to address the question.
The
proposal should include the following:
Section
1
Introduction
Literature review (short)
Research questions
Section
2
Methods
Participants
Instrument(s)
Methods
Procedures
Option
3
Measurement
Paper
Identify
an area within measurement that interests you and conduct an extensive
literature review. The paper may be a literature review, a theoretical paper,
or a paper outlining specific applications of measurement techniques.
Possible
topics include:
Item Response Theory
Issues related to analysis of scales
used in behavioral sciences
Test administration and scoring
Professional standards for test use
Score transformation
Test equating
Generalizability Theory
Other
topics may be acceptable for a measurement paper. If you identify a topic of
interest, we will discuss the appropriateness for this class. Approval of the
topic by the instructor is needed.
The
grade earned will be based on the portfolio.
The sections will be evaluated and weighted as outlined below.
Instrument
Evaluations 25%
Assignments
25%
Analyses
and Interpretations 15%
Project
35%
Grading
Scale:
90%
or above A
80%-89% B
70%-79% C
60-69% D
Below
60% F
Incompletes and Withdrawals. Grades associated with incomplete course
work or withdrawal from class will be assigned in strict conformity to
University policy (see Auburn University Bulletin, 1996-1997, p. 20.). If you wish to drop this course you may do so
by the 10th class day with no grade assignment. From the 10th class day to
mid-quarter a W (withdrawn-passing) grade will be recorded in your transcripts.
After this period withdrawal from the course will only be granted under unusual
circumstances and must be approved by the Dean of the
Academic Misconduct, The Department of EFLT recognizes university policy
regarding academic misconduct. Violations include, but are not limited to:
plagiarism, unauthorized assistance during examinations, submitting another’s
work product as your own, using another’s words as your own without appropriate
citation, sharing unauthorized materials with another that contain questions or
answers to examinations, altering or attempting to alter assigned grades. In
accordance with University policy regarding academic misconduct, students may
be subject to several sanctions upon violations of the Student Academic Honesty
Code. See the Tiger Cub publication for the current year (pp. 113-116) for
specifics regarding academic misconduct as well as student’s rights and
responsibilities associated with the Code.
The instructor will comply with the
Course
Content
Classical
True Score Theory
Conceptual
Framework for Understanding Validity
Content
Analysis
ANOVA
Procedures for Establishing Validity
Correlation
and Regression Procedures for Establishing Validity
Discriminate
Analysis
Factor
Analytic Procedures
Multitrait-Miltimethod Procedure
Generalizability Theory
The
Role of Reliability
Reliability
Procedures
Test
Construction
Data
Collection Methods
Scaling
Norming Procedures
Standard
Setting
Item
Analysis
Classical
Item Response Theory
Logistic
Regression
Item
Response Theory
Bias
in Testing
Bias
in Selection
Detecting
Item Bias
Test
Score Equating
Other:
Diversity and Assessment (Suzuki et al.)
Learning
Disability Assessment (Research)
Intelligence
Testing (Research on Current Issues)
Future
Directions in Measurement (Embertson)