Track Sessions: Innovative Approaches to Assessment
ADVERTISMENT
Innovative Approaches to Assessment Sessions in Alphabetical Order
Anti-Racist Assessment: Interrupting Patterns of Privilege and Oppression in Educational Measurement
In this session we present a number of new efforts aimed at leveraging the power of assessment to advance racial equity. We explore the role of educational measurement in perpetuating inequities and discuss our strategies for interrupting these patterns. Susan Lyons introduces the session by providing a brief introduction to the history and continuing legacy of racial oppression in the design and use of educational assessments. Fiona Hinds then presents recommendations from Cognia’s Equitable Assessment toolkit for advancing practice in five key areas: culturally-responsive classroom assessment, large-scale test design, psychometrics, reporting, and test use. We hear from state education leaders in Alabama and Maryland who share the work that is happening in their states related to addressing racial inequity in assessment. The session closes with commentary and reflection from Jennifer Randall who re-frames the ideas from the session through the lens of her anti-racist validity framework.
Session Presenters: Susan Lyons, Fiona Hinds, Sandy Ledwell, Jennifer Randall, Brianna Creed
Session Date & Time: Wednesday, June 23, 2021 at 11:00 AM – 12:30 PM ET
Building a Balanced Assessment System from the Perspectives of Policy, Practical Assessment Program Matters, and Technical Quality
The Every Student Succeeds Act (ESSA) offers the opportunity to change K-12 assessments with considerable flexibility for the better. It has never been more important for states and school districts to develop a balanced assessment system that provides meaningful information about student achievement that can guide curriculum and instruction decisions and help improve student learning. A balanced assessment system is a set of coordinated tests that produce overlapping but not redundant information to serve different users to achieve their common goal of improving educational achievement. When a variety of measures are needed to accommodate multiple objectives and more information is expected within a system, there are tremendous challenges. In this session, three states and a well-known measurement expert will discuss these challenges and possible solutions, based on their experiences in the process for design, development, and implementation of balanced assessment systems from policy, development, practical, and technical perspectives.
Session Presenters: Katia Foret, Vince Verges, Theresa Bennett, Kelly Bacher, Steve Ferrara
Session Date & Time: Wednesday, June 23, 2021 at 11:00 AM – 12:30 PM ET
ADVERTISMENT
ADVERTISMENT
Early Returns: Lessons Learned from Administering Assessments to Students during the Coronavirus Pandemic
The coronavirus pandemic has disrupted nearly every aspect of life including education and assessment. To safely provide education to students, local education agencies have developed flexible learning options. As states desire to have valid and reliable information about student learning and growth to inform various policy priories, an important question to ask is whether the assessments within their assessment system are robust enough to be administered flexibly. This session discusses the journey taken by California, the largest school district in the state, and the state’s assessment contractor to develop the people, process, and technical infrastructure to support flexible assessment options.
Session Presenters: Terran Brown, Mao Vang, Moses Jorge, William Piferrer
Session Date & Time: Wednesday, June 23, 2021 at 1:00 PM – 2:00 PM ET
Embedded Standard Setting: Analytics and Practical Applications to Item Development and Item Alignment
Embedded Standard Setting (ESS; Lewis & Cook, 2020) is a relatively new methodology situated in Principled Assessment Design. ESS utilizes item-achievement-level alignments hypothesized by subject matter experts during item development or resulting from an independent alignment study in lieu of the judgments made in a traditional item-based standard setting workshop
Practitioners will present tools and applications that advance ESS theory into ESS practice
- Explore ESS analytics that summarize and graphically reflect the degree to which the assumptions of ESS are met, and in either case provide actionable information to improve item alignment
- Describes the results of a SME critical review of ESS-inconsistent items—items identified as having alignments not supported by empirical data—to understand the nature of, and to resolve, the inconsistencies.
- Leverage ESS to enhance a commonly-used item-based standard-setting method—ID Matching (Ferrara & Lewis, 2012)—to reduce the cognitive load of panelists
Session Presenters: Daniel Lewis, Amanda Brice, Christina Schneider, Ellen Forte
Session Date & Time: Monday, June 21, 2021 at 2:10 PM – 3:10 PM ET
Examining Validity, Technical Properties, and Accessibility of Innovative Item Types in State Assessments
With extensive applications of advanced technology in online testing, innovative items have been widely explored and implemented in online testing. These new item types promote substantial changes in item structure, response style, and offer interactive activities in state assessments. Innovative item types intentionally measure higher level of cognitive complexity and are usually scored with partial-credit models. The psychometric properties of innovative items, however, are rarely studied with empirical evidence and reported in measurement literature. The current session incorporates three empirical studies on innovative item types. Using simulated and operational data from state assessments, validity evidence, psychometric properties, and accessibility of a variety of innovative item types are investigated in online testing. Advantages and practical issues of using those new item types to measure student performance and for students with disabilities, are discussed for the improvement in item development, scoring, accessibility, and the technical quality of assessments.
Session Presenters: Mark Hansen, Shudong Wang, Tim Davey, Liru Zhang
Session Date & Time: Tuesday, June 22, 2021 at 2:10 PM – 3:10 PM ET
Growing Innovative K-2 Assessment Programs: Engaging Students and Educators
Many educational organizations have worked to incorporate K-2 students into their assessment and accountability systems. This presentation explores the experiences, perspectives, and innovative approaches to create meaningful K-2 assessments across three states, and addresses any modifications made for virtual or hybrid learning experiences.
Michigan first implemented its K-2 assessment in 2012. The assessment features educator written items and will soon be linked to grade 3 reading scales.
The Georgia Department of Education has developed game based assessments for grades 1 and 2 which are designed to engage learners in formative assessment of reading and mathematics standards.
Alabama is in the early stages of developing an early grade assessment system. As a first step, the State developed sets of instructional strategies that will lead towards eventual summative assessments for these early grades.
Each state will discuss the key policy drivers for their K-2 assessment systems, challenges, and plans for the future.
Session Presenters: Holly Baker, Maggie Hicks, Julie Murphy, Jan Reyes, Kara Courtney
Session Date & Time: Wednesday, June 23, 2021 at 1:00 PM – 2:00 PM ET
Helping Local Educators Learn to Use the Formative Assessment Process
Several states are engaging educators in learning about the formative assessment process, based on the CCSSO FAST SCASS (2017) formative assessment definition. State efforts include workshops, print and video resources, online learning opportunities, and websites. Each state has approached the challenge of providing educator professional learning in a somewhat different manner.
This session will focus on the different approaches that Maryland, Michigan, and Oregon use, leading the panel and the audience to consider the advantages and challenges of each approach, and how other states can learn how to apply the best of the different approaches to their work in promoting educator use of the formative assessment process.
Session Presenters: Edward Roeber, Dan Farley, Jennifer Wojcik, Kim Young, Caroline Wylie
Session Date & Time: Tuesday, June 22, 2021 at 3:40 PM – 4:40 PM ET
Implementing reliable performance assessment systems with educator involvement in creation, implementation and collaborative scoring (at scale).
Implementing performance assessment at scale, with involvement of educators in the creation, implementation and collaborative scoring, is not an easy task. This session will highlight the stories of three innovative organizations as they work through implementing common performance assessment programs. The three organizations sharing their stories will be the Colorado Department of Education, Michigan Assessment Consortium, and Millard Public Schools. Although the implementation, approach and strategy employed is different, they all involved teachers in the development and collaborative scoring of the assessments.
Highlighted in each assessment program is the importance of educators collaboratively evaluating student work: when educators see student work created outside of their classrooms it can serve as one of the most powerful professional learning opportunities available to teachers. Participation is invaluable for teachers to understand what high performance looks like, thus helping them provide challenging levels of instruction to assure equity for all students.
Session Presenters: Zac Henrich, Angela Landrum, Edward Roeber, Darin Kelberlau
Session Date & Time: Wednesday, June 23, 2021 at 1:00 PM – 2:00 PM ET
Lessons Learned from Innovative Assessment Pilots in Three States
Georgia, Louisiana, and North Carolina are in the midst of ambitious, multi-year innovative assessment pilots. While each seeks to leverage multiple testing events throughout the year to reduce dependence on year-end assessments, their three approaches differ from one another in significant ways. Georgia is working with district consortia to pilot two distinct solutions designed to provide immediate feedback that informs instruction in ELA, math, and science; Louisiana is measuring student understanding of pre-identified knowledge and texts from ELA and social studies; and North Carolina is exploring the use of a through-grade model to support more appropriate summative assessments in ELA and math. Each state will share its pilot design, and the lessons learned from their first two years of implementation, with the goal of providing actionable information and advice to other states considering ways to introduce and bring to scale innovations to their own assessment systems.
Session Presenters: Andy Latham, Allison Timberlake, Chanda Johnson, Tammy Howard
Session Date & Time: Tuesday, June 22, 2021 at 2:10 PM – 3:10 PM ET
Now More Than Ever: The Case for Open Standards to Achieve Interoperability and Accessibility in Digital Assessments
As more educational systems shift to online learning and assessment due to COVID-19, interoperability and accessibility are key to building fair, valid and reliable digital assessments. The newly released, version 3.0, Question and Test Interoperability (QTI) standard addresses these challenges by including advanced accessibility support, better support of Computer Adaptive Testing (CAT), as well as portable custom interactions for technology-enhanced items.
An overview of the main components and supports of the QTI interoperability standard as well as improvements with the newly released QTI 3.0 standard will be described including the value propositions to states, technology vendors and content publishers.
Presenters will describe their experiences building assessment systems with open standards and best practices on how to solve key interoperability challenges when working with technology vendors, including language that can be used in contracts and request for proposals (RFPs) for states and districts looking to move to 100% online assessment.
Session Presenters: Dawn Cameron, Susan VanGundy, Dale Cornelius
Session Date & Time: Wednesday, June 23, 2021 at 1:00 PM – 2:00 PM ET
Promising Approaches to Virtual Standard Setting
As a result of the COVID-19 pandemic, online solutions for carrying out educational activities and functions that were historically conducted in-person, are needed. In the winter of 2020, a new English language proficiency assessment was administered to students with a plan to conduct an in-person standard setting event the following May. Due to the pandemic, the event was rescheduled to the fall and was to be conducted virtually. The procedures used to conduct a virtual standard setting event for the new English language proficiency assessment will be discussed, along with successful practices and lessons learned in the facilitation of the virtual panel meetings. A representative from the state department of education will reflect on feedback from participating educators, observations of the virtual meeting and implications for future events.
Session Presenters: Brooke Nash, Arthur Thacker, Gail Tiemann, Beth Fultz
Session Date & Time: Tuesday, June 22, 2021 at 3:40 PM – 4:40 PM ET
Reinventing Assessment Development Work in a Virtual World
This session will cover approaches to completing assessment development activities that traditionally take place via in-person meetings. Approaches and considerations for conducting synchronous and asynchronous virtual item development, testlet reviews, alignment studies, and item content, bias and sensitivity reviews will be covered. Attendees will learn various approaches to getting assessment development work done and leave with considerations for implementation, and suggested practices that may remain even after in-person development activities are allowed to resume.
Session Presenters: Toni Wheeler, Audra Ahumada, Tracy Gooley, Marjorie Wine, Trisha Klein
Session Date & Time: Monday, June 21, 2021 at 2:10 PM – 3:10 PM ET
Remote Testing Options for Students and Teachers: Are They Here to Stay?
With the rise of remote teaching and learning during the COVID-19 pandemic, K-12 education leaders have been challenged to adopt new and creative approaches to assessment administration. A number of states have begun to implement remote testing options when traditional assessment administration in a brick-and-mortar setting is not possible. While at-home testing fills a critical need during the pandemic, it comes with new challenges: How do we ensure the security of tests that are administered remotely? What new technologies are required to support educators and students? How do we engage parents/guardians in their new role in the at-home testing process? In this session, we will discuss how two states – Texas and Pennsylvania – implemented remote testing during the 2020-2021 school year. We will explore the successes and challenges in their states, and discuss the future role of remote testing as a part of distance learning beyond the pandemic.
Session Presenters: John Bandy, Roberto Manzo, Brian Campbell
Session Date & Time: Tuesday, June 22, 2021 at 3:40 PM – 4:40 PM ET
Remote Testing with Fidelity: Is it equitable, valid, and reliable?
Is it possible to equitably deliver remote K–12 assessments securely and with fidelity? In a remote setting considerations include, among others:
- Student equity and safety
- Data privacy
- Technology access
- Measurement reliability
- Test content security
In this session, Minnesota and Massachusetts will share their testing observations from a remote testing and remote proctoring feasibility study and recommendations for policy decisions on how data can be used from this mode of administration. Assessment experts will discuss the test delivery and measurement considerations.
Session Presenters: Trent Workman, Michol Stapel, Jennifer Dugan, Ye Tong
Session Date & Time: Monday, June 21, 2021 at 3:40 PM – 4:40 PM ET
Stackable, Instructionally-embedded, Portable Science (SIPS) Assessments Project: A Partnership to Create Innovative Assessments that Generate Meaningful, Interpretable, and Actionable Results
Presenters will share the goals, research-based approach, and national significance of the Stackable, Instructionally-embedded, Portable Science (SIPS) Assessments project funded through the US Department of Education’s Competitive Grants for State Assessments. SIPS brings together a collaborative of eight states, five organizations, and a panel of experts to address simultaneously states’ needs for large-scale science assessments and the needs of educators, parents, and students for resources that support science learning throughout the school year. Presenters will share how SIPS partners are applying current research, theory, and best practice to establish replicable and scalable processes, tools, and resources to drive meaningful shifts to science instructional practice and assessment use as envisioned by the National Research Council’s (NRC) A Framework for K12 Science Education (Framework; NRC, 2012). Presenters will share benefits of the SIPS Model of Assessment—a model in which assessment tasks are stackable, portable, and aligned to a standards-based instructional model.
Session Presenters: Erin Buchanan, Rhonda True, Ellen Forte, Jim Pellegrino
Session Date & Time: Tuesday, June 22, 2021 at 2:10 PM – 3:10 PM ET
States’ Professional Development Model: Educators’ Deep Dive into Developing Innovative Science Assessments and Why the Model Works
New science standards often require students to engage in new item types, which results in a need for innovative assessments that measure students’ progress toward mastery of the standards. Understanding new item types measuring new standards requires a deep dive by educators into the standards and how the standards link to instruction. Three states have approached this through professional development that focuses on teachers writing innovative technology-enhanced items and evidence-based items, and the development and use of instructional supports. This session focuses on each state’s approach to educator involvement in the writing of science items that measure the new standards. The session will include insightful lessons learned regarding how best to engage educators in this process.
Session Presenters: Patty McDivitt, Maggie Hicks, Jackie Sampsell, Lisa Sireno
Session Date & Time: Monday, June 21, 2021 at 3:40 PM – 4:40 PM ET
Students Doing Science -- Summative Simulation-based Innovative Assessment Pilot
With the acceptance of Innovation Assessment Development Authority (IADA) Application, Massachusetts is supporting a locally driven set of deeper learning measures that provide teachers and students with a more authentic and engaging experience that measures by having students actually do science. To tie engaging science classroom experiences to assessment, Massachusetts is supporting the development and delivery of innovative, authentic, and engaging simulation-based science assessments built with teachers and students at the center.
Session Presenters: Trent Workman, Matt Gaertner, Jennifer Riley, Kevin King, Sam Ribnick
Session Date & Time: Wednesday, June 23, 2021 at 2:10 PM – 3:10 PM ET
Validity Arguments Meet Artificial Intelligence in Innovative Online Assessments
Over the past several years, there has been a growing interest in Artificial Intelligence (AI) enabled assessments. These innovative assessments have been used in credentialing and the workforce for several years and are now becoming more common in education. While these types of solutions immediately offer their users the promise of efficiency and a “wow factor” in their assessments, users need to maintain high standards for validity and fairness in these high stakes assessments and conform to modern developments in this space—specifically principled design and validity argumentation. This session brings together a multi-disciplinary panel of thought leaders in AI-enabled assessment and validity to raise awareness of issues for those interested in either developing or using AI-enabled assessments and suggest paths forward.
Session Presenters: Hillary Michaels, Steve Ferrara, Kadriye Ercikan, David Dorsey
Session Date & Time: Tuesday, June 22, 2021 at 2:10 PM – 3:10 PM ET