Message: RE: Admin memo re EOCs

RE: Admin memo re EOCs
FromVandeven, MargieDate  Tuesday, September 26, 2017 5:26 PM
To
Preis, Stacey
Cc
Neale, Chris;Henningsen, Blaine;Thornton, William;Sireno, Lisa
SubjectRE: Admin memo re EOCs

Much better!! Just a couple of minor edits.

 

From: Preis, Stacey
Sent: Tuesday, September 26, 2017 4:37 PM
To: Vandeven, Margie
Cc: Neale, Chris; Henningsen, Blaine; Thornton, William; Sireno, Lisa
Subject: Admin memo re EOCs

 

Commissioner,

 

Here is the latest draft that I believe incorporates all of the changes we discussed this afternoon. Once we are finished with the text, I will ask Robin to put on my letterhead and send to Communications. (If there are additional tweaks to formatting/alignment, I will leave that to Communications to adjust once the content is final.)

 

Stacey


TO: School Administrators

FROM: Stacey Preis, Deputy Commissioner, Division of Learning Services

  Blaine Henningsen, Assistant Commissioner, Office of College and Career Readiness

  Chris Neale, Assistant Commissioner, Office of Quality Schools

SUBJECTS: LS-17-001

DATE:  September 26, 2017

Thank you for your patience as we have worked to reach a resolution regarding spring 2017 assessments in Algebra I and English II. We have included several pieces of information in this update. 

I. Modifications to the 2017 APR

The adjustments outlined below will be incorporated into the 2017 APR. Due to the removal of the Algebra I and English II data, Standard 1 Academic Achievement, Standard 2 Subgroup Achievement and Standard 3 High School Readiness will be calculated in the following manner:

A.  English Language Arts and Mathematics Assessment Hold Harmless

Standard Applicable: Standards 1 and 2 – English language arts (ELA) and Mathematics

Updated Policy due to exclusion of Algebra I and English II data: The 2017 APR will be calculated using existing 2015 and 2016 data plus 2017 data with Algebra I and English II removed. The points earned in Standard 1 and 2 will be compared to the points earned in 2016 using this methodology. The district or charter local education agency (LEA) will receive the higher of 2016 or 2017 points for the APR. The same calculation process will be followed for Standard 1 Academic Achievement and Standard 2 Subgroup Achievement.

Example – A district or charter LEA receives 12 points in Standard 1 ELA in 2016, but earns 16 points in 2017 (with Algebra I and English II data excluded). The 16 points for 2017 would be used in the APR calculation. The opposite is also true; if a district or charter LEA earned 16 points in 2016, but only 12 in 2017, 16 points from 2016 would be used for the APR calculation.

Participation in Algebra I and English II during the 2017 administration window satisfies the participation rate requirement. Districts and charter LEAs will not receive a level not determined (LND) for students who took Algebra I and English II assessments in the 2016-17 school year.

B. High School Readiness – Removal of Algebra I Scores

Standard Applicable: Standard 3 High School Readiness

Updated Policy: The results from the 2017 End-of-Course assessments (EOCs) in Algebra I and English II do not produce year-to-year comparable results. For purposes of the 2017 APR, if a student was assessed on the Algebra I EOC, DESE will substitute a student’s qualifying proficient score on the prior year 7th grade mathematics assessment. (After analysis of the 2017 assessment data, it was determined that K-8 districts did not administer the English II EOC).

II. APR Release Schedule

DateDescriptionNotes
September 29 (5:00)Algebra I and English II corrections deadline COBSecure Mode
October 4Public hearing – changes to the Comprehensive Guide for the Missouri School Improvement ProgramGovernor’s Office Building

Room 450

Jefferson City, MO

8:30-10:30 a.m.

October 6 (5:00 p.m.)Data freeze for preliminary APRSecure Mode
October 20APR preview (excluding Algebra I and English II)Secure Mode
October 20 – October 27Appeals WindowStandards 3-5 only
October 27 (5:00 p.m.)Data freeze for Final APRFreeze
November 8Final APR (Excluding Algebra I and English II) behind secure loginSecure Mode
November 13Embargoed data available to pressEmbargoed
November 15Final APR release (Excluding Algebra I and English II)Public

The following data were available for school districts and charter LEAs beginning September 22, 2017:

III. Sunshine Law request

DESE has received a Sunshine Law request regarding the 2017 EOC results in Algebra I and English II. We must honor this request.

The data will be provided with the following caveats:

  1. The Missouri Assessment Technical Advisory Committee (TAC) made up of nationally recognized assessment experts recommended that the data not be reported.
  2. DESE agreed with the TAC recommendation and will not publish the data on our website for these two tests.
  3. Districts and charter LEAs are provided time to verify data sets (i.e., to make any needed corrections to the data and to appeal test scores). Since these data were not to be reported by DESE, districts and charter LEAs may not have certified the data for these two tests. This means that, aside from the year-to-year comparability issues, the data could inaccurately reflect the performance of a particular district or charter LEA in other capacities as well.

In short, we believe that it would be irresponsible to publicly report data obtained by DESE in its current form. Should someone contact you about these data, please refer to the above caveats.

IV. Algebra I AND English II Verification Process

Consistent with past practice, prior to producing public reports based on accountability data, districts and charter LEAs have the opportunity to verify and/or correct the related data. In light of the Sunshine Law request, information related to the administration of Algebra I and English II will be provided to school districts and charter LEAs for review. Districts and charter LEAs will have an opportunity to make corrections to the 2017 Algebra I and English II data. Please remember that Algebra I and English II data will not be used in the calculations of the 2017 APR. DESE does not believe that the public release of these data is advisable.

V. Moving Forward

Once again, we want to convey that we share your frustration around spring 2017 assessments in Algebra I and English II. We expect more from our assessment vendors. The EOC vendor, Questar, has recently been purchased by Educational Testing Services (ETS), and we are able to share with you key elements of the resolution we reached. First, districts and charter LEAs will not be billed for 2017-18 EOCs in Algebra I and English II, up to the amount paid for those assessments in 2016-17.

Second, for the remainder of the contract year, DESE will require the following additional assurances regarding the 2017-18 EOCs, and Questar has agreed to fulfill these obligations:

  1. Engage the psychometrics services of its parent company, ETS, to review the preliminary aggregate analysis, provide consultative services to the Missouri Assessment TAC on the program status, and perform a psychometric quality audit of the overall program.
  2. Establish a process to verify bandwidth requirements and the allocation of bandwidth for as many as 50,000 users prior to the beginning of each assessment window.
  3. Provide daily reports to DESE during each assessment window which include the number of students tested that day in each course, cumulative test totals, cumulative percent of the number of students anticipated during the test window, a brief narrative summary of test issues encountered during the day, and available test administration analytics. 
  4. Provide preliminary reports to DESE regarding test completion throughout each assessment window.

For additional questions regarding accountability and APR production, please contact please contact staff in our School Improvement section:

Jocelyn Strand  jocelyn.strand@dese.mo.gov573-751-4104
Kevin Freemankevin.freeman@dese.mo.gov573-751-4426
Amanda Cashamanda.cash@dese.mo.gov573-526-4886

For additional questions regarding 2017-18 assessments, please contact staff in our Assessment section:

Lisa Sireno or Shaun Batesassessment@dese.mo.gov573-751-3545