Message: RE: EOC Scores Delayed?

RE: EOC Scores Delayed?
RE: EOC Scores Delayed?

From | Henningsen, Blaine | Date Thursday, September 14, 2017 1:59 PM

To | 'Kilgore, Christina'
Cc |
Subject | RE: EOC Scores Delayed?

EOC FAQ from webinar_9.1.2017 bh.docx (39 Kb )

Christina,

Here is what the Commissioner sent out. Webinar link shown below and attachment.

https://www.youtube.com/watch?v=NS_ZpeDPUAg&feature=youtu.be

Dr. Blaine Henningsen

Assistant Commissioner, Office of College and Career Readiness

Missouri Department of Elementary and Secondary Education

Phone 573-751-2660

The mission of the Office of College and Career Readiness is to guide and support Missouri schools in preparing success ready students.

A success ready student has the knowledge, skills, behaviors, and experiences to achieve personal goals and contribute in all facets of life.

From: Kilgore, Christina [mailto:KilgoreC@parkhill.k12.mo.us]
Sent: Thursday, September 14, 2017 12:52 PM
To: Henningsen, Blaine
Subject: EOC Scores Delayed?

Dr. Henningsen,

My name is Christina Kilgore and I am responsible for coordinating assessments for the Park Hill School District in Kansas City. I heard through the gossip channel that DESE will be delaying the release of the EOC scores this year and that scores will not be available to buildings for use in course grade calculations. I searched for a message from Commissioner Vandeven or an FAQ document but could not find evidence in my files nor on the DESE website that this is the case, so I’m looking for some confirmation as to whether or not this is accurate.

Thank you in advance for clearing up my confusion,

Dr. Christina Kilgore

Coordinator of Curriculum and Student Services

----------------------------------
Park Hill School District

7703 NW Barry Road

Kansas City, Missouri 64153

Phone: 816-359-6962

kilgorec@parkhill.k12.mo.us

EOC FAQ

INTRODUCTION

We are extremely frustrated with our vendor’s inability to deliver usable results. We know that Missouri students, teachers and districts work hard throughout the year, and they are counting on us to get this right. We know you have questions. Below are the most common questions we heard from districts in our discussions about this issue.

RAW V. DISTRICT EOC SCORES

Can you clarify with more detail exactly what was wrong that triggered the review? Lisa (Many of the questions in this section are very similar. Please group together as much as possible.)

2018 is the first year the new Missouri Learning Standards in Math and ELA are to be assessed. Is there a chance that was a reason for the test not comparing? Did teachers start teaching the new standards too soon? Lisa

Is it possible to release the state's passing percentages for EOCs in spring 2017 in order to understand the comparability issue? Lisa/Stacey (?)

Why are raw scores okay, but final EOC scores are not?

The raw scores accurately reflect a student’s performance. The assessments themselves were deemed accurate and reliable by DESE’s Technical Advisory Committee and would produce the same results year after year. The district and statewide scores are not comparable because the cut scores in 2017 changed from 2016.

So we do disregard the previous data that we saw? Lisa

Are raw scores still valid to use for student grades in Algebra I and English II?

Yes, raw scores may be used for students’ grades. The tests themselves were valid and reliable.

Are the score comparable across students? Lisa

What specifically was not comparable?

The 2017 cut scores do not align with cut scores from the 2016 assessments, rendering comparisons unreliable.

Does that not indicate a problem with the scale but not a problem with the scores? Why not adjust the scale? Lisa

From previous correspondence, we understand that the forms issue was within the forms of the 2017 Algebra I and English II EOCs, but was it really between the forms from 2016 to 2017? Lisa

We give the English 1 EOC also. Is E1 also not comparable? What about Algebra II?

English II and Algebra I are the only EOCs that are not comparable.

You keep mentioning the pattern. What exactly was the pattern? Lisa

If these were not new forms, one could assume that they have been used more than once in the past. How do the scale score data on these differ from when these forms were used in the past? Lisa

Do we need to send home new reports with only the raw scores for the Algebra 1 and English 2 EOCs?

What type of data/file set will we receive from DESE? We upload EOC scores to our school information system to track completed EOC assessments. Will we receive a file for Algebra I and ELA II for 16-17 to upload? Chris

A+ ELIGIBILITY

Please address how this will impact A+.

Districts may use EOC raw scores for determining A+ scholarship eligibility. Student record labels may be used as appropriate.

But you ARE comparing forms when all A+ students are being held accountable to the same achievement levels. One student may have had a harder test and scored lower, missing proficient, compared to a student who may have taken an easier test. Lisa

Will districts be able to make their own proficiency levels for the A+ levels or should they use a lower proficiency cut score? Chris ?

Since this will be valid for A+, does this mean we will get a list of the results and include it in our assessments scores/upload it in SIS? Lisa or Chris

Since A+ requires proficiency on Algebra I, can that be changed to a required raw score expectation since the achievement levels seem to be bad? Chris

For students that did NOT attain a status in Algebra I that would allow for A+ scholarship money, what rights do they have? Will they be allowed to appeal? How can you throw out the entire test, but tell certain kids their score will still count? Chris

Who will pay for the re-take for A+ if needed? (Chris, please group as many of the A+ questions together that you can and provide a response that can address them to the extent we can.)

Is it fair to tie A+ money to test scores when the validity of test scores can be called into question like this? Especially considering (a) some students have already graduated and not met the Algebra I requirement; and (b) the students who might be forced to retake the test aren't in Algebra I class anymore, so how can they reasonably be expected to "test better" now? Chris

ALGEBRA I/ALGEBRA II/8th GRADE MATH

Why does the test change each year? Isn't Algebra 1 the same concepts that need to be taught each year and be tested on? Why change if the past worked? Stacey and Lisa

Were not the 8th grade Algebra I scores used in the place of 8th grade MAP Math scores for those 8th graders taking the Algebra I EOC?

Yes. Eighth-grade students who took the Algebra I EOC are not included in 8th grade math scores.

By throwing out all Algebra I, does that mean students who were required to take Algebra II are shouldering all the accountability? Chris

Will the fall 2017 Algebra I test be like the past spring 2017 test, or will it be comparable to the new 2018 spring Algebra I test?

Fall 2017 will be the first administration of new EOCs in math and English language arts under the new Missouri Learning Standards.

Since this is the first year that Algebra II was being used for graduation requirements, could it be entertained to drop this from APR (taking it in isolation without the Algebra I EOC)? Chris

Is excluding 8th grade math overall in the APR calculation an option on the table? Chris

Explain the "no effect" for a middle school - the 8th grade MAP scores will then carrying the weight for the building. Usually the Algebra I scores HELP middle schools overall. Chris

How does this issue affect 8th graders who took the Algebra I EOC instead of the 8th grade MAP?

For purposes of grades, raw scores from the Algebra I EOC may be used. The raw scores are accurate and give students, teachers and parents an idea how students are performing.

If the majority of our 8th graders take the EOC, how can that not affect us in a negative way? Chris

For districts that have the 8th grade Math/ Algebra I issue, what will be done to overcome the gaping hole that the Algebra I EOCs traditionally fill? Chris

All of our 8th graders take Algebra I. We utilize their scores to determine eligibility to take the Algebra II EOC for high school accountability or re-take Algebra I (if received Basic or BB). How will this be affected? Chris

Will all districts receive 8th grade math scores?

Yes. The 8th grade Grade-Level Assessment results will be used in the APR.

Since 8th grade scores are so district dependent regarding Algebra I, have you considered the possibility of looking more at grades 3-7? LEA's are more alike on those variables. Chris

What is the impact for K-8 with Algebra I High School Readiness and Grade-Level Math? Chris

Will Algebra II count towards the APR for the kids who took Algebra I prior to high school?

Yes, Algebra II is the required high school mathematics assessment for accountability purposes for students who took the Algebra I EOC assessment prior to high school.

Our high school Algebra I scores were the highest math scores in our district. How will that impact our APR? Chris

Due to the disruptions that occurred during testing, will students that did not score proficient on Algebra I have the opportunity to take the test again? Chris

If we have students re-take the Algebra I EOC, is there anything we need to do to retract the old score? Chris

What about seniors that have already graduated? How will they retake the Algebra I test? Chris

Algebra II is optional for some. If we are negatively impacted by the inclusion of those scores, will there be an appeal process? Chris

We gave our 8th grade students who took Algebra I the 8th MAP test this past spring rather than the Algebra I test. If we give those students the Algebra I test this December, will this test be a valid test? OR should we wait until the spring and test them on Algebra I and Algebra II so that the kinks in the new testing system can be worked out? Lisa

If the MPI of Algebra II in isolation brings down our APR, would you consider exempting it for a district? Chris

2017 APRs (Chris, this whole section is yours.)

As a single high school district, can you explain how APR will be calculated?

What about the statute that states that if a change occurs to the assessment or the MSIP standard, the previous year's APR can remain in place?

As we communicate with our stakeholders (staff, parents, community), we always like to convey our message with confidence. When do you estimate we can confidently provide some concrete information about APR? Is there a chance that there could just be NO APR for 2016-17?

We plan to release APRs in late October.

When our district is judged by overall APR scores, taking away these scores will again negatively impact our performance. Once the APR is released to the media, how do we respond?

Please elaborate on this statement: Status targets and progress calculations will be adjusted to accommodate the loss of the two EOCs.

If you take out some of the scores, won't that in effect increase the impact of the remaining scores (Biology and Government)?

If we scored higher on the English II can we use that for our APR?

Can we sub in geometry for Algebra II?

Will additional EOCs be used for APR as stated in the commissioner’s video message yesterday?

What will the impact be for districts that do not take the optional EOCs? What impact will this have on districts that rely on high school scores as a part of achieving full progress/status points to achieve full accreditation points?

The 2017 APR will be based on 100 points instead of the usual 140 points. Status targets and progress calculations will be adjusted to accommodate the loss of the two EOCs.

How will this affect K-8 districts APR scores for high school readiness?

The High School Readiness Standard will not be used in the 2017 K-8 APRs.

To clarify, will the middle school APR be based only on 8th grade MAP since Algebra I is taken out?

So, prior year APR won't be recalculated?

No. However, to gain an apples-to-apples comparison, we will need to develop a data set that removes Algebra I and English II EOC results from prior years to calculate 2017 APRs.

If English II and Algebra I were our highest scores, then how will this impact our APR?

What will be used in APR calculation if we only take the required Algebra I and English II EOC and do not take the optional EOC's?

Now that the Assessment TAC has given advice, will the MSIP TAC visit about what this means to the 2017 APR?

The MSIP TAC will convene on Sept. 12.

Can you explain in detail how a district-level decision has been made about withholding scores but DESE is unable to make the same decision for building-level APRs?

What data will be used in the 18, 19, & 20 APRs with different data sets in all three years?

Is there a process to contest our APR if ours is negatively impacted?

HOLD HARMLESS (Chris, this whole section is yours.)

GLA Assessments changed forms several times and those were used in APR calculations over time with Hold Harmless in place. What is different about this situation?

If the decision is to remove Algebra I and English 2 from previous years MPI calculations, will the new Hold Harmless policy still be utilized to calculate 2017 APR points?

Will Hold Harmless now include 2014 results?

During the APR review period, will each district's result be looked at individually to give the best scenario possible....similar to hold harmless?

WHAT’S NEXT?

Bottom line, what happened last year is bad, but we are scheduled for Fall EOC testing. What do I go tell my counselor and teachers? How should we move forward? Stacey or Sarah

For planning purposes, what is your estimate on timelines?

We plan to have achievement data to districts and charter schools during the week of Sept. 5. The appeals window for data corrections for EOCs closes Sept. 22.

Will there be a comment/feedback period after the department and TAC have come up with possible solution/changes to the formula?

Comprehensive Guide revisions will be made during the week of Sept. 18. A public hearing on the Comprehensive Guide will be held on Oct. 4.

Moving forward, are test coordinators supposed to attend the trainings for Nextera? Lisa

GENERAL QUESTIONS

How does this connect to federal reporting for math and ELA? Lisa or Chris

How much will removing a large group of scores change the thresholds for 2020 Target/On Track/Approaching”? Chris

Are summer (2016 or 2017) assessments affected? Lisa

Was this an issue in the fall? If so, wasn't the issue apparent at that point? Lisa

Were the scores higher or lower? Lisa

Were the tests easier or harder? Lisa

How do we need to record EOC levels and scores on student transcripts/records? Chris

Are we still going to receive item analysis? When? Lisa

Yes.

Will we see item analysis on Grade-Level Assessments also? Yes

Will DESE be providing a press release to the media?

DESE will be talking with reporters who cover education for wire services and major newspapers.

Will DESE be providing talking points when talking with media requests or parent questions?

Yes

Do we need to change the student reports from the EOCs? No.

With adjustments to APRs --has DESE run the data across multiple districts to see possible impact of draft changes? Chris

How does this affect C.T.E. Perkins summary data? Blaine

It should not have any impact at this time, most likely it will impact summary data beginning in the 2019-2020 school year. We use Algebra I and English II scores as measures of academic attainment for Perkins funding accountability. There will be no impact this year because we don’t pull scores for students until they have graduated from high school. Usually students take Algebra I and English II in their Freshman or Sophomore years in school, therefore the impact of not having these scores available will not be a factor until two or three years from now. Our plan at this time is to ask for a waiver from the U.S. Department of Education when these scores become reportable.

Has DESE considered taking the outages that took place during the testing process into consideration for their validity? Lisa/Stacey

Do we distribute the scores to students/parents? Chris

Will districts be reimbursed for the cost of last year's EOC tests in these 2 subjects? Stacey/Bill

Can you share all the possible solutions on the table? Chris

If our scores are already in our Student Information system should we remove them? No.

Should we contact parents and let them know what happened?

DESE prepared a letter to parents notifying them of changes to the 2017 APR and how those changes affect their students. Districts may use this in their communications however they choose.

Should we still send out score reports for Algebra I and English II to parents? Lisa

How will this impact next year's tests? Isn't the same vendor in line to do them? Stacey/Bill

How can cut scores be released in May but you do not see it until late July? Lisa

How do we explain this situation to teachers and students? We would encourage you to share the messages from the Commissioner with your teachers.

Will districts be able to review data after adjustments have been made? Chris

Questar is the vendor for Nextera. Is the state confident that Questar will not repeat this problem with 2018 results? Stacey/Bill

Am I correct in my understanding that we should not expect to see an updated workbook until October? Chris or Lisa (I’m not sure what “workbook” means.)

Will Algebra 1 and English 2 be available this fall? Lisa

We post the scores directly to our transcripts, should we remove these scores for Algebra I and English II? Consistent with previous guidance from DESE, we encourage districts to use raw scores. (Lisa, response OK?)

We reimburse students half their tuition based on achievement level. How are we able to do this if the test is invalid? Tuition?? Ideas anyone?

Why are we waiting for multiple iterations of MSIP to achieve our "desired state" rather than inititiating change through MSIP VI? Chris

Has there been discussions about possibly using ACT instead of EOC? Lisa

When will the decision be made and updated MSIP progress/status information be shared with districts? Chris

What procedures are being put in place to ensure this does not happen again? What level of confidence do you have that this irregularity will be corrected for the 2017-18 school year? Stacey/Bill

How do we know that the other tests were not impacted? Lisa

When will we have state proficiency levels for all other EOCs? Chris?

We are thinking of additional EOCs as being Algebra II and Geometry and English I. Will these be used? Yes. These tests will be used as normal.

Will reports be updated that show which EOCs students have taken so that we know which seniors need to be tested in which areas before they graduate? Chris

Can we expect this webinar, with all questions and answers, to be posted on DESE's website?

You should receive a link to the webinar by email.

I have 9th graders that took MAP math 8 instead of Algebra I. They are currently in Algebra II. Should I have them take the Algebra I EOC in the spring for A+ purposes? We usually test them in the fall of their 9th grade year. Chris

Is DESE in litigation with Questar? Aren't they the same vendors for the new test this fall? Stacey/Bill

Are you asking for us to all email you the dollars spent on Algebra I and ELA II for the 2016-17 EOCs? Stacey/Bill

How will we track mandated EOCs between districts - still the EOC History Report? Chris?

Any thought into using this as a platform to illustrate concerns regarding standardized testing and impact MSIP 6? Chris

Does the department plan to begin collecting data points from vendors more than once a year, to monitor quality assurance? Stacey/Bill

Since the Q&A indicated that the participation level was met, will we receive a file that shows that students took the test in order to meet the requirement that graduates take the 4 required tests? Chris

Appreciate the comment about DESE taking a step back to reconsider our approach to assessment, but haven't you received significant feedback from school districts in how best to approach assessment through the MSIP revision feedback? Chris

How can we use the student reports from these EOC's that list the "wrong" proficiency levels? Lisa

When will MPI scores for grades 3-8 be released? Or, are we able to forecast our own MPI based on the data we have received for grades 3-8? Chris

Should we put the Algebra I and English II scores on the student's permanent records? Consistent with previous guidance from DESE, we encourage districts to use raw scores.

What will be the process for district input on APR proposed solutions? Chris

What type of data/file set will we receive from DESE? We upload EOC scores to our school information system to track completed EOC assessments. Will we receive a file for Algebra I and ELA II for 16-17 to upload? Jeff, Chris, and/or Lisa

author:Bowles, Nancy
last saved by:hennib
revision number:2
last print date:2017-09-01T20:30:00Z
creation date:2017-09-05T13:43:00Z
Last Saved Date:2017-09-05T13:43:00Z
document template:Normal
edit minutes:3
page count:9
word count:2885
character count:16446
source:Microsoft Office Word
security:4
Count Lines:137
Count Paragraphs:38
Scale Crop:no
company:State of Missouri
links up to date:no
Count Characters with Space:19293
shared doc:no
Links Dirty:no
Application Version:14.0000
_DocHome:3031134010