Message: EOC FAQ

EOC FAQ
FromBowles, NancyDate  Thursday, August 31, 2017 12:40 PM
To
Potter, Sarah
Cc
SubjectEOC FAQ
EOC FAQ.docxEOC FAQ.docx (33 Kb )

Sarah,

 

I was hoping to get this finished before I retire, and it looks like I have succeeded. J There are MANY unanswered questions, but I was able to answer a few. Also, you will see red text with a question about the Top 10 goals. I was trying to remember what Commissioner said yesterday, but that’s as far as I got. Didn’t know what the official wording will be.

 

Nancy Bowles | Communications Specialist | 573-522-8311 |  dese.mo.gov | facebook.com/MOEducation | twitter.com/MOEducation | youtube.com/MODeptofEducation

 


EOC FAQ

INTRODUCTION

We are extremely frustrated with our vendor’s inability to deliver usable results. We know that Missouri students, teachers and districts work hard throughout the year, and they are counting on us to get this right. We know you have questions. Below are the most common questions we heard from districts in our discussions about this issue.

RAW V. DISTRICT EOC SCORES

Can you clarify with more detail exactly what was wrong that triggered the review?

2018 is the first year the new Missouri Learning Standards in Math and ELA are to be assessed. Is there a chance that was a reason for the test not comparing? Did teachers start teaching the new standards too soon?

Is it possible to release the state's passing percentages for EOCs in spring 2017 in order to understand the comparability issue?

Why are raw scores okay, but final EOC scores are not?

The raw scores accurately reflect a student’s performance. The assessments themselves were deemed accurate and reliable by DESE’s Technical Advisory Committee and would produce the same results year after year. The district and statewide scores are not comparable because the cut scores in 2017 changed from 2016.

So we do disregard the previous data that we saw?

Are raw scores still valid to use for student grades in Algebra I and English II?

Yes, raw scores may be used for students’ grades. The tests themselves were valid and reliable.

Are the score comparable across students?

What specifically was not comparable?

The 2017 cut scores do not align with cut scores from the 2016 assessments, rendering comparisons unreliable.

Does that not indicate a problem with the scale but not a problem with the scores? Why not adjust the scale?

From previous correspondence, we understand that the forms issue was within the forms of the 2017 Algebra I and English II EOCs, but was it really between the forms from 2016 to 2017?

We give the English 1 EOC also. Is E1 also not comparable? What about Algebra II?

English II and Algebra I are the only EOCs that are not comparable.

 

You keep mentioning the pattern. What exactly was the pattern?

If these were not new forms, one could assume that they have been used more than once in the past. How do the scale score data on these differ from when these forms were used in the past?

Do we need to send home new reports with only the raw scores for the Algebra 1 and English 2 EOCs?

What type of data/file set will we receive from DESE? We upload EOC scores to our school information system to track completed EOC assessments. Will we receive a file for Algebra I and ELA II for 16-17 to upload? 

A+ ELIGIBILITY

Please address how this will impact A+.

Districts may use EOC raw scores for determining A+ scholarship eligibility. Student record labels may be used as appropriate.

But you ARE comparing forms when all A+ students are being held accountable to the same achievement levels. One student may have had a harder test and scored lower, missing proficient, compared to a student who may have taken an easier test.

Will districts be able to make their own proficiency levels for the A+ levels or should they use a lower proficiency cut score?

Since this will be valid for A+, does this mean we will get a list of the results and include it in our assessments scores/upload it in SIS?

Since A+ requires proficiency on Algebra I, can that be changed to a required raw score expectation since the achievement levels seem to be bad?

For students that did NOT attain a status in Algebra I that would allow for A+ scholarship money, what rights do they have? Will they be allowed to appeal? How can you throw out the entire test, but tell certain kids their score will still count?

Who will pay for the re-take for A+ if needed?

Is it fair to tie A+ money to test scores when the validity of test scores can be called into question like this? Especially considering (a) some students have already graduated and not met the Algebra I requirement; and (b) the students who might be forced to retake the test aren't in Algebra I class anymore, so how can they reasonably be expected to "test better" now?

ALGEBRA I/ALGEBRA II/8th GRADE MATH

Why does the test change each year? Isn't Algebra 1 the same concepts that need to be taught each year and be tested on? Why change if the past worked?

Were not the 8th grade Algebra I scores used in the place of 8th grade MAP Math scores for those 8th graders taking the Algebra I EOC?

Yes. Eighth-grade students who took the Algebra I EOC are not included in 8th grade math scores.

By throwing out all Algebra I, does that mean students who were required to take Algebra II are shouldering all the accountability?

Will the fall 2017 Algebra I test be like the past spring 2017 test, or will it be comparable to the new 2018 spring Algebra I test?

 

 

Since this is the first year that Algebra II was being used for graduation requirements, could it be entertained to drop this from APR (taking it in isolation without the Algebra I EOC)?

Is excluding 8th grade math overall in the APR calculation an option on the table?

Explain the "no effect" for a middle school - the 8th grade MAP scores will then carrying the weight for the building. Usually the Algebra I scores HELP middle schools overall.

How does this issue affect 8th graders who took the Algebra I EOC instead of the 8th grade MAP?

For purposes of grades, raw scores from the Algebra I EOC may be used. The raw scores are accurate and give students, teachers and parents an idea how students are performing.

If the majority of our 8th graders take the EOC, how can that not affect us in a negative way? 

For districts that have the 8th grade Math/ Algebra I issue, what will be done to overcome the gaping hole that the Algebra I EOCs traditionally fill?

All of our 8th graders take Algebra I. We utilize their scores to determine eligibility to take the Algebra II EOC for high school accountability or re-take Algebra I (if received Basic or BB). How will this be affected?

Will all districts receive 8th grade math scores?

Yes. The 8th grade Grade-Level Assessment results will be used in the APR.

Since 8th grade scores are so district dependent regarding Algebra I, have you considered the possibility of looking more at grades 3-7? LEA's are more alike on those variables. 

What is the impact for K-8 with Algebra I High School Readiness and Grade-Level Math?

Will Algebra II count towards the APR for the kids who took Algebra I prior to high school?

Yes, Algebra II is the required high school mathematics assessment for accountability purposes for students who took the Algebra I EOC assessment prior to high school.

Our high school Algebra I scores were the highest math scores in our district. How will that impact our APR?

Due to the disruptions that occurred during testing, will students that did not score proficient on Algebra I have the opportunity to take the test again?

If we have students re-take the Algebra I EOC, is there anything we need to do to retract the old score?

What about seniors that have already graduated? How will they retake the Algebra I test?

Algebra II is optional for some. If we are negatively impacted by the inclusion of those scores, will there be an appeal process?

We gave our 8th grade students who took Algebra I the 8th MAP test this past spring rather than the Algebra I test. If wegive those students the Algebra I test this December, will this test be a valid test? OR should we wait until the spring and test them on Algebra I and Algebra II so that the kinks in the new testing system can be worked out?

If the MPI of Algebra II in isolation brings down our APR, would you consider exempting it for a district?

2017 APRs

As a single high school district, can you explain how APR will be calculated?

What about the statute that states that if a change occurs to the assessment or the MSIP standard, the previous year's APR can remain in place?

As we communicate with our stakeholders (staff, parents, community), we always like to convey our message with confidence. When do you estimate we can confidently provide some concrete information about APR? Is there a chance that there could just be NO APR for 2016-17?

We plan to release APRs in late October.

When our district is judged by overall APR scores, taking away these scores will again negatively impact our performance. Once the APR is released to the media, how do we respond?

Please elaborate on this statement: Status targets and progress calculations will be adjusted to accommodate the loss of the two EOCs.

If you take out some of the scores, won't that in effect increase the impact of the remaining scores (Biology and Government)?

If we scored higher on the English II can we use that for our APR?

Can we sub in geometry for Algebra II?

Will additional EOCs be used for APR as stated in the commisioner’s video message yesterday?

What will the impact be for districts that do not take the optional EOCs? What impact will this have on districts that rely on high school scores as a part of achieving full progress/status points to achieve full accreditation points?

The 2017 APR will be based on 100 points instead of the usual 140 points. Status targets and progress calculations will be adjusted to accommodate the loss of the two EOCs.

How will this affect K-8 districts APR scores for high school readiness?

The High School Readiness Standard will not be used in the 2017 K-8 APRs.

To clarify, will the middle school APR be based only on 8th grade MAP since Algebra I is taken out?

So, prior year APR won't be recalculated?

No. However, to gain an apples-to-apples comparison, we will need to develop a data set that removes Algebra I and English II EOC results from prior years to calculate 2017 APRs.

If English II and Algebra I were our highest scores, then how will this impact our APR?

What will be used in APR calculation if we only take the required Algebra I and English II EOC and do not take the optional EOC's?

Now that the Assessment TAC has given advice, will the MSIP TAC visit about what this means to the 2017 APR?

The MSIP TAC will convene on Sept. 12.

Can you explain in detail how a district-level decision has been made about withholding scores but DESE is unable to make the same decision for building-level APRs?

What data will be used in the 18, 19, & 20 APRs with different data sets in all three years?

Is there a process to contest our APR if ours is negatively impacted?

HOLD HARMLESS

GLA Assessments changed forms several times and those were used in APR calculations over time with Hold Harmless in place. What is different about this situation?

If the decision is to remove Algebra I and English 2 from previous years MPI calculations, will the new Hold Harmless policy still be utilized to calculate 2017 APR points?

 

Will Hold Harmless now include 2014 results?

During the APR review period, will each district's result be looked at individually to give the best scenario possible....similar to hold harmless?

WHAT’S NEXT?

Bottom line, what happened last year is bad, but we are scheduled for Fall EOC testing. What do I go tell my counselor and teachers? How should we move forward?

For planning purposes, what is your estimate on timelines?

We plan to have achievement data to districts and charter schools during the week of Sept. 5. The appeals window for data corrections for EOCs closes Sept. 22.

Will there be a comment/feedback period after the department and TAC have come up with possible solution/changes to the formula?

Comprehensive Guide revisions will be made during the week of Sept. 18. A public hearing on the Comprehensive Guide will be held on Oct. 14.

Moving forward, are test coordinators supposed to attend the trainings for Nextera?

GENERAL QUESTIONS

How does this connect to federal reporting for math and ELA?

How much will removing a large group of scores change the thresholds for 2020 Target/On Track/Approaching”?

DESE is moving away from Top 10 by 20 with new strategic priorities. Our goal remains the same: that all Missouri students graduate ready for college and career.

Are summer (2016 or 2017) assessments affected?

Was this an issue in the fall? If so, wasn't the issue apparent at that point? 

Were the scores higher or lower?

Were the tests easier or harder?

How do we need to record EOC levels and scores on student transcripts/records?

Are we still going to receive item analysis? When?

Yes.

Will we see item analysis on Grade-Level Assessments also?

 

Will DESE be providing a press release to the media?

DESE will be talking with reporters who cover education for wire services and major newspapers.

Will DESE be providing talking points when talking with media requests or parent questions?

Yes

Do we need to change the student reports from the EOCs?

With adjustments to APRs --has DESE run the data across multiple districts to see possible impact of draft changes?

How does this affect C.T.E. Perkins summary data?

Has DESE considered taking the outages that took place during the testing process into consideration for their validity?

Do we distribute the scores to students/parents?

Will districts be reimbursed for the cost of last year's EOC tests in these 2 subjects?

Can you share all the possible solutions on the table?

If our scores are already in our Student Information system should we remove them? 

Should we contact parents and let them know what happened?

DESE is preparing a letter to be sent to parents notifying them of changes to the 2017 APR and how those changes affect their students.

Should we still send out score reports for Algebra I and English II to parents?

How will this impact next year's tests? Isn't the same vendor in line to do them?

How can cut scores be released in May but you do not see it until late July?

How do we explain this situation to teachers and students?

Will districts be able to review data after adjustments have been made?

Questar is the vendor for Nextera. Is the state confident that Questar will not repeat this problem with 2018 results?

Am I correct in my understanding that we should not expect to see an updated workbook until October?

Will Algebra 1 and English 2 be available this fall?

We post the scores directly to our transcripts, should we remove these scores for Algebra I and English II?

We reimburse students half their tuition based on achievement level. How are we able to do this if the test is invalid?

 

Why are we waiting for multiple iterations of MSIP to achieve our "desired state" rather than inititiating change through MSIP VI?

Has there been discussions about possibly using ACT instead of EOC?

When will the decision be made and updated MSIP progress/status information be shared with districts?

What procedures are being put in place to ensure this does not happen again? What level of confidence do you have that this irregularity will be corrected for the 2017-18 school year?

How do we know that the other tests were not impacted?

When will we have state proficiency levels for all other EOCs?

We are thinking of additional EOCs as being Algebra II and Geometry and English I. Will these be used?

Will reports be updated that show which EOCs students have taken so that we know which seniors need to be tested in which areas before they graduate?

Can we expect this webinar, with all questions and answers, to be posted on DESE's website?

You should receive a link to the webinar by email.

I have 9th graders that took MAP math 8 instead of Algebra I. They are currently in Algebra II. Should I have them take the Algebra I EOC in the spring for A+ purposes? We usually test them in the fall of their 9th grade year.

Is DESE in litigation with Questar? Aren't they the same vendors for the new test this fall?

Are you asking for us to all email you the dollars spent on Algebra I and ELA II for the 2016-17 EOCs?

How will we track mandated EOCs between districts - still the EOC History Report?

Any thought into using this as a platform to illustrate concerns regarding standardized testing and impact MSIP 6?

Does the department plan to begin collecting data points from vendors more than once a year, to monitor quality assurance?

 

Since the Q&A indicated that the participation level was met, will we receive a file that shows that students took the test in order to meet the requirement that graduates take the 4 required tests?

Appreciate the comment about DESE taking a step back to reconsider our approach to assessment, but haven't you received significant feedback from school districts in how best to approach assessment through the MSIP revision feedback?

How can we use the student reports from these EOC's that list the "wrong" proficiency levels?

When will MPI scores for grades 3-8 be released? Or, are we able to forecast our own MPI based on the data we have received for grades 3-8?

Should we put the Algebra I and English II scores on the student's permanent records?

What will be the process for district input on APR proposed solutions?

What type of data/file set will we receive from DESE? We upload EOC scores to our school information system to track completed EOC assessments. Will we receive a file for Algebra I and ELA II for 16-17 to upload?