Giving an online test is like giving a bubble sheet test up to a point. Instead of ....
students will see (ABCD, ABCDE, ABCD/FGHJ, etc.). For bubble sheet tests, the ...
Quick Start Guide for ROS This quick start guide for ROS v8.1 should help everyone understand the basics of ROS. If you need additional help or information about anything, please contact support at 866-724-9722, click Us under the Help tab, or email
[email protected]. In this document, we will be covering the following topics: • • • • • • • •
New Features Notification Working with Tests Giving a Paper Test Giving Online Tests Copying Library Tests Reports on Tests Creating Named Reports Reading ROS Reports
• • • • • • • •
Building New Tests Test Info Page Roster Files User Preferences Notes on User Preferences Using Tools with Online Testing Student Test Review Usage Reports
New Features Notification If we make a big change in the software, we will let you know. The first time you log in after we have made a feature update you see our Help New! page that describes the new features and gives you a quick link to your Test Tree page. The next time you log in you will go directly to your Test Tree page like normal. If you missed something or just want to see the latest information, click on Help at the top of ROS, and then click on New! Working with Tests In ROS, we organize your tests by year, subject, grade, and owner. We call this section of ROS the Test Tree. If you do not see the grade, you are interested in, click Show All Grades and you will see all the grades for that subject. ROS also lets you display only the subjects and grades you are interested in, for more information about this, see the User Preferences section.
Click on a grade and a name to see a list of tests. Each test is shows a checkbox for Version: 2010 10 04
Page 1
More questions?
Toll free – 866.724.9722
[email protected]
working with multiple tests), a color block for the assessment type, a bubble sheet icon and maybe an online testing icon. When you click on a test’s name, it shows three buttons: info, give, and reports. If you need to give a test, see reports on a test, or get more information about a test, click the appropriate button. If you need to do this for multiple tests, first select the checkboxes next to each test name and click the info, give, or reports button below any of the test names. For more information about what happens after clicking the info, give, or reports buttons, see below for the appropriate section. Giving a Paper Test After selecting your test and clicking the give button, you will be taken to a list of your classes. This section of ROS is called the Users and Entities Tree (Users are people: administrators, teachers, and students; entities are groups: districts, schools, classrooms, and seats.) Click on the class name to open it and see the students. Once you have chosen the right class of students and selected whom you wish to test, click the Print Sheets button to view the sheets on a PDF form, which you can then print on a local laser printer.
If you need to manually enter or correct a student’s answers click the Make CADE Entries button. If the test you selected is available for online testing, there will also be an Online Version: 2010 10 04
Page 2
More questions?
Toll free – 866.724.9722
[email protected]
Testing Monitor button to start an online testing session. See below for instructions. On the page above, the numbers in the Taken column show how many of students in the group(s) shown have taken this test. The Virtual column shows how many students have taken this test in another class. The Pending column shows which students are currently taking this test now, as an online test. Giving an Online Test Giving an online test is like giving a bubble sheet test up to a point. Instead of clicking Print Sheets, click Online Testing Monitor. Before you do that, make sure all of your students have their UserID and passwords. To get student UserIDs and passwords go to the Users section. Once there, navigate to the correct classroom and click the View link under the Settings column. In the classroom info screen that follows, click the link next to the Students line. This link produces a list of students’ names, ClientIDs, UserIDs, and passwords. Either print this screen or download, save, and print an MS Excel file. Students login to ROS in like you do, using their ClientID, UserID, and password. Clicking the Online Testing Monitor button starts an online testing session for your class. See the screen below.
The screen above shows an ongoing testing session. Six students need to start the test, one is working, and one has finished. One student has submitted his/her Version: 2010 10 04
Page 3
More questions?
Toll free – 866.724.9722
[email protected]
results, and two students’ test sessions have been stopped. Stopping a test session permits to the viewing of reports. You can stop a test session for all students or for individual students. (If you cannot see a student’s results, probably you have not stopped their test session.) To get the most recent information about these testing sessions click the refresh information button. (This screen does not update automatically). When students have submitted their tests you need to click the stop ALL sessions button when the entire class has finished. Stopping testing sessions saves all of the students’ responses and makes reports available immediately. Sometimes a student’s testing session becomes blocked. This means the student’s testing session has stopped and has been started again. For testing security reasons, ROS allows a student’s testing session to be started only once. Blocking can happen when two students are using the same UserID. Alternatively, it can happen because the student has used the web browser’s back button, the browser window was closed, or the computer was rebooted. When the student tries to log on again, the testing session will be blocked. Click the restart button in the Online Testing Monitor screen to allow the student to restart their session. Copying Library Tests ROS Stores sets of tests available to multiple clients in the Library. The test library often includes the Tests for Higher Standards (TfHS) tests, and may include your state's released tests, or other tests you may have access to. Test availability is set according to your district's permissions. To change these contact ROS support. Administrators may have access to different tests than teachers. Note that all Library tests are read and copy only. Only ROS can change these tests. Click the Library section at the top
Version: 2010 10 04
Page 4
More questions?
Toll free – 866.724.9722
[email protected]
of the page to see the list of tests available. Tests are organized by subject and grade. If you do not see the grade you want, click the Show All Grades link. Click to open the grades and tests you are interested in. Click the info button to review the test’s information and to ensure it is the correct test. Then click the copy button to copy the test to your own folder. Once you have copied the test, you will see all available options for a ROS test. You own the copy you just made and can modify it. See other sections of this document to use the Test Info Table features. Reports on Tests After selecting the desired test(s) in the Test Tree and clicking the reports button, you will see a list of classes that have taken this test. Click the report link on the right to view reports. You can also open one of your classes and view a report on an individual student. This will take to the Pick Report Type screen. Let’s look at the four common standard ROS reports first. The following screen shows when you click the ROS Tab. The four standard reports are these: the Item Analysis — shows details of student responses to each test item; the Matrix report — which shows performance on each item, standard, strand, and/or cognitive level, and on the total test and can be broken down by NCLB categories; the Progress report — which tracks performance on more than one test: and the Multi-Category report — which is a specially formatted Matrix Report for use in AYP calculations.
Version: 2010 10 04
Page 5
More questions?
Toll free – 866.724.9722
[email protected]
While viewing individual reports, do not forget to select the correct options to display the information you need. Check the needed options and click the rebuild button to view the updated report. Note that a few of the options you may have used in viewing your report can be saved as a user preference. See the section on User Preferences. Creating Named Reports Any report that ROS can produce can be saved as a Named Report. After clicking the Reports button the Test Tree, if you click the Teacher link on the left you will see all the Named Reports (custom formatted reports) that are available to all teachers. If you produce a Named Report of your own, there will be a third tab called Your where you can find all of the reports you have personally formatted.
Generally, to create a Named Report, you will start with one of the four standard ROS reports. Using the standard and or advanced control boxes, experiment with the various options available to get the report desired. After you make each change click the Rebuild button to view the report. When you get the report you want, click on Name This Report at the bottom of the screen. You will be taken to the Name this Report screen where you will enter a short report name (32 characters, max.) and a description of the report, which can be longer. Reading ROS Reports The Matrix Report: This tabular report is organized with either students or entities, (classrooms, schools, or demographic groups), on the left identifying the rows and test scores identifying the columns. The scores shown are often test items (questions) or groupings of questions such as standards, strands, or cognitive-skill groups. The total test score is always shown. The relatively simple report below is an example. Ten students from Jane Smith’s Math Section 1 class who took a 10-item math test in 09-10 are shown. In addition to their names, their gender is displayed. The numbers in the right-hand portion of the table are test scores. In this case, the numbers in each cell are percentage correct. Instead, if we wish we could display a number correct score or a fraction (3/4, read as, 3 correct out of a possible 4). The number in the top left Version: 2010 10 04
Page 6
More questions?
Toll free – 866.724.9722
[email protected]
cell is 0. That means that Abigail received a 0 percent correct on the first item. The item numbers are shown on the bottom blue row. Moving to the right, we find that she got 100 percent correct on the second item. (Most multiple-choice items are 1point items, so the only possible scores are 0% or 100%.) However, the third cell shows 50%. That is because it is a summary score based on two items. The average of 0% and 100% is 50%. The cell summarizes the two items that measure Standard 1.a.3. (found on the center blue row labeled “Std”). In the right-most column, labeled “Total”, is the total test score. In this case, Abigail on received a total score of 20% correct, that is 2 out of 10 items. These scores range from 10% correct to 100 % correct. The average score for the class was 56% correct ((lower right cell). What about the cell colors? These colors are set in the Test Info Box for this test. in this case, red means scores from 0% to 49%; orange means 50% to 59%, yellow means 60% to 79%, and white is 80% and above. The cut points and colors are customizable.
Below the report itself is the control box for the report. In the Group rows by: section of the box, we notice that student names and gender boxes are checked. These are displayed the report above. In the Group columns by: section, we see that the test, standard, and Item # boxes are checked. These controls are reflected in the columns on the right side of the report. If you change what is shown in the check boxes, click rebuild to display the new report. The Advanced Options section of the control box is shown below. We will not discuss it in detail here, but will point out that the two boxes together make possible a very great variety of variations on each basic report. Version: 2010 10 04
Page 7
More questions?
Toll free – 866.724.9722
[email protected]
The Item Analysis Report: This simple report shows the same ten students answered the same 10-item test. The ten items are indicated in the left-most column. For item #1, we see that 40% of the students (4 students) chose answer “B”; 10% of the students chose option “C”; and 50% chose option “D”. Option “B” is shows in green because it is the correct answer. This is also shown by the asterisk (*). We might want to look at item 1 on the test as more students chose a wrong answer. Make sure the item’s answer key is correct. Item 2 was answered correctly by all ten students. Notice that the average test score is still 56% correct. Below the report itself, is its control box.
Version: 2010 10 04
Page 8
More questions?
Toll free – 866.724.9722
[email protected]
Building New Tests Building new tests is quick and easy. Click the Build New Test button at the top of the Test Tree page. ROS will prompt you to enter the name of the new test and select the subject, grade, and the form type. The form type is an important piece of information. On all tests, the form-type sets the number of items and the type of answer options students will see (ABCD, ABCDE, ABCD/FGHJ, etc.). For bubble sheet tests, the formtype also sets the layout of the bubbles. You cannot change the form type once you have selected it, so choose carefully. If you are not sure about how many items will eventually be on a test, choose a higher number. For example, if you now have 40 item to put on a test, but may choose to add a few more later, choose a 60-item test form. After you click the Save This Test button, it is ready to use. You can give this test as a bubble sheet test right away, but do not forget to set the key. (If you do forget, all students will get all questions wrong, as there are no correct answers indicated. But don’t panic, you can add the answers later and all the tests will instantly rescore.) The Test Info Page
Version: 2010 10 04
Page 9
More questions?
Toll free – 866.724.9722
[email protected]
The Test Info Page lets you set all kinds of information about the test. You can change the test’s name (1), set the answer key, make a test an online, store files with the test, change the cut/pass scores, change the keep status, change the subject and grade, change the assessment type, or set special test options. Change or enter the key (2). Click on the key information. Then click on the edit key button. Scroll down and fill in the updated key information. Click the save changes button. Online Information (3). Clicking the online information takes you to an advanced feature. Enable or disable the students’ ability to review test results (4).Set test type (Test, Quiz, or Benchmark) (5). Change the test owner (6). Make various test status settings (7). Store associated files, such as the MS Word document from which you printed you test (8). Noted: At ROSworks, we rarely remove tests from the system. You can mark a test as deleted and it will show up in a different part of the Test Tree, but it is not actually removed from the system, only hidden, and you can retrieve the tests. Also noted: With ROS, you can give any test, even if it does not belong to you (except a Library test). You never need to copy a test just to give it to your students. By default, we automatically set all tests to not copyable for this reason. Making extra copies of tests can make it harder to view reports across grades or schools. If a teacher copies a test that was to be given to all students at a grade level and then gives it to his/her students, that teacher’s results will not be aggregated with the other teachers results in the school, as ROS “assumes” it is a different test. You need to make a copy of a test only to make changes to it or to set different options. If the test is yours, you can always copy it; otherwise contact the test owner.
Roster Files To operate, ROS needs information about students, teachers, and their class schedules. ROS gets this information from what we call a roster file. Administrative users take care of this for everyone so that you never have to add this information yourself. If you are an Administrative user and you need to update class schedule information, you need to generate and upload a new complete roster file. After you have generated the data export from you Student Information System, save the report to your computer. What you need for the upload file to contain is detailed in Roster File Spec link in the Docs section, under the ROS Help tab. To upload you saved file, look for the line that includes the word District and the current school year in the User Tree and click on the entity link. That will usually be the entity (school or district) directly below your own login. So, look for the line that includes the school name or district name and the current school year in the User Tree and click on the entity link. Version: 2010 10 04
Page 10
More questions?
Toll free – 866.724.9722
[email protected]
After clicking the entity link, you will see the Entity Information Table. The last row of this table has a link at the bottom that reads upload. Click it and follow on-screen directions. Normally, we process these roster files overnight, because the loading process slows down our servers. However, the files can be processed during the day, if necessary. Please contact Support if you need the Roster file processed quickly. After processing the roster file, ROS sends you a report in email about the information we loaded. Look it over. Especially look at the part of the table that list teachers deactivated. If there are a lot of these, it usually means that something went wrong. Any teachers deactivated will not be able to login. Call us and let us know, as we can roll back the upload. Be sure we have your correct email address(es). We can also automate this upload process. If you can generate the Roster File for us and place it on a password protected website, we can automatically copy it from you and process it. Contact Support by email for more information about this.
User Preferences ROS lets you save some report options you view often. ROS also lets you customize your Test Tree. An Administrative user may have already set your Report Preferences. You can override these settings with your own. An Administrative user may have also already set your Test Display Preferences. We encourage you to select the preferences that work for you. You can set which subjects and grades you want to see. You can also choose to see deleted tests, tests from previous years, or you can add the Taken, Virtual, and Pending columns to your Test Tree. For Report Preferences setting preferences at different levels have different effects. Setting preferences with a district or school Administrative user only affect those accounts. Setting preferences on the district or school entity can affect all accounts in the district or school. Setting preferences on a school entity will affect all the teachers at that school and the school Administrative user. All teachers and school Administrative users can override preferences by setting their own. For Test Display Preferences setting preferences at different levels have different effects, and these preferences work differently from the report preferences. Setting preferences with a district or school Administrative user only affect those accounts. Setting preferences for the district entity will affect all teachers at all schools (but not school administrators). Setting preferences for a school entity will affect all teachers at that school. Version: 2010 10 04
Page 11
More questions?
Toll free – 866.724.9722
[email protected]
All teachers can override any preset preferences by setting their own. Some suggestions: Elementary teachers will usually want to see all subjects for their own grade, and sometimes for the grade before. A middle or secondary teacher will usually want to see his/her subject area for all grades/courses he/she teaches. Administrators will usually want to see all grades, all subjects for the grades under their supervision. Administrative users also have access to a tab called Student Test Review. This controls what specific information students are permitted to see if student test review is turned on for a given test. There are five possible bits of information available for students. Using Tools with Online Testing ROS has released a new student-testing engine. We are keeping the standard testing available (this requires no plugins to web browsers). Our new student-testing engine has tools like Virginia’s end of year SOL testing. Using this student testing engine requires Flash version 10.1 plugin (or later), which is probably already installed on most computers. With this testing engine, we require that JavaScript be enabled in the browser. In Windows XP or more recent: users of Microsoft’s IE browser should be using version 7 or 8. Other browsers will also work: Firefox 2 or higher, Safari 3 or higher. Most other upto-date browsers (Chrome, Opera, Mozilla, etc.) will also work. On the Mac: IE 5.5 will not work. Other browsers will work: Safari 3 or higher, Firefox 2 or higher. Most other upto-date browsers (Chrome, Opera, Mozilla, etc.) will also work. On Linux/Unix/Ubuntu: Firefox: 2 or higher, Konqueror 3 or higher will work. Most other up-to-date browsers (Chrome, Opera, Mozilla, etc.) will also work. To allow students to use this new testing engine you need to set the options on your online test. To set the options on your test go to the test information page for the test and click on the This is an online test link. On the Online Information page for this test, click on the Test Options button. On the Online Options, page look for the Testing Software area. Select the testing engine (standard or tool enabled) and select the tools you want to enable for this individual test, and then click Save Options. When using the tool enabled testing engine we recommend you use the Eliminate Options and Highlight areas of screen for all tests. Click the others as appropriate.
Version: 2010 10 04
Page 12
More questions?
Toll free – 866.724.9722
[email protected]
Also note that on the table above is a setting called Item Numbering. This is a setting designed to increase online test security. If the option labeled Different Start is chosen, students will start with different items, so that not all students will be working on the same item at the same time. However, the first item shown to students is always labeled “1.” Reports are shown as though all students started with the actual #1. Student Test Review Because ROSworks believes that feedback to students (and parents) is an important part of the assessment process. ROS has a feature that permits students to review test results, subject to the limitations imposed on these results by the test owner and the top-level Administrative user. Student Test Review settings See the discussion under User Preferences under control the information teachers (test owners) are allowed to show students. By default, no results are available to students. The possible types of information that can be enabled for student viewing are: score percentage ( points / possible points), right and wrong test items, the display reporting category, standard identifier, cognitive level, and point value for each test Version: 2010 10 04
Page 13
More questions?
Toll free – 866.724.9722
[email protected]
item, the actual test item (if available), an image of the student's bubble sheet (if available). If the top-level Administrative user has enabled any of these categories of information, test owners can decide if they want to show this information (under Online Security Options on the Test Info Page) for their tests. The default setting for all tests is off. Each test has to be enabled individually by the test owner. If the feature is enabled, students (and parents) can log in to ROS as if they were going to take a test and see information about tests they've taken. Usage Reports Under the Help tab the final item is Usage. Clicking this shows four different types of usage reports for Administrative users. Testing Sources shows from where and how test data has been entered, on a daily basis. System usage is a weekly report which shows all tests scored by ROS and where they've come from. User Stats is also a weekly report which shows how many ROS accounts are used (not how frequently individual accounts are used). Finally, the last report, Test Stats, is also a weekly report that shows (nearly) all possible information about tests in the account. It may or may not be useful. The first part shows basic features used by the tests, who owns the tests, and test usage. The second part goes into painstaking detail about various other parts of the tests.
Version: 2010 10 04
Page 14
More questions?
Toll free – 866.724.9722
[email protected]
