Ofc of Testing & Evaluation Oper

What is the program’s mission statement or statement of purpose? Why does the program exist?

The Office of Testing & Evaluation supports the educational mission of the University by providing high quality testing, exam scoring, evaluation and survey services for faculty, students, and staff. In addition, the Office of Testing & Evaluation provides testing services to the local community. Services for all constituents are provided in a professional, secure and accessible environment that supports academic success.

How does the program mission align with the university mission?

The Office of Testing & Evaluation aligns with university mission of rigorous inquiry by students, excellence in teaching and research opportunities for instructors by providing quantitative feedback as a means to measure student performance (exam results) as well faculty performance (evaluation results). Our program supports faculty and graduate student research by providing quatitative data reports generated from surveys.

Whom does the program serve?

Faculty, staff, students, non-eiu students and local community.

What are the services provided? How do these services align with the university mission and program mission?

• Testing for current and newly admitted students. (ie: Math Placement Exams, Nelson-Denny Reading Comprehension assessments
• Accommodated testing for students registered withe the Office of Student Disability Services
• Faculty evaluations
• Exam scoring services for faculty
• Survey services

Data is provided to departments

Describe the program’s origins (e.g. year established, purpose, expectations).

It is unclear when the Office of Testing & Evaluation (or some variation of the name) was established. As far back as can be determined, the office has always been charged with the responsiblity of producing and scanning faculty evaluations, scoring faculty classroom exams and administering various exams to students.

How has the unit changed or adapted over time?

Technology has greatly enhanced the services provided by the Office of Testing & Evaluation. Faculty members may now choose to have their evaluations administered online, allowing students to complete them with smart phones, tablets, computers, etc outside the traditional classroom setting; exam results are provided to instructors via a secure website;evaluations results are also distributed to department chairs via a secure website; exam proctoring services are now available for qualified students, providing a distraction free testing environment. By providing results electronically, the department has realized cost savings in not only paper and printing costs, but also personnel dollars.

If needed, provide supplemental comments to help the reader understand the program’s history and relevance to university mission.

Provide data, if available, on the numbers of students, faculty, staff, or others served by the program during the past four years. If no data are available, please estimate the numbers served annually.

2013
EXAMS Administered
Test Center - Pilot Program AY13

• 35 Students participated
  68 Faculty members participated
  223 Exams administered

651 non Test Center exams administered (Math Placement/TOEFL/CLEP/DANTES/ACT Residual)

Non-EIU student proctored exams:
19 outside university/college or home-schooled students/exams
9 City of Charleston police promotional exams

EXAMS Scored

95,716 exam sheets across all colleges
540 Nelson-Denny Reading Assessment

FACULTY EVALUATIONS

60,295 evaluations processed: 50,351 scannable; 11,227 on-line

MISC Scanning

Academic Challenge: 1015 sheets scanned
44 Surveys created (2,572 sheets scanned)

_______________________________________________________

2012
EXAMS Administered

639  exams administered (Math Placement/TOEFL/CLEP/DANTES/ACT Residual)

Non-EIU student proctored exams:
20 outside university/college or home-schooled students/exams
12 City of Charleston police promotional exams

EXAMS Scored

100,583 exam sheets across all colleges
643 Nelson-Denny Reading Assessment

FACULTY EVALUATIONS

64,979 evaluations processed: 52,146 scannable; 12,833 on-line

MISC Scanning

Academic Challenge: 1012 sheets scanned
47 Surveys created (6,368 sheets scanned)
_______________________________________________________

2011
EXAMS Administered

549  exams administered (Math Placement/TOEFL/CLEP/DANTES/ACT Residual)

Non-EIU student proctored exams:

11 outside university/college or home-schooled students/exams
12 City of Charleston police promotional exams

EXAMS Scored

99,709 exam sheets across all colleges

FACULTY EVALUATIONS

69,012 evaluations processed: 58,731 scannable; 10,281 on-line

MISC Scanning

I Sing the Body Electric: 488 sheets scanned
Academic Challenge: 1016 sheets scanned
10 Surveys created (5,967 sheets scanned)
_______________________________________________________

Does the program offer any co-curricular opportunities for students (e.g., internships, RSO support, service-learning activities)? Please describe.

No co-curricular activities are available to students due to the nature of the services we offer.  We deal with faculty exams and faculty evaluations, both of which are confidential in nature. We do have student workers, but they are not allowed access to evaluations and only delivered sealed exams to be scored.

Does the program contribute to the delivery of academic programs (e.g., providing professional expertise, serving as adjunct faculty)? Please describe.

All of our services contribute to the delivery of academic programs.  We provide exam results for scanned exams to instructors; we distribute and process faculty evaluations which are used for faculty to meet contractual obligations; we also process advisor evaluations used for Merit/PBI.

Our newest service provides a secure/proctored testing environment for students who receive testing accommodations as part of their accommodations provided by the Office of Student Disability Services.  By using this service, instructors do not have to provide these services themselves.

We also administer the ACT Residual to students seeking admission to EIU; provide survey services to faculty and graduate students.

The director teaches one section of EIU1111 - University Foundations for new, first time freshmen every fall.

If needed, provide supplemental comments to help the reader understand the internal demand for the program. Note any clarifications or special circumstances (e.g., curriculum changes made by another program) that should be considered when reviewing the above data.

Is the program accredited or approved by a recognized external agency or otherwise certified to meet established professional standards? Provide an executive summary of and link to the program’s most recent accreditation or certification report, if available.

The newly enhanced Test Center (now providing accommodated testing to qualified students), follows the professional guidelines and standards set forth by the National College Testing Association (NCTA). We have not yet, however, gone thru the official certification process.

Is the program required to meet any regulatory or legal requirements? Is the program subject to any special auditing requirements?

FERPA regulations are followed as are any testing regulations established by national testing and assessment companies such as ACT, ETS (TOEFL), and CAE (CLA). None of our services are subject to any special auditing requirements.

What are the most important outreach or public service activities supported by the program?

• Provide test proctoring services for non-eiu students taking courses for colleges and universities as well as students who are being home schooled.
• City of Charleston Police Department, promotional exams - administration and scoring
• Provide scoring services for the Academic Challenge Competition sponsored by WYSE (Worldwide Youth in Science and Engineering) & EIU PreEngineering Studies

How do the local community and the region benefit from the program?

By providing proctoring services to non-eiu students, students are able to further their education via non-traditional methods, while maintaining academic integrity and completing exams in a proctored setting. 

Administering and scoring the police promotional exam is a service we offer to the City of Charleston. By administering this test on campus, officers are able to to test locally and results are generated in a very timely manner. 

If needed, provide supplemental comments or data sources to help the reader understand the external demand for the program.

What are the two or three more important measureable outcomes tracked to assess program quality? Does the program conduct an outcomes assessment, and if so, what has been the impact?

The only true measure of program quality OTE has is the continued use services by faculty, staff and students.  OTE continues to scan large numbers of faculty exams each year with no measurable errors; faculty/course evaluations are produced/results scanned in a timely manner, meeting UPI contractual deadlines; and test students in a proctored environment that reduces the chances of cheating.  OTE does not conduct an outcomes assessment on a regular basis. Satisfaction surveys are conducted occasionally, but without a significant response rate.

What are the two or three most effective best practices that the program has implemented? What benefits have been gained from implementing these best practices?

Purchasing Class Climate software for faculty/course evaluations has been instrumental in giving instructors flexiblity in the type of evaluation (online or paper), but also allows for online evaluations to able to be completed on smartphones, tablets, pcs, etc.  Department chairs are able to access evaluation results approximately 2 or 3 weeks earlier than before implementation of this software, and saves innumerable hours of staff overtime. 

What external recognitions (e.g., awards, accommodations, professional certifications, references in trade publications) have the program and its staff received in the past three years?

none

Is the program active with any regional, national, or international professional organizations?

NCTA - National College Testing Association

Note any presentations, publications, or offices held in the last three years.

If needed, provide supplemental comments or data sources to help the reader understand the quality of program outcomes.

Note any special benefits (e.g., personnel support, equipment, permanent improvements) that the program has received in the past three years from its grants and other sponsored programs.

n/a

How does the program benefit from donor gifts (e.g., scholarships, endowed chairs)? Does donor support provide a significant percentage of the program’s overall funding?

N/A

List two or three key relationships that the program maintains with external constituencies (e.g., community colleges, other universities, government bodies). How do these relationships advance the university mission or otherwise benefit the university?

Administration of the City of Charleston Police Promotional Exam each year is an integral part of maintaining a relationship between the city and the university. 

OTE maintains a working relationship with many university and colleges by offering proctoring services to both eiu and non-eiu students. Students are able to complete online exam requirements without having to travel to the university/community college hosting the course, allowing opportunities beyond the EIU campus.

If needed, provide supplemental comments to help the reader understand the resources generated by the program. Note any clarifications or special circumstances (e.g., revenue pass-throughs) that should be considered when reviewing the above data.

Provide an executive summary of and link to any metrics or benchmarks that the program tracks to measure productivity.

http://www.eiu.edu/~cats/vpaa/auap12-13/index.php

http://www.eiu.edu/~cats/vpaa/auap11-12/index.php

OTE measures productivity by tracking the number of evaluations and exams scanned as well as the number of tests administered in the department.  The links above show these numbers for each year.

What initiatives has the program implemented to enhance staff productivity (e.g., access to training, workflow improvements)? Briefly describe the costs and benefits of these initiatives.

2013

Development of Test Center for students needing accommodated testing. This is the first full year of the Test Center so costs are only estimated at approximately $15,000 but has not, as yet, been fully funded.  Instructors are able to use the proctoring services we provide instead of providing the accommodations to students themselves.

2012

A new evaluation software was purchased (Class Climate). 3 staff members attended training for this software

2011

An OSS position was moved to a centralized location in order to assist two departments, saving the cost of additional clerical support.

If needed, provide supplemental comments to help the reader understand the productivity of the program. Note any clarifications or special circumstances (e.g., accreditation requirements, curricular changes, program restructuring) that should be considered when reviewing the above data.

If needed, provide supplemental comments to help the reader understand the costs associated with the program. Note any clarifications or special circumstances (e.g., expenditures made centrally or externally, expenditures made on behalf of other units) that should be considered when reviewing the above data.

129300 commodities expenditures are primarily for scantron forms used by faculty university-wide for exams and for printing faculty evaluations.

Contractual expenses are related to software/hardware maintenance agreements related to exam scoring and evaluation processing - a service available to all faculty and staff members.

229301 2012 costs included computers for the testing lab.

The salary lines in this account are primarily proctoring fees paid that have income generated by the exam off-setting the expense.

How does the program seek to distinguish itself from similar programs at other institutions?

There are no other programs at EIU that produce/process faculty evaluations or electronically score exams.  Some departments have areas where students that receive testing accommodations can take exams, but OTE offers these accommodations in a controlled environment with trained proctors.

Note any unique and/or essential contributions that the program makes to the university.

• OTE is solely responsible for the processing of faculty/course evaluations; evaluations are printed/distributed to faculty members/results processed and sent to department chairs as required by union contract
• The Test Center provides a secure/proctored exam environment for students that are eligible for testing accommodations as part of their services as outlined by Office of Student Disability Services
• Electronic exam scoring is free to all faculty members

Provide any program-specific metrics that help to document program contributions or program quality. Examples of some commonly used program-specific metrics may be found here.

If needed, provide supplemental comments to help the reader understand the program impact on the university mission.

Provide a link to or listing of the program’s goals and/or strategic plan.

http://www.eiu.edu/~casa/CASA%20Report%202013%20all.pdf

What role will the program have in the implementation of the university’s strategic plan (provide link to strategic plan)?

http://www.eiu.edu/strategicplanning/Strategic_Plan_Final.pdf - page 6 - "Be the institution that creates the best opportunities for success including degree completion, advanced studies, and career preparation". With adequate funding, an expansion of the services offered by the test center will help achieve this goal. Services can be expanded to not only meet the needs of students requiring testing accommodations, but also of those choosing to pursue their education in a non-traditional manner. 

page 7 -"Strengthen our support structures that serve students and employees with disabilities,
significant personal challenges, and other special needs."  The  focus of the test center that was started FA13 is the student with disabilities. As long as this effort is funded, OTE will be instrumental in helping the university implement this area of the strategic plan.

In the next two or three years, what best practices, improvements in operations, or other opportunities to advance the university’s mission are likely to be implemented?

•  An expansion of the testing area of the Office of Testing & Evaluation; this has already been started AY14, but with limited funding.  With an ever increasing demand in the number of students requiring testing accommodations, the University will need to find a way to support this endeavor.
• Online classes present an opportunity for our department to provide proctoring services to students pursuing their college education in a non-traditional manner.

If needed, provide supplemental comments to help the reader understand future opportunities for the program.