From Self-Assessment to Accreditation -- Overview

The self-assessment process is a major enterprise for the agency or school. It is an educational endeavor, which requires time to reflect and engage in critical thinking about the agency’s current position with regard to its mission, activities, and outcomes. The process uses NAC’s standards as a way to help evaluate the operations and services provided. The comprehensive evaluation process, which is periodically and jointly conducted by the agency, school or program, and NAC, includes the following steps:

1.      The Executive Director or a representative of NAC communicates with the agency or school CEO. Eighteen (18) months before an on-site peer review (OSPR) team visit, NAC contacts the CEO regarding dates, appointment of peer review team chair, appointment of a “mentor” from NAC, and the comprehensive evaluation process.

2.      The agency or school analyzes itself through a comprehensive self-assessment outline provided by NAC. The Self-assessment is completed and sent to reviewers and to the NAC office30 days prior to the OSPR team visit. Electronic or printed copies are acceptable (NAC & most reviewers prefer electronic copies) of the self-assessment report.

Benefits of the Self-Assessment Study

§        An understanding the strengths of an organization with a clearer sense of direction;

§        Development of a plan for assessing the outcomes or results of programs and services offered;

§        Strengthening the planning process, resulting in a workable system of ongoing analysis, improvement, and monitoring;

§        A plan for the systematic evaluation of curricula, study guides, equipment and materials, financial resources, space and facilities;

§        A plan for the systematic use and evaluation of the agency's policies, procedures, and records.

The Accreditation Process

1.    Peer reviewers are selected from other agencies with similar experience and training to perform the on-site reviews. The team reviews the comprehensive self-assessment study, visits the agency, verifies ratings and supporting documents, evaluates findings, prepares a written report and submits it to the NAC Executive Director.

2.    A draft OSPR team’s report and findings is prepared by NAC and sent to the CEO. The CEO is given an opportunity to correct factual errors. Correct information with documentation or explanations provided in writing to NAC’s Executive Director.

3.    The Executive Director of NAC provides the institution’s CEO with a copy of the final draft of the OSPR team report. 

4.    The CEO of the agency, school or program provides NAC with a written response to the final draft team report. The CEO’s written response is attached to the report and included in the final report for the Board’s review and decisions.

5.    The NAC Board of Directors reviews the team’s written report and any written response from the agency to the report. By teleconference the Board receives verbal statements of the evaluating team chair and the CEO of the institution, which are followed by questions from Board members. In executive session the Board considers and discusses the OSPR team’s confidential recommendation before taking action on the accreditation status of the school or agency.

6.    Once the Board of Directors has made a decision regarding accreditation of an agency, school or program, The NAC office will provide a finale OSPR report, a letter stating the Board’s accreditation decision and a Certificate of accreditation within one month of the determination.

7.    The improvement process continues as the agency, school or program considers the results of its own self-assessment study and commendations and recommendations found in the OSPR final report and creates an action plan.

Careful consideration should be given to the amount of time needed to complete the steps listed above. Complex organizations may need a full year between application and the OSPR to thoroughly conduct their self-assessment study. If done well a self-assessment study is abundantly informative, beneficial, and rewarding to the agency itself. 

Options for Board Actions

§                    Grant full 5-year accreditation;

§                    Reaffirm accreditation;

§                    An annual progress report is mandatory to maintain accreditation. Based on the annual reports the NAC Board may advise a focused interim visit or additional documentation;

§                    Defer action on accreditation until a correction action plans is submitted;

§                    Issue or continue astatement about the termination of accreditation unless the school or agency has demonstrated to the satisfaction of the Board of Directors that it has satisfied all concerns by a specified date;

§                    Deny accreditation due to lack of compliance;

§                    Terminate accreditation when compliance is refused.

All Board actions set forth above are published in the minutes of the Board meeting at which the action took place.

REAFFIRMATION OF ACCREDITATION: Members are not accredited permanently. They are to conduct a comprehensive self-assessment study andbe reviewed by an OSPR team generally every five years. Accredited members are required to submit an annual progress report to the NAC office to maintain the tenure of their accreditation in good standing.

Part II

Organizing a Comprehensive Self-Assessment Study

The self-assessment study is the most significant part of the accreditation process. Its aims are to understand, evaluate, and improve services to individuals with blindness or low vision, not merely to defend what already exists. The self-assessment study is viewed as an ongoing process. NAC requires an agency seeking initial accreditation or re-accreditation to complete this process prior to each on-site visit. The capability of an agency to conduct a satisfactory self-assessment study is in itself a test of the qualities and capacities of an agency.

The general purposes of the self-assessment studies are to:

§        Analyze the resources and effectiveness of the agency in achieving its mission and goals;

§        Determine whether the expectations of clients served are met to their satisfaction;

§        Appraise the relationship of the agency’s activities to its mission, goals and resources;

§        Provide a sound basis for strategic planning and enhancing the quality of the agency;

Ultimately, a well-planned and executed self-assessment study suited to the complexity of the organization provides a basis for demonstrating that the agency warrants accreditation. If the agency, school or program seeks additional accreditation from another accrediting body, a joint or concurrent on-site review can often be arranged through NAC's Executive Director.

Administration:    The self-assessment study process begins with the commitment and support of the chief executive officer and the governing board. It is the administration’s responsibility to select and appoint members of a Coordinating Committee that represents the departments and structures of the agency. (This committee should reflect the selection in the standards manual that will be applied during the accreditation process.)

Agency Administrative Tasks

§                    Review the agency's existing committee structure and existing processes for evaluation and planning,

§                    Review NAC's Handbook of Accreditation.

§                    Create Committees for rating each applicable standard.

§                    Identify the focus, chair, and membership of each committee.

§                    Sets a schedule for completion of assignments.

§                    Adjusts its expectations and timelines as needed.

Coordinating Committee: In most cases responsibility for the self-assessment study will be delegated to groups representing various departments or divisions of the agency, school or program. The committee, under direction of the administration, should outline expectations regarding characteristics of an acceptable report (e.g. format, focus, editorial style). This committee will provide documents, information, and assistance to staff to assure the self-assessment is complete and accurate and that written reports are organized for review by the OSPR team. 

Responsibilities of Coordinating Committee

§                    Monitor progress of the groups/subcommittees;

§                    Develop editorial guidelines for the self-assessment report;

§                    Provide documents, data, and other materials needed;

§                    Organize and maintain a centralized file of pertinent self-assessment information and data;

§                    Report regularly to the agency chief executive officer on progress of the study;

§                    Review the draft reports of each group;

§                    Determine a date by which all final draft reports must be completed;

§                    Provide an agency-wide review of the final draft report;

§                    Complete final editing, printing, and distribution of the self-assessment study (at least 30 days prior to the on-site review);

§                    Assist with preparations for the on-site review;

§                    Assist with the development of a plan for follow-up to insure that an annual progress report is prepared and submitted to NAC and oversees the implementation of the on-site review recommendations.

Subcommittees: Members of each committee are the front line reviewers of the agency’s performance in meeting its mission and goals, outcomes, and particular challenges. They evaluate and rate their performance on the NAC standards format. Each applicable standard receives a rating and/or comments that will be reviewed and verified by the OSPR team. The amount of work assigned the different committee will vary. 

SUBCOMMITTEE ACTIVITIES:

§                    Review agency policies pertinent to their assignment.

§                    Conduct interviews, make observations, and verify outcomes.

§                    Submits reports according to established timelines.

§                    Meet regularly during of development of their report.

§                 Report progress to Coordinating Committee.

§                    Report on issues and questions raised by the self-assessment.

§                    Submit material that supports ratings.

§                    Assist editing and review of the final self-assessment report.

On-Site Peer Review Team

FUNCTION: Members of an on-site peer review team function as colleagues. The purpose is to produce a team report, which will be useful to the agency or school and to the NAC Board of Directors fordetermining an agencies accreditation status. The report should also be written with the next on-site visit in mind as repeated recommendation may result in non-accreditation.

Team Make-Up: The peer review team generally consists of two or more reviewers depending on the size of the organization and whether the review will be conducted concurrently with another accrediting body, on the complexity of the program and on the scope of the review. Reviewers are assigned primarily from professionals who come from out of state and have credentials and experience related to the services being reviewed.

Attention is given in selecting reviewers to avoid a real or perceived conflict of interest. Generally NAC’s executive director matches the reviewers with the specific programs of an agency and submits names to that agency’s chief executive officer six to eight weeks in advance of the on-site review. The chief executive officer is requested to notify the NAC executive director of any concern about the composition of the team. This includes past and present contacts with proposed team members that may strongly bias the team and preclude an objective review.

Role of OSPR Team Members: The contribution of the individual members on an OSPR team is critically important.   A brief review of the process, as a whole, will help to explain the role of a team member.

Stage One: Reviewers are to read and familiarize themselves with the agency’s self-assessment document as well as give careful attention to other supporting documentation provided such: as schedules, manuals, budgets, annual reports, policy manuals, previous site visit reports, etc.

Stage Two: During the on-site visit, the team’s charge is to make an independent analysis to determine whether the agency meets the NAC standards for accreditation. Each agency is to be evaluated in terms of its adherence to its mission and goals statement. Team members are to approach the task as a professional with specialized expertise and as a colleague in the field. Team members need to look for coherence between what is stated in the agency’s self-assessment document and what it does in practice. Each member is advised to approach the task humbly, keeping in mind that no one has all the answers.

Stage Three: Team members should have completed their individual reports by the end of the on-site visit. Each team member’s role is to assess their assigned sections objectively, critically, constructively, and accurately with balanced professionalism. The team’s focus must be on compliance with eligibility requirement and the NAC standards and policies.

Stage Four: The primary function of the OSPR report is to give agency staff, administration, and the governing board a critical analysis of all areas measured against the standards. The OSPR report is also the instrument that provides NAC’s Board a basis for deciding upon accreditation status. Members of the team complete a confidential recommendation about the level of accreditation to be sent to NAC’s executive director concerning accreditation. The Board’s deliberation procedure provides for the chair of the OSPR team and for the CEO of the reviewed agency to report orally to the NAC Board in an executive session to answer questions and share comments on the report and the on-site review visit via teleconference.

Preparation for an On-Site Peer Review

Dates and Arrangements: Dates for the on-site peer review are determined by the NAC’s executive director in consultation with the agency’s chief executive officer.

Hotel and Local Transportation: Arrangements for local transportation and hotel reservation and team meeting space is made by the host agency and communicated to the on-site review team in a timely manner either via e-mail or telephone communications.

Travel Arrangements: Team members make their own flight   arrangements. All incurred travel expenses are reimbursed by the host agency. Travel costs should be parsimonious.

Detailed Schedule: A section assignment/schedule is prepared well in advance by the NAC office in consultation with the agency allotted time for interviews with key administrators, representatives of the governing board, staff, clients/students, and representatives of appropriate external agencies and the reading of agency documents.

On-Site Work Room:  The Agency provides a work room for the on-site review team large enough to give the evaluators adequate private work space that is secure so confidential materials can be left safely throughout the review. In addition, the agency/school makes arrangements so as to provide for confidential interviews for each team member.

Team Member Preparation: The following ten (10) items are strongly recommended by NAC to assist team members in preparing for the visit:

§                    Review all materials such as the previous OSPR report if this is not the first review, the self-assessment document and the section-assignments schedule prior to on-site arrival.

§                    Consult with the team chair and the NAC office if you have questions after reviewing the information.

§                    Plan your schedule to be available for the entire evaluation, typically form Sunday afternoon to Wednesday noon.

§                    Prepare a general outline of your assigned areas including questions to be asked when you conduct interviews.

§                    List items you will want to verify from sources other than the self-assessment document or interviews.

§                    Evaluate the quality and thoroughness of the self-assessment document and be prepared to comment or questions at the team’s organizational meeting. Give particular attention to the important involvement of staff, clients, students and consumers.

§                    Share a list of planned interviews with staff, clients, etc. other members of the team.

§                    Make your own travel arrangements to the host state. Arrive in time to participate in the evaluation team’s organizational meeting.

§                    Keep receipts of expenses incurred during your on-site visit and submit to the host agency for reimbursement.

§                    The team organizational meeting is very important in setting the tone for the visit, clarifying assignments, and confirming the evaluation schedule. 

Pre-Visit Meeting of the On-Site Peer Review Team: In the afternoon or evening before the first day of the on-site review, the team will convene for an organizational meeting at their hotel. The purpose of the meeting is to coordinate the team’s activities during the visit: 

§        Charge team members with responsibility to evaluate compliance with the eligibility requirements, standards, and policies of NAC.

§        Make individual assignment for report writing that will assure completion of the draft before the end of the review. (The NAC office will provide the team members with a report template to insure uniformity of format, font and style of the written report.)

§        Schedule activities, conferences or interviews as needed. The agency is responsible for scheduling in-person or phone interviews with consumers and collateral agency representatives.

§        Format and discuss objectives for the exit conference.

§        Confirm that each team member has developed an individual schedule of evaluation activities.

§        Reinforce the importance of sharing information as well as maintaining confidential communications among team members.

§        Discuss the meals, transportation and reimbursement.

§        Provide an opportunity for questions and interaction on topics of concern following the organizational meeting.

Some institutions will prefer to host a reception/dinner evening previous to beginning of the on-site review. If this is the case, the event can be scheduled between 6:30 – 8:00 p.m. The purpose for such a meeting is to have team members meet with key administrators, members of the agency’s self-assessment coordinating committee, principal staff members, and consumer representatives on a less formal basis. This can also be accomplished at a brief early morning meeting the first day of on-site activities if a reception is not planned.

On-Site Review Activities: This section outlines a general schedule for the team’s activities during the on-site visit. The team chair may modify the schedule and team’s activities in accordance with unique features of the institution, local circumstances, and the composition of the team.

First Day Of Review on-site

Generally:     Arrive at agency by    8:30 a.m.  

             Return to Hotel at     3:30 p.m. 

Brief Meeting: Usually, the on-site review begins with a brief meeting to complete introductions of key staff. This is an opportunity for the CEO to make further introductions if needed and to speak to the purpose of the review. The team Chair will introduce the team members, indicate the areas they will be evaluating, and review the purpose of accreditation. There will be a brief overview of the procedures to be followed and the schedule. An orientation tour of the facility or campus may be scheduled.

Team Activities: The team will spend the remainder of the workday verifying the results of the self-assessment document and seeking answers to any questions they may have regarding the materials received prior to arriving. Follow the general schedule that has been developed and shared with the agency. Specifically the team will:

§                    Conduct interviews.

§                    Review organization documents, budges, procedure manuals and reports.

§                    Inspect facilities and equipment.

§                    Observe classes and operations.