Custom Development

Customized_Solutions

Customized Solutions

NOCTI offers customized assessment solutions to meet the unique needs of a school, district, college, state, or association. Whether being used for end-of-program accountability or as part of a certification program, NOCTI follows rigorous industry guidelines for standards and test development to ensure validity and reliability for every assessment. Working with your Subject Matter Experts, NOCTI facilitates a development workshop based on the eight steps listed below. Workshops can be facilitated onsite or via web meeting.

Our Development Process

NOCTI has earned full accreditation from the International Certification Accreditation Council (ICAC). This accreditation ensures a neutral third party has properly evaluated the organization and signifies that NOCTI’s credentials follow the best international industry practices and standards outlined in ISO 17024.

1
Develop Standards & Competencies

Standards clearly outline what students should know and be able to do when they complete a program. Subject Matter Expert (SME) involvement ensures NOCTI industry-based credentials and those we develop for other organizations measure what matters. SME teams review existing national, industry, and state standards to determine critical content. Specific standards are identified to serve as the foundation for the development process. When standards do not exist, NOCTI facilitates a Job and Task Analysis process to define the content.

2
Develop Blueprint

The Blueprint serves as the roadmap for the development process. To create a Blueprint, SME teams rate the critical content to determine importance in the industry and distribution of test items across the competencies. This important step is necessary to assist in creating a valid and reliable credentialing assessment.

3
Develop Items/Jobs

Using the established blueprint as a foundation, the SMEs work with a NOCTI facilitator to select and/or create items for inclusion on the knowledge-based assessment. Some competencies are more appropriately measured through a skill-based, hands-on assessment which contains a selection of jobs within an occupation. During performance job development, SMEs are charged with developing student, evaluator, and job set-up instructions, a list of materials, supplies, and equipment as well as a comprehensive scoring rubric for evaluating student work.

4
Pilot Administration

Pilot testing is a critical phase in the development process and represents an actual assessment administration with a pilot test group. The group must be representative of the population that will be tested including appropriate geographic, ethnic, and gender demographics. The valuable data gathered from pilot testing is used to ensure every test element is performing according to industry guidelines.

5
Analyze Pilot Results

NOCTI thoroughly reviews the pilot data which includes item-level information such as difficulty, distractor analysis, and discrimination between high and low scorers. Test-level information is also reviewed and includes mean, maximum and minimum scores, several measures of variance (e.g., standard deviation, standard error of measurement, skew, kurtosis) as well as the reliability of the assessment. A report is provided to the SME team for additional review and feedback.

6
Establish Cut Score

Criterion-referenced cut scores are established by a qualified team of SMEs. A cut score is the point on a test at which a student is expected to perform to be considered “minimally competent” or ready for an entry position in a technical field. Cut scores are established for both the knowledge-based and skill-based components.

7
Review & Incorporate Edits

Edits are based on the results of pilot administration, item analysis, and cut score activities. Approved edits are incorporated into a final version of the credentialing assessment.

8
Finalize

Finalization of the credentialing assessment includes integrated academic alignment, conversion of the knowledge-based test to Text-to-Speech format, and release for general usage.