Background

Assessment plays a pivotal role in all educational programs, particularly in the realm of medical education. This process is complex, involving a diverse array of tools and methods employed by assessors to evaluate, measure, and document the academic readiness and skill acquisition of medical graduates. In India, medical education has traditionally adhered to conventional assessment methods, relying on a limited number of structured clinical cases. This approach can be inadequate and subjective (1, 2), potentially negatively impacting students' learning abilities and, by extension, public health outcomes. Therefore, there is a pressing need for a robust and revised evaluation system.

The assessment blueprint is a meticulously crafted document designed prospectively to comprehensively compile all of the educational content and its anticipated outcomes. Its purpose is to ensure the assessment process encompasses all aspects of the curriculum and their respective domains. Presently, various assessment methods are employed, including short-case and long-case assessments, logbooks, audits, simulated patient surgeries and procedures, video assessments, simulators, and standardized patients (3). Using multiple assessment methods is imperative to cover the most crucial aspects of clinical competency. It has been recognized that relying solely on a single method would be insufficient to help students grasp and acquire skill-based knowledge.

The blueprint represents a highly efficient tool that seamlessly integrates subject content, learning objectives (including skills, knowledge, and attitude), tools, and methods necessary for assessing the content. Currently, it stands as the gold standard for evaluation, facilitating fair assessments by offering clear guidelines to students (4).

This study was conducted to develop an assessment blueprint for clinical skill competencies among undergraduates in the Department of Dermatology, Venereology, and Leprosy (DVL). Additionally, it seeks to analyse faculty perceptions regarding this blueprint.

Objectives

The primary objective of the present study was to develop a blueprint for assessing clinical skill competencies within the DVL department for medical students enrolled at the Apollo Institute of Medical Sciences and Research (AIMSR) in Hyderabad, India. This study aims to enhance medical education for undergraduates through the creation of standardized assessment methods and the validation of these methods by soliciting feedback from subject experts.

Methods

The faculty members from the DVL department at 3 medical schools, namely AIMSR, Osmania Medical College, and Gandhi Medical College, along with the curriculum coordinator from AIMSR, participated in the development of the blueprint. This cross-sectional study was done over 1 year. Since 2019, nearly all Indian medical schools have initiated the implementation of a competency-based curriculum as proposed by India's National Medical Council (NMC). The curriculum comprises 3 phases: pre-clinical, para-clinical, and clinical phases, with dermatology (DVL) falling under the phase 3 clinical subject category.

As part of the Early Clinical Exposure (ECE) module in Competency-Based Medical Education (CBME), medical students during their course undergo 45 days of clinical postings in the DVL department, with 15 days each allocated to phase II, part 1 of phase III, and part 2 of phase III. Table 1 provides the details of the competency content, while a step-by-step description of how the blueprint was formulated can be found in Figure 1.

Blueprint Planning: This blueprint was developed to evaluate the clinical competencies of medical students during clinical postings in the DVL department
(from phase 2 to phase 3) at AIMSR, Hyderabad.

Sensitization Workshop

The DVL faculty was sensitized by the Institutional Medical Education Unit on Competency-based medical education and the significance of blueprinting in medical education, including both teaching and assessments. The blueprint includes all the recommended competencies, along with well-defined learning objectives, teaching and learning methods, and assessment details.

A total of 10 subject experts from 3 medical colleges (3 of whom are heads of the departments, 1 is a professor, 1 is an associate professor, 4 are assistant professors, and 1 is a senior resident) participated in the development of the blueprint in coordination with the curriculum coordinator. The process is explained in 3 steps as follows.

Step 1: All competencies covering the syllabus included in phases II and III of MBBS were reviewed
by the subject experts. First, the learning objectives were formulated, followed by the design of teaching/learning (T/L) methods and assessment tools for each competency.

Figure 1. Flowchart to explain the methodology

Table 1. Final Blueprint for Dermatology, Venereology, and Leprosy for Clinical Assessment for Medical Undergraduates

DOAP: Demonstrate, observe, assist perform; OSCE: Objective structured clinical examination; T/L: Teaching/Learning; SNAPPS: summarize, narrow, analyze, probe, plan, and select

Step 2: The weightage for each competency was determined based on 2 parameters: first, the impact of the topic on health (I), and second, its frequency of occurrence (F).

Perceived impact (I) of the topics on health were categorized under one of the following:

1) Non-urgent, 2) serious but not life-threatening, and 3) life-threatening emergency.

The frequency (F) of occurrence of a particular disease was categorized under the following:

1) Rarely seen, 2) relatively common, and 3)
very common.

Based on the I × F, the topics were classified as “must know” with I × F 6-9, “should know” with I×F of 3-4, and "good to know" with I × F of 1-2.

The weightage coefficient for each competency was calculated as I×F/T. The sum of I and F is labelled as T. The number of questions for each competency was calculated by multiplying the weightage coefficient
by the total number of items in the assessment. The total marks for each topic were calculated by multiplying the corresponding value of weightage (W) by the total marks assigned to the students according to the blueprint.

Step 3: Contents of Blueprint: Each competency was described in detail to ensure that the medical student is adequately trained and assessed to meet the minimum required level. Table 2 provides details about the division of competencies included in the blueprint. The assessments were divided into objective structured clinical examination (OSCE) and case-based discussion. Furthermore, OSCE was subdivided into history taking, physical examination, clinical procedure, clinical reasoning, and communication skills.

Step 4: Feedback: In this step, a faculty feedback questionnaire was prepared to obtain the opinions and expertise of the faculty regarding the assessment blueprint. The questionnaire underwent peer review and was further validated by the medical education unit's team. The feedback form was constructed with validated standard questions to gather a valid and accurate perception of the faculty's views, which would assist in evaluating the significant role of the blueprint and its necessity in assessing the clinical skills of medical students. Data analysis was conducted using SPSS version 20.0.

The faculty's perceptions were graded using a
5-point Likert scale. To measure the reliability of the faculty feedback, the Cronbach's alpha test was used. According to SPSS, if the score is α < 0.5, it is considered unacceptable; α ≥ 0.5 is poor; α ≥ 0.6 is questionable; α ≥ 0.7 is acceptable; α ≥ 0.8 is good, and α ≥ 0.9 is excellent (5). The obtained value was α = 0.874, which is considered good based on the rule of thumb for results.

Results

Table 3 provides details about the number of competencies, including all those suggested by the NMC as per the revised curriculum. The assessment pattern includes OSCE and multiple-choice questions (MCQs), covering phases II and III (part 1 and part 2). These competencies can be taught and assessed in either one phase or multiple phases. The table is enriched with comprehensive information regarding all the included competencies, their alignment with T/L methods, the phase of assessment, and the suitable assessment format. Additionally, the table offers specifics about the allotted marks for each competency, which are determined based on the impact score and the frequency of application of specified competencies in clinical practices.

Table 4 presents information about the curriculum contents, which are the product of IxF and categorized as knows, including Must know (MK), Should Know (SK) and Good to Know (GK). In the present study, 50% of the competencies fall under must know, 33.3% fall under should know, and 16.7% fall under good to know. This table can serve as a template for assessors, assisting them in selecting the competencies for assessment.

The faculty feedback form was employed to gather the opinions of the subject experts on the designed blueprint, as depicted in Figure 2. The results of the feedback form are presented in terms of percentages. Among the findings, 70% of the faculty expressed the view that blueprint assessment enhances the validity of assessments, and they appreciated the allocation of proper weightage to topics related to public health importance. Additionally, 60% of the faculty participants conveyed that blueprinting contributes to fairness in assessment, as it ensures the equitable distribution of questions across all significant topics. They also noted that the blueprint serves as a valuable guide for constructing clinical exams.

Discussion

A blueprint serves as a map that precisely defines the content of assessments. It has been proven to be advantageous in the successful implementation of the curriculum and assessments.

Table 2. Best Suitable Method for Assessing Clinical Skills in DVL

Table 3. Best Suitable Method for Assessing Clinical Skills in DVL

Table 4. Categorization of Competencies with Phase-Wise Teaching

It aids in assigning appropriate weightage to all competencies, ensuring comprehensive coverage of the syllabus for individual subjects and phases of learning. The primary objective of the present study was to develop a blueprint for the DVL subject to assess second and third-year medical undergraduate students. The study aimed to include both topic-based and competency-based questions with credible weightage, covering all subject topics in accordance with NMC guidelines.

The developed blueprint was then presented to subject experts, and their valuable feedback on the blueprint design was obtained, as detailed in the results section.

Most previous literature on blueprint development and implementation has highlighted the advantages of blueprints in teaching and assessing medical subjects, which aligns with the outcomes of our study. A study conducted by Sunita et al in 2015 evaluating blueprints and their implementation showed that approximately 89% of the faculty believed that blueprints contribute to fair evaluation and assist in aligning assessment objectives. They also share the same opinion as our study that blueprinting should be an integral part of any type of assessment, whether formative, summative, or final assessment (6).

Figure 2. Measures of the faculty feedback form

The blueprint serves as a guide for planning and constructing questions from eligible chapters, ensuring that the required weightage is given to various competencies within a specific subject. It is a clear, fair, explicit, and transparent assessment tool that guides students on the right path of learning and acquiring the desired knowledge, which can further enhance their clinical skills in terms of observation, analysis, and implementation (6-8). The use of the blueprint makes assessments more meaningful by directing students toward evidence-based learning.

Furthermore, implementing this blueprint in the second and third academic years of medical undergraduates and gathering feedback from the students on its effectiveness can help establish a high-quality assessment tool (blueprint) for the DVL subject.

Limitations: There are certain limitations in our study. First, the validation of the blueprint is subjective, which may introduce some degree of bias. Additionally, we did not collect student feedback, which could have provided valuable insights into their perception of the assessment process.

Conclusion

Using a blueprint in assessments both for summative and formative purposes, enhances the quality, validity, and reliability of the assessment process. It is widely accepted and contributes to a balanced approach to

learning and implementing concepts for medical students. The blueprint is a valuable tool that offers flexibility in selecting the I (impact) and F (frequency) parameters, and these choices may vary among assessors.