Strides in Development of Medical Education

Document Type : Original Article

Authors

1 MD, Assistant Professor, Department of Dermatology, Venereology & Leprosy, School of Apollo Institute of Medical Sciences & Research, University of KNRUHS, Hyderabad, Telangana, India

2 MD, Senior Resident, Department of Dermatology, Venereology & Leprosy, Apollo Institute of Medical Sciences & Research, Hyderabad, Telangana, India

3 MD, Associate Professor, Department of Dermatology, Venereology& Leprosy, Osmania Medical College, Hyderabad, Telangana, India

4 MD, Professor, Department of Dermatology, Venereology, & Leprosy, Gandhi Medical College, Secunderabad, Telangana, India

5 MD, Professor & HOD, Department of Dermatology, Venereology & Leprosy, Apollo Institute of Medical Sciences & Research, Hyderabad, Telangana, India

6 MD, Principal, Professor & HOD, Department of Biochemistry, Mamta Institute of Medical Sciences, Bachupally, Telangana, India

7 PhD, Associate Professor, Department of Biochemistry, Dr. Patnam Mahender reddy Institute of Medical Sciences, Chevella, Telangana, India

Abstract

Background: The assessment blueprint is a calibrated dossier, which is a meticulously designed document that proactively compiles all educational content and its anticipated outcomes comprehensively. This blueprint is a powerful tool that integrates subject matter, learning objectives (including skills, knowledge, and attitude), tools, and methods to assess the content.
Objectives: The primary goal of this study was to develop a blueprint for clinical skill competencies within the Department of Dermatology, Venereology, and Leprosy (DVL) for medical undergraduates at the Apollo Institute of Medical Sciences and Research (AIMSR) in Hyderabad, India.
Methods: The faculty of DVL and the curriculum coordinator developed a blueprint for the mentioned department at AIMSR in collaboration with Gandhi and Osmania Medical College. Faculty members provided feedback on the blueprint.
Results: The feedback results are presented in terms of percentages. Notably, 70% of the faculty members expressed the view that blueprint assessments enhance the validity of assessments and appropriately assign weightage to topics of public health significance.
Conclusion: Incorporating blueprints into both summative and formative assessments enhances quality, validity, and reliability. It represents one of the most acceptable approaches for medical students to balance learning and practical application of concepts.

Highlights

Harshita Reddy. B: (Google Scholar) (PubMed)   

Prattilohi. S. Rajlaxmi: (Google Scholar) (PubMed)   

Sudha Vani Damarla: (Google Scholar) (PubMed)   

Sudharani Chintagunta: (Google Scholar) (PubMed)   

Manmohan Gavvala: (Google Scholar) (PubMed)   

Neetha Kundoor: (Google Scholar) (PubMed

Srilatha Bashetti: (Google Scholar) (PubMed

Keywords

Background

Assessment plays a pivotal role in all educational programs, particularly in the realm of medical education. This process is complex, involving a diverse array of tools and methods employed by assessors to evaluate, measure, and document the academic readiness and skill acquisition of medical graduates. In India, medical education has traditionally adhered to conventional assessment methods, relying on a limited number of structured clinical cases. This approach can be inadequate and subjective (1, 2), potentially negatively impacting students' learning abilities and, by extension, public health outcomes. Therefore, there is a pressing need for a robust and revised evaluation system.

The assessment blueprint is a meticulously crafted document designed prospectively to comprehensively compile all of the educational content and its anticipated outcomes. Its purpose is to ensure the assessment process encompasses all aspects of the curriculum and their respective domains. Presently, various assessment methods are employed, including short-case and long-case assessments, logbooks, audits, simulated patient surgeries and procedures, video assessments, simulators, and standardized patients (3). Using multiple assessment methods is imperative to cover the most crucial aspects of clinical competency. It has been recognized that relying solely on a single method would be insufficient to help students grasp and acquire skill-based knowledge.

The blueprint represents a highly efficient tool that seamlessly integrates subject content, learning objectives (including skills, knowledge, and attitude), tools, and methods necessary for assessing the content. Currently, it stands as the gold standard for evaluation, facilitating fair assessments by offering clear guidelines to students (4).

This study was conducted to develop an assessment blueprint for clinical skill competencies among undergraduates in the Department of Dermatology, Venereology, and Leprosy (DVL). Additionally, it seeks to analyse faculty perceptions regarding this blueprint.

Objectives

The primary objective of the present study was to develop a blueprint for assessing clinical skill competencies within the DVL department for medical students enrolled at the Apollo Institute of Medical Sciences and Research (AIMSR) in Hyderabad, India. This study aims to enhance medical education for undergraduates through the creation of standardized assessment methods and the validation of these methods by soliciting feedback from subject experts.

Methods

The faculty members from the DVL department at 3 medical schools, namely AIMSR, Osmania Medical College, and Gandhi Medical College, along with the curriculum coordinator from AIMSR, participated in the development of the blueprint. This cross-sectional study was done over 1 year. Since 2019, nearly all Indian medical schools have initiated the implementation of a competency-based curriculum as proposed by India's National Medical Council (NMC). The curriculum comprises 3 phases: pre-clinical, para-clinical, and clinical phases, with dermatology (DVL) falling under the phase 3 clinical subject category.

As part of the Early Clinical Exposure (ECE) module in Competency-Based Medical Education (CBME), medical students during their course undergo 45 days of clinical postings in the DVL department, with 15 days each allocated to phase II, part 1 of phase III, and part 2 of phase III. Table 1 provides the details of the competency content, while a step-by-step description of how the blueprint was formulated can be found in Figure 1.

Blueprint Planning: This blueprint was developed to evaluate the clinical competencies of medical students during clinical postings in the DVL department
(from phase 2 to phase 3) at AIMSR, Hyderabad.

Sensitization Workshop

The DVL faculty was sensitized by the Institutional Medical Education Unit on Competency-based medical education and the significance of blueprinting in medical education, including both teaching and assessments. The blueprint includes all the recommended competencies, along with well-defined learning objectives, teaching and learning methods, and assessment details.

A total of 10 subject experts from 3 medical colleges (3 of whom are heads of the departments, 1 is a professor, 1 is an associate professor, 4 are assistant professors, and 1 is a senior resident) participated in the development of the blueprint in coordination with the curriculum coordinator. The process is explained in 3 steps as follows.

Step 1: All competencies covering the syllabus included in phases II and III of MBBS were reviewed
by the subject experts. First, the learning objectives were formulated, followed by the design of teaching/learning (T/L) methods and assessment tools for each competency.

Figure 1. Flowchart to explain the methodology

 

Table 1. Final Blueprint for Dermatology, Venereology, and Leprosy for Clinical Assessment for Medical Undergraduates

Competency - TOPIC

T/L - method

Impact (I)

Frequency of occurrence (F)

I × F

W = IxF/T

Total marks (100)

No. of questions = W x 45

Rounding off (culminate closer to the next value)

OSCE

MCQ (1Mark)

Phase 2

Phase 3 (part1)

Phase 3 (part 2)

Identify and grade various types of acne

Bedside clinic

1

3

3

0.04

4

1.8

2

1 (3M)

1

ü

 

ü

Identify and differentiate vitiligo from other causes of hypopigmented lesions

Bedside clinic, SNAPPS

2

2

4

0.05

5

2.25

2

1 (4M)

1

ü

ü

 

Identify and distinguish psoriatic lesions from other causes

Bedside clinic, SNAPPS

2

2

4

0.05

5

2.25

2

1 (4M)

1

ü

ü

ü

Demonstrate the Grattage test

DOAP

2

3

6

0.07

7

3.15

3

1 (5M)

2

ü

ü

ü

Identify and distinguish lichen planus lesions from other causes

Bedside clinic, SNAPPS

1

2

2

0.02

2

0.9

2

0

2

 

 

ü

Identify and differentiate scabies from other lesions in adults and children

Bedside clinic, SNAPPS

2

3

6

0.07

7

3.15

3

1 (5M)

2

ü

ü

 

Identify and differentiate pediculosis from other skin lesions in adults and children

Bedside clinic

1

2

2

0.02

2

0.9

1

1 (2M)

0

ü

ü

 

Identify candida species in fungal scrapings and KOH mount

DOAP

1

3

3

0.04

4

1.8

2

2 (2M)

0

ü

 

 

Identify and distinguish herpes simplex and herpes labialis from other skin lesions

Bedside clinic

2

3

6

0.07

7

3.15

3

1 (5M)

2

 

ü

 

Identify and distinguish herpes zoster and varicella from other skin lesions

Bedside clinic

2

3

6

0.07

7

3.15

3

1 (5M)

2

ü

ü

 

Identify and distinguish viral warts from other skin lesions

Bedside clinic, DOAP

2

3

6

0.07

7

3.15

3

1 (5M)

2

ü

 

ü

Identify and distinguish molluscum contagiosum from other skin lesions

Bedside clinic, DOAP

1

3

3

0.04

4

1.8

2

1 (3M)

1

ü

 

ü

Enumerate the indications, describe the procedure, and perform a Tzanck smear

DOAP

1

2

2

0.02

2

0.9

1

1 (2M)

0

ü

 

ü

Demonstrate and classify based on the clinical features of leprosy, including an appropriate neurologic examination

Bedside clinic, DOAP

3

2

6

0.07

7

3.15

3

1 (5M)

2

ü

ü

ü

Enumerate the indications and observe the performance of a slit skin smear in patients with leprosy

Bedside clinic, DOAP

3

2

6

0.07

7

3.15

3

1 (5M)

2

 

 

ü

Identify and classify syphilis based on the presentation and clinical manifestations

Bedside clinic

2

2

4

0.05

5

2.25

2

1 (4M)

1

ü

ü

 

Identify spirochete in a dark ground microscopy/VDRL

DOAP

3

2

6

0.07

7

3.15

3

1 (5M)

2

ü

ü

 

Counsel in a non-judgemental and empathetic manner to patients on the prevention of sexually transmitted disease

Role-play, DOAP

3

3

9

0.11

11

4.95

5

3 (3M)

2

ü

ü

ü

DOAP: Demonstrate, observe, assist perform; OSCE: Objective structured clinical examination; T/L: Teaching/Learning; SNAPPS: summarize, narrow, analyze, probe, plan, and select

 

Step 2: The weightage for each competency was determined based on 2 parameters: first, the impact of the topic on health (I), and second, its frequency of occurrence (F).

Perceived impact (I) of the topics on health were categorized under one of the following:

1) Non-urgent, 2) serious but not life-threatening, and 3) life-threatening emergency.

The frequency (F) of occurrence of a particular disease was categorized under the following:

1) Rarely seen, 2) relatively common, and 3)
very common.

Based on the I × F, the topics were classified as “must know” with I × F 6-9, “should know” with I×F of 3-4, and "good to know" with I × F of 1-2.

The weightage coefficient for each competency was calculated as I×F/T. The sum of I and F is labelled as T. The number of questions for each competency was calculated by multiplying the weightage coefficient
by the total number of items in the assessment. The total marks for each topic were calculated by multiplying the corresponding value of weightage (W) by the total marks assigned to the students according to the blueprint.

Step 3: Contents of Blueprint: Each competency was described in detail to ensure that the medical student is adequately trained and assessed to meet the minimum required level. Table 2 provides details about the division of competencies included in the blueprint. The assessments were divided into objective structured clinical examination (OSCE) and case-based discussion. Furthermore, OSCE was subdivided into history taking, physical examination, clinical procedure, clinical reasoning, and communication skills.

Step 4: Feedback: In this step, a faculty feedback questionnaire was prepared to obtain the opinions and expertise of the faculty regarding the assessment blueprint. The questionnaire underwent peer review and was further validated by the medical education unit's team. The feedback form was constructed with validated standard questions to gather a valid and accurate perception of the faculty's views, which would assist in evaluating the significant role of the blueprint and its necessity in assessing the clinical skills of medical students. Data analysis was conducted using SPSS version 20.0.

The faculty's perceptions were graded using a
5-point Likert scale. To measure the reliability of the faculty feedback, the Cronbach's alpha test was used. According to SPSS, if the score is α < 0.5, it is considered unacceptable; α ≥ 0.5 is poor; α ≥ 0.6 is questionable; α ≥ 0.7 is acceptable; α ≥ 0.8 is good, and α ≥ 0.9 is excellent (5). The obtained value was α = 0.874, which is considered good based on the rule of thumb for results.

Results

Table 3 provides details about the number of competencies, including all those suggested by the NMC as per the revised curriculum. The assessment pattern includes OSCE and multiple-choice questions (MCQs), covering phases II and III (part 1 and part 2). These competencies can be taught and assessed in either one phase or multiple phases. The table is enriched with comprehensive information regarding all the included competencies, their alignment with T/L methods, the phase of assessment, and the suitable assessment format. Additionally, the table offers specifics about the allotted marks for each competency, which are determined based on the impact score and the frequency of application of specified competencies in clinical practices.

Table 4 presents information about the curriculum contents, which are the product of IxF and categorized as knows, including Must know (MK), Should Know (SK) and Good to Know (GK). In the present study, 50% of the competencies fall under must know, 33.3% fall under should know, and 16.7% fall under good to know. This table can serve as a template for assessors, assisting them in selecting the competencies for assessment.

The faculty feedback form was employed to gather the opinions of the subject experts on the designed blueprint, as depicted in Figure 2. The results of the feedback form are presented in terms of percentages. Among the findings, 70% of the faculty expressed the view that blueprint assessment enhances the validity of assessments, and they appreciated the allocation of proper weightage to topics related to public health importance. Additionally, 60% of the faculty participants conveyed that blueprinting contributes to fairness in assessment, as it ensures the equitable distribution of questions across all significant topics. They also noted that the blueprint serves as a valuable guide for constructing clinical exams.

Discussion

A blueprint serves as a map that precisely defines the content of assessments. It has been proven to be advantageous in the successful implementation of the curriculum and assessments.

 

Table 2. Best Suitable Method for Assessing Clinical Skills in DVL

Competency number

Competency topic

History taking

Physical examination

Clinical procedure

Clinical reasoning

Communication skills

MCQs

DR 1.2

Identify and grade various types of acne

 

ü

 

ü

 

ü

DR 2.1

Identify and differentiate vitiligo from other causes of hypopigmented lesions

ü

ü

 

ü

 

ü

DR 3.1

Identify and distinguish psoriatic lesions from other causes

ü

ü

ü

 

 

ü

DR 3.2

Demonstrate the Grattage test

 

 

ü

 

 

ü

DR 4.1

Identify and distinguish lichen planus lesions from other causes

ü

ü

 

ü

 

ü

DR 5.2

Identify and differentiate scabies from other lesions in adults and children

ü

ü

 

ü

 

ü

DR 6.2

Identify and differentiate pediculosis from other skin lesions in adults and children

 

ü

 

ü

 

 

DR 7.2

Identify candida species in fungal scrapings and KOH mount

 

 

ü

 

 

 

DR 8.2

Identify and distinguish herpes simplex and herpes labialis from other skin lesions

ü

ü

ü

 

 

ü

DR 8.3

Identify and distinguish herpes zoster and varicella from other skin lesions

ü

ü

ü

 

 

ü

DR 8.4

Identify and distinguish viral warts from other skin lesions

 

ü

ü

 

 

ü

DR 8.5

Identify and distinguish molluscum contagiosum from other skin lesions

 

ü

ü

 

 

ü

DR 8.6

Enumerate the indications, describe the procedure, and perform a Tzanck smear

 

 

ü

 

 

 

DR 9.2

Demonstrate and classify based on the clinical features of leprosy, including an appropriate neurologic examination

 

ü

ü

ü

 

ü

DR 9.3

Enumerate the indications and observe the performance of a slit skin smear in patients with leprosy

 

 

ü

 

 

ü

DR 10.1

Identify and classify syphilis based on the presentation and clinical manifestations

ü

ü

 

ü

 

ü

DR 10. 2

Identify spirochete in a dark ground microscopy/VDRL

 

 

 

ü

 

ü

DR 10.5

Counsel in a non-judgemental and empathetic manner to patients on the prevention of sexually transmitted disease

 

 

 

ü

ü

ü

 

Table 3. Best Suitable Method for Assessing Clinical Skills in DVL

Competency number

Competency topic

OSCE (number in the bracket represents marks)

MCQs (1 mark each)

Total marks (100)

1.2

Identify and grade various types of acne

1 (3M)

1

4

2.1

Identify and differentiate vitiligo from other
causes of hypopigmented lesions

1 (4M)

1

5

3.1

Identify and distinguish psoriatic lesions from other causes

1 (4M)

1

5

3.2

Demonstrate the Grattage test

1 (5M)

2

7

4.1

Identify and distinguish lichen planus lesions from other causes

0

2

2

5.2

Identify and differentiate scabies from other lesions
in adults and children

1 (5M)

2

7

6.2

Identify and differentiate pediculosis from other skin
lesions in adults and children

1 (2M)

0

2

7.2

Identify candida species in fungal scrapings and KOH mount

2 (2M)

0

4

8.2

Identify and distinguish herpes simplex and herpes
labialis from other skin lesions

1 (5M)

2

7

8.3

Identify and distinguish herpes zoster and varicella
from other skin lesions

1 (5M)

2

7

8.4

 Identify and distinguish viral warts from other skin lesions

1 (5M)

2

7

8.5

Identify and distinguish molluscum contagiosum from
other skin lesions

1 (3M)

1

4

8.6

Enumerate the indications, describe the procedure,
and perform a Tzanck smear

1 (2M)

0

2

9.2

Demonstrate and classify based on the clinical features of
leprosy, including an appropriate neurologic examination

1 (5M)

2

7

9.3

Enumerate the indications and observe the performance
of a slit skin smear in patients with leprosy

1 (5M)

2

7

10.1

Identify and classify syphilis based on the presentation
and clinical manifestations

1 (4M)

1

5

10.2

Identify spirochete in a dark ground microscopy/VDRL

1 (5M)

2

7

10.5

Counsel in a non-judgemental and empathetic manner to
patients on the prevention of sexually transmitted disease

3 (3M)

2

11

 

Table 4. Categorization of Competencies with Phase-Wise Teaching

I x F

Category

%

Phases

Phase 2

Phase 3 (part 1)

Phase 3 (part 2)

6 - 9

Must know

50

7

7

5

3 - 4

Should know

33.3

6

2

3

1 - 2

Good to know

16.7

2

1

2

Total

 

100

 

 

 

 

It aids in assigning appropriate weightage to all competencies, ensuring comprehensive coverage of the syllabus for individual subjects and phases of learning. The primary objective of the present study was to develop a blueprint for the DVL subject to assess second and third-year medical undergraduate students. The study aimed to include both topic-based and competency-based questions with credible weightage, covering all subject topics in accordance with NMC guidelines.

The developed blueprint was then presented to subject experts, and their valuable feedback on the blueprint design was obtained, as detailed in the results section.

Most previous literature on blueprint development and implementation has highlighted the advantages of blueprints in teaching and assessing medical subjects, which aligns with the outcomes of our study. A study conducted by Sunita et al in 2015 evaluating blueprints and their implementation showed that approximately 89% of the faculty believed that blueprints contribute to fair evaluation and assist in aligning assessment objectives. They also share the same opinion as our study that blueprinting should be an integral part of any type of assessment, whether formative, summative, or final assessment (6).

Figure 2. Measures of the faculty feedback form

 

The blueprint serves as a guide for planning and constructing questions from eligible chapters, ensuring that the required weightage is given to various competencies within a specific subject. It is a clear, fair, explicit, and transparent assessment tool that guides students on the right path of learning and acquiring the desired knowledge, which can further enhance their clinical skills in terms of observation, analysis, and implementation (6-8). The use of the blueprint makes assessments more meaningful by directing students toward evidence-based learning.

Furthermore, implementing this blueprint in the second and third academic years of medical undergraduates and gathering feedback from the students on its effectiveness can help establish a high-quality assessment tool (blueprint) for the DVL subject.

Limitations: There are certain limitations in our study. First, the validation of the blueprint is subjective, which may introduce some degree of bias. Additionally, we did not collect student feedback, which could have provided valuable insights into their perception of the assessment process.

Conclusion

Using a blueprint in assessments both for summative and formative purposes, enhances the quality, validity, and reliability of the assessment process. It is widely accepted and contributes to a balanced approach to

learning and implementing concepts for medical students. The blueprint is a valuable tool that offers flexibility in selecting the I (impact) and F (frequency) parameters, and these choices may vary among assessors.

  1. Gujarathi AP, Dhakne-Palwe S, Patil RN, Borade-Gedam P, Mahale MS, Gosavi SV, et al. Preparation of blueprint for formative theory assessment of undergraduate medical students in community medicine. MVP Journal of Medical Sciences. 2015; 2(2): 100-3. doi:18311/mvpjms/2015/v2/i2/782.
  2. Medical Council of India. Competency Based Assessment Module for Undergraduate Medical Education. [cited 2019 Feb 15]. Available from: https://wwwmciindiaorg/CMS/information-desk/for-colleges/ug-curriculum.
  3. Kundoor N, Hussain F, Lingam S, Reddy VN, Arcot R, Bondugula H, et al. Developing and Validating an Assessment Blueprint for Clinical Skill Competencies in General Surgery for the Medical Undergraduates. Strides Dev Med Educ. 2023 February; 20(1): 8-15. doi: 22062/sdme.2023.198234.1156.
  4. Adkoli BV, Deepak KK. Blueprinting in assessment. In: Singh T, Anshu, Editors. Principles of assessment in medical education. New Delhi: Jaypee Brothers Medical Pub; 2012:205-13.
  5. Tavakol M, Dennick R. Making Sense of Cronbach’s Alpha. Int J Med Educ. 2011; 2: 53-5. doi: 5116/ijme.4dfb.8dfd. [PMID: 28029643] [PMCID: PMC4205511]
  6. Patil SY, Gosavi M, Bannur HB, Ratnakar Blueprinting in assessment: A tool to increase the validity of undergraduate written examinations in pathology. Int J Appl Basic Med Res. 2015 Aug; 5 (Suppl 1): S76-9. doi: 10.4103/2229-516X.162286. [PMID: 26380218] [PMCID: PMC4552073]
  7. Adkoli B. Attributes of a good question paper. In: Sood R, editor. Assessment in Medical Education: Trends and Tools. New Delhi: KL Wig Center for Medical Education and Technology, AIIMS; 1995.
  8. Epstein RM. Blueprinting in medical education. N Engl J Med. 2007 Jan 25;356(4):387-96. doi: 1056/NEJMra054784. [PMID: 17251535]