Introduction
Evaluation of clinical skills of Speech-Language Pathology (SLP) students is an important duty for related departments, and it is the case for their students of health advocates because of the community needs for their services [1]. Upgrade of the quality of clinical skills can lead to improvement of health services [2]. Clinical exams are very important duties for related departments in universities with the aim of increasing the quality of clinical education. Results of good exams can determine weaknesses and strengths of clinical education and lead to appropriate reformation of education and increase students’ motivation. Giving feedback to students after exams can help them to learn better [3, 4, 5]. Evaluating students’ clinical skills during working with real patients can help them to reach desired skills. Mastery of these skills is necessary for the promotion of community health [6].
Traditional methods for evaluation of students’ clinical skills are more subjective or according to the general impression of university professors [7, 8, 9]. Different studies have shown that continuous evaluations can have undeniable effects on the development of clinical education; therefore, more precise methods have been emphasized [10]. In response to more community demand about responsibilities of medical treatment staff, it seems necessary to select more real evaluation methods for clinical skills of students working in educational hospital centers [11].
Different methods proposed and used for evaluation of students’ clinical skills are portfolios, Objective Structured Clinical Examination (OSCE), Mini-Clinical Evaluation Exercise (Mini-CEX), and Direct Observation of Procedural Skills (DOPS) [12].
Direct Observation of Procedural Skills (DOPS) is a test, which is used for the assessment of procedural skills and increases students’ awareness of their educational needs. DOPS requires direct observation of the performance of any student during real work on a predetermined procedure or clinical skill in a real environment, simultaneously evaluated by an academy staff using a checklist [13]. During the DOPS test, the examiner’s attention concentrates on the important points about the related skill. By repetition of the DOPS exam, the progress of students can be tracked by comparison of the results. This exam can facilitate better learning of clinical skills, because of the feedback in the exam, instead of general comments, and because of the objective behaviors of students in a real clinical environment [14, 15].
Speech therapy students who study in this field must successfully pass different clinical exams during their educational period. These students need to have good clinical skills for the assessment and treatment of different clients with communication disorders [16, 17, 18]. Better evaluation methods can help them to learn better.
There are different speech, language, and swallowing disorders, which speech therapists work with and try to eliminate or alleviate their problems. One of these disorders is stuttering. Stuttering is non-fluent speech with different characteristics: aberrations in the fluency of speech, blocks, repetitions, and prolongations, with some other symptoms, such as eye blinking, weak eye contact, and abnormal movements in hands, head, or other parts of the body [18, 19]. 
Stuttering begins usually in 2-6 years old children, and 4-5 percent of school-aged children may be stutterers [20]. One percent of adults may be stutterers [21]. Environmental, motor, mental, cognitive, and linguistic factors can have a role at the beginning of stuttering [18]. Assessment and treatment of stuttering are very complicated, and clinicians must be skillful in a thorough assessment for designing an appropriate treatment plan. With this description, students must show their clinical skills by participating in exact and appropriate evaluations for the determination of their strengths and weaknesses. When their weaknesses are determined, they can try to participate in special programs for the promotion of their clinical skills.
Stuttering Severity Instrument–Fourth Edition (SSI-4) is a reliable and valid norm-referenced stuttering assessment that can be used for both clinical and research purposes. It measures stuttering severity in both children and adults in the four areas of speech behavior: frequency, duration, physical concomitants, and naturalness of the individual’s speech [22].
With respect to the fact that different subjective clinical exams are used for clinical exams of speech therapy students, in this study we decided to design and assess the psychometrics of direct observation of procedural skills (DOPS) exam for evaluating clinical skills of students in the assessment of the severity of stuttering using the Persian version of SSI-4. 
Materials and Methods
The data of this descriptive-analytic non-interventional study was gathered in the second semester of 2020-2021, in the School of Rehabilitation Sciences of the Iran University of Medical Sciences. The sample was 19 students of speech therapy who were passing the units related to clinical skills. The census method was used for the selection of samples, after explaining some related details and taking their consent.
A blueprint of procedural skills for assessing stuttering was provided and presented to ten academy staff. They were requested to rate the items according to the clinical importance of any item. According to the resulting rankings after the suggestions of the academy staff of the speech therapy department of Iran University of Medical Sciences, this skill was selected: severity of stuttering using SSI-4. 
The SSI-4 checklist for measurement of severity of stuttering was prepared according to the DOPS format. Twelve items were designed for this checklist. The details of the items were determined via studying different educational literature, such as books related to stuttering, and then via confirmation of academy staff. The checklist was prepared with special details for documentation of results, i.e. scoring of each item was according to students’ performance rating from 0 to 10 (unacceptable, lower than expected, limited, expected, and higher). 
Ten academy staff examined the face and content validity of the designed test. The experts determined their opinions about the impact score of any question for calculation of the face validity. For the determination of content validity in this study, the Lawshe method was used. Content validity was evaluated by the determination of content validity ratio (CVR) and content validity index (CVI). The simplicity, transparency, and relevance of the questions were examined for CVI, and the necessity and usefulness were examined for CVR. 
In the next step, two examiners who were students’ clinical supervisors with a Master of Science (MS) degree were selected to perform the test on students. These evaluators were trained by a written guide about the DOPS exam, for uniformity of examiners’ behavior during judgments, scoring, and using of the criteria in each part of the checklist. For training of the students, they were informed in individual sessions about the evaluators, the procedure, the purposes of the research, and its process. Students filled out the consent form. Each student was informed about the time for the DOPS exam, whenever he/she felt enough competent. 
The examiners were observing the student’s work directly. They recorded their judgments in the structured checklist and at the end of the session, they presented appropriate feedback to any student about his/her weaknesses and strengths. The required time for this observation and filling of the DOPS questionnaire, and presentation of the feedback was nearly 15 and 5 minutes, respectively.
Finally, after filling in the checklists by the evaluators observing the students and providing feedback to them, the reliability of the test was checked. For evaluation of the reliability of the test, two reliability methods were used: reliability between the evaluators and internal consistency. Internal consistency was calculated using Cronbach’s alpha.
Results
As mentioned, the opinions of a group of experts were used to examine the face validity. Participants were first asked to rate the importance of each item in the 5-point Likert scale from 1 (not important at all) to 5 (absolutely important). Then, their impact scores were calculated. To accept the face validity of each item, its impact score should not be less than 1.5 and only questions in terms of face validity are acceptable if their score is higher than 1.5. 
According to Table 1, it can be seen that the results of the impact scores of all questions were more than 1.5.


Also, the impact scores of all designed questions were above 4. Therefore, all questions had appropriate formal validity and were included in the questionnaire. 
According to the Lawshe table for ten experts, the CVR should be 0.62 or more. Therefore, we followed this command to obtain the CVR. In this test, to obtain the CVI, the number of people who reported the items as relevant or completely related was divided by the total number of experts and the value was 0.7 or more (Table 2).


As can be seen in Table 2, for most questions, the CVR value was reported above 0.62. All the questions of this test had a CVI of more than 0.7, which was desirable in terms of content validity.
In order to evaluate the reliability of the evaluator, two evaluators examined 19 undergraduate students in a test designed for the assessment of stuttering severity (Table 3).


The results of Table 3 show the ICC values ​​and the results indicated that the two assessors had a good agreement in scoring learners (ICCs<0.8).
Cronbach’s alpha coefficient was also calculated to evaluate the reliability of the internal consistency method (Table 4).


Cronbach’s alpha value for the stuttering intensity test using SSI-4 was 0.971, which confirmed the internal consistency of DOPS for the stuttering intensity test using the SSI-4 and the total DOPS score.
Discussion
In the present study, we used the Persian version of SSI-4 in order to prepare and validate the DOPS exam for evaluating the clinical skills of students in the assessment of the severity of stuttering. As mentioned, one of the evaluation methods is DOPS. In this research, we designed a test according to the structure of the DOPS test that examined the clinical performance of the student in assessing stuttering and then examined the validity and reliability of the test.
In this test, for face validity, the views of speech therapists were used and these individuals validated clinical and practical skills assessment through DOPS on the patients. These findings are consistent with most studies in this field.
In a study conducted in 2012 by Sahib al-Zamani et al. at Zahedan University School of Nursing and Midwifery on the validity and reliability of the DOPS test, eight procedures were used to perform this test and its high validity was mentioned. They stated that students’ familiarity with this method during the academic year and its inclusion in the curriculum of colleges, according to the students’ level of experience, increase the validity of the test and compared to previous methods, such as the logbook, which shows only the quantity of the procedure, or the OSCE, which is performed on the artificial model, the face validity of DOPS is higher [23]. This finding is consistent with the studies conducted by Kuhpayehzadeh et al. in 2014 at the Faculty of Nursing and Midwifery of Kashan University of Medical Sciences [24]. In this study, after examining the results of internal consistency of the desired skill, if any of the items are removed, there will be no noticeable change in the alpha value of the entire skill so that all items to measure the relevant skill have the appropriate internal consistency [25], which is consistent with the study by Alborzi et al. on audiometry students in 2015 in the School of Rehabilitation Sciences of the Iran University of Medical Sciences [26].
In this study, we used two experts in the field of speech therapy for the reliability of the evaluators, and according to the results related to the internal correlation coefficient (ICC<0.8), the two used assessors had a good agreement in grading the students and these results indicate the appropriateness of this test. Also, according to Naeem et al., one of the best ways for a test to be reliable is using two different assessors to observe a student’s performance on the same skill [13].
Cronbach’s alpha coefficient was also calculated to evaluate the reliability of the internal consistency method. Cronbach’s alpha value for the stuttering intensity test using SSI-4 was 0.971, which confirms the internal consistency of DOPS for the stuttering intensity test using the SSI-4 and the total DOPS score. In the study by Alborzi et al., Cronbach’s alpha coefficient was used for assessing the reliability of the test, which was measured as 0.788 [26].
Conclusion
The findings of this study showed the suitable validity and reliability of the DOPS test for real and objective assessment of clinical skills in the field of speech therapy. Also, students and professors agreed on the applicability of this method and acknowledged that this method has positive educational effects due to giving constructive feedback and conscious intervention by professors as well as educating students. Therefore, these positive points indicate that using this method, the person’s strengths, and weaknesses can be identified and he/she will upgrade his/her strengths and cover weaknesses to a great extent, which can increase the clinical skills of the person, leading to the improvement of the level of health and safety. This improvement is one of the goals of medical education. It is therefore suggested that faculty members of speech therapy focus more on this method of assessment and use the DOPS test to assess the clinical skills of speech therapy students. The following suggestions may be helpful:
• Using the DOPS test to assess the clinical skills of speech therapy students in the field of stuttering along with other methods.
• Building DOPS tools for other areas of speech therapy.

Ethical Considerations
Compliance with ethical guidelines
This study was approved by the Ethics Committee of Iran University of Medical Sciences (Code: IR.IUMS. REC.1398.889).

Funding
The paper was extracted from the MSc. thesis of the first author at the Department of Speech Therapy, School of Rehabilitation Sciences, Iran University of Medical

Authors' contributions
Conceptualization and supervision: Younes Amiri Shavaki, Shahnam Sedigh Maroufi, Jamileh Abolghasemi, Mohammad-Sadegh Jenabi; Methodology: Jamileh Abolghasemi; Investigation, writing - review & editing: All authors; Writing - original draft: Zahra Kahani; Funding acquisition: Zahra Kahani, Younes Amiri Shavaki.

Conflict of interest
The authors declared no conflicts of interests.

Acknowledgments
We sincerely thank the cooperation of the Iran University of Medical Sciences, School of Rehabilitation Sciences.


References

1. Lincoln MA, Adamson BJ, Cant RV. The importance of managerial competencies for new graduates in speech pathology. Advances in Speech Language Pathology. 2001; 3(1):25-36. [DOI:10.3109/14417040109003706]
2. Kendall D. The development of a graduate program in rehabilitation sciences in Ghana: A case report about a global health academic partnership [Msc. thesis]. Seattle: University of Washington; 2020. https://digital.lib.washington.edu/researchworks/handle/1773/45738?show=full
3. Bari V. Direct observation of procedural skills in radiology. American Journal of Roentgenology. 2010; 195(1):W14-8. [DOI:10.2214/AJR.09.4068] [PMID]
4. Franko DL, Cousineau TM, Trant M, Green TC, Rancourt D, Thompson D, et al. Motivation, self-efficacy, physical activity and nutrition in college students: Randomized controlled trial of an internet-based education program. Preventive Medicine. 2008; 47(4):369-77. [DOI:10.1016/j.ypmed.2008.06.013] [PMID] [PMCID]
5. Yousefzadeh S, Golmakani N. [The midwifery students’ perspective about clinical evaluation based on log book (Persian)]. Journal of Research Development in Nursing Midwifery. 2012; 9(1);103-11. http://nmj.goums.ac.ir/article-1-218-en.html
6. Shahgheibi S, Pooadi A, BahramRezaei M, Farhadifar F, Khatibi R. Evaluation of the effects of direct observation of procedural skills (dops) on clinical externship students’ learning level in obstetrics ward of Kurdistan University of Medical Sciences. Journal of Medical Education. 2009; 13(1-2):29-33. https://brief.land/jme/articles/105385.html
7. Wilkinson TJ, Frampton CM. Comprehensive undergraduate medical assessments improve prediction of clinical performance. Medical Education. 2004; 38(10):1111-6 [DOI:10.1111/j.1365-2929.2004.01962.x] [PMID]
8. Shumway JM, Harden RM. Association for Medical Education in Europe. AMEE Guide No. 25: The assessment of learning outcomes for the competent and reflective physician. Medical Teacher. 2003; 25(6):569-84. [DOI:10.1080/0142159032000151907] [PMID]
9. Sanatkhani M, Molla Z, Akbari M. [Evaluation of the students’ perception about clinical education and examination in Mashhad School of Dentistry (Iran) in 2009 (Persian)]. Journal of Mashhad Dental School. 2012; 36(3):211-22. [DOI:10.22038/JMDS.2012.481]
10. Delaram M, Tootoonchi M. [Comparing self- and teacher-assessmentin obstetric clerkship course for midwifery students of Shahrekord University of Medical Sciences (Persian)]. Iranian Journal of Medical Education. 2009; 9(3):231-8. https://ijme.mui.ac.ir/article-1-1055-en.html
11. Tahernezhad K, Javidan F. [Advanced assessment of medical students clinical performance: challenges methods and approaches (Persian)]. Strides in Development of Medical Education. 2008; 5(1):58-70. http://sdme.kmu.ac.ir/article_90139.html
12. Kariman N, Heydari T. [The effect of Portfolio’s evaluation on learning and satisfaction of midwifery students]. Journal of Arak University of Medical Science. 2010; 12(4):81-8. http://jams.arakmu.ac.ir/article-1-434-en.html
13. Naeem N. Validity, reliability, feasibility, acceptability and educational impact of direct observation of procedural skills (DOPS). Journal of the College of Physicians and Surgeons--Pakistan. 2013; 23(1):77-82. [PMID]
14. Bindal N, Goodyear H Bindal T, Wall D. DOPS assessment: A study to evaluate the experience and opinions of trainees and assessors. Medical Teacher. 2013; 35(6):e1230-4. [DOI:10.3109/0142159X.2012.746447] [PMID]
15. Weller JM, Jolly B, Misur MP, Merry AF, Jones A, Crossley JGM, et al. Mini-clinical evaluation exercise in anaesthesia training. British Journal of Anaesthesia. 2009; 102(5):633-41. [DOI:10.1093/bja/aep055] [PMID]
16. Bourbonnais FF, Longford S, Giannatonio L. Development of a clinical evaluation tool for baccalaureate nursing students. Nurse Education in Practice. 2008; 8(1):62-71. [DOI:10.1016/j.nepr.2007.06.005] [PMID]
17. Bhugra D, Malik A. Workplace-based assessment in psychiatry. London: Royal College of Psychiatrists; 2007. https://www.google.com/books/edition/_/qs2PcHLANCIC?hl=en&sa=X&ved=2ahU
18. Guitar B. Stuttering: An intergrated approach to its nature and treatment. Baltimore, MD: Lippincott Williams & Wilkins; 2006. https://www.google.com/books/edition/_/X0d80oJKEjsC?hl=en&
19. Heydari-Nasrabadi M, Kamali M, Arrani-Kashani Z. [Others’ presence on the life experiences of people stuttering (Persian)]. Archives of Rehabilitation. 2015; 15(4):78-88. http://rehabilitationj.uswr.ac.ir/article-1-1262-en.html
20. Fekar Gharamaleki F, Shahbodaghi MR, Jahan A, Jalayi S. [Investigation of acoustic characteristics of speech motor control in children who stutter and children who do not stutter(Persian)]. Archives of Rehabilitation. 2016; 17(3):232-43. [DOI:10.21859/jrehab-1703232]
21. Rahimi SS, Farazi M, Darouie A, Bakhshi E, Abdi S, Valinejad V, et al. [Comparison the quality of life among adults with and without stuttering: An emphasis on the severity of stuttering (Persian)]. Archives of Rehabilitation. 2016; 17(4):300-7. [DOI:10.21859/jrehab-1704300]
22. Everard RA, Howell P. We have a voice: Exploring participants' experiences of stuttering modification therapy. American Journal of Speech-Language Pathology. 2018; 27(3S):1273-86. [DOI:10.1044/2018_AJSLP-ODC11-17-0198] [PMID]
23. Sahebalzamani M, Farahani H, Jahantigh M. [Validity and reliability of direct observation of procedural skills in evaluating the clinical skills of nursing students of Zahedan nursing and midwifery school (Persian)]. Zahedan Journal of Research in Medical Science. 2012; 14(2):e93588. https://brief.land/zjrms/articles/93588.html
24. Kuhpayehzade J, Hemmati A, Baradaran H, Mirhosseini FS, Akbari H, Sarvieh M. [Validity and reliability of direct observation of procedural skills in evaluating clinical skills of midwifery students of Kashan Nursing and Midwifery School (Persian)]. Journal of Sabzevar University of Medical Sciences. 2014; 21(1):145-54. http://jsums.medsab.ac.ir/article_413.html?lang=en
25. Wilkinson JR, Crossley JG, Wragg A, Mills P, Cowan G, Wade W. Implementing workplace‐based assessment across the medical specialties in the United Kingdom. Medical Education. 2008; 42(4):364-73. [DOI:10.1111/j.1365-2923.2008.03010.x] [PMID]
26. Alborzi R, Koohpayehzadeh DJ, Roozbahani M. [Validity and reliability of the persian version of direct observation of procedural skills tool in audiology (Persian)]. Scientific Journal of Rehabilitation Medicine. 2021; 10(2):346-57. [DOI:10.22037/JRM.2020.113471.2372]