Science for Education Today, 2024, vol. 14, no. 1, pp. 125–148
© Abramova V. Y., Veksler A. K., Glubokova E. N., Nazmutdinova T. S., Popova R. I., Chernyshova E. P., Shelukhina O. A., Kalabina I. A., 2024
UDC: 
316.75+303.687+378.146

Objective assessment of learning outcomes: Peculiarities of design and application of assessment tools in modern conditions of digital development

Abramova V. Y. 1 (St. Peterburg, Russian Federation), Veksler A. K. 2 (St. Petersburg, Russian Federation), Glubokova E. N. 2 (St. Petersburg, Russian Federation), Nazmutdinova T. S. 1 (St. Peterburg, Russian Federation), Popova R. I. 1 (St. Peterburg, Russian Federation), Chernyshova E. P. 2 (St. Petersburg, Russian Federation), Shelukhina O. A. 1 (St. Peterburg, Russian Federation), Kalabina I. A. 2 (St. Peterburg, Russian Federation)
1 Herzen State Pedagogical University
2 Herzen State Pedagogical University of Russia
Full text (Open Access) — Downloaded 276
Abstract: 

Introduction. The article deals with the problem of designing and application of assessment tools in the context of objective assessment of learning outcomes in higher education. The aim of the article is to identify the peculiarities of design and application of assessment tools in modern conditions of digital development for the objective assessment of learning outcomes.
Materials and Methods. In order to achieve the purpose of the research, the authors used theoretical (analysis of theoretical sources) and empirical (survey of 222 teachers) methods. The data obtained were subjected to quantitative and qualitative analysis.
Results. The authors analyzed the approaches to the interpretation of assessment procedures and means in theoretical studies and regulatory documents. The main results of the study allowed to identify a number of problems in the development of assessment tools banks: substitution of the subject of the assessment for the knowledge component; orientation to the development of assessment materials at the discipline level; assessment of the validity and reliability of the designed assessment tools; assessment
of teachers’ labor input; the ratio of the used template assessment tools and practical tasks; difficulties in the design of interdisciplinary assessment tools and competence assessment in general; underestimation of the potential of digital environment in the development of assessment tools. Evaluating the problems of designing and applying assessment tools by teachers depended on the subject area they teach and their experience of participation in the development of assessment tools at various levels. The dependence on teachers’professional experience and the position held has not been revealed.
Conclusions. Based on the results of the study, conclusions about the types of problems in designing and application of assessment tools in modern conditions are drawn. Practical recommendations are provided.

Keywords: 

Learning outcomes; Competence assessment; Validity of assessment tools; Reliability of assessment tools; Design of assessment tools; Assessment tools banks.

For citation:
Abramova V. Y., Veksler A. K., Glubokova E. N., Nazmutdinova T. S., Popova R. I., Chernyshova E. P., Shelukhina O. A., Kalabina I. A. Objective assessment of learning outcomes: Peculiarities of design and application of assessment tools in modern conditions of digital development. Science for Education Today, 2024, vol. 14, no. 1, pp. 125–148. DOI: http://dx.doi.org/10.15293/2658-6762.2401.06
References: 
  1. Efremova N. F. The principles of independent evaluation system in higher education. Higher Education in Russia, 2015, no. 7, pp. 63–67. (In Russian) URL: https://elibrary.ru/item.asp?id=24002989
  2. Ibragimov G. I., Ibragimova E. M. Competence assessment: Challenges and solutions. Higher Education in Russia, 2016, no. 1, pp. 43–52. (In Russian) URL: https://www.elibrary.ru/item.asp?id=24993105
  3. Sabelnikova E. V. The evolution of competence research in the international context. Psychological Science and Education, 2021, vol. 26 (2), pp. 78–89. (In Russian) DOI: https://doi.org/10.17759/pse.2021260207 URL: https://www.elibrary.ru/item.asp?id=46207522
  4. Cross V., Hicks C., Barwell F. Exploring the gap between evidence and Judgement: Using video vignettes for practice-based assessment of physiotherapy undergraduates. Assessment & Evaluation in Higher Education, 2001, vol. 26 (3), pp. 189–212. DOI: https://doi.org/10.1080/02602930120052369
  5. Mislevy R. J., Steinberg L. S., Almond R. G. Focus article: On the structure of educational assessments. Measurement: Interdisciplinary Research and Perspectives, 2003, vol. 1 (1), pp. 3–62. DOI: https://doi.org/10.1207/S15366359MEA0101_02
  6. Chernyshova E. P., Veksler A. K., Kalabina I. A., Glubokova E. N. Designing assessment funds for the bachelor's degree program “Art education in visual arts”: Problems and solutions. Perspectives of Science and Education, 2023, no. 5, pp. 227–244. (In Russian) DOI: https://doi.org/10.32744/pse.2023.5.14 URL:  https://www.elibrary.ru/item.asp?id=54799664 
  7. Varakuta A. A. A flexible approach to assessing students' learning outcomes taking into account working efforts of the teacher. Novosibirsk State Pedagogical University Bulletin, 2018, vol. 8 (4), pp. 187–202. (In Russian) DOI: http://dx.doi.org/10.15293/2226-3365.1804.12 URL: https://elibrary.ru/item.asp?id=35605578
  8. Rawlusyk P. E. Assessment in higher education and student learning. Journal of Instructional Pedagogies, 2018, vol. 21, pp. 1–34. URL: https://files.eric.ed.gov/fulltext/EJ1194243.pdf
  9. Coates H. Assessment of learning outcomes. In: Curaj A., Matei L., Pricopie R., Salmi J., Scott P. (eds)  The European Higher Education Area, 2015, pp. 399–413. DOI: https://doi.org/10.1007/978-3-319-20877-0_26

10. Goss H. Student learning outcomes assessment in higher education and in academic libraries: A review of the literature. The Journal of Academic Librarianship, 2022, vol. 48 (2), pp. 102485. DOI: https://doi.org/10.1016/j.acalib.2021.102485

11. Wiliam D. Studies in educational evaluation. Studies in Educational Evaluation, 2011, vol. 37 (1), pp. 3–14. DOI: https://doi.org/10.1016/j.stueduc.2011.03.001

12. Dawson P., Bearman M. Concluding comments: Reimagining university assessment in a digital world. In: Bearman M., Dawson P., Ajjawi R., Tai J., Boud D. (eds). Re-imagining University Assessment in a Digital World. The Enabling Power of Assessment, 2020, pp. 291–296. DOI: https://doi.org/10.1007/978-3-030-41956-1_20

13. Richter T.V., Shestakova L. G., Zentsova I. M., Sugrobova N. Yu. Using mobile learning applications for the development of students’ universal competencies: Evaluation of the effectiveness. Science for Education Today, 2020, vol. 10 (6), pp. 181–199. (In Russian) DOI: http://dx.doi.org/10.15293/2658-6762.2006.10 URL: https://elibrary.ru/item.asp?id=44478888

14. Maaliw R. R. Adaptive virtual learning environment based on learning styles for personalizing e-learning system: Design and implementation. International Journal of Recent Technology and Engineering, 2020, vol. 8 (6), pp. 3398–3406. DOI: https://doi.org/10.35940/ijrte.F8901.038620

15. Mustafa A., Sharif S. M. An approach to adaptive e-learning hypermedia system based on learning styles (AEHS-LS): Implementation and evaluation. International Journal of Library and Information Science, 2021, vol. 3 (1), pp. 15–28. URL: https://www.semanticscholar.org/paper/An-approach-to-Adaptive-E-learning-Hypermedia-based-Eltigani Mustafa/2f354240d2ffb22d73ec195042f4335be582e591

16. Owan V. J., Abang K. B., Idika D. O., Etta E. O., Bassey B. A. Exploring the potential of artificial intelligence tools in educational measurement and assessment. Eurasia Journal of Mathematics, Science and Technology Education, 2023, vol. 19 (8), pp. em2307. DOI: https://doi.org/10.29333/ejmste/13428  

17. Matthews K., Sherwood C., Enright E., Cook-Sather A. What do students and teachers talk about when they talk together about feedback and assessment? Expanding notions of feedback literacy through pedagogical partnership. Assessment & Evaluation in Higher Education, 2024, vol. 49 (1), pp. 26–38. DOI: https://doi.org/10.1080/02602938.2023.2170977

18. Fernandes E. L., Flores M. A. Assessment in higher education: Voices of programme directors. Assessment & Evaluation in Higher Education, 2022, vol. 47 (1), pp. 45–60. DOI: https://doi.org/10.1080/02602938.2021.1888869

19. Efremova N. F. Evidence-centered approach to competences assessment in vocational education. Koncept, 2023, no. 8, pp. 1–14. (In Russian) URL: https://www.elibrary.ru/item.asp?id=54357555

20. Balve P., Ebert L. Ex post evaluation of a learning factory–competence development based on graduates feedback. Procedia Manufacturing, 2019, vol. 31, pp. 8–13. DOI: https://doi.org/10.1016/j.promfg.2019.03.002

21. Sharov A. A., Konovalov A. A. Vocational education teachers’ soft-competences: Assessment methodology and correlation analysis. Science for Education Today, 2022, vol. 12 (5), pp. 7–21. (In Russian) DOI: http://dx.doi.org/10.15293/2658-6762.2205.01 URL: https://elibrary.ru/item.asp?id=49623125 

22. Gadušova Z., Hašková A., Szarszoi D. Teachers` competences evaluation: Case study. Science for Education Today, 2020, vol. 10 (3), pp. 164–177. DOI: http://dx.doi.org/10.15293/2658-6762.2003.09 URL: https://elibrary.ru/item.asp?id=43091315

23. Shakhova E. Yu., Nester E. V., Sitov I. S. The centralized structure of fund of assessment tools of educational program. Higher Education in Russia, 2017, no. 1, pp. 69–77. (In Russian) URL: https://www.elibrary.ru/item.asp?id=28099030

24. Dudyrev F. F. Maksimenkova O. V. Training simulators in vocational education: Pedagogical and technological aspects. Educational Studies, 2020, no. 3, pp. 255–276. (In Russian) DOI: https://doi.org/10.17323/1814-9545-2020-3-255-276 URL: https://www.elibrary.ru/item.asp?id=44007519

Date of the publication 29.02.2024