• A
  • A
  • A
  • ABC
  • ABC
  • ABC
  • А
  • А
  • А
  • А
  • А
Regular version of the site

Impact of the Self-Assessment Process on the Improvement of the Bachelor’s Level Training Programme

 

Self-assessment is the 1st phase of the multiphase process of TP analysis started at the Faculty of Sociology in November 2012 with the emphasis on the training results. At this phase, special attention is paid to the implementation of the programme of competence-based approach, which so far has no off-the-shelf solutions in Russia or abroad.

Work on the implementation and development of the new approach in sociological education is supervised by HSE pro-rectors Vadim Radaev and Sergey Roshchin as well as the Dean of the Faculty of Sociology Alexander Chepurenko.

Research and methodology development in this field is the result of active cooperation of the employees of the Centre for Development of Sociological Education (headed by Margarita Bershadskaya) and the teams of the Faculty departments with consulting support and continuous creative collaboration with the University’s Methodology Office (headed by Alexandra Serova).

This section provides specific examples of the following three important aspects:

·        how the process of self-analysis proper impacts TP improvement (subsections 3.3.2 and 3.3.3);

·        how the process of self-analysis will be applied for the enhancement of the quality of training in the future (subsection 3.3.8.);

·        the process of self-analysis as a powerful tool for involving the F&S in TP assessment  and improvement (subsections 3.3.7, 3.3.8) .

 

 

Development of National Requirements for the Competences of a Bachelor and Master of Sociology

 

Development of common cultural and professional competences of a sociologist is one of the major goals of the bachelor’s programme TP, which to the most extent corresponds to the institutional goals of the Faculty of Sociology.

According to this goal, self-analysis shall focus on the monitoring of the fulfilment of requirements for the structure of developed competences and methods of their development and assessment.

It should be emphasized here that development of national requirements for the structure of bachelor competences (and master competences as continuation) is the best practice of the previous experience of analytical work of the Faculty of Sociology. In 2009 the Faculty organized and chaired the Consortium of universities that developed the current Fedreral educational standards for bachelor and master training in sociology[1]. The team of authors of FS 2010 officially included five employees and leading faculty members of the Faculty of Sociology: Vadim Radaev (head of the Department of Economic Sociology, chairman of the Consortium of FS authors); Alexander Chepurenko (Dean of the Faculty, vice chairman of the Consortium); Andrey Kozhanov (Deputy Dean for academic affairs and methodology), Inna Devyatko (head of the Department of Analysis of Social Institutions, head of the master’s programme), Margarita Bershadskaya (head of the Centre for Development of Sociological Education).

The fact that the Ministry of Education and Science of the Russian Federation has approved the FSs developed by the initiative and under supervision of the HSE’s Faculty of Sociology is a sure evidence of the competitive advantage of the Faculty and recognition of its experience in analytical work in the field of TPs for training in sociology.

The same is true about methodological support of the FS implementation, in particular,  recommendations in the form of approximate TPs for bachelor and master training (they include suggested curricula and training programmes for all disciplines)[2].

These activities of the Faculty are going on: development of FS 2012 by the initiative of the Ministry – bachelor’s, master’s and postgraduate programmes. When performing these highly responsible activities, the Faculty mainly uses the results of analysis of its own TPs.

 

 

Implementation of the Requirements for the Competences of a Bachelor in Sociology in the Bachelor Programme TP

 

The HSE uses its own Original educational standards for training bachelors and masters in sociology, which were developed by the Faculty based on the FS requirements (not lower than the state requirements).
The curriculum is in full compliance with the structure of the TP specified in the original educational standard. The situation with a set of training programmes is more complicated. Already the first self-assessment phase showed that the programme authors were mainly focused on the discipline content and rational distribution of training load based on the curriculum, but not always understood the competence-based approach to training results, which was new for them.

The shortcomings identified at this phase showed that it was necessary to immediately do the following:

·       enhance the qualification of faculty members;

·       correct the training programmes of disciplines;

·       repeat TP monitoring (second self-assessment phase).

The mechanism of detecting shortcomings is implied in the system of programme self-assessment including:

·       development and analysis of the matrix of competences, which distributes the competences a graduate should have by the basic programme disciplines (i.e., partial development of competences in some disciplines);

·       creation and analysis of the matrix of knowledge and skills, which distributes them by competences (this forms a basis for creating the resource base for each of the competences of the resource base);

·       development of suggestions on the correction of discipline training programmes to eliminate the shortcomings detected.

Taking into account the importance of the specified elements of self-analysis, they are described in more detail below (subsections 3.3.3 and 3.3.4).

This subsection just gives an overview of general results of the first self-assessment phase (Table 3.8) and shows the role of self-analysis and its impact on TP improvement as a consequence of the general results of the second self-analysis phase (Table 3.9).

Table 3.8 lists the TP strengths and weaknesses detected at the first self-assessment phase and gives corresponding suggestions on how to make corrections in the training programmes (see subsection 3.3.2.).

 

Table 3.8

Results of the First Self-Assessment Phase
Bachelor Level Training Programme

STRENGTHS

SHORTCOMINGS

EVENTS

The curriculum is in full compliance with the requirements of the standard for the programme structure

 

 

The training programmes of the disciplines and practical training provisions are in compliance with the curriculum

 

 

The training programmes of the basic disciplines, developed by the Faculty of Sociology, are written based on a common template that focuses on training results

The training programmes, developed at other faculties are not template-based and  contain no information about the competences to be developed and indicators of their mastery

Recommendations on how to correct the training programmes of other faculties

A high level of the content and efficiency of the methods for knowledge and skill development (according to the TP expertise)

Not all the competences specified in the standard are included in the training programmes of the basic disciplines

Distribution of uncovered or insufficiently covered competences by  discipline

The template-based

training programmes of the basic professional disciplines formulate the descriptors of the competences to be mastered within the discipline

- Insufficient number of the descriptors of the competences developed in the basic professional disciplines;

- Some descriptors are worded incorrectly

- make and analyse lists of descriptors;

- suggestions on how to correct the wording

The F&S of the Faculty were trained on how to develop tests for  competence assessment

Insufficient number of tests for competence assessment

Develop tests for competence assessment

The training programmes contain requirements for the knowledge and  skills to be acquired upon mastering the discipline

- Some standard requirements for the knowledge and skills are not completely reflected in the programme;

- The knowledge and skills are not specified in the competence descriptors

Suggestions on how to create the resource base of the competences based on the requirements for the knowledge and skills -  formulate the appropriate descriptors and distribute them by competence

The qualification of faculty members is to be improved based on the self-analysis results

 

Table 3.9 gives examples of some shortcomings, detected during the first self-analysis phase, to show how the self-analysis process impacts the programme improvement.

 

Table 3.9

Results of the Second Self-Assessment Phase: Examples of Eliminating the Shortcomings Detected during the First Phase

PROGRAMME WEAKNESSES

BEFORE

NOW

1. The TPs of the basic professional disciplines, which contain no information about competence development

2

0

2. Competences not covered in the programme

2

0

3. PCs partially developed in one or two disciplines

 

in one discipline

in two disciplines

 

 

3

4

 

 

0

0

4. Incorrect wording of the indicators of competence mastery (descriptors)

8

Update and correction

5. Insufficient number of the descriptors of the competences developed within the basic professional disciplines (16 mandatory competences)

total

average per competence

 

 

 

 

57

3.6 - from

 0 to10

 

 

 

 

139

8.7

6. Insufficient number of tests for competence assessment

80

400

7.Insufficient number of descriptors specified based on the requirements for the knowledge and skills

 

 

19

 

 

51

 

 

Analysis of the Matrix of Competences and its Impact on the Process of TP Improvement

The main tool for assessing the competence-based approach by the entire set of TP elements (training programmes of the basic disciplines, practical training and FSC provisions) is creation and analysis of the matrix of competences.

To illustrate the efficiency of TP self-assessment with aggregation of results and further measures for eliminating the shortcomings detected, below there are the results of monitoring the competence-based approach for mastering the basic professional disciplines.

The TP template, suggested to faculty members since 2011, includes not only the list of the competences developed when training in the discipline (fully or partially), but also the indicators of competence mastery (descriptors), which are tested both when studying the discipline and within various types of testing by the discipline.

Unfortunately faculty members do not always understand the new approach and its role in the training process, so when developing training programmes, they often just formally list competences and descriptors trying (intentionally or not) to make the number of competences, developed during the discipline study, minimal. Hence, a real risk of not fulfilling the requirements for the competences to be acquired by a programme graduate. The matrix of competences created during the first self-assessment phase identifies some competences that are insufficiently or not at all covered in the programme. By way of example, let’s give the results of the analysis of the initial matrix of competences developed in the basic part of the programme – when studying the basic disciplines of the professional cycle during theoretical training. The matrix showed that more than five out of 16 competences are not sufficiently covered in the discipline TPs. These include two common cultural competences (CC) and three professional competences (PC):

·       CC-12 Understanding of essence and value of information in the development of modern society, recognize risks and dangers, satisfy information security requirement, including protection of official secrets.

·       CC-16 Knowledge of safeguards for personnel and public protection from the consequences of accidents and disasters.

·       PC-6 Ability to participate in elaboration of recommendations based on professional sociological knowledge aimed at social problems solutions, elaboration of mechanisms for coordinating the interests of social groups and communities.

·        PC-10 Ability to apply basic theoretical knowledge and practical skills for participation in scientific and applied research, analytical work and consulting.

·       PC-11 Ability to use methods of collection, processing and interpretations of complex social information to solve the administrative tasks including those beyond the immediate sphere of activities.

In addition, two TPs were found to be missing information about the competences to be developed:

·       “Economic and social statistics”

·        “Social structure and social stratification”

As is seen from the first phase matrix, the shortcomings, specified in items 1, 2, 3 of Table 3.9 in the previous subsection, were identified. This allowed carrying out some work on the programmes during the second phase, which fully eliminated the bottlenecks of the first matrix.

The first and second self-analysis phase matrixes of competences, which give a summary of this subsection, can be found in Annexes A.3.8 and A3.9.

 

Analysis of the Matrix of Knowledge and Skills and its Impact on the Process of TP Improvement

 

Knowledge and skills are the necessary and very important component of any developed competence and its resource base.

At the first self-analysis phase it was found out that the discipline TPs quite fully reflect the process of obtaining knowledge and skills during discipline study. One of the shortcomings is a weak correlation between the knowledge and competences developed when training in the discipline. This results from the distribution of the knowledge and skills by the competences developed, i.e., the matrix of knowledge and skills. The matrix was created based on the appropriate analysis of the competence descriptors that reflect their resource base. The mandatory competences of the professional cycle are described considering the standard requirements for the cycle resource base. Formalabsence of the correlation between the knowledge and skills and mandatory competences is quite significant: the three requirements for the knowledge and skills were not reflected in any competences; 10 out of 16 competences had no descriptors of the resource base.

Although mastery of the knowledge and skills, implied by the training programmes, is undoubtedly the basis for developing the competences, the absence of the formal correlation with the competences did not allow evaluating them as part of the competences (the resource base and the competences are evaluated separately).

As a result of the analysis of the resource base and the content of the discipline training programmes, over 30 descriptors of the resource base were formulated additionally for all the mandatory competences of the professional cycle. Correction of the training programmes in compliance with the suggestions made dramatically changed the matrix (see Table 3.10). The competences that contained the descriptors of the resource base at least in one of the basic disciplines in the original matrix are marked with *. The # character indicates changes in the second programme self-assessment phase.

As is seen, at the second self-analysis phase, the matrix of knowledge and skills has no gaps: all the knowledge and skills are included in the competences (one or several); each of the 16 mandatory competences now contains the descriptors of the knowledge and skills.

Table 3.10

Matrix of Knowledge and Skills at the Second Self-Analysis Phase

Requirements for knowledge and skills

CC

PC

8

9

10

12

16

1

2

3

4

5

6

7

8

9

10

11

12

Know (resource base):

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

main classic and modern sociological theories and schools

 

 

*

 

 

#

 

 

 

 

#

 

 

 

 

 

#

main factors in integrated social processes and functioning mechanisms of major social communities

 

 

#

 

 

 

 

 

#

 

#

 

 

 

#

 

#

Methodology basics of sociological research, main methods of sociological information collection and analysis

 

 

 

 

 

*

 

 

#

 

 

 

 

 

 

 

 

theoretical foundations of sociological branch disciplines

#

 

#

 

 

 

 

 

#

 

 

 

 

 

 

 

#

main factors of socio-economic, political and management processes, principal approaches to their study, as well as peculiarities of the application in Russia

 

 

*

 

 

 

 

 

 

 

 

 

 

 

 

 

 

main theoretical models and  research methods that describe social action, social perception, communication and interaction at micro/micro levels

 

 

*

 

 

 

 

 

 

 

 

 

 

 

 

 

 

be able to:

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

produce, select, process and analyse data on social processes and social communities

 

#

 

 

 

 

 

 

 

 

 

 

 

 

 

*

 

apply relevant methods of data collection and analysis, recognize limitations, evaluate quality (validity and reliability) of sociological data

 

#

 

 

 

 

#

 

 

 

 

 

 

*

 

 

 

participate in project work and accomplish own analytical projects

 

 

#

 

 

 

*

*

 

 

 

 

 

*

 

 

 

present the results of research and analytical work to public and professional audience

 

 

 

 

 

#

#

 

 

*

 

 

 

 

 

 

 

have skills of:

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

application of fundamental sociological knowledge in practical work

 

 

 

 

 

 

 

 

 

 

#

 

 

 

 

 

 

social data analysis by applying statistical software systems

 

#

 

 

 

 

*

 

#

#

#

 

 

 

 

 

 

obtaining professional information from various sources, including Internet and foreign literature

 

 

 

 

 

 

*

#

 

 

 

#

 

#

#

#

#

 

 

Suggestions on Correction of Training Programmes of Other Faculties Based on the Requirements for Sociology Training Programme

 

To make sure the requirements of our TP resulting from the analysis of the matrix of competences and matrix of knowledge and skills are met, we were to develop suggestions on how to correct the TPs of the basic professional disciplines of other faculties (“Economic and social statistics”; “Social structure and social stratification”). The aim was to formulate the competences, required by a sociologist, based on the specifics of the content of these courses, which, as our preliminary analysis shows, contains all necessary conditions for their development.

There are two different ways to formulate competences developed by disciplines and their corresponding descriptors. The first approach is to limit the number of such competences to 5-6. Most authors of training programmes of other faculties are committed to this principle. The second approach, which is applied by our Faculty to the basic professional disciplines, implies step-by-step development of system and professional competences in most disciplines of the cycle. We believe (and were supported in the matter by psychologists during one of the meetings on methodological issues), that such approach is more effective, the more so because, as a rule, none of the disciplines can be responsible for the development of all professional competences.

It was this approach that we followed to develop recommendations on how to make changes in the above-named disciplines. Recommendations on how to fill the table of competences in these disciplines’ TPs are given in the Annex. Considering the importance of these courses for sociology students, the recommendations contain the wording of descriptors of all the mandatory competences of the professional cycle (more than 30 descriptors per discipline). The recommended descriptors (see tables in Annexes A.3.10 and A.3.11) are in full compliance with the content of the disciplines and are easy to verify.

3.3.6. Development of the Methodology School at the Faculty of Sociology

         The HSE’s Faculty of Sociology, renowned for its academic school, in recent years has gained prominence for its methodology development, too. Promotion of the methodology school at the Faculty is carried out under the guidance and in creative collaboration with the HSE’s Methodology Office.

First recognition of our methodology school is its winning of the competition for the development of state educational standards in sociology (2009) and further development of approximate TPs for Russian universities and colleges that train bachelors and masters in sociology. The Centre for Development of Sociological Education was opened at the Faculty to coordinate the activities on the implementation of new educational standards in Russian universities and colleges.

The next important development phase was working out of our own original standards introduced in 2010.

Then followed a great volume of work on the implementation of the new standards along with the development of methodology materials, organization of professional development courses, in-house and field seminars for faculty members and the members of the Consortium of universities with experience shared during round tables, All-Russian and inter-university conferences, social networks, publication of developed methodologies in journals and conference files.

Professional development of the F&S of the Faculty is mainly aimed to solve the most critical problem with the implementation of standards – that of competence-based approach to training results. The HSE’s Methodology Office has developed very good methodologies, templates and algorithms for competence evaluation[3]. Every two weeks there are events called “Methodology Wednesdays” where the methodology materials are explained and discussed. The F&S of the Faculty of Sociology take an active part in sharing the experience. It should be noted that there was a clear increase in their activity during TP self-assessment. This is a strong evidence of the fact that the process of TP self-assessment is a powerful tool for integration of the F&S of the Faculty of Sociology in TP evaluation and improvement.Things, that first were not accepted by the faculty members (competences, descriptors, etc.), are gradually not just entering their vocabulary but also more actively used to formulate the course objectives and requirements for mastering it (see sections devoted to TP improvement in the process of self-analysis: 3.3.2.; 3.3.5; 3.3.6). The most active employees and faculty members participate in such projects of the Methodology Office as development and expertise of tests for competence evaluation, development of a competence-based course profile, certificates of system competences and so on.

Faculty events on the issues of sociological education proper are held by the University’s Methodology Office (or jointly). The nature of discussions indicates that the process of TP self-assessment makes the faculty members more and more interested in methodology issues. One should just see the expressions on their faces when they are discussing the tasks in the field of methodology development (slides on page 65) – this is so unlike many meetings and seminars of the kind when the faces of the participants express nothing but indifference, incomprehension and no interest.

These and other similar slides  summarize the results of  2009-2012 and tasks for the future (2013-2016), some of which are already in progress during self-assessment.

 

 


[1] http://ecsocman.hse.ru/sociology/bac.html

[2] http://www.hse.ru/data/2011/04/07/1211727134/plan_ba.pdf

http://www.hse.ru/org/hse/socedu/plany?_r=299391305232597.03483&__t=1063857&__r=OK

 

 

[3] http://www.hse.ru/studyspravka/programmauchdisc

 


 

Have you spotted a typo?
Highlight it, click Ctrl+Enter and send us a message. Thank you for your help!
To be used only for spelling or punctuation mistakes.