The questions of validity and reliability are often raised with regard to Discus and other similar systems. This section of the site reproduces a South African study, A Reliability and Validity Study on the Discus Personality Profiling System, by Karin Roodt of the prestigious Technikon Natal. Note that this study – known as The Roodt Report – relates not just to DISC in general but specifically to the Discus software.
A Reliability and Validity Study on the Discus Personality Profiling System
K. Roodt (Ms)
M Ed (Counselling and Guidance) (UNISA)
Psychologist (SAMDC)
Senior Lecturer: Department Human Resources Management, Technikon Natal
Abstract
The purpose of the investigation was to determine whether the Discus measuring instrument could be considered a reliable and valid instrument. The test-retest method was used in the reliability study and was administered to 90 employees from a variety of companies in Kwa Zulu-Natal and Gauteng. The Pearson’s product-moment correlation coefficient was used and correlation scores of 0.728 (Dominance), 0.645 (Influence), 0.730 (Steadiness) and 0.550 (Compliance) were established. The p-value in all the cases was as low as 0.0001. This indicates significance at alpha = 0.001. It can therefore be concluded with 99.9% level of confidence that the Discus instrument is reliable.
In the validity exercise criterion-related validity was used. An exploratory study was undertaken in order to determine which of the 15 Factors (Factor B excluded) of the 16-PF correlated with the four dimensions of the Discus. One hundred and twenty respondents in South Africa were involved for this purpose. The Pearson’s product-moment correlation coefficient was applied. It was found that Factors Q1, X=G, L, Q1 and X=Q2, E; E, Q2 and -I show significant correlations with Dominance at the 1% and 5% level of significance. Factors A, -Q2, H, F and -Q3 show significant correlations with Influence at the 1% and 5% level of significance. Factors -E and -Q1 show a correlation with Steadiness at the 5% level of significance. Factors -E, Q2, -H, -G and O show significant correlations with Compliance at the 1% and 5% level of significance. It can therefore be concluded that the correlations were significant.
What follows is the outcome of a reliability and validity study as performed by Karin Roodt, M Ed (Counselling and Guidance) UNISA, Registered Psychologist (SAMDC), Senior Lecturer, Technikon Natal (serving as Project Manager), assisted by Charles Robert, BSc (Hons)(Stats): Statistician, Director of South Africa on Line.
I would like to thank the following companies and individuals for participating in this study and in some way contributing to the final report:
- Edgars Group Gauteng
Wayne Suelz, Recruitment and Assessment Specialist: Human Resources Department - Toyota SA
François Viljoen, Mark de Wet and the rest of the dynamic team in the Department of Human Resources Management - BB Cereal
Gill Gibson (Mrs) - NBS
Natasha Carsens (Mrs) - Technikon Natal
Leslie Jordaan, Director: Personnel (Durban), Craig Stewart: Personnel (Pietermaritzburg) and Ann du Toit (editing) - HSRC
Barbara Miller and Berryll McIntyre for assisting with the marking of Technikon Natal’s 16-PF answer sheets - Labour Lawyer
Graham Giles for scrutinising this report from a law perspective
A study of this nature must be approached with caution because no single psychological tool is able to yield everything about a person’s personality. A test battery, i.e. a collection of several tests, reveals far more about an individual than does any single test of personality, preference, interests, intelligence or general personality dynamics.
K. Roodt 31.01.1997
“The Discus personality profiler is a completely computerised assessment tool, designed to describe the different roles a person fulfils in the work environment.”
(Axiom 1994:1)
The word ‘personality’ has been debated for centuries. Everybody has their own idea about exactly what it means. In Discus terms a personality is defined as the sum of all a person’s varying response styles to varying stimuli (Swanepoel 1995:1). In practical terms, however, it is impossible to measure and evaluate every one of a person’s possible responses to every possible stimulus.
The purpose of this study is to determine whether the Discus instrument may be considered a reliable and valid instrument for assessing personality.
3.1 Reliability
In the reliability exercise, the test-retest reliability technique was used. According to this method the same instrument is applied to the same respondents at a later stage and the correlation between the two scores is then calculated (Huysamen 1980:54; Mulder 1981:211).
The questionnaire was administered by the respective people participating in the exercise. All of these participants are trained in Discus and how to administer the instrument. The instrument was administered for the first exercise to obtain a pretest score. The exercise was then repeated with the same respondents after a period of three months in order to obtain a post-test score.
A statistical evaluation of the raw data, resulting from the exercise, was then obtained by using the SAS system, reflecting Pearson’s Product-moment correlation coefficient (coefficiency of stability).
3.1.1 The Questionnaire
The questionnaire consists of 24 questions each of which presents the respondents with four options. The respondents’ task is to select one of the options that most closely resembles themselves, and one that least closely describes them. The respondents are required to focus on the role they fulfil in their work environment and answer all the questions in relation to that role.
- Phrase-based
The phrase-based question set contains questions of the form ‘Behaving compassionately towards others’ or ‘Persuading others to your point of view’. - Adjective-based
The adjective-based question set contains words such as ‘kind-hearted’, ‘persuasive’ and ‘modest’.
For the purpose of this exercise the phrase-based questionnaire was used because it is easier to understand.
3.1.2 Sampling Technique and Size
Various companies were approached to assist with the exercise as reflected in table 1.
The questionnaire was administered to 90 respondents. These respondents were randomly selected from the respective companies reflected in table 1. A statistical evaluation of the raw data resulting from the testing was then obtained by using the SAS system reflecting Pearson’s product-moment correlation coefficient.
3.1.3 Results
The correlation analyses are reflected in table 2.
The questionnaire was administered to 90 respondents. These respondents were randomly selected from the respective companies reflected in table 1. A statistical evaluation of the raw data resulting from the testing was then obtained by using the SAS system reflecting Pearson’s product-moment correlation coefficient.
3.1.3 Results
The correlation analyses are reflected in table 2.
The significance level chosen for this instrument is alpha = 5%. Where the p-value is less than 0.05, the scores show a significant correlation. In the reliability analysis the p-value in all the cases is as low as 0.0001. This indicates significance at alpha = 0.001. It can therefore be said the correlation is significant at 1% level.
The reliability coefficient of the measuring instrument is close to 1 and can therefore be seen as reliable.