Canadian Social Science

A Development of the Evaluation Model for Faculty Organizational Effectiveness in Public Universities

Samaporn Sirilap[a], *; Prawit Erawan[b]; Prasert Ruannakarn[c]

[a] Faculty of Education, Mahasarakham University, Thailand.

[b] Associate Professor, Mahasarakham University, Thailand.

[c] Assistant Professor, Mahasarakham University, Thailand.

*Corresponding author.

Received 17 July 2012; accepted 12 September 2012

Abstract

The purposes of this research were to: 1) studying current conditions, problems, and needs to develop, and studying appropriate elements and indicators of the evaluation for organization in public universities, 2) developing the evaluation model for faculty organizational effectiveness in public universities, and 3) evaluating the proposed evaluation model for faculty organizational effectiveness in public universities. The research was conducted through a process of research and development procedures consisting of 3 steps including: step 1. model’s investigation and analysis, from administration instructor officer and student total 729 persons from 36 faculty organization, step 2. model’s verification, from 7 experts about evaluation organization or quality assurance, and step 3. model’s implementation and evaluation, from assessor administrator instructor officer and stakeholders total 255 persons. Tools of research consisted of: questionnaires, Interview, and record observations. The statistics used in this study were percentage, mean and standard deviation. The results were as follow: 1) The results of the evaluation organization is currently and needs were found at a high level, the assessment problems summarizes 9 main points, and appropriate elements and indicators were found at a high level. 2) The results of the evaluation model for faculty organizational effectiveness found consisted of 8 sections: Principles of assessment present the objective assessment and evaluation model, Structure of assessment, Elements of assessment organizational effectiveness, Assessment methods consisted of evaluation process and audit process, Evaluation criteria and scoring system, Assessment tools and how to use them, The operating system to support the assessment and The use of the assessment results. 3) According to the evaluation of using the model, it was found that 3.1 Effectiveness of the organization Education Faculty, Mahasarakham University found at a highest level. 3.2 The results of the implementation were found at a moderate to high level. and 3.3 The evaluation results of appropriateness and the ability to use the evaluation were found at a highest level.

Key words: evaluation model; organizational effectiveness; public university

Samaporn Sirilap, Prawit Erawan Prasert Ruannakarn. (2012). A Development of the Evaluation Model for Faculty Organizational Effectiveness in Public Universities. Canadian Social Science, 8(5), 183-191. Available from: http://www.cscanada.net/index.php/css/article/view/j.css.1923669720120805.6353
DOI: http://dx.doi.org/10.3968/j.css.1923669720120805.6353.

INTRODUCTION

Traditionally, evaluation organizational effectiveness in Thailand, in higher education, follows The Guide to Quality Assurance in Higher Education edited in B.E. 2553 (Office of the Higher Education Commission (OHEC), 2011). It defines that the assessment focuses on the potential and the availability of education, and standards for the implementation of the mission of higher education institutions. Mostly, the education organization uses assessment criteria to define plans, projects, activities and goals of the organization as it is oblivious to the true effectiveness of the organization to achieve its aims. Traditionally, organizational effectiveness has been defined in relation to the degree of goal attainment. Organizational goals can be defined simply as the desired states that the organization is trying to attain. Goals provide direction and motivation, and they reduce uncertainty for participants and represent standards for assessing the organization. (Wayne K. Hoy & Cecil G. Miskel, 2008) and organization effectiveness is mainly a problem-driven construction rather than a theory-driven construct. (Kim Cameron, 2005, p. 313) and goals are used. A variety of purposes, diversity goals, strict monitoring and control to evaluate organizational activities as well as to motivate and direct them (Scott, 2003). indicate the importance of the effectiveness, especially in state universities. Consequently, in higher education we should have an evaluation model composite: aim, approach, performance, criteria, tools, define “What to expect”, together with comparison and reflection. The beget to the reflex performance and the true effectiveness of the organization. Based on the Goal-centered judgment to compare the performance of organizations using the indicator the goals of the organization. (Cameron and Whetten, 1983). Because to clearly define the aim made it necessary that we can clearly both define the operations and goals as well, and compare the performances and what to expect of the aim.

PURPOSES OF THE STUDY

• study current conditions, problems and needs of the evaluation for faculty organization effectiveness in public universities,

• develop the evaluation model for faculty organizational effectiveness in public universities,

• evaluate the proposed evaluation model for faculty organizational effectiveness in public universities.

METHODOLOGY

The research methodology used is research and development and is classified into three stages.

Step 1: models investigation and analysis by researcher conducting the study and analysis of the three parts including:

1) Analysis of the theories of the texts, articles and related researchs. Framing the research and synthesizing the evaluation organizational effectiveness,

2) studies of the opinions concerning the current conditions, problems and evaluate developing requirements, and studies of the appropriate elements and indicators from administrators, instructors, officers and students. They represent a total of 729 persons from 36 faculties from 6 large public universities and

3) evaluation of the model design (draft) of the organizational effectiveness for faculties< The results of the analysis consist of two parts.

Step 2: model’s verification by experts of the two parts including 1) verifying the appropriateness, accord and possibilities of the evaluation model by 7 experts who are professionals in evaluation or quality assurance in higher education, and evaluating the model by interview. 2) improving the evaluation model (draft) to make it more accurate and complete.

Step 3: model’s implementation and evaluation. It is the model used in the faculty organization in public universities. The purpose of the evaluation model focuses on reflecting the actual operation result. So, in stage 1 and 2 of the implementation, the faculties operated regularly by using the actual operation results from one past academic year. Stage 3 The assessment of the faculty consists of 4 steps: 1) Information check, 2) The actual location, 3) Analysis and conclusions and 4) Evaluation report, by using approach and instruments of the evaluation model reported by faculties, observations records by auditors, questionnaires about image, attachment, performances of graduates by instructors, officers, students, people and stakeholders. In addition, evaluation of the results of the model of those involved to investigate the impact on instructors, officers, and administrators and the model as a whole is made by using interviewing, questioning and the collecting of feedback for improving various components of the model in order to find defects on stage 4 and 5.

RESULTS

1) The results of the evaluation model is currently and needs to develop were found at a high level, the assessment problems summarizes 9 main points, and appropriate elements and indicators were found at a high level.

In addition, the assessment problems summarizes 9 main points with: The purpose of the assessment, The appropriateness of the valuation model, Composition assessment, An indicator of the evaluation, Determining the weights of evaluation, The scoring, The assessment tools, The use of the assessment results, and The problems and areas for improvement.

The result of the appropriate elements and indicators consisted of 7 elements as follow: 1) Management and resource allocation with 5 indicators, 2) Education and development of students with 8 indicators, 3) Development of academic, professional, and quality for students with 8 indicators, 4) Development of academic, professional, and quality for instructors and officers with 6 indicators, 5) Employment of administrators, instructors and officers with 6 indicators, 6) The open system interacts with the community with 6 indicators, and 7) Organization Health with 16 indicators, were found at a high level.

The researcher used data from step 1 for design evaluation model, and model’s verification from 7 experts. So evaluation model for faculty organizational effectiveness in public university has gone to confirm by experts. as follows

2) Evaluation model for faculty organizational effectiveness in public university found consisted of 8 sections as follow: (1) Principles of assessment present the objective assessment and evaluation model (2) Structure of assessment (3) Elements of assessment organizational effectiveness (4) Assessment methods consisted of evaluation process and audit process (5) Evaluation criteria and scoring system (6) Assessment tools and how to use them (7) The operating system to support the assessment and (8) The use of the assessment results detail as follows :

(1) The principles of evaluation showed the objective assessment and evaluation model consisted of 7 elements as follows:

• Aim (A): the aim of evaluation, designated in accordance with the aims of the organization, need to be clear and consistent with the actual work.

• Approach (A): Guidelines or conditions of what to expect, criterions, indicators, methods, tools and what is intended.

• Performance (P): Operating follows to design and collect performance data.

• Comparison (C): The performance is comparable to what is expected, or a means to evaluate organizational effectiveness.

• Reflection (R): The results of the evaluation report are to be given to the relevant authorities, such as the faculties, universities, federations or professional organizations.

• Current of education (C): Consideration and analysis of current legal, regulatory, political, economic, social and educational values.

• Improvement consisted (I): Improving and developing guidelines and conditions for assessing the effectiveness of the faculty organizations. As showed in Figure 1.

Table 1

The Level of the Evaluation Model Is Currently and Needs to Develop in the Public Universities

Evaluation element	current conditions			Needs to develop
	S.D.	Level		S.D.	Level
1. Principles for evaluation	3.75	0.88	high	3.86	1.00	high
2. Concept of evaluation	3.56	0.93	high	3.94	0.98	high
3. Objective for evaluation	3.58	0.92	high	4.03	0.96	high
4. Organizational evaluation structure	3.53	0.87	high	3.88	0.93	high
5. The process of organization evaluation	3.65	0.86	high	3.89	0.97	high
6. Tools to evaluate the organization	3.52	0.89	high	4.00	0.97	high
7. How to use the tool	3.57	0.93	high	3.96	0.98	high
8. To use the assessment results	3.53	0.95	high	3.78	0.98	high
Total	3.57	0.91	high	3.93	0.97	high

Figure 1

The Evaluation Model for Faculty Organization Effectiveness in Public Universities

Figure 2

Structure of Evaluation Effectiveness for Faculty Organization in Public Universities

4.2) audit process consisted of 4 steps that is 1. Information check 2. The actual location 3. Analysis and conclusions and 4. Evaluation report.

Figure 5

Assessment Process for Faculties in Public Universities

(5) Evaluation organization effectiveness criteria consisted of 5 levels that are: highest, high, moderate, obtainable and should be improved. The scoring system focus on results of process and results with faculty according to approach, deployment, learning, integration of process and level, trend, comparison, integration of result.

Figure 6

Evaluation Organization Effectiveness Criteria for Faculties in Public Universities

(6) Evaluation tools and how to use them consisted of 6 types:

• The performance report.

• Questionnaire about attachment to the faculty organization.

• Interview about operation the faculty organization.

• Record observing the atmosphere and environment of the faculty organization

• Questionnaire about faculty organization image.

• Questionnaire performance of graduates from the supervisors.

(7) The operating system supports the evaluation of organizational effectiveness in public university. It is guideline for Organization management and quality improvement of organization by used SIPOC model and Quality Assurance Triangle for improved to operations of the faculty.

Figure 7

The Sample to Operating System for Faculty in Public University

(8) To use the evaluation results for improving and development of strategies, operation plans, management, communication systems, solving problems, activities and project design. Focus on the mission of faculty with graduates manufacturing, research, academic services, and preservation of arts and culture, and professional standard as well.

3) The results of the evaluation model as follows:

• The results of evaluation the effectiveness of the Faculty of Education, Mahasarakham University were found on average at the highest level: the model can be a reflection of the actual performance of the faculty organization. It is confirmed by information from reference sources that have credibility and reliability.

• The results of implemented model were found: the administrators had an opinion about using the evaluation effectiveness at the high level, instructors and officers had overall opinion about using the evaluation effectiveness at a moderate level.

• Concerning the result of the model, their appropriateness and ability for evaluation for faculty organizational effectiveness were assessed, the assessors and administrators overall opinion were found at a highest level, instructors and officers overall opinion were found at a high level. In conclusion, the evaluation results of appropriateness and the ability for use the evaluation model were found at a highest level.

CONCLUSION

The results of this study are that the evaluation enables to measure the effectiveness of the organization’s department in the best manner and is appropriate. Because the model is consistent with the theory concept and the needs of users and experts. Therefore, this model should be used to reflect the actual performance and reflect the overview effectiveness level of the faculty organization in public universities to solve the problem of management for faculty organization in public universities.

For summarize, the evaluation model for faculty organizational effectiveness in public universities was developed by researcher can use to evaluate effectiveness organizations for faculty organization in public university.

DISCUSSION

The research findings should be of interest to the discussion, as follow: 1) Overall current conditions and needs to develop were found at a high level; this is probably because evaluation and quality assurance process cannot truly respond to the within demand, and cannot reduce vulnerability because most had been in needs to develop at a high level. In addition, education statute prescript a quality assurance or evaluation as part of the operation in educational institutions; therefore educational institutions action follow requirements in order to comply with regulations as stated in The Guide to Quality Assurance in Higher Education B.E. 2553 (Office of the Higher Education Commission (OHEC). 2011). It defined the assessment on focusing on the potential and the availability of education, and fix standards for the implementation of the mission of higher education institutions. Based on feedback found, there are two approaches that are a). The assessment is appropriate and comprehensive of the operation organization. b). Assessment was not appropriate, not covering and not consistent with the operation of the organization. It cannot indicate the effectiveness of the organization and to be the cause of assessment, probably because of the level of understanding of the different evaluation. So, the administrator will have to perform a management of quality and use communication, and definition of individual goals as well, in order to facilitate understanding. And it should be added that, to reflect the results of operations and to illustrate overview of the organization, the organizational effectiveness is able to reflect the ability of the organization as a whole. Therefore, it is important to assess organizational effectiveness, focusing on the feelings of the people within the organization. That is consistent with Small, Joan E. (2009) who found that effectiveness of nonprofit organizations is important for understanding the relationship between organizational performance and organizational characteristics that may affect performance. And that will increase the effectiveness of organizations by using the evaluation, which is consistent with Capps Patricia (2008), Chang, Ya-Hui Elegance (2010), Ludmila Praslova (2010), A. Gregory (1999), and Newman, David (2011),etc.

2) The elements and indicators appropriate for evaluating the organizational effectiveness consist of 7 elements: management and resource allocation, education and development of students, development of academic and professional quality for students, development of academic and professional quality for instructors and officers, employment of administrators, instructors, and officers; the system interacts with the community, and organization health. These are consistent elements for Cameron (1986), Kwan & Walker (2003), and Ruetinan Samuttai (2006), elements that reflect organizational effectiveness, but indicate difference because indicators present the needs of the organization. They also focus on finding the factors and relationships, affecting organizational effectiveness, which is consistent with Cameron (1978), Clott (1995), Kwan & Walker (2003), Parhizgari and Gilbert (2004) and Pounder (1995), etc.

3) The evaluation model for faculty organizational effectiveness consists of 8 compositions. This research focus on individuals associated with the organization, and their relationships within the organization, and on the feelings of the people in the organization. We have been using the people who reflect both internal and external organizations. Similarly, Ronald H. Heck (2005) found that schools with higher quality educational environments consist of a principal leadership, have high expectations, are frequently monitoring student progress and climate and that they produced higher than expected achievement gains. This is something which is consistent with Ruja Rodkhem (2004) who concludes that variables that influence organizational effectiveness consist of organizational climate, organizational culture, environment, organizational commitment and motivation. Yuwaranee Sukwinya (2006) found that organization effectiveness indicators consist of individual outcome such as performance, satisfaction and involvement, goal achievement, high performance, and stakeholders’ satisfaction. So, the tools that reflect performance of both internal and external organization are:

4) The implementation and evaluation. An effective evaluation model must be able to measure the effectiveness of the organization department at the highest level

Thus, the evaluation effectiveness model that the researchers created is appropriate. Results of the study and related documents will be helpful for administrators and other concerned parties to realize the importance of effectiveness evaluation and useful for administrators to control quality and manage their organization effectively. This gives organizations that are aware of result, image, attachment and credibility, an aim to operate with good results, and the ability to communicate with the public, the community and the society. And it deserves to be studied and developed continuously.

Suggestions for further research

Should be research the concept of quality assurance Quality Assurance Triangle for used to improve the quality service of the government and to increase organizational effectiveness studies further.

Should be research an indicators of the effectiveness organization to reduce to a minimum and the development of indicators that are clear concrete.

ACKNOWLEDGEMENTS

The author thanks Mahasarakham University, Thailand, for its grant.

REFERENCES

Gregory A. (1999). The Road to Integration. Reflections on The Development of Organizational Evaluation Theory and Practice. University of Humberside, Hull, UK.

Cameron, K. S. & Whetten, D. A. (1983). Organizational effectiveness: a Comparison of Multiple Models. New York: Academic Press.

Cameron, K. (1978). Measuring Organizational-Effectiveness in Institutions of Higher-Education. Administrative Science Quarterly, 23, pp. 604-632.

Cameron, K. (1986). A Study of Organizational Effectiveness and Its Predictors. Management Science, 32, 87-112.

Cameron, K. (2005). Organizational Effectiveness: Its Demise and Re-emergence through Positive Organizational Scholarship. In K. G. Smith and M. A. Hitt, Eds. Great Minds in Management: The Process of Theory Development (pp. 394-429). New York: Oxford University Press.

Capps, Patricia (2008). The use of Kirkpatrick’s four levels of evaluation by performance improvement practitioners (Doctoral Dissertation). Indiana University, USA.

Chang, Ya-Hui Elegance (2010). An Empirical Study of Kirkpatrick’s Evaluation Model in the Hospitality Industry (Doctoral Dissertation). Florida International University, USA.

Clott, Christopher B. (1995). The Effects of Environment, Strategy, Culture, and Resource Dependency on Perceptions of Organizational Effectiveness of Schools of Business. The Annual Meeting of the Association for the study of Higher Education 20th (Vol. 32, pp. 2-5). Orlando.

Heck, R. H. (2005). Examining School Achievement over Time: A Multilevel, Multi-group Approach. In W.K. Hoy and C.G. Miskel (Eds.), Contemporary Issues in Educational Policy and School Outcomes (pp. 1-28). Greenwich, CT: Information Age.

Hoy, Wayne K. & Miskel, Cecil G. (2008). Educational Administration: Theory, Research, and Practice (8thed). New York: McGraw-Hill.

Kwan, P., & Walker, A. (2003). Posting Organizational Effectiveness as a Second-Order Construct in Hong Kong Higher Education Institutions. Research in Higher Education, 44(6), 705-726.

Ludmila Praslova (2010). Adaptation of Kirkpatrick’s Four Level Model of Training Criteria to Assessment of Learning Outcomes and Program Evaluation in Higher Education. Anguard University, Southern California.

Lynne Miller Franco, et al. (2002). Sustaining Quality of Healthcare: Institutionalization of Quality Assurance. Quality Assurance Project.

Newman, David (2011). An Empirical Validation of Guskey’s Professional Development Evaluation Model Using Six Years of Student and Teacher Level Reading Data (Doctoral dissertation). Cleveland State University.

Parhizgari, A. M., & Gilbert, G. R. (2004). Measures of Organizational Effectiveness: Private and Public Sector performance. Omega, 32, 221-229.

Pounder, D. G., Ogawa, R. T., & Adams, E. A. (1995). Leadership as an Organization-Wide Phenomena: Its Impact on School Performance. Educational Administration Quarterly, 31(4), 564-88.

Rashad Massoud, et al. (2001). A Modern Paradigm for Improving Healthcare Quality. Quality Assurance Project.

Ruetinan Samuttai. (2006). Development of a Multi-dimensional and Integrated causal Model of Faculty Effectiveness in Higher Education Institutions (Doctoral Dissertation). Chulalongkorn University, Thailand.

Ruja Rodkhem (2004). The Development of an Evaluation Model for the Organizational Effectiveness of Colleges under the Ministry of Public Health By the Balanced Scorecard Approach (Doctoral Dissertation). Chulalongkorn University, Thailand.

Scott, W. R. (2003). Organizations: Rational, Natural, and Open Systems (5thed). NJ: Prentice Hall.

Small, Joan E. (2009). Determinants of Organizational Effectiveness and an Integrated Performance Evaluation Model for Nonprofit Organizations. D.P.A., University of Illinois, Springfield.

Yuwaranee Sukwinya (2006). A Development of the Organization Effectiveness Causal Model of Bursing Colleges under the Ministry of Public Health (Doctoral Dissertation). Chulalongkorn University, Thailand.

(2) Structure of evaluation followed on organizational structure and operational. As shown in Figure 3.

(3) Elements of evaluation organizational effectiveness consisted of 7 elements and 52 indicators. As shown in Figure 3.

• Management and resource allocation with 5 indicators: mobilizing students in the area to study at the Institute, mobilizing instructors to work in the Institute, mobilization of financial resources, the ability to provide resources, and success in acquiring research funding.

• Education and development of students with 7 indicators: satisfaction with life in learning, satisfaction with the education program, development opportunities for students to interact with groups of friends, the development of life skills, the development of moral and ethics, the extra-curricular activities, and a focus on the development of students as individuals.

• Development of academic and professional quality for students with 7 indicators: level of achievement, conformity of the graduate attributes to faculties/universities/professional organizations, efforts to learn, response level to the professional goals of the students, the importance of professional education, employment of graduates, have been highly appreciated by employers of graduates.

• Development of academic and professional quality for instructors and officers with 6 indicators: amount of academic development, professional attendance of instructors and officers, supporting the development of instructors and officers, works published by instructors being recognized and highly respected in academic circles, and the appropriate attribute of instructors.

• Employment of administrators, instructors and officers with 6 indicators: The level of attachment to the institution by instructors and officers, Satisfaction in hiring instructors / staff are happy in their work, The attraction of the research to instructors, Satisfaction with the course and the surroundings / The work of the instructors and officers, Satisfaction of the administrator, instructors and officers to the faculty, and The best employer, of the university

• The system interacts with the community with 6 indicators: Community services of personnel / enthusiasm for community service, Professional activities outside the institution / an emphasis on the needs of employers, Emphasis on community relations / sustain excellent relationships with industrial and other institutions, The organization is modified according to the context and environment, Linking culture and the learning development, and showing appreciation to the faculty in the cultural activities in the community

• Organizational health with 15 indicators: The relationship between students and faculty, Relations between each department, Environment for collaboration, Flexibility on management, Level of trust, Amount of conflict and anxiety, Solution approach, How to supervise and monitor quality, Procedures and the adequacy of awareness and rewarding, The method in decision making, Energy with the relations of organization participation, Long-term determination and plans, Focus on the importance of mentality, Survival of the institution, and Linking processes within the organization

(4) Assessment methods consisted of evaluation process and audit process detail as follows:

4.1) evaluation process consisted of 5 steps as follows: 1. Preparing for evaluation 2. Operation/ data collection 3. Evaluating organizational effectiveness 4. Reflect the results of evaluation and 5. Improvement and development.

Figure 3

Elements of Evaluation Organizational Effectiveness for Faculty in Public Universities

Figure 4

Evaluation Process for Faculties in Public Universities