How to Evaluate and Diagnose Organizations

Written by johntalaguit
Sections of this topic
    Coworkers working together in an office

    Much of the content
    of this topic came from this book:
    Consulting and Organization Development - Book Cover

    © Copyright Carter McNamara, MBA, PhD, Authenticity Consulting, LLC.

    Strongly Suggested Pre-Reading

    Organizational Performance Management

    Sections of this Topic Include:

    Description

    (Be sure to read the description in Organizational Performance Management to understand where organizational evaluation typically fits into the cycle of activities in ensuring strong performance in an organization.)

    You always want your organization to be operating at an optimum level, whether the organization is your family or where you work for a living. But what does an optimum level really look like? How do you know whether your organization is already at that level — or could it be doing even more?

    As you read in Organizational Structures, there are numerous types of organizations. All of them are systems and yet they have unique characteristics among them. Also, there are vastly different personalities of the people within them. So how do we ensure our organizations are always high-quality? We can’t even seem to agree on what organizational effectiveness is.

    Experts have suggested various best practices and standards of excellence for their respective type of organization, whether they focus on families, teams, or overall organizations. However, there is not a strong consensus about whether their suggestions are the right ones for that type and everybody in that type.

    Yet, it is not enough to proclaim that there is no standard, and so evaluations are not worth the effort. Perhaps the benefit of evaluations is not the consistent conclusions that they come to, but rather what we learn about the organizations and ourselves along the way.

    Indeed, organizations are unique. That is why the approaches to evaluating them should be highly customized, as well. This topic in the library aims to help you carefully customize and implement your own plan for evaluating your organization.

    The purpose of the information in this topic is to convey the core concepts in evaluating organizations. Your proficiency in the concepts would come from applying them over time, especially under the guidance of a person who is highly experienced in applying them, as well.

    What is an Organizational Evaluation and Diagnosis?

    An organizational evaluation includes the activities to improve an organization, usually by comparing the quality of its operations to some standard of high performance (this is an organizational assessment) and then recommending what changes should be made in order to bring that quality up to that standard (this expands the assessment into an organizational evaluation).

    Leaders and managers are often doing some form of organizational evaluation. Perhaps they are doing them implicitly and in a manner that unfolds as they go along. In that situation, they might not view their activities as doing an “organizational evaluation” at all.

    Or, they might be following a carefully designed evaluation plan that specifies desired results to be achieved by a certain time, a standard of quality in achieving the results, methods of measuring progress toward the results, and guidelines for rewarding the achievement of the results — or helping those who somehow are behind.

    The latter approach is very similar to a research project because it includes clarifying a research question to be answered, for example, “How can we improve our organization?” or “What is the real cause of our ongoing problems with cash flow?” Then it includes planning what data to collect and how to analyze it in order to come to careful conclusions and recommendations.

    You might decide to hire an organizational consultant to help you. In that case, the consultant would very likely go through the phases described in How to Consult in Organizations, especially the Discovery Phase of Consulting. Some of the information on this topic about organizational evaluation references certain topics in that phase.

    The information in this overall topic explains how to do the latter, including how to plan and implement your organizational evaluation, including how to report your findings. An example is included near the end, along with typical findings and recommendations that the example might produce.

    Benefits of Organizational Evaluation and Diagnosis

    Sometimes people have a negative reaction to the term “evaluation.” That is an unfortunate reaction because there are many benefits from evaluation. Actually, the process of evaluation is similar to the process of how adults learn, so there can be tremendous learning generated during evaluations. It might be useful for you to reference the following benefits when explaining how evaluation is important. The benefits of an evaluation include the it:

    • Mobilizes employees for organizational change as they feel their opinions are being heard and respected.
    • Facilitates meaningful communication among participants – perhaps the most important benefit of the evaluation process.
    • Cultivates realistic expectations for change as participants continually think about the organization’s situation and what can realistically be done about it.
    • Enhances learning for participants as they continue to collect and reflect on feedback about the organization’s performance and their role in it.
    • Improves performance as participants continue to make adjustments to what they are doing based on the results of their learning.
    • Improves the organization’s credibility among its stakeholders at a time when organizations are always competing for a positive image.

    Guidelines for Successful Evaluation

    • Ensure the evaluation design matches the nature and needs of your organization. One of the best ways to ensure a close match is to involve members of the organization as much as possible in the design and implementation of the evaluation plan. That highly collaborative and participatory approach can ensure their strong buy-in and participation in the evaluation. They typically would not be involved in collecting data later on during the evaluation because that could inhibit other members from sharing honest opinions.
    • Discuss evaluation with members in the early phases of the evaluation. The best forms of evaluation include the commitment and participation of members of the organization. Therefore, it is best if you involve them as soon as possible in the design and implementation of evaluation plans. Also, the sooner that you discuss the evaluation, the sooner that you can begin collecting useful information for the evaluation. In the early discussions, mention the benefits of evaluation.
    • Focus on relevance, utility, and practicality as much as on “scientific” priorities. Scientific priorities are in regard to accuracy, validity, and reliability. However, far too many highly “scientific” evaluation reports sit collecting dust on shelves because the reports have little utility and practicality and, thus, little relevance to the readers of the reports. Therefore, it is far better to err on the side of less scientific value and more on the usefulness of the report by focusing on relevance, utility, and practicality.
    • Integrate organizational evaluation with other ongoing evaluations in the organization. Many organizations conduct various forms of evaluation as part of their ongoing management activities, for example, as part of ongoing employee performance management, program evaluations and strategic planning. Integrating your organizational evaluation with these other activities in the organization helps members to leverage their evaluation activities and, thus, save time and energy.
    • Include a mix of methods to collect information. For example, review relevant documentation such as strategic plans, policies, procedures and reports. Then administer practical questionnaires to quickly collect information anonymously if appropriate. Follow up your questionnaires with various interviews. Interviews might be closed (by asking specific questions that evoke specific answers) or open-ended (asking general questions that can evoke a wide range of responses) and with individuals or groups.
    • Place high priority on capturing learning during evaluations. Learning involves gaining new knowledge, skills, or perspectives – learning is not merely finding new things to do. The best forms of learning from evaluations are those that are focused on solving the problems or achieving the goals that are the primary focus of your organizational evaluation.
    • Share learning from evaluations as soon as you have them. There is a tendency to put off acting on the results or conclusions from evaluations until the evaluation has been finished. That approach treats the evaluation like a “black box” that should not be tampered with until it is over. That approach also minimizes the tremendous value that evaluations have for making continuous improvements. By promptly sharing results, you can make ongoing adjustments to the evaluation to ensure that it remains high quality. You also help to ensure strong buy-in among members.

    Preparation

    Adjust for Your Personal Biases

    All of us have biases or natural ways that we automatically perceive and interpret things in the world, including how we come to conclusions about them. Many times, we are not aware of those biases, despite the significant role they play in what we see and do not see. Biases can significantly affect what you conclude as being important and unimportant in an organizational evaluation. Meanwhile, another person with different biases might have very different conclusions about the same organization. See

    Don’t Design Your Evaluation Plan Alone

    Instead form a small Project Team of members to share their expertise, time, and attention to planning the evaluation. The best team includes some members of the organization that is being evaluated. They can answer preliminary questions about the organization, especially about its culture. They can review drafts of documents from the evaluation to be sure they are understandable. Perhaps most important, they can help other members of the organization to see the evaluation as coming from one of their own.

    Train the team members about the basics of organizational evaluation. Be clear about their roles in the evaluation, especially to work with the evaluator to design the evaluation plan. If the evaluation is to be done by the team members themselves, then discuss how that might adversely affect participation in the data collection.

    Critical Role of Diagnostic Models in Evaluations

    An evaluation could collect a vast range of information. However, unless there is some framework around which to know what information to collect and how to make judgments about that information, the analysis will likely become a very overwhelming and confusing endeavor. This is where a diagnostic model is very useful. A good diagnostic model will:

    1. Suggest some standard of performance about how a high-quality organization should be operating, including the quality of its overall intended outcomes, practices within the organization, and how those practices are integrated with each other. The standards might be, for example, best practices or standards of excellence.
    2. Suggest what types of information need to be collected in order to compare the current performance of the organization with the suggested standard of performance.
    3. Facilitate the comparison of the current performance of the organization to the preferred standard of performance in order to generate recommendations to improve the performance of the organization.

    You can select your diagnostic model early on when designing your evaluation plan in the next topic. There are numerous types to choose from. Guidelines for selecting diagnostic models are included in the next section.
    Some Types of Organizational Diagnostic Models.

    Design Your Organizational Evaluation and Diagnosis Plan

    As you and your Project Team answer the following questions to design your plan, fill in this Template for Designing Your Organizational Evaluation and Diagnosis Plan.

    1. Who are the primary audiences? For example, is the information for Board members, management, employees, investors, or customers? Each of these audiences might have very different interests in the evaluation and its results. Also, each might require that the evaluation results be organized and presented in a certain manner to be most useful to them.
    2. What are the primary purposes of the organizational evaluation?
      For example, the purpose of the evaluation might be to answer certain management questions, such as: How do we find the cause of our recurring cash problems? Or, what strengths and weaknesses do we have to include in the SWOT information during our strategic planning?
    3. What types of information are needed? For example, if you want to find the causes of the cash problems, then you might need to get information, especially about organizational sustainability, for example, revenue, expenses, quality of products and services, how well the marketing is being done, and how realistic the vision and goals are for the organization. The information you collect could be suggested by an organizational diagnostic model. See Free Online Organizational Assessment Tools for Businesses and Free Online Assessment Tools for Nonprofits for collecting information. You will need a diagnostic model to analyze the information. See Some Types of Organizational Diagnostic Models.
    4. From what sources should the information be collected?
      For example, should it be collected from individual employees, individual customers, groups of customers, groups of employees, or also from program documentation? Attempt to get information from people in the focus of your evaluation, but also those who are most affected by that area. See Research Planning and Data Collection.
    5. What are the best methods to collect the information? Can you get the information by reviewing documentation, using questionnaires, and conducting interviews? Are there assessment instruments that you can use? Should you use an instrument that has already been developed or should you develop your own? Is it best to use a mix of these methods? See Selecting Which Business Research Methods to Use and How to Select from Among Public Data Collection Tools.
    6. What context-sensitive considerations must be made? Each organization has unique features, such as its culture, nature of leadership, rate of change in its environment, nature of programs and services, and size. How do these features influence how you will gather your information? See What Makes Each Organization Unique?
    7. What is the best time for getting the information? Do you need to provide a report by a certain date? Are there problems that need to be addressed right away? How often can you get access to the sources of the information that you need? How long will it take to collect information?
    8. Who should collect the information? Ideally, someone from outside the organization does the information collection, analysis, and reporting. That approach helps to ensure that the evaluation is carried out in a highly objective and low-bias manner. However, it is often unrealistic for small- to medium-sized organizations to afford an outside evaluator. Consequently, it is important to select personnel who can conduct the evaluation in a manner as objectively as possible. Equally as important is to ensure that the evaluation is designed to collect the most important information as quickly as possible. See How to Successfully Hire and Work With an Excellent Consultant.
    9. How will you analyze the information? Analysis usually includes comparing the results of the assessment to the standard suggested by the diagnostic model. See Analyzing, Interpreting and Reporting Basic Research Results.
    10. How will you make interpretations and generate recommendations? Your recommendations will likely be about the kinds of activities (suggested by the diagnostic model) that the organization needs to do to achieve the standard (suggested by the diagnostic model). See Traits of Useful Recommendations and Generating Recommendations Based on Results of Discovery used by consultants. You also might recommend which strategies they need to evolve to the next stage of organizational development. See Basic Overview of Life Cycles in Organizations.
    11. How will you report the information? How the evaluation results are reported depends on the nature of the audience and the decisions that must be made about the evaluation. For example, you might provide an extensive written report or a presentation to a group of people. See Contents of a Research Report — An Example and Sharing Findings and Recommendations With Clients and
    12. Should you test the design of the evaluation before you use it with many people? Depending on the complexity of your plans, you might benefit from field-testing them by using data-collection tools with a certain group of people to discern if the tools are understandable to them.
    13. What ethical considerations must be made? For example, do you want to report any information unique to any of the participants? If so, then you should get their expressed and written consent.

    It might be useful to review an additional perspective on designing research plans.
    Planning Your Research

    (The manner in which those recommendations are implemented is out of the scope of the activities in an organizational evaluation and is more a matter of the activities in guiding and supporting organizational change. See Guidelines, Methods, and Resources for Organizational Change Agents).

    Implement Your Organizational Evaluation and Diagnosis Plan

    Announce the Evaluation to Members of the Organization

    To increase the number of respondents in your data collection, it is critical that it maintains their strong and ongoing buy-in. Probably the most critical point in which to start cultivating that is when first announcing the evaluation to others in the organization. The announcement must be done carefully to help employees quickly understand and appreciate the need for the evaluation – so that they do not react to it as just another fad or “silver bullet” intended to save the day. Here are some suggestions to consider.

    1. The Chief Executive Office and a Board member should announce the evaluation to the employees (in the case of a small to medium-sized corporation). They should mention:

    • Its purpose and benefits
    • How the employees are expected to participate in it
    • When they will get the results of it
    • How they can share their ongoing feedback about how its recommendations will be implemented
    • The members of the Project Team, especially the members of the organization
    • A primary contact person, if they have any further questions

    Special care should be given to ensure sufficient time for reactions, questions, and suggestions.

    2. Accompany the announcement with an official memo. The memo should soon follow the announcement and be signed by upper management. It should reiterate the information shared during the announcement.

    Prepare Participants Before Data Collection

    Carefully prepare those who will be providing responses to the evaluation — you should not start simply by asking them for input. Consider the following guidelines.

    1. Management should introduce the evaluator(s) to the organization.
    One of the most powerful ways to do this introduction is in a group, for example, in an employee meeting. The introduction should include the evaluator’s description of how information will be collected, along with any terms of confidentiality. Include time for their questions and suggestions.

    2. Tell participants what is expected of them during the evaluation. Explain how information will be collected, and when and how they can participate. Mention any pre-work that would be useful for them to undertake and any topics or activities that they should think about before participating in the evaluation.

    3. Contact each participant before conducting any interviews. Interviews can be a rather personal way to get useful information. It helps a great deal if the evaluator calls each participant before the actual interview in order to introduce themselves, verify the timing of the upcoming interview and understand if the participant has any questions.

    4. Review useful organizational documentation before contacting anyone. The review of documentation is a major form of data collection. The evaluator can learn a great deal about the organization from the documents. That understanding is an advantage because participants soon realize that the evaluator already knows a great deal about the organization. See How to Review Documentation.

    If You Encounter Questionable or Illegal Practices

    See a video about principles for successful consulting, defining “success”, principles for ethical consulting, managing risks and liabilities, and knowing when to leave. From the Consultants Development Institute.

    Occasionally, an evaluation uncovers organizational activities that seem immoral, for example, a violation of your professional standards and those in society, significant lies in the workplace or intentionally withholding very useful information from others. You might even encounter activities that are illegal, such as misappropriation of funds, fraud, theft, or violation of employment laws. This video shares guidelines for how to deal with those kinds of situations.

    Sharing Findings and Recommendations from the Evaluation

    There is a variety of common mistakes made when sharing the results of research, including organizational evaluations. As mentioned above, the nature of activities in an organizational evaluation is very similar to that done in the Discovery Phase of consulting. Thus, the following guidelines for sharing results in that phase also apply here.
    Sharing Findings and Recommendations With Clients

    Example Application of a Diagnostic Model for a Systems Analysis

    Assessment Tool and Associated Diagnostic Model

    Later on, below are two sets of a sample assessment and an associated diagnostic model – one set for for-profit and another for nonprofits. The assessment tool can be used to assess the quality of activities in each of the major functions in an organization, such as strategic planning, human resources, and financial management. Its results can suggest which functions need to be improved. This diagnostic tool can suggest in which order to improve them. First, read the following section to learn how to use both tools.

    How to Use the Tools for Systems Analysis

    In a topic in Understanding Organizations, you learned that an organization is a system that is comprised of other systems — of other subsystems. They all are highly integrated with each other. The outputs of some are the inputs of others. This graphic displays that type of integrated and systematic relationship. So, if management reports a problem with a particular function, for example, with a certain program, then a good systems analysis would look at the inputs to that program (for example, strategic planning) and the outputs from the program (for example, its effect on finances) and ensure that each of the three has high-quality activities as indicated by the results of the assessment tool.

    If there is a problem with the quality of the strategic planning activities (for example, they have very unrealistic expectations of programs), then that could be a major cause of the recurring problems with the programs. If there are recurring problems with finances, then the problems with the programs could be what is causing the problems with the finances. Thus, this systems approach can also suggest in order to improve the various functions in an organization.

    A key benefit of the following assessments and diagnostic models is that they reference various terms that are usually quite familiar to leaders in the organization and, thus, can be very helpful when orienting them to the tool and getting their buy-in to the diagnostic process.

    Sample Assessment and Diagnostic Model for For-Profit Organizations

    1. Assess the for-profit organization using this assessment tool.
    Assessment of For-Profit Organizations

    2. Then use this diagnostic tool to identify which management functions need to be improved and in which order.
    Systems-Based Model to Diagnose For-Profit Organizations

    Sample Assessment and Diagnostic Model for Nonprofit Organizations

    1. Assess the nonprofit organization using this assessment tool.
    Assessment of Nonprofit Organizations

    2. Then use this diagnostic tool to identify which management functions need to be improved and in which order.
    Systems-Based Model to Diagnose Nonprofit Organizations

    Potential Types of Issues Found in Organizations

    The list of issues in each of the following documents focuses especially on functions within each organization. They are the types of issues that might be analyzed with the above-mentioned assessments and diagnostic tools.

    Common Types of Recommendations to Improve Organizations

    The list of recommendations in each of the following documents focuses especially on the functions within each organization. They are the types of recommendations that might be included in an assessment report about the results of using the above-mentioned diagnostic tools.

    Free, Online Organizational Assessments Tools for For-Profits

    Selecting from Among Publicly Available Assessments

    Evaluating Overall For-Profit Organizations

    Evaluating Various Management Functions (For-Profit)

    Free, Online Organizational Assessments Tools for Nonprofits

    Selecting from Among Publicly Available Assessments

    Evaluating Overall Nonprofit Organizations

    Evaluating Various Management Functions (Nonprofit)

    Next Steps?

    Your organizational evaluation might have identified various aspects of your organization that you want to improve. The next topic that will be useful to you is
    Improving Organizations: Guidelines, Methods, and Resources for Organizational Change Agents

    Suggested Additional Readings

    Learn More in the Library’s Blogs Related to Organizational Performance

    In addition to the articles on this current page, also see the following blogs that have posts related to organizations. Scan down the blog’s page to see various posts. Also, see the section “Recent Blog Posts” in the sidebar of the blog or click on “Next” near the bottom of a post in the blog. The blog also links to numerous free related resources.

    For the Category of Organizational Development:

    To round out your knowledge of this Library topic, you may want to review some related topics, available from the link below. Each of the related topics includes free, online resources.

    Also, scan the Recommended Books listed below. They have been selected for their relevance and highly practical nature.


    johntalaguit