1.10. Analyse and Process the Information Collected to Draw Valid Conclusions

Every day we are bombarded with problems to solve and decisions to make. And the quality of our solutions and decisions is only as good as the information they are based on. With so much information at our fingertips, how can we systematically analyse the information for better results? How can we judge the accuracy of certain pieces of information that are floating everywhere that everyone is using? The ability to analyse information prevents professionals from becoming overwhelmed by the volume.

We have to constantly analyse the data gathered, so the information requested by businesses is accurate, streamlined, and not overwhelming. 

Analysing information involves examining it in ways that reveal the relationships, patterns, trends, etc. that can be found within it. That may mean subjecting it to statistical operations that can tell you not only what kinds of relationships seem to exist among variables, but also to what level you can trust the answers you’re getting.  It may mean comparing your information to that from other groups (a control or comparison group), to help draw some conclusions from the data.  The point, in terms of your evaluation, is to get an accurate assessment in order to better understand your work and its effects on those you’re concerned with, or in order to better understand the overall situation.

There are two kinds of data you’re apt to be working with, although not all evaluations will necessarily include both.  Quantitative data refers to the information that is collected as or can be translated into, numbers, which can then be displayed and analysed mathematically.  Qualitative data are collected as descriptions, anecdotes, opinions, quotes, interpretations, etc., and are generally either not able to be reduced to numbers, or are considered more valuable or informative if left as narratives.  As you might expect, quantitative and qualitative information needs to be analysed differently.  

Quantitative data. As we’ve discussed, quantitative data are typically collected directly as numbers.  Some examples include:

• The frequency (rate, duration) of specific behaviours or conditions 
• Test scores (e.g., scores/levels of knowledge, skill, etc.) 
• Survey results (e.g., reported behaviour, or outcomes to environmental conditions; ratings of satisfaction, stress, etc.) 
• Numbers or percentages of people with certain characteristics in a population (diagnosed with diabetes, unemployed, Spanish-speaking, under age 14, grade of school completed, etc.) 

Data can also be collected in forms other than numbers and turned into quantitative data for analysis.  Researchers can count the number of times an event is documented in interviews or records, for instance, or assign numbers to the levels of intensity of an observed event or behaviour.  For instance, community initiatives often want to document the amount and intensity of environmental changes they bring about – the new programs and policies that result from their efforts. Whether or not this kind of translation is necessary or useful depends on the nature of what you’re observing and on the kinds of questions your evaluation is meant to answer.  

Quantitative data is usually subjected to statistical procedures such as calculating the mean or average number of times an event or behaviour occurs (per day, month, or year).  These operations, because numbers are “hard” data and not interpretation, can give definitive, or nearly definitive, answers to different questions.  Various kinds of quantitative analysis can indicate changes in a dependent variable related to – frequency, duration, timing (when particular things happen), intensity, level, etc. They can allow you to compare those changes to one another, to changes in another variable, or to changes in another population.  They might be able to tell you, at a particular degree of reliability, whether those changes are likely to have been caused by your intervention or program, or by another factor, known or unknown.  And they can identify relationships among different variables, which may or may not mean that one causes another.

Qualitative data. Unlike numbers or “hard data”, qualitative information tends to be “soft,” meaning it can’t always be reduced to something definite.  That is in some ways a weakness, but it’s also a strength.  A number may tell you how well a student did on a test; the look on her face after seeing her grade, however, may tell you even more about the effect of that result on her.  That look can’t be translated to a number, nor can a teacher’s knowledge of that student’s history, progress, and experience, all of which go into the teacher’s interpretation of that look.  And that interpretation may be far more valuable in helping that student succeed than knowing her grade or numerical score on the test.

As explained above, qualitative data can sometimes be changed into numbers, usually by counting the number of times specific things occur in the course of observations or interviews, or by assigning numbers or ratings to dimensions (e.g., importance, satisfaction, ease of use).

The challenges of translating qualitative into quantitative data have to do with the human factor.  Even if most people agree on what 1 (lowest) or 5 (highest) means in regard to rating “satisfaction” with a program, ratings of 2, 3, and 4 may be very different for different people.  Furthermore, the numbers say nothing about why people reported the way they did.  One may dislike the program because of the content, the facilitator, the time of day, etc. The same may be true when you’re counting instances of the mention of an event, such as the onset of a new policy or program in a community based on interviews or archival records.  Where one person might see a change in the program he considers important another may omit it due to perceived unimportance. 

Qualitative data can sometimes tell you things that quantitative data can’t.  It may reveal why certain methods are working or not working, whether part of what you’re doing conflicts with participants’ culture, what participants see as important, etc.  It may also show you patterns – in behaviour, physical or social environment, or other factors – that the numbers in your quantitative data don’t, and occasionally even identify variables that researchers weren’t aware of. 

It is often helpful to collect both quantitative and qualitative information. 

Quantitative analysis is considered to be objective – without any human bias attached to it – because it depends on the comparison of numbers according to mathematical computations.  Analysis of qualitative data is generally accomplished by methods more subjective – dependent on people’s opinions, knowledge, assumptions, and inferences (and therefore biases) – than that of quantitative data.  The identification of patterns, the interpretation of people’s statements or other communication, the spotting of trends – all of these can be influenced by the way the researcher sees the world.  Be aware, however, that quantitative analysis is influenced by a number of subjective factors as well.  What the researcher chooses to measure, the accuracy of the observations, and the way the research is structured to ask only particular questions can all influence the results, as can the researcher’s understanding and interpretation of the subsequent analyses.