Sunday, November 05, 2006

Survey Problems

I'm kicking myself. I just realized today what an inadequate job I did teaching the participants with certain parts of how to conduct surveys. For the past month I have been entering the survey data that they collected and coded and finding some significant, worrisome problems and contradictions with their data. I've been spotting the problems, explaining what the problems are to them individually, and asking them to go back and correct the problems. A couple days ago I finally got all the corrections back. This morning I started to re-enter the data into the spreadsheet in which I've been recording the results. I have spent hours recording the results, and the participants have probably spent dozens of hours coding them. Some of the people who corrected the problems decided to completely rewrite their results. The problem I noticed this morning was that the information on most of the respondents has completely changed since the participants revised the responses. For example, respondent number nine may have at first been a fifty year old man with a household of ten people. Upon revision, respondent number nine was suddenly a thirty year old woman with a household of two people. Major problem.
 
At this point I realize that there are some gaps in what information I gave them about surveying. We discussed administration of the surveys in quite a bit of detail, and I watched them administer the surveys. They did a great job. That's not the problem. The problem was recording the information and coding it. I did not give them near enough instruction on exactly how to do that. In retrospect I can clearly see why we're having the problems we are having now. I neglected to give them a template for recording the information, I did not give them classroom practice with recording, and I did not give them practice with coding.
 
The good part is that they administered the surveys well. They asked the right questions to respondents and listened carefully to their answers. They benefited from having the experience of each having talked to twenty-five people about the problems they have in the community. That is valuable and proved to be useful for identifying problems in the community. The bad part is that we may be unable to quantify that data and analyze it. I feel terrible considering how much time they have put into this. If we had more time, I would go back and properly reteach how to code, which would hopefully enable us to salvage the results of the survey. But at this point I've only got five weeks left. We can't afford to go back just to work on the surveys. We need to focus on getting the projects up and running.
 
I guess I can't expect to have done everything perfectly. Live and learn, right? But I still hate the fact that I wasted so much of their time to have to learn a lesson that I could have prevented by thinking things out better. I will say that I am fairly confident that we will not have this problem with any other things that I am teaching them in the workshops. Since the surveys I have been careful to think out every single thing that they need to know to be able to accomplish the task at hand, and I've modeled the process first, which makes a big difference.

0 Comments:

Post a Comment

<< Home