Online Research: New Challenges and Opportunities

I delivered a guest lecture this week on online research - both online methodology and researching online phenomena. Below are the slides from my presentation and a coding exercise of online discourse I had the students do.


 

Online Research: New Challenges and Opportunities from Glen Farrelly

Methods to Study Digital Media

I'm teaching a class next week on "Online research: New challenges & opportunities". In preparation, I was thinking of all the ways to study online (and by this I mean both Internet and mobile) phenomena and to use online methods to study other things.

As I love typologies, I thought I'd prepare one on this.  Although such a list doesn't express the complexity of online research, I thought it would demonstrate the diverse ways to investigate digital media.

Online methods to examine online phenomena:

• computer-captured and compiled data (e.g., web metrics)
• email, virtual reality, VoIP telephone or video call, or instant messaging interviews
• diarying or user logs
• remote observation (participant or nonparticipant)
• online focus groups
• web-based or email surveys
• audience response systems
• remote user testing or experiments
• autoethnography (can be conducted through blogging)

Methods to analyze online phenomena:

• social network analysis
• semiotics or visual analysis
• content analysis
• discourse analysis 
• hermeneutics
• ethnography

Online methods to study offline phenomena:

• web-based or email surveys
• email, virtual reality, VoIP, or instant messaging interviews
• diarying (e.g. through special software or blogs)
• photo documentation via mobile device

Offline methods to study online phenomena (which may or may not involve having users interact with digital media while capturing data):

• face-to-face interviews or focus groups
• nonparticipant observation
• contextual inquiry
• verbal protocol analysis (talk-aloud method)
• eye-tracking studies
• user testing or experiments

As a colleague pointed out there's also design science and participatory design that involves creating digital media as research tool - this type of research could be place in every category.

Please let me know if I missed a major method, as I'd like this typology to evolve iteratively.

Research Proposals Methodology Checklist

Earlier this month, I finished work as a teaching assistants for my department's research methods class. I've TAed this for the past 2 years and I love it. There are so many (almost countless) research methods and planning a research project is probably the most fun part of the whole thing.

It's also great to read about what the masters students plan to study and the various dimensions of research possible in the field of Information. I've graded a large number of research proposals now and have been impressed by students' approaches and their research interests. However, even in the best proposals they are often missing key elements.

I put together a checklist of items to include in a research proposal for a guest lecture I did for the class. Some students found it useful, so I thought I'd share it here. Surprisingly, I haven't found a concise checklist like this in research methods texts.

I'm covering the major items that apply to common methods, such as interviews, observation, focus groups, discourse analysis, and surveys.

Sampling

• Population - describe the characteristics of the group you will be studying, mention any pertinent demographic (e.g. age, gender, location, occupation) and pyschographics (habits, attitudes, hobbies, opinions) -  if studying texts or artifacts, describe the unifying characteristics
• Sampling strategy - e.g. census, random, stratified random, or nonprobability sampling such as  convenience, purposive diversity/dissimilar, snowballing, key informant, etc.
• Sample size - number and if this number is appropriate (or the stoping point, e.g. saturation)
• Access - how you will get access to this group and permission to study them
• Recruitment techniques to be used - e.g. posters, web posts, emails, etc.
• Incentives - if using why is it necessary, how much, and how to be distributed
• Inclusion and exclusion criteria - conditions to be met to qualify or exempt people or texts from study

Method considerations

• Format - online, print, email, instant messaging - which one(s) and why
• Interview guide or questionnaire - how will questions be determined (if appropriate)
• Questioning style (e.g. structured, semi-structured, conversational for interviews) and question format (i.e. open or closed ended)
• Probing or follow-ups if used and when (e.g. during or after)
• Discussion aids - will think-aloud protocol be used, artificts (e.g. diaries, photos) or cultural probes
• Rapport - how to establish trust, confidence, and ease amongst participants

Session preparation

• Meeting location details - discuss any pertinent details such as the type of place to be held and why (e.g. convenient for participant, comfortable setting), logistics, arrangement of furniture, noise levels, position of camera, presence of others, etc.
• If communicating or observing online discuss the technology to be used and any applicable norms or constraints

Data collection

• Recording of session - audio or video taping, photodocumenting, log files
• Note-taking - during or after session?
• Third-party observers or facilitators - whether or not used, relationships to researcher, training provided, and any issues that may result
• Role of researcher - researcher biases, how will the researcher be involved in and shape data collection and any steps to mitigate or place the researcher (e.g. passive observer or participant observer)

Data analysis

• Transcription - style to be used (e.g. naturalistic, selective) and whom will do it
• Approach to data analysis (e.g. grounded theory, statistical tests, etc.)
• Software - name and describe how used
• Coding technique used and how codes determined
• Reliability measures - particularly if more than one person is coding results

Data presentation & dissemination

• Anticipated findings
• Presentation formats - e.g. case study, charts, narrative, performance, etc.
• Outcomes - e.g.recommendations, program evaluation, etc.
• Dissemination - planned conference presentations, sharing among an applicable association, publication plans, etc.
• How might the findings be shared other than a traditional paper (if applicable)?
• Sharing results with participants - e.g. send them the final paper and - will they get permission to edit?, post summary on blog,or participant community (e.g. trade association)

Ethics

• Considerations when studying a given group
• Informed consent - how it will be obtained and special situations (e.g. minors or other who may not be able to provide it)
• Deception - if used, explain why necessary
• Harm - plans to avoid any emotional or physical harm to participans, e.g. if sensitive topics raised how will this be handled, referals to specialists, etc.
• Debriefing plans (if applicable)
• Anonymity or confidentiality - steps taken in data collection,storage, and presentation to protect participants' privacy

Obviously an entire book, or several, could be written on this topic. This is meant to touch upon the major and common areas. But please let me know if I missed something crucial.