AI in Research Writing
Research workstation showing statistical output and coded transcripts

Part 2

Method & Data Sense-making

Key question: Where does AI make my analysis more rigorous — and where does it become misconduct?

The hardest line in the whole workflow

AI may help you design, justify, interpret and explain your analysis. It may never produce, extend, impute or adjust your data.

Everything below sits firmly on the safe side of that line — and it is precisely where most researchers under-use AI, because they only think of it as a writing tool.

Prompts to copy and try

1. Pressure-test your methodology

Best in Claude

Act as Reviewer 2 for a Q1 journal in [FIELD].

My study design: [DESCRIBE DESIGN, SAMPLING, INSTRUMENT, ANALYSIS]
My research question: [PASTE]

1. List every threat to validity in this design (internal, external, construct, statistical conclusion).
2. For each threat, say whether it is fatal, fixable before data collection, or acknowledgeable as a limitation.
3. Tell me the three questions a reviewer is most likely to ask that I cannot currently answer.
4. Suggest the minimum change that would most improve rigour without changing my scope.

Turns AI into a pre-reviewer. Finding the fatal flaw before data collection is worth more than any writing speed-up.

2. Justify your method choice in writing

Best in ChatGPT or Claude

I chose [METHOD, e.g. explanatory sequential mixed methods] for a study on [TOPIC] with [SAMPLE].

Write a 200-word methodological justification suitable for a journal Methods section that:
- links the choice to my research question and epistemological stance,
- names the two most credible alternative methods and states why I rejected them,
- uses hedged academic register, no marketing language.

Then list the seminal methodology sources I should cite for this choice (author + year + book/article title only) so I can locate and verify them myself.

The last line is the safe way to get citation leads: names to look up, never a ready-made reference list to paste.

3. Draft and refine an instrument

Best in ChatGPT

I need a [survey / interview protocol / observation rubric] measuring [CONSTRUCT] among [POPULATION].

1. Break the construct into dimensions, citing the theoretical model I named: [MODEL].
2. Draft 4-6 items per dimension on a [5-point Likert] scale.
3. Flag any item that is double-barrelled, leading, culturally loaded, or too abstract for this population.
4. Suggest 3 items that could serve as attention or consistency checks.
5. Note which items would need reverse coding.

Instrument drafting is generative work on your own construct — no data involved, and the item-flagging step catches classic design errors.

4. Interpret statistical output correctly

Best in ChatGPT (Advanced Data Analysis) or Claude

Here is real output from my analysis in [SPSS / R / jamovi]:
[PASTE OUTPUT TABLES]

Sample: [N, design, variables]

1. Explain in plain language what each coefficient, p-value and effect size actually means for my variables — not generic definitions.
2. State the assumptions this test requires and which of my output rows tell me whether they were met.
3. Write the results in APA 7 reporting format.
4. List three interpretations that would OVERSTATE these results, so I can avoid them.
Do not speculate beyond the numbers I gave you.

Point 4 is the quality move: most reviewer complaints are about over-claiming, not about the statistics themselves.

5. Use AI as a second coder (qualitative)

Best in Claude or NotebookLM

Here is my codebook: [CODE NAMES + DEFINITIONS + EXAMPLES]
Here are [N] anonymised transcript excerpts: [PASTE]

1. Apply my codebook to each excerpt. Quote the exact text segment for every code applied.
2. Where an excerpt fits no existing code, say so and propose a candidate new code with a definition.
3. Where my code definitions overlap or are ambiguous, tell me which pairs are confusable.
4. Do not merge, paraphrase or invent quotes.

I will compare this against my own coding to check consistency.

AI becomes a consistency check on your own coding, not a replacement coder. Report it honestly in your methods.

Tool spotlight for this stage

ChatGPT

Advanced Data Analysis runs real Python on a file you upload — descriptives, assumption checks, clean charts.

Tip: Ask it to show the code it ran. If you cannot read the code, do not report the result.

Claude

Best for careful reasoning about design, validity threats and qualitative nuance.

Tip: Ask 'what would make this conclusion wrong?' before you ask it to write anything.

Gemini

Strong on statistics explanation and works well with data in Google Sheets and Colab.

Tip: Use it to re-explain an unfamiliar test three ways until the assumptions are clear.

NotebookLM

Load your codebook, methodology chapter and transcripts as sources for grounded, cited answers.

Tip: Great for checking that your Methods section actually matches your protocol document.

Two more tools worth knowing

  • jamovi or JASPFree, transparent statistics packages. Pair them with AI: run the analysis in the software, use AI to interpret and report the output.
  • Julius AIConversational data analysis over your uploaded dataset with visible code and charts — useful for exploration, provided you audit the code.

Integrity boundary for Part 2

Do this

  • Anonymise or de-identify data before uploading anything, and check your ethics approval first.
  • Ask for the code or the reasoning behind every number the AI reports.
  • Report AI-assisted coding or interpretation in your methods section.

Never do this

  • Never ask AI to generate, simulate as real, or 'fill in' missing responses.
  • Never accept a p-value, effect size or theme the AI produced without reproducing it yourself.
  • Never upload identifiable participant data to a public AI tool.

Your notes are saved in this browser only.