Working with LLMs

In this course you are expected to use large language models (LLMs) such as GPT UiO: for setting up Python, writing and debugging analysis code, understanding methods, and writing reports. This is how much research and technical work is done today. What the course assesses is your own understanding, and that is checked in the oral defenses and the final exam.

What an LLM is good and bad at here

Good at: installing packages and fixing error messages, writing first versions of analysis code, explaining what a function or a parameter does, suggesting how to plot or fit something, and turning your notes into readable text.

Bad at: knowing what your data look like, knowing whether a number is physically reasonable, and noticing when code runs but gives the wrong answer. It will often answer confidently when it is wrong.

Five habits

  1. Give it the physics and the numbers. “Beads of 0.51 µm, 0.1837 µm per pixel, 15 fps, I expect D ≈ 0.9 µm²/s” gets you far better help than “how do I use trackpy?”.
  2. Ask it to explain, not only to write. For every step, ask why. If you cannot explain a line of your code, you do not yet own it.
  3. Test on data with a known answer. Synthetic data, a limiting case, a simple estimate. If the method fails there, it fails on your data too.
  4. Estimate before you compute. Know the order of magnitude you expect before you look at the result.
  5. Keep a log. Note what you asked, what went wrong and how you found out. It becomes the LLM log in your report and good material for the defense.

The LLM log in your reports

One page appended to each report:

  • What you used an LLM for (code, debugging, explanations, writing).
  • At least one example where it was wrong, misleading or incomplete, and how you found out.
  • What you checked yourself, and how.

The log is not graded on how much or how little you used an LLM. It shows that you know where the result comes from.

Data and privacy

Use GPT UiO or another tool approved by UiO for anything that includes data or text that is not yours to share. Do not paste personal information into any LLM.