ChatGPT-4 & Questions from a Materials Thermodynamics Course

ChatGPT-4 and Thermodynamics

I took several problems from a graduate-level materials thermodynamics course that I’ve taught and fed them to ChatGPT-4.¹ The goal of this exercise was to gain insight into how to incorporate the use of ChatGPT into assignments in order to:

1. Support student learning of materials thermodynamics, in general; and
2. Help students become savvy users of generative AI.

Not surprisingly, there is a wide range in the quality and correctness of ChatGPT-4’s responses to questions. This fact should be shared with students, and they should be discouraged from using generative AI without critical review.

Below are five problem statements, my rationale for asking students to answer these questions, ChatGPT-4’s responses, and subsequent exchanges and iterations. Articulating your reasoning for asking specific questions on a pset, before reviewing genAI’s responses can help you to more effectively modify your assignments to better support your goals for student learning.

---

Questions 1 & 2 Fundamental Equations & Equations of State

These two questions go in opposite directions: in one case, they are given the fundamental equation and need to create the equations of state; in the other, they are given equations of state and must come up with a fundamental equation.

As explained by Callen² pp. 26 & 28:

Although this is a remarkable and theoretically powerful statement, there is a practical issue in that it is typically quite difficult (impossible) to directly measure entropy. Thus, much of thermodynamics involves creating and using alternate representations of the fundamental equation. In particular, for any given fundamental equation, three equations of state (expressions for the conjugate intensive parameters of the specific fundamental equation) provide equivalent information. These relationships are typically more readily measurable and, therefore, more useful in studying real systems.

The rationale for asking students to solve this problem

1. We want them to see that there is no entropy meter – but that we can work around this.
2. We want them to confront the power of the fundamental equation.
3. We want students to understand both conceptually and operationally what an equation of state is (and to ultimately be able to explain the equivalence of 3 equations of state and a fundamental equation.

How did ChatGPT-4 do?

Below is ChatGPT-4’s response and my subsequent follow-up questions. It does a reasonable job with this question. However, it glosses over some key points. My follow-up questions pull this out, but students could be asked these same questions during an in-class discussion after engaging with the problems.

The rationale for asking students to solve this problem

1. We want to help students realize that thermodynamics is not only mathematical manipulations. Rather, it is essential to consider both the mathematical description and physical properties of the system in question. By engaging with this problem, students must confront the definition of a fundamental equation, which sets them up to accurately develop and use fundamental equations associated with alternate energy functions later in the course.

How did ChatGPT-4 do?

ChatGPT-4 doesn’t provide a correct response, even after significant prodding. ChatGPT-4 acts like a novice problem solver, which should not be surprising, but this problem, in particular, really exposes its limitations.

Of course, It doesn’t know what a fundamental equation actually is (a fundamental equation in U or S must only be a function of extensive parameters [U=U(S, V, N) or S=S(U, V, N)]) so it’s just grasping at equations and making substitutions.

In order to solve the problem, one has to recognize that the entropic differential equation is the most practical since it allows a separation of variables and a straightforward integration:

dS =( 1/T )dT + (P/T )dV – (μ/T)dN

(with N constant)
1/T = abN/U
P/T = bN^2/V^2

dS = abN/U dU + bN^2/V^2 dV

Integrating, we get:
S – S_o = abNln(U/N) – bN^2/V

Working with students to help them understand that although thermo can be “just math,” in order to solve problems successfully, it is important to consider the details of the system and the connections between its mathematical description and physical properties.

As a follow-up activity for questions 1 & 2, either as an in-class activity or as part of the pset, or in a short quiz, require that they consider the significance of the answers.

1. Why do we care about T, P, & μ?
2. In question 1, the equation of state for temperature is in terms of entropy – why does this do us any good?
3. Does the answer to question 2 make conceptual sense? Why or why not? What assumptions were made in the derivation of S(U, V, N)?

The rationale for asking students to answer this question

We want students to grapple with the unintuitive nature of the expression.

dU which is an exact differential, a State Function (always), wherein dU is dependent only on its starting and ending states (the differential change in energy does not depend on the process(es) that took it from one state to another). Yet, the first law of thermodynamics states that this change in internal energy, dU is equal to the change in the value of 2 process variables, heat and work. Heat and work have no physical meaning outside of a process (one cannot measure the heat or the work of a given system – only the changes in those quantities during a specific process).

It is very easy for students to successfully apply the first law without ever considering the strangeness of this relationship.

I ask this question early on in the semester so that students grapple with this idea (what is implied about the relationship between heat and work?) at least a bit. I usually ask it as a pre-class question that we discuss in class, but it could be included in a pset (with subsequent class discussion).

How did ChatGPT-4 do?

Below is ChatGPT-4’s response. Not surprisingly, ChatGPT-4 completely misses the nuance. The answer it provided is not incorrect, but it basically just restates the equation in words.

Question 4

The rationale for asking students to solve this problem

It is important for students to wrestle with the concept of stability: what it means mathematically and for a physical system, how to determine if a system is stable, and the explicit connection between stability and phase transformations. Although students are asked to solve many problems related to stability, this problem helps them connect physically measurable quantities (isothermal compressibility and constant pressure heat capacity to stability, and ideally would require they sketch the isotherms for the van der Waals equation on a graph of P v. T and identify the regions where the system is unstable.

How did ChatGPT-4 do?

Below are ChatGPT -4’s responses and my subsequent follow-up questions. ChatGPT-4 did a reasonable job with this question but errs in its writing of the partial derivative for the constant pressure heat capacity. This is not a huge error, and ChatGPT-4 corrects itself after some basic prodding. [NB. I mistyped the equation of state when I entered the problem, yet it based its response on the correct expression.] An additional error is that ChatGPT-4 leaves off the “equality condition” for the values of C_P and K_T, stating only that both values should be > 0 (not ≥ 0).

It does a pretty good job with follow-up questions. Note: students could be asked to consider the follow-up questions as part of an in-class discussion of the problem set.

The rationale for asking students to solve this problem

This question is important because it helps students to see:

1. That fundamental equations can be written in terms of a combination of intensive and extensive variables; and that there must be at least one extensive variable.
2. It gives students practice generating alternate energy functions – by Legendre Transforming the equation for internal energy.
3. It gives them practice in deriving relevant Gibbs-Duhem equations for specific systems.

How did ChatGPT-4 do?

Below are ChatGPT -4’s responses and my subsequent follow-up questions. ChatGPT-4 had some trouble with this. For this question, it was very important to keep prodding it to redo its responses. It didn’t even attempt part c.

For students to do this (prod) effectively, they really do have to grapple with the answer themselves. It is only possible to know it is incorrect by working through the mathematical manipulations independently.

Note: I don’t know why it says there is an error in my derivation above – it is exactly the same as the correct derivation it eventually provides. Perhaps because I didn’t complete the simplification?

---

¹ Note ChatGPT-4 costs $20/month – a cost that is not an option for some students. ChatGPT-3.5 is “free” (for now), but considerably less accurate at responding to prompts throughout the post – I will try to suggest options that level the proverbial playing field wrt access to these products (ignoring student-use of these products is not an equitable solution).
² Callen, Herbert. Thermodynamics and an Introduction to Thermostatistics, 2nd Edition. Wiley (1991).