The Ideal Gas, Mixing Entropy and Gibbs’ Paradox, the Canonical Ensemble

IV.D The Ideal Gas As discussed in chapter II, micro-states of a gas of N particles correspond to points µ ≡ {p~i , ~qi }, in the 6N -dimensional phase space. Ignoring the potential energy of interactions, the particles are subject to a Hamiltonian H= N � � p~i 2 2m i=1 � + U (q~i ) , (IV.26) where U (~q) describes the potential imposed by a box of volume V . A microcanonical ensemble is specified by its energy, volume, and number of particles, M ≡ (E, V, N ). The joint PDF for a micro-state is 1 p(µ) = · Ω(E, V, N ) � 1 0 for q~i ∈ box, and otherwise � i p~i 2 /2m = E (±ΔE ) . (IV.27) In the allowed micro-states, coordinates of the particles must be within the box, while �N the momenta are constrained to the surface of the (hyper-)sphere i=1 p~i 2 = 2mE. The allowed phase space is thus the product of a contribution V N from the coordinates, with √ the surface area of a 3N -dimensional sphere of radius 2mE from the momenta. (If the microstate energies are accepted in the energy interval E ± ΔE , the corresponding volume � in momentum space is that of a (hyper-)spherical shell of thickness ΔR = 2m/EΔE.) The area of a d-dimensional sphere is Ad = Sd Rd−1 , where Sd is the generalized solid angle. A simple way to calculate the d–dimensional solid angle is to consider the product of d Gaussian integrals, Id ≡ �� ∞ −x2 dxe −∞ �d = π d/2 . (IV.28) Alternatively, we may consider Id as an integral over an entire d–dimensional space, i.e. Id = � � d i=1 � � dxi exp −x2i . (IV.29) The integrand is spherically symmetric, and we can change coordinates to R2 = � i x2i . Noting that the corresponding volume element in these coordinates is dVd = Sd Rd−1 dR, Id = � ∞ dRSd R 0 d−1 −R2 e Sd = 2 � ∞ dyy d/2−1 e−y = 0 79 Sd (d/2 − 1)! , 2 (IV.30) where we have first made a change of variables to y = R2 , and then used the integral representation of n!. Equating expressions (IV.28) and (IV.30) for Id gives the final result for the solid angle, Sd = 2π d/2 . (d/2 − 1)! (IV.31) The volume of the available phase space is thus given by Ω(E, V, N ) = V N 2π 3N/2 (2mE)(3N−1)/2 ΔR . (3N/2 − 1)! (IV.32) The entropy is obtained from the logarithm of the above expression. Using Stirling’s formula, and neglecting terms of order of 1 or ln E ∼ ln N in the large N limit, results in � � 3N 3N 3N 3N S(E, V, N ) =kB N ln V + ln(2πmE) − ln + 2 2 2 2 � � �3/2 � (IV.33) 4πemE =N kB ln V . 3N Properties of the ideal gas can now be recovered from T dS = dE + P dV − µdN , � 3 N kB 1 ∂S �� = = . (IV.34) � T ∂E N,V 2 E The internal energy E = 3N kB T /2, is only a function of T , and the heat capacity CV = 3N kB /2, is a constant. The equation of state is obtained from � P ∂S �� N kB = = , =⇒ P V = N kB T. V T ∂V �N,E (IV.35) The unconditional probability of finding a particle of momentum p~1 in the gas can be calculated from the joint PDF in eq.(IV.27), by integrating over all other variables, p(p~1 ) = � 3 d q~1 N � i=2 d3 ~qi d3 p~i p({q~i , ~pi }) V Ω(E − p~1 2 /2m, V, N − 1) = . Ω(E, V, N ) (IV.36) The final expression indicates that once the kinetic energy of one particle is specified, the remaining energy must be shared amongst the other N − 1. Using eq.(IV.32), (3N/2 − 1)! V N π 3(N−1)/2 (2mE − p~1 2 )(3N−4)/2 � � · N 3N/2 p(p~1 ) = (2mE)(3N−1)/2 V π 3(N − 1)/2 − 1 ! �3N/2−2 � 1 (3N/2 − 1)! p~1 2 � � . = 1− 3/2 2mE (2πmE) 3(N − 1)/2 − 1 ! 80 (IV.37) � � From Stirling’s formula, the ratio of (3N/2 − 1)! to 3(N − 1)/2 − 1 ! is approximately (3N/2)3/2 , and in the large E limit, p(p~1 ) = � 3N 4πmE �3/2 � � 3N p ~1 2 exp − . 2 2mE (IV.38) This is a properly normalized Maxwell-Boltzmann distribution, which can be displayed in its more familiar form after the substitution E = 3N kB T /2, � � 1 p~1 2 . p(p~1 ) = exp − 2mkB T (2πmkB T )3/2 (IV.39) IV.E Mixing Entropy and Gibbs’ Paradox The expression in eq.(IV.33) for the entropy of the ideal gas has a major shortcoming in that it is not extensive. Under the transformation (E, V, N ) → (λE, λV, λN ), the entropy changes to λ(S + N kB ln λ). The additional term comes from the contribution V N , of the coordinates to the available phase space. This difficulty is intimately related to the mixing entropy of two gases. Consider two distinct gases, initially occupying volumes V1 and V2 at the same temperature T . The partition between them is removed, and they are allowed to expand and occupy the combined volume V = V1 + V2 . The mixing process is clearly irreversible, and must be accompanied by an increase in entropy, calculated as follows. According to eq.(IV.33), the initial entropy is Si = S1 + S2 = N1 kB (ln V1 + σ1 ) + N2 kB (ln V2 + σ2 ), (IV.40) where, σα = ln � 4πemα Eα · 3 Nα �3/2 , (IV.41) is the momentum contribution to the entropy of the αth gas. Since Eα /Nα = 3kB T /2 for a monotonic gas, σα (T ) = 3 ln (2πemα kB T ) . 2 (IV.42) The temperature of the gas is unchanged by mixing, since E1 E2 3 3 E1 + E 2 = = = kB T. kB Tf = N1 N2 2 2 N1 + N2 81 (IV.43) The final entropy of the mixed gas is Sf = N1 kB ln(V1 + V2 ) + N2 kB ln(V1 + V2 ) + kB (N1 σ1 + N2 σ2 ). (IV.44) There is no change in the contribution from the momenta which depends only on temper­ ature. The mixing entropy, ΔSMix � � N2 V2 V V N1 V1 = Sf − Si = N1 kB ln + N2 kB ln = −N kB ln + ln , (IV.45) V2 N V N V V1 is solely from the contribution of the coordinates. The above expression is easily generalized � to the mixing of many components, with ΔSMix = −N kB α (Nα /N ) ln(Vα /V ). Gibbs’ Paradox is related to what happens when the two gases, initially on the two sides of the partition, are identical with the same density, n = N1 /V1 = N2 /V2 . Since removing or inserting the partition does not change the state of the system, there should be no entropy of mixing, while eq.(IV.45) does predict such a change. For the resolution of this paradox, note that while after removing and reinserting the partition, the system does return to its initial configuration, the actual particles that occupy the two components are not the same. But as the particles are by assumption identical, these configurations cannot be distinguished. In other words, while the exchange of distinct particles leads to two configurations • | ◦ A | B and ◦ | • , A | B a similar exchange has no effect on identical particles, as in • | • A | B and • | • . A | B Therefore, we have over-counted the phase space associated with N identical parti­ cles by the number of possible permutations. As there are N ! permutations leading to indistinguishable micro-states, eq.(IV.32) should be corrected to Ω(N, E, V ) = V N 2π 3N/2 (2mE)(3N−1)/2 ΔR , N ! (3N/2 − 1)! (IV.46) resulting in a modified entropy, � � eV S = kB ln Ω = kB [N ln V − N ln N + N ln e] + N kB σ = N kB ln +σ . N 82 (IV.47) As the argument of the logarithm has changed from V to V /N , the final expression is now properly extensive. The mixing entropies can be recalculated using eq.(IV.47). For the mixing of distinct gases, V2 V V V1 + N2 kB ln − N1 kB ln − N2 kB ln N N2 N1 N2 �1 � � � V N2 V N1 · · = N1 kB ln + N2 kB ln N1 V1 N2 V2 � � N2 V2 N1 V1 ln + ln , = −N kB N V N V ΔSMix = Sf − Si = N1 kB ln (IV.48) exactly as obtained before in eq.(IV.45). For the ‘mixing’ of two identical gases, with N1 /V1 = N2 /V2 = (N1 + N2 )/(V1 + V2 ), ΔSMix = Sf − Si = (N1 + N2 )kB ln V1 + V2 V1 V2 − N1 kB ln − N2 kB ln = 0. (IV.49) N1 + N2 N1 N2 Note that after taking the permutations of identical particles into account, the available volume in the final state is V N1 +N2 /N1 !N2 ! for distinct particles, and V N1 +N2 /(N1 + N2 )! for identical particles. • Additional comments on the microcanonical entropy: 1. In the example of two-level impurities in a solid matrix (sec.IV.C), there is no need for the additional factor of N !, as the defects can be distinguished by their locations. 2. The corrected formula for the ideal gas entropy in eq.(IV.47) does not affect the com­ putations of energy and pressure in eqs.(IV.34) and (IV.35). It is essential to obtaining an intensive chemical potential, � � � �3/2 � µ ∂S �� S 5 V 4πmE =− = − + kB = kB ln T ∂N �E,V N 3N N 2 . (IV.50) 3. The above treatment of identical particles is somewhat artificial. This is because the concept of identical particles does not easily fit within the framework of classical mechanics. To implement the Hamiltonian equations of motion on a computer, one has to keep track of the coordinates of the N particles. The computer will have no difficulty in distinguishing exchanged particles. The indistinguishability of their phase spaces is in a sense an additional postulate of classical statistical mechanics. This problem is elegantly resolved within the framework of quantum statistical mechanics. Description of identical particles in quantum mechanics requires proper symmetrization of the wave function. The 83 corresponding quantum microstates naturally yield the N ! factor, as will be shown later on. 4. Yet another difficulty with the expression (IV.47), resolved in quantum statistical me­ chanics, is the arbitrary constant that appears in changing the units of measurement for q and p. The volume of phase space involves products pq, of coordinates and conjugate momenta, and hence has dimensions of (action)N . Quantum mechanics provides the ap­ propriate measure of action in Planck’s constant h. Anticipating these quantum results, we shall henceforth set the measure of phase space for identical particles to dΓN = 1 h3N N ! N � d3 ~qi d3 p ~i . (IV.51) i=1 IV.F The Canonical Ensemble In the microcanonical ensemble, the energy E, of a large macroscopic system is pre­ cisely specified, and its equilibrium temperature T , emerges as a consequence (eq.(IV.7)). However, from a thermodynamic perspective, E and T are both functions of state and on the same footing. It is possible to construct a statistical mechanical formulation in which the temperature of the system is specified and its internal energy is then deduced. This is achieved in the canonical ensemble where the macro-states, specified by M ≡ (T, x), allow the input of heat into the system, but no external work. The system S, is maintained at a constant temperature through contact with a reservoir R. The reservoir is another macroscopic system that is sufficiently large so that its temperature is not changed due to interactions with S. To find the probabilities p(T,x) (µ), of the various micro-states of S, note that the combined system R ⊕ S, belongs to a microcanonical ensemble of energy ETot ≫ ES . As in eq.(IV.3), the joint probability of micro-states (µS ⊗ µR ) is 1 · p(µS ⊗ µR ) = ΩS⊕R (ETot ) � 1 for HS (µS ) + HR (µR ) = ETot . (IV.52) 0 otherwise The unconditional probability for micro-states of S is now obtained from p(µS ) = � {µR } p(µS ⊗ µR ) . 84 (IV.53) Once µS is specified, the above sum is restricted to micro-states of the reservoir with energy ETot − HS (µS ). The number of the such states is related to the entropy of the reservoir, and leads to � � � � � � ΩR ETot − HS (µS ) 1 ∝ exp SR ETot − HS (µS ) . p(µS ) = kB ΩS⊕R (ETot) (IV.54) Since by assumption the energy of the system is insignificant compared to that of the reservoir, � � ∂SR HS (µS ) . = SR (ETot) − SR ETot − HS (µS ) ≈ SR (ETot) − HS (µS ) ∂ER T (IV.55) Dropping the subscript S, the normalized probabilities are given by p(T,x) (µ) = e−β H(µ) . Z(T, x) (IV.56) The normalization, Z(T, x) = � e−β H(µ) , (IV.57) {µ} is known as the partition function, and β ≡ 1/kB T . (Note that probabilities similar to eq.(IV.56) were already obtained in eqs.(IV.25), and (IV.39), when considering a portion of the system in equilibrium with the rest of it.) Is the internal energy E, of the system S well defined? Unlike in a microcanonical ensemble, the energy of a system exchanging heat with a reservoir is a random variable. Its probability distribution p(E), is obtained by changing variables from µ to H(µ) in p(µ), resulting in p(E) = � {µ} e−β E � p (µ) δ (H(µ) − E) = δ (H(µ) − E) . Z (IV.58) {µ} Since the restricted sum is just the number Ω(E), of micro-states of appropriate energy, � � � � Ω(E)e−β E 1 S(E) E 1 F (E) p(E) = = exp = exp − , − kB T Z kB T Z Z kB (IV.59) where we have set F = E − T S(E), in anticipation of its relation to the Helmholtz free energy. The probability p(E), is sharply peaked at a most probable energy E ∗ , which minimizes F (E). Using the result in sec.(II.F) for sums over exponentials, Z= � {µ} e−β H(µ) = � E 85 e−βF (E ) ≈ e−βF (E ) . ∗ (IV.60) The average energy computed from the distribution in eq.(IV.59) is hHi = � µ H(µ) e−β H(µ) ∂ ln Z 1 ∂ � −β H =− =− . e Z Z ∂β µ ∂β (IV.61) In thermodynamics, a similar expression was encountered for the energy (eq.(I.37)), � � � F ∂(βF ) ∂F �� 2 ∂ = . E = F + TS = F − T = −T � ∂T x ∂T T ∂β (IV.62) Eqs.(IV.60) and (IV.61), both suggest identifying F (T, x) = −kB T ln Z(T, x) . (IV.63) However, note that eq.(IV.60) refers to the most likely energy, while the average energy appears in eq.(IV.61). How close are these two values of the energy? We can get an idea of the width of the probability distribution p(E), by computing the variance hH2 ic . This is most easily accomplished by noting that Z(β) is proportional to the characteristic function for H (with β replacing ik) and, − � ∂Z He−β H , = ∂β µ and � ∂ 2Z H2 e−β H . = ∂β 2 µ (IV.64) Cumulants of H are generated by ln Z(β), hHic = 1 � −β H 1 ∂Z ∂ ln Z He =− =− , Z µ Z ∂β ∂β (IV.65) and 1 1 � 2 −β H hH2 ic = hH2 i − hHi2 = H e − 2 Z µ Z � � µ He−β H �2 = ∂ 2 ln Z ∂hHi − . = ∂β 2 ∂β (IV.66) More generally, the nth cumulant of H is given by hHn ic = (−1)n 86 ∂ n ln Z . ∂β n (IV.67) From eq.(IV.66), � � ∂hHi 2 ∂hHi � hH ic = − = kB T , � ∂T x ∂(1/kB T ) 2 ⇒ hH2 ic = kB T 2 Cx , (IV.68) where we have identified the heat capacity with the thermal derivative of the average energy hHi. Eq.(IV.68) shows that it is justified to treat the mean and most � likely energies interchangeably, since the width of the distribution p(E), only grows as hH2 ic ∝ N 1/2 . � √ The relative error, hH2 ic /hHic vanishes in the thermodynamic limit as 1/ N . (In fact eq.(IV.67) shows that all cumulants of H are proportional to N .) The PDF for energy in a canonical ensemble can thus be approximated by � � 1 −βF (E ) 1 (E − hHi)2 √ p(E) = e ≈ exp − . 2 Z 2kB T Cx 2πkB T 2 Cx (IV.69) The above distribution is sufficiently sharp to make the internal energy in a canonical ensemble unambiguous in the N → ∞ limit. Some care is necessary if the heat capacity Cx is divergent, as is the case in some continuous phase transitions. The canonical probabilities in eq.(IV.56) are unbiased estimates obtained (as in sec.(II.G)) by constraining the average energy. The entropy of the canonical ensemble can also be calculated directly from eq.(IV.56) (using eq.(III.68)) as S = −kB hln p(µ)i = −kB h(−βH − ln Z)i = E −F , T (IV.70) again using the identification of ln Z with the free energy from eq.(IV.63). For any finite system, the canonical and microcanonical probabilities are distinct. However, in the so called thermodynamic limit of N → ∞ limit, the canonical probabilities are so sharply peaked around the average energy that they are essentially indistinct from microcanonical probabilities at that energy. The following table compares the prescriptions used in the two ensembles. Ensemble Microcanonical Canonical Macro-state (E, x) (T, x) p(µ) � � δΔ H(µ) − E � �/Ω exp − βH(µ) /Z Normalization S(E, x) = kB ln Ω F (T, x) = −kB T ln Z Table 3: Comparison of canonical and microcanonical ensembles. 87