Navigation

Not What You Meant? There are 43 definitions for Energy. Also try: Disorder or Entropia.

Entropy

About 5 pages (1,399 words)

Entropy Summary

Know this topic well? Help others and get FREE products!

Encyclopedia of Nonlinear Science

ENTROPY

Entropy is a quantity characterizing disorder, or randomness. This conclusion is the result of successive advances since the pioneering works by Sadi Carnot on the fundamentals of steam engines in 1824 and by Rudolf Clausius, who, between 1851 and 1865, developed the concept of entropy (from τρoπη meaning “transformation” in Greek) as a thermodynamic state variable. Called S, this quantity varies as dS=dQ/T in an equilibrium system at temperature T exchanging a quantity dQ of heat and no matter reversibly with its environment.

Thereafter, Ludwig Boltzmann (1896/1898), Max Planck (1901), Josiah Willard Gibbs (1902), and others discovered the statistical meaning of entropy. Boltzmann and Gibbs introduced the concepts of probability and statistical ensemble in the context of mechanics. If P_α is the probability (satisfying 0≤P_α≤1 and ∑_αP_α=1) that the system is found in the microstate a specified by a set of observables such as energy and linear or angular momenta, the thermodynamic entropy is given by

(1)

where k_B=1.38065 10⁻²³J K⁻¹ is the so-called Boltz-mann constant, although it was originally introduced by Planck (Sommerfeld, 1956).

The entropy S measures the disorder in a statistical ensemble composed of a very large number, of copies of the system. Each copy is observed in a microstate α occurring with the probability P_α. The microstates of all the copies form a random list Typically, copies are found in the microstate α, and The copies being statistically independent, the total number of possible lists of the microstates of the ensemble is thus equal to

(2)

where denotes the factorial of the integer In the limit according to Stirling’s formula where e=2.718…denotes the Naperian base, entropy (1) is given in terms of the logarithm of the number W of possible lists (originally called complexions by Boltzmann):

(3)

If the system is perfectly ordered, all the copies in the ensemble would be in the same microstate, and there would be a single possible complexion W=1 so that the entropy would vanish (S=0). In contrast, if the system were completely disordered with A equiprobable microstates α, the entropy would take the maximum value S=k_B In A. For partial disorder, the entropy takes an intermediate value.

In spatially extended homogeneous systems, the thermodynamic entropy is an extensive quantity. If the system is covered by disjoint windows of observations of volume V, the microstates in the windows form one among W possible lists (2). If the volume V of the observation window is large enough, the thermodynamic entropy is again given by Equation (3). In this case, the entropy per unit volume obtained by dividing the entropy S by the volume V is a measure of spatial disorder.

Entropy can also be interpreted as a quantity of information required to specify the microstate of the system. Indeed, the recording of the random microstates of the statistical ensemble requires one to allocate at least I=log₂ W bits of information in the memory of a computer. This number of bits is related to the entropy (1) by Such connections between entropy and information have been developed since the works by Leo Szilard in 1929 and Léon Brillouin around 1951.

Equation (1) for the entropy is very general. It applies not only to equilibrium but also to out-of-equilibrium systems provided the states α are understood as coarsegrained states. In a classical system of N particles, the coarse-grained states α should correspond to cells of volume h^3N in the phase space of the positions and momenta of the particles, where h=6.626 10⁻³⁴ Js is Planck’s constant of quantum mechanics.

In 1902, Gibbs suggested that the second law of thermodynamics is a consequence of a dynamics having the mixing property according to which the statistical averages of observables or the coarse-grained probabilities P_α converge to their equilibrium values. Not all systems are mixing, but for those that are, entropy (1) converges toward its equilibrium value. In mixing systems, the statistical correlations in the initial probability distribution tend to disappear on finer and finer scales in phase space and are shared among more and more particles during a causal time evolution. The approach to the thermodynamic equilibrium may thus be described in terms of asymptotic expansions of the probability distributions in the long-time limits t→±∞. Both limits are not equivalent because, in the limit t→+∞, the probability distributions remain smooth in the unstable phase-space directions but become singular in the stable ones and vice versa in the other limit t→−∞, which may appear as an irreversibility or time arrow in the long-time description. The irreversibility in the increase of the entropy is thus closely related to the problems of identifying all the degrees of freedom guaranteeing the causality of the time evolution and of reconstructing the initial conditions, which is of great importance for the understanding of historical processes such as biological and cosmological evolution.

During recent decades, it has been shown that the increase of entropy does not preclude the formation of structures in equilibrium or non-equilibrium systems, nor in self-gravitating systems. At equilibrium, the homogeneity of pressure, temperature, and chemical potentials does not prevent inhomogeneities in the densities as is the case in crystals, in vortex states of quantum superfluids, and in mesomorphic phases of colloidal systems where equilibrium self-assembly occurs. Besides, open non-equilibrium systems can remove entropy to their environment, leading to far-from-equilibrium self-organization into spatial structures such as Turing patterns, self-sustained oscillations in such systems as chemical clocks, or complex processes such as biological morphogenesis.

While the entropy per unit volume characterizes spatial disorder at a given time, a concept of entropy per unit time was introduced in 1949 by Claude Shannon in development of his information theory, as a measure of temporal disorder in random or stochastic processes. It is defined in the same way as standard entropy but replacing space by time. In 1959, Andrei N.Kolmogorov and Yakov G.Sinai applied Shannon’s idea to deterministic dynamical systems with an invariant probability measure, and they defined a metric entropy per unit time in analogy with Equation (1), considering the states α as the sequences ω₁ω₂…ω_n of the phase space cells ω_j successively visited at time intervals Δt by the trajectories during the time evolution of the system. In order to get rid of the arbitrariness of the coarse-grained cells ω_j, Kolmogorov and Sinai considered the supremum (least upper bound) of the entropy per unit time with respect to all possible partitions of the phase space into cells ω_j, defining

(4)

where the probability P is evaluated with the given invariant measure (Cornfeld et al., 1982).

In isolated chaotic dynamical systems, the temporal disorder of the trajectories finds its origin in the sensitivity to initial conditions because the Kolmogorov-Sinai entropy per unit time is equal to the sum of positive Lyapunov exponents as Proved by Yakov B. Pesin in 1977 (Eckmann & Ruelle, 1985). We notice that the entropy per unit time hKS differs from the irreversible entropy production defined by the time derivative of the standard thermodynamic entropy S in an isolated system. Indeed, the entropy per unit time may take a positive value for a system of particles already at thermodynamic equilibrium where entropy production vanishes. In spatially extended chaotic systems, the spatiotemporal disorder can be characterized by a further concept of entropy per unit time and unit volume.

A so-called topological entropy has also been introduced as the rate of proliferation of cells in successive partitions iteratively refined by the dynamics (Eckmann & Ruelle, 1985). In finite chaotic systems, the topological entropy is the rate of proliferation of periodic orbits as a function of their period. The topological entropy is not smaller than the Kolmogorov-Sinai entropy: h_top≥h_KS.

PIERRE GASPARD

Entropy

ENTROPY

Further Reading