Maximum Entropy

Given that a web of quantum interactions forms the basis of objective reality we are left with a puzzle as to the existence of complex structures, ourselves included. Complexity is an extremely special state and is therefore rare. This notion is a basic law of physics known as the second law of thermodynamic. It says that the quantity of entropy or disorder, in a closed system, will tend to increase. That in the random walk entailed by the evolution of systems they are much more likely to enter states of lesser complexity than states of greater complexity due to the fact that less complex configurations are much more numerous.

How to explain not only the existence of the many complex systems we see around us but also their increase in numbers and degree of complexity over time? The usual explanation is that entropy need only increase in a closed system and that the complex systems we are considering are not closed but rather part of a larger system whose overall entropy is increasing. For example the complex biological system on earth is not a closed system but must be considered together with the sun. In this sun/earth system entropy is increasing as the decrease on the earth is more than compensated for by the increase of the sun.

Still this is only a partially satisfying explanation as it leads us only to more questions: How are designs for extremely unlikely complex systems such as earth's biology found, how are those designs instantiated and how can they survive and evolve ever greater complexity?

A first step to answering these questions might involve a scientific principal that is a refinement of the second law of thermodynamics known as Maximum Entropy. This principal, although it appears to only tweak the second law, offers enormous insight. It says that the evolution of a system will result in all aspects of the system increasing their entropy except where they are constrained by scientific law.

As an illustration we might consider a historical scientific mystery concerning the atom. The atom was first envisioned as a classical entity with an electron orbiting a nucleus. According to classical scientific law the atom should achieve lesser complexity by radiating away the electron's energy and the electron spiralling inward and collapsing into the nucleus within a fraction of a second. This is a disastrous model as it predicts all matter is unstable. Only when it was accepted as a scientific law that an electron's orbital energy came in quanta or packets and could not go to zero was it understood that an atom remained stable because its electron orbital energy was constrained from falling below its ground state. This quantum scientific law constrains the system's tendency to go to states of higher entropy and the situation conforms to the predictions of maximum entropy; that the system will go to the states of highest entropy available to it subject to the constraints of scientific law.

Although some of the underlying principles of Maximum Entropy have been used by practicing scientist at least since the time of Laplace, it only became formalized during the last century largely in the work of E.T. Jaynes.[i] Since then it has become an indispensible implement in the scientific toolbox as it guides modelling of a system on the basis of the second law of thermodynamics and any other applicable scientific laws.

Maximum Entropy is usually considered as a principle providing a tractable method for making predictions on a system's evolution subject to scientific law but it may also be turned around and used to inform us as to the nature of scientific law.

Consider that in the very early universe scientific subject matter was quite limited. I am using the term 'scientific subject matter' in an objective sense; in the sense that scientific law greatly preceded humanity and has only recently been partially discovered by us. In the early universe the only scientific law in operation was probably that of quantum gravity. Atomic physics, cosmology, chemistry, biology and culture, in fact the bulk of scientific subject matter, had not yet emerged in the web of reality.

From this view it is clear that the bulk of scientific law came into existence along with the complex entities that could endure and now compose much of scientific subject matter. Scientific law can be considered as the specific designs found by nature that are capable of retaining and evolving their complexity. Let us remember that complex systems are not forbidden, they are only extremely rare. A complex design able to maintain and evolve its complexity is not impossible, in fact we have overwhelming evidence that such systems exist. When we examine their nature we see that there are scientific laws operating that constrain these systems from evolving to more simple states. These scientific laws however did not pre-date the scientific subject matter composing the complex system rather they evolved with it. The applicable scientific laws might be considered as their design specifications.

Thus we can consider scientific law concerning such things as chemistry or genetics as the design specifications of those systems nature has found and instantiated capable of retaining and evolving their complexity. They specify the constraints imposed by nature that deny the evolution of these designed complex systems access to states of lesser complexity.

[i] Jaynes, E. T., 1979, `Where do we Stand on Maximum Entropy?' (2.6Mb) in The Maximum Entropy Formalism, R. D. Levine and M. Tribus (eds.), M. I. T. Press, Cambridge, MA