The kind way to say it is: “Humans are really good at detecting patterns.” The less kind way is: “Humans are really good at detecting patterns, even when they don’t exist.”
I’m going to blatantly swipe these two pictures from Peter Coles, but you should read his post for more information. The question is: which of these images represents a collection of points selected randomly from a distribution with uniform probability, and which has correlations between the points? (The relevance of this exercise to cosmologists studying distributions of galaxies should be obvious.)
The points on the right, as you’ve probably guessed from the set up, are distributed completely randomly. On the left, there are important correlations between them.
Humans are not very good at generating random sequences; when asked to come up with a “random” sequence of coin flips from their heads, they inevitably include too few long strings of the same outcome. In other words, they think that randomness looks a lot more uniform and structureless than it really does. The flip side is that, when things really are random, they see patterns that aren’t really there. It might be in coin flips or distributions of points, or it might involve the Virgin Mary on a grilled cheese sandwich, or the insistence on assigning blame for random unfortunate events.
Bonus link uncovered while doing our characteristic in-depth research for this post: flip ancient coins online!
These are all great questions. Let me answer them in turn.
So, as you know, given a probability distribution rho_i, where i labels the possible states, the entropy is defined by:
entropy = -(sum i) rho_i log rho_i.
It vanishes for a pure state–that is, when rho_i is equal to one for a single state and zero for all other states. The entropy is maximal if we have no information and must assign equal probability to all states. Given knowledge of the average energy of the system, we must maximize the entropy (i.e., our ignorance) subject to the constraint that has the given value, and we then obtain the familiar canonical ensemble of thermodynamics.
Entropy can be measured in bits, provided that we take our logarithm base 2. (Changing the base is equivalent to multiplying the formula by a uniform constant.) As an example, the entropy of an unknown binary string n bits long, and therefore having 2^n possible states each with equal probability 1/2^n, is just n bits, as expected.
Meanwhile, the algorithmic complexity of a single state can be defined in several ways. One simple definition is that it’s the number of bits in the shortest string needed to communicate the value of the state to a second individual. A set of 10 vertical arrows obviously requires a shorter description than a set of 10 randomly oriented arrows.
Another definition of the complexity of a state is that its the number of bits in a miinimal algorithm or computer program needed to generate the value of the state.
Now, you are correct that there remains some ambiguity in the definition of complexity. I could always change my labelings so that a state |1010111001> became |1111111111>. Obviously the definition of a complexity function depends on our labeling scheme for the states.
But the crucial fact is that once we’ve chosen a complexity scheme, whatever our choice, there will always be exponentially more states of high complexity than low complexity, because the complexity function defines a one-to-one correspondence between the states of the system and the set of minimal binary messages representing them. That’s the crucial feature of complexity. That fact ensures the various properties I described earlier.
For example, the exponential property of the complexity ensures that for a system with large entropy in the thermodynamic limit, all but a negligible number of states in the probability distribution will have high complexity, of order the entropy. One can show that the complexity is peaked at an average value of order the entropy.
Also, if our memory device is very limited, then we have no choice but to describe very complex states by means of a probability distribution with comparable entropy.
Finally, given an initial single state with low complexity, the system is highly likely to be found in a state of high complexity later on, again because there are exponentially many more states of high versus low complexity. This is the complexity version of the 2nd law of thermodynamics, valid even for perfectly closed systems with unitary time evolution.
@Aatash,
I’m not sure if you were trying to correct me, or just adding to my comment; but to clarify, I’m well aware that Technical Analysts do not consider stock price movements random (by ‘adding to the list’, I meant that TAs see patterns in random fluctuations). My reference to Mandelbrot was a nod to the fact that it has been shown that many (perhaps all) of the features that a TA would look for in stock price movements can be generated by, for example, multifractal models of the stock market. With regards to “Who gives a damn [how you make money]”, I think the key issue here is that if you have been making money by chance, based on a deeply flawed idea (be it technical analysis or some inappropriate stochastic model), then one day you may find that those flaws catch up to you, and relieve you of your capital.
Old topic.
Maybe, in terms of characteristics favored by evolution, correctly perceiving that the situation is not random can alert one to possible danger and perhaps save one’s life, whereas incorrectly thinking that things are not “normal” may merely lead to a minor waste of time and energy and a bit of unneccessary angst.
Pingback: Randomness: our brain deceive us « Mirror Image
May be humans try to find a patern were SEAMS to be random
What we usually think is random (like the stars in the sky) is not
For that reason we are good finding patterns were they are but we are not good at generating random series
Pingback: » Randomness is the materialist’s delusion
Pingback: Arrows and Demons « In the Dark
Carl Pilkington seems to be a supported of Boltzmann brains.
>>the two pictures both have zero entropy, since they are both exact states and not represented by probability distributions<<
Yay! I've been saying this for ages but it rarely seems to get through. Another example: a data set is never Gaussian.
Pingback: Perception, Piero and Pollock « In the Dark
Why do you believe that random-without-correlations is ‘more random’ than random-with-correlations? The output, for instance, of a Markov chain can have any correlation length whatsoever.
But you’re not alone. My physics colleagues almost universally say random when they mean uncorrelated.
Pingback: Looking For Patterns - Tips for Creative Problem Solving « ZenStorming - Where Science Meets Muse…
Pingback: Random Links XXXXI « Random Musings of a Deranged Mind