Sebastian Zearing

how to be progressive without being a stupid liberal

Statistical Invariances and Hierarchical and Dimensional Variation

Look around you.

How many repeating things can you find? In my room, I can count several dozen individual blinds over the window, two windows, several dozen buttons on a remote control, several dozen keys on my keyboard, thousands of carpet threads, six guitar strings, several dozen books, four pillows, and a partridge in a pear tree [kidding]. Interestingly, if I had the right equipment, I could tell that all of these things are made from 10^bignumber electrons, quarks, and pions, though if I go back to using my eyes, I’m only interacting with these subatomic particles through photons. These subatomic particles make atoms and ions in exceedingly regular ways. These in turn make molecules in slightly more complicated ways. These molecules in turn make bulk materials in even more complicated ways (or sometimes the pattern jumps straight from atoms to bulk materials, as with most metals). And then these materials go to make all kinds of different things. We can also take a detour through biology, where the molecules, in breathtakingly complicated ways, make cells, which then make tissues, which make organs, which make bodies. So we have a world replete with all kinds of different things, the vast majority of which are not unique, isolated things but rather similar to other things.

Essentialism is true

I argued several posts ago that essentialism is false. Well, I take it back. Barely. Essentialism, though false for any everyday thing you can think of, is true for one specific class of subatomic particles, elementary particles, of which there are 17 known. And even though we might one day learn that the electron, for instance, is not an elementary particle, essentialism will then be true for whatever elementary particles compose it. Incidentally, we have done better than Plato because these essences have no need for accidents. In stark contrast to things like rabbits or roses or asteroids, every elementary particle of a certain type is exactly the same as every other one of that type. So there is at least one facetious sense in which the Universe is perfect: all the elementary particles are perfect!

Some figures

I’ll use some simple figures to illustrate what a statistical invariance is. In these images, consider the right and left edges as being connected, as well as the top and bottom edges.


Figure 1. 50 “elementary particles” randomly strewn about.


Figure 2. 50 “elementary particles” randomly strewn about.


Figure 3. 50 “elementary particles” randomly strewn about.


Figure 4. 50 “elementary particles” randomly strewn about.


Figure 5. 50 “elementary particles” randomly strewn about.

In case it bears repeating, each figure contains 50 “elementary particles” randomly strewn about. But they are obviously different. In figure 2-4, we see that the particles come in pairs. In figure 2, the pair is separated by a fixed horizontal distance. In figure 3, the distance is subject to a little noise. In figure 4, both the horizontal distance and the vertical displacement are subject to a little noise. Finally in figure 5, we have complex objects constructed from three parts: the top pairs of dots are as in figure 2, the bottom pairs of dots are as in figure 4, and they come with one additional dot to the left.

Though there is only the one, same elementary particle in each figure, what one, same thing repeats in each figure is quite different. That which repeats is the statistical invariance. <Man>, as we learned a few posts ago, is no essence. I will now reveal what <man> actually is: <man> is a statistical invariance. Similarly with everything else Plato might ascribe an essence to, there is no essence at all, merely statistical invariance.

Nested invariances and hierarchical v. dimensional variation

Many statistical invariances, certainly most everyday ones, are invariant over things that are themselves invariances, which are invariant over things that are themselves invariances, … several times over all the way down to the elementary particles. This is overtly obvious with examples such <nation-state>, <moon>, <bouquet>, and <genealogy>. Even the most seemingly straightforward noun oozes with such statistical abstractness. In fact, basically all non-subatomic nouns are highly abstract, in the conventional sense of abstract, and the list of nouns conventionally known as abstract are really just those whose sub-invariances are not fixed together spatiogeometrically. In other words, <nation-state> is colloquially referred to as abstract not because it refers to something “immaterial,” but because there’s no geometrically invariant matter that is the nation-state. The matter is certainly invariant in other ways that makes the nation-state as material as any hunk of granite. Can’t have a nation-state without matter!

But that’s a tangent. So there’s the obvious way in which invariances are nested: body made of organs made of tissues made of cells… There’s another way invariances can be nested: instead of nesting over building blocks, you can can nest over the strictness of the invariance. The classic example is the Linnaean taxonomy. <Dog> is a very specific invariance, and it nests with the invariances <wolf> and <fox> to form the less specific invariance <canine> which nests with <cat> and other invariances to form <carnivore> then <mammal> then <vertebrate> then <animal> then <eukaryote> then <terrestrial biota>. This nested hierarchy portrays the way in which life exhibits variation. This is also the primary way that conceptual variation in the brain varies. <Rocking chair> is a type of <chair> which is a type of <furniture>, <man-made thing>, <inanimate thing>.

The other primary way things vary is dimensionally. Here, the various [hierarchically restricted] possibilities of hierarchical variation are available to all instead of to just one particular subpopulation. For example, only mammals produce milk, and only vertebrates are mammals. Consequently, everything that produces milk has a backbone. In dimensional variation, this stricture doesn’t exist, and things without a backbone are also free to produce milk. Man-made objects tend to exhibit dimensional variation more commonly. Thus any Ford, Honda, or Fiat can have leather seats or be red, and whether or not a microwave has a turnstile does not restrict it from having a touchpad. You can mix and match freely, such that the full dimensional space of possibilities can be realized. Interestingly, sexually reproducing biological populations exhibit dimensional variation. Asexual ones revert to hierarchical variation, unless they’re DNA-exchanging sluts like bacteria, in which case we’re back to dimensional variation. In fact, sexual reproduction probably evolved to recapture the benefits of dimensional variation after multicellularity and embryology cemented the impossibility of freely exchanging DNA. And finally, if we restrict ourselves to eukaryotic/multicellular evolution, one could assert that macroevolution is the level where hierarchical variation is prominent, and microevolution is the level where dimensional variation is prominent. Indeed, the observation that life varies hierarchically is one of the best pieces of evidence against special creation: though an omnipotent Yahweh can create life forms with either kind of variation, dimensional variation offers more possibility and therefore more creativity. But hierarchical variation is what we have. Q.E.D.


How words map onto this ontology is very complicated. In English, nouns common nouns usually denote some specific invariance. Adjectives usually describe dimensional variation (e.g. tall man) but can also describe hierarchical (e.g. flowering plants). Verbs usually denote some specific invariance just as nouns, but they’re a class of invariances orthogonal to nouns, and often have a relationship with time and timecourses that nouns simply don’t. Verbs are what nouns do, but now we’re getting circular. And how language more broadly maps onto this ontology is an even more jumbled mess than the last two sentences.

Learning statistical invariances

So statistical invariances exist and provide an exceedingly rich way of thinking about the world. But how can we come to know which invariances exist in the world? Thankfully, animals that are good at finding invariances tend to leave more offspring, so evolution takes care of it. For the most part, you don’t have to try to find an invariance, you just look around you, and your cognitive system finds them for you. In fact, when you were a little baby, your auditory system automatically sifted through the vocalizations you heard from everyone around you, and it discovered the phonemes of your native language. Phonemes are statistical invariances and you found them all by yourself! This means that you are a machine that turns implicit information into explicit information and most of the implicit information has to do with statistical invariances.

The fantastic advance of machine learning in the past couple of years is predicated on the fact that we have finally learned how to program computers to find statistical invariances in data. In the foremost example of deep learning, these algorithms function by iteratively finding low-level clusters in the data and then using these to find higher-level clusters in the data, until more and more and ideally all of the statistical structure within a training data set is found. That the best thinkers in the early days of computing in the 50s thought this rudimentary aspect of cognition could be implemented in one summer goes to show how incredibly counter-intuitive our experience of being in the world is, and how thankful we must be that we are standing on the shoulders of giants.

P.S. See BBC’s Look Around You for laughs.


3 responses to “Statistical Invariances and Hierarchical and Dimensional Variation

  1. Pingback: Essentialism is False | Sebastian Zearing

  2. Pingback: Social Constructs and Social Wrappers | Sebastian Zearing

  3. Pingback: Sexuality, androphilia, gynephilia, and gay genes | Sebastian Zearing

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: