26. Psychology II: A Totally Different Pardigm

Simple Truths:

  • If you want to study the most complex system in the world, do not rely on the simplest tools/methodology or mathematics a fifth grader could apply.
  • If you want to study an individual, you need to study an individual.
  • If you claim to study behavior, you need to predict behavior (> 40% of the uncertainty).
  • If you want to know if you’re right about someone, you could ask them.

In my last Blog, I made four basic points.  Psychology:

  • Uses crude tools and inappropriate methodology to understand behavior/thoughts.
  • Rather than focus on people, it focuses on groups.
  • The utility (squared effect size) of their studies, although it is not zero, it is near zero (≤ 10%).  Their studies are practically useless.
  • Many theories are more descriptive of their author, than for people in general.

I realized as a neophyte graduate student, that in order to study individuals one needed an advanced methodology.  One cannot make advances in any field until the methodology is available.  For example, surgery couldn’t advance if the only tools available were flint-napped rocks.  Nor could you have any hope for success until you understood sterilization and anesthesia.  In Psychology, comparing two means (or a two way ANOVA) does NOT allow any adequate models of individuals, but, at best, groups.

If you want a real science of individuals, you need to study an N-of-One, or expand it to many N-of-One studies.  Is this heretical?  Nah.  Binet, the author of the first IQ test, based it on his observations of his daughter, an N-of-One.  The original studies in Psychophysics used ‘observers’, often N-of-One.  Of course, the original clinical theories of psychopathology, like those of Freud, devised their theories based on N-of-One observations.  B. F. Skinner used N-of-One.  The list can go on.

Mathematical Methodological Tools:

Analyses across individuals might tell you something common across all of them, but ignore any and all individual differences.  Means and correlations using groups of people will not work.  Empirically, predictions based on studying groups have had little prediction for individuals.  More to the point, p-values (inferential statistics) only applies to groups, but not N-of-One studies.   This is because the basic assumption of independence can never be met, as a person knows all of their prior behaviors.  Independence of observations is the cornerstone of inferential statistics (p-values).  One cannot use p-values in an analysis of a single individual.

For N-of-One research, one can still use means, medians, modes, and categorical descriptive analyses.  One can still correlate one parameter with another, and use all the correlational tools (e.g., factor analysis, cluster analysis, multidimensional scaling, regression, discriminant analysis).  One could still do time-series analyses.

Concrete Example:

Before I get too esoteric let me describe my one and only human study done at the University of Illinois about 40 years ago, an N-of-One study – Asking My Wife What She Thinks.  This used some of the methodology of Seymour Rosenberg of Rutgers.  I will not talk about the details too much.  I first asked my ‘subject’ (my wife) to name all the people she knew well enough to describe.  This included different aspects of herself, friends, relatives, classmates, fictional people, me, etc.  There were 110 ‘individuals’ she thought about.  I asked her to list adjectives for a random subset of 25 ‘individuals’.  Next she identified the polar opposite of each adjective (e.g., Imaginative vs. Dull, angry vs. happy).  We also expanded the list of adjectives pairs.  Finally she reduced the set of adjective/opposites to get rid of duplicates/synonyms and adjectives unique to a specific individual.  There were 73 adjective/opposite pairs.  Finally she rated each ‘individual’ on each  adjective/opposite.  That was the hard part, as there were over 8,000 ratings.  She spent about 60 hours making the ratings.  (*What can I say, she loved me.*)  I then did a (correlated or oblique rotation) factor analysis on the data.

It needs to be stressed that what I learned is (potentially) completely unique to her and perhaps unique to her at the age of 27 – an intelligent, Educational Psychology graduate student, focusing on counseling from the Bronx.  She is beyond WEIRD (Western, Educated, Industrialized, Rich, and Democratic).  She is my wife.  Her ‘theory of psychology’ may only be unique to her and may be even more unique to her at that stage of her development, 38 years ago.  Furthermore, in the last 38 years, she has thrice grown past her ‘professional career’, her parents and those of that generation are gone, our two children were born and have successfully ‘left the nest’, she has mellowed, and grown wiser.  In other words, the way she views people now is very likely different from then.

What did I learn and how does it relate to Theories of Psychology?

  • She tended to use the extreme end of her descriptors (and not the central part of her own ‘scales’).  This was more noted for the original adjective rather than the bipolar opposite.  To speak ‘psychologese’, this is partial confirmation of Kelly’s Dichotomy Corollary, and rejection of the normal distribution assumption endemic in much of psychological testing.
  • Her ‘Theory’ was much more complex than Eysenck’s.  Eysenck proposed two uncorrelated factors: Introversion v Extraversion and Neuroticism v Adjusted.  Her five factors include these factors, but they are both slightly different (e.g., her I-E factor had a clear pushy-loud component) and those two factors were correlated (0.32).
  • Osgood’s three factors are also too few to describe my wife.  Osgood’s theory wasn’t adequate to describe the complexity of my wife’s own theory of people and wasn’t clearly represented among my wife’s factors.
  • Of the four independent factors of Guilford’s theory, only one was seen to relate to my wife’s.
  • She generated, what I deduced to be five moderately correlated factors.  The (correlated) factors (named by my wife) were:
    • Distant v Caring
    • Nervous, tense v Calm and related
    • Immature v Mature
    • Unique, assertive v Passive and calm
    • Intelligent, good looking v Dull and plain – Note: she did not agree that this fifth factor was a true representation of the way she view people.
  •  The range of correlations among the factors went from 0.21 to 0.84, and all positive.  The second order factor she identified as whether the person has ‘gotten it all together’ or not.  The highest loading factor was the second Nervous v Calm factor (loading was 0.98) and the lowest loading was the fourth Unique v Passive factor (loading was .33).
  • Given that the factors were all positively correlated, many might dismiss the results as a mere ‘halo’ effect.  However the unique factor loading correlations ranged from 0.20 to 0.94 with a median of 0.78.  Yes, my wife tended to view people as ‘gotten it all together’ or not, but she was much more complex than that.
  • These five factors explained 59% of the variance of her ratings.  Not too shabby and SIX FOLD BETTER than psychology’s typical 10%.

Let me again stress, that the ‘individuals’, the adjectives, the factor ‘descriptions’ and the entire framework, were all unique to her and constructed by her, although I did the analyses and was guided by Ledyard Tucker – the world’s leading authority on factor analysis at the time.

Relevance:

  • Any individual’s theory of psychology is likely to have a HUGE unique component.
  • Any theory is likely to be quite malleable over time.
  • It will take a very large investment in time to elicit the data for even a single person.
  • Such analyses require very advanced mathematical training to execute.
  • Any general theory of psychology MUST include a large number of such N-of-One studies.

Post-1976 thoughts:

Can alternative data generation be done?  Unforeseen in 1976, Google Glass and similar continuous data collection methods are currently available.  In essence we can now record everything we see throughout the day.  Continuous recordings (e.g., Google Glass) can also hear everything we hear.  Computers are learning how to convert pictures into digital descriptions.  Computers can transcribe spoken words into digital data.  Our smart phones automatically know our locations, so it is possible to continuously monitor our location.  Perhaps the 2015 version might be called ‘Watching and Listening to my Wife to Determine What She Thinks‘.

Factor analysis may not have been the best statistical approach.  Perhaps another integrative approach might be better, especially approaches dealing with more ‘granular’ data and unipolar scales.  Perhaps completely novel statistical approaches are needed.  Repeated measurements on the same subject would be fascinating (and hell to analyze – although Tucker & Messick Three Mode Factor Analysis might be an initial start, if it were integrated with time series analyses).

The data I had my wife collect was ideally suited for factor analysis.  In 2015 other data collection techniques (e.g., Google Glass) would imply other or new statistical techniques.

It is my strongly held belief that “Psychology is a Crock”, until 1) every psych graduate student is completely proficient in time series analysis, three-mode factor analysis, cluster analysis and newer statistical multivariate methodologies; 2) every psych PhD student has done at least one ‘N of 1’ research project; and 3) full professors would be expected to have integrated many ‘N of 1’ studies to demonstrate a theory.  On the other hand, like the proliferation of individual DNA genome databases, the posting of individual theories of personality would make such integrations easier.

Any psychologist who confirms a theory by comparing two averages (across many people) or computes a correlation (across many people) at a single or a couple of time points should be laughed at, or pitied!  Given that maturation takes decades, I can forgive ignoring time/situations, but never ignoring people – individuals. You cannot study people (Psychology) by computing averages. Those pseudo-psychologists who are unable to make the transition, should be moved over to Sociology or Biology, where group amalgams are appropriate.

Am I being harsh?  Would you trust an airplane built by an engineer who didn’t understand algebra or use computers/calculators?  Would you trust a cardiologist who didn’t understand how to read an EKG or measure blood pressure?  Would you trust a psychologist who never formally understood a single individual and ONLY used ‘intuition’ to make their insights?  Yes, there may be an occasional old-school psychologist who made a cunning insight.  But certainly not many insights and certainly not most psychologists.

This entry was posted in Psychology. Bookmark the permalink.

6 Responses to 26. Psychology II: A Totally Different Pardigm

  1. J Toby Mordkoff says:

    Let me address your four “basic points” about psychology as a science.

    1. [Psychology] uses crude tools and inappropriate methodology to understand behavior/thoughts.

    I’m really tempted to write “if you don’t understand the difference between (overt) behavior and (covert) thoughts, then you’re not ready to comment on psychology as a science,” but that wouldn’t be helpful. Instead, I’ll ask what you’d use to study behavior if not what the subject does, including how long it takes them to do it. (Please note that the most-popular measure in psychology is response time.)

    With that said, I’d agree that many operational definitions in psychology still need work. But you need to address this issue in a less superficial manner for your opinion of psychology to have any value to anyone.

    2. Rather than focus on people, it [psychology] focuses on groups.

    This is completely and utterly false. The study of group behavior is sociology; psychology is the study of individual behavior. Yes, we sometimes study the effects of other people (plural) on the behavior of an individual, and we sometimes study how an individual’s behavior changes when he or she is in a group, but the focus is always on the individual. How could you not know this?

    What can confuse the naive or overly-critical is how psychologists use sample sizes greater than one. This is because our target of generalization is people from the sampling population, so we take samples and conduct statistical tests in which the interaction between the effect and subjects is always in the error term. Again, not because we want to say anything about groups; because we want to says things are generally true about individuals.

    3. The utility (squared effect size) of their studies, although it is not zero, it is near zero (≤ 10%). Their studies are practically useless.

    Yes. The eta-squares are usually quite small (as long as you ignore the “laws” of psychology, such as Hick/Hyman Law, which often have a squared effect size in the .90 range). But this is because most studies only include a small number of manipulations and rarely include covariates for all other influences that we’ve already found. When you put together all of the effects sizes that we have for a given behavior, then the total explanatory power is quite good, especially considering the huge range of options for human behavior.

    It seems like you just don’t get the “divide and conquer” strategy that psychologists use. We deal with a small number of factors at a time, holding everything else constant or allowing them to vary randomly (with the hopes that random assignment protects us from confounds). When someone wishes to stop doing theory-testing and do something applied, they can apply all of the effects that we know of that are relevant. If you have a principled reason why this approach won’t work (and I’ve seen people try to argue this before), then have at it. But just laughing at the small effect sizes ONE AT A TIME is to completely miss the point.

    4. Many theories are more descriptive of their author, than for people in general.

    Ha ha. And you write naive and inflammatory blog posts. So what? An ad hom is not a valid argument.

    – JTM

    • Thank you very much for replying to my blog. I love thoughtful replies.

      In any case, let me address your comments.

      1) I agree that the only thing one can measure is (overt) behavior. We can ask about (covert) thoughts. William James and the early psychologist asked Observers about their thoughts, but that approach disappeared a century ago. I mentioned nothing about thoughts within either blog, albeit my single graduate empirical human study was entitled ‘Asking my wife what she thinks’.

      My comment focused on ‘crude tools and inappropriate methodology’. I talked about how psychology considers all subjects identical, like lab rats, with a single mean/variance. I talked about analyzing means. I talked about the effect of a single individual when N gets large, i.e., the effect of an individual approaches zero. The crude tools pertains to the ‘t-test mentality’ endemic to almost all of psychology. [Note: the t-test mentality also refers to 3 way ANOVAs. BTW, a F[1,30] is identical to a t-test squared with 30 df.] In the stat class you teach, all the topics in the syllabus refers to one dependent variable.

      You asked what I’d use to study behavior. To be frank, I don’t know, everything JOINTLY I guess. Reaction time is valid. Asking people questions is valid. Observing them is valid. I mentioned Google Glass and location from our cell phones as valid sources of data. All are valid. But psychology (e.g., Psychology 5050) collects and focuses on one parameter (e.g., reaction time) and ignore all others. How can a true science of a complex structure exist by the inappropriate methodology of measuring one, AND ONLY, parameter, in isolation of all others. I am reminded of the Indian parable of the blind men looking at an elephant. My blog pertained to crude tools and inappropriate methodology. My feeling is that psychology cannot proceed until ALL parameters can be JOINTLY synthesized. Until then the conclusion that an elephant is long, very hard and pointed cannot be dismissed as a description of an elephant. In the parable, the blind men spent their time arguing on the true nature of the elephant. If they were psychologists, they’d have multiple publications on the tensile strength/hardness of the elephant (i.e., tusk). Others would talk about the similarities of the elephant (i.e., ear) and different samples of leather. This completely refutes blind man A’s conclusions.

      2. Psychology says it studies the individual, but does it? I should have been clearer. I am not talking about the environment in which the study was run, whether the subject was isolated or among other people. Yes, your RT studies (e.g., Chen and Mordkoff, 2012) put subjects in a “dimly lit, soundproof chamber”. However, when you do your studies, do you actually investigate the response of a single subject in-depth? Or do you get a few measurements on a person, then pool, mingle, dilute that individual until s/he is impossible to distinguish from all other individuals? Do you test your theory within each subject? No, you don’t study individuals, but herds of individuals. The summary statistic in your ANOVA, the mean, is sum of X divided by N. When you sum, the individual gets aggregated with all others and lost. When N is large, the influence of that individual is completely lost. You said yourself, psychology’s ‘target of generalization is people from the sampling population’. The key word here is population. You are trying to generalize to a population, not a person. The first Google definition of Generalize is “make general or broad statement by inferring from specific cases.” This reminds me of Box’s comment “All models are wrong, but some are useful”. This brings us to comment 3.

      3. Psychology: Explains Virtually Nothing. In my first Animal Learning class the professor, a Behaviorist, commented on statistics “You don’t need statistics to prove that a thrown brick will smash a window”. The Hicks/Hyman Law (the more alternatives a person has the longer it takes them to choose) is such a triviality. “If you prick us, do we not bleed” is not very insightful about medicine.

      Yes, I see that psychology uses the ‘divide’ approach, but never the ‘and conquer’ approach. My central thesis is that ‘most studies only include a small number of manipulations and rarely include covariates for all other influences’. By taking such very thin single slices of a complex organism you get back to the blind men and the elephant parable. When has psychology EVER applied all of the effects that we know are relevant and tested how much it explains? When you do that I’d be curious on the effect size. I’d certainly bet you a dollar that it would still be an effect size much, much less than 50%. I would guess the adjusted effect size less than 15%. Psychology explains almost nothing.

      4. An ad hominem attack is not a valid argument. True. But if each of us are unique to a large extent, how can one unique individual ever understand a second? Central to my position is that people are highly predictable. My wife and children know me. They can easily predict what I’d say/do in virtually any situation. But could you? Could you know what I like to eat? Watch in the movies/TV? Say about Obama-care? My reaction time to any stimuli? By being trapped in the sum of X divided by N methodology (i.e., limited to ANCOVA and p-values), a psychologist could only describe the herd behavior, not me. Only by acutely observing an individual could you understand individuals – effect sizes less than 10%.

      • J Toby Mordkoff says:

        I’ll probably respond one piece at a time, starting with the first point, when I have time. I’m hoping that this will be useful (although your references to my grad course in stats makes me worried that you’re somewhat creepy, as well as an avid user of the ad hom argument).

        I did not see you as arguing that “psychology considers all subjects identical, like lab rats, with a single mean/variance” because that’s too inaccurate to take seriously. Maybe I was being too charitable. Maybe you really do think that there are psychologists of note who believe something like this. In any case, you’re wrong.

        First, there’s the huge divide within psychology between, for example, those who ended up in the Psychonomic Society and those who formed the Psychometric Society. The former treat individual differences as part of the error term; the latter are focused on these differences. To say that psychology – as a whole – “considers all subjects identical” is just nonsense. Even Psychonomes know about individual differences; we’re just working on something else first. And pretty all that Psychomets study comes down to individual differences.

        Second, you are, again, treating a deep question in a naive and superficial manner. There are many ways that one can think of the question of whether people are all the same. One is to focus on, for example, mental architectures (which is the place where you’ll come closest to having a point). The second Paradigm Assumptions of (modern) cognitive psychology is that all (normal) people have the same mental architecture – i.e., the same set of components. This is often called The Universal Mind Hypothesis. (FYI: the first Paradigm Assumption is the idea that the mind is decomposable into components; I hope that you can see why they are in the order that they are.) Note that Universal Mind does not to say that the *contents* of every mind is the same. No-one would say that. The assumption is that all minds are divisible into the same set of components. That the contents of these common mechanisms are different across people – which we all acknowledge – makes it clear that we don’t expect the same behavior from everyone. We don’t even believe this about Norwegian white rats.

        You then answered my question about what measures to use with an argument that more than one should be used at the same time (and then got real creepy). This, too, is simplistic and misleading. You gave no principled reason why. My reply to your reply is to ask what current theories make simultaneous and dependent predictions for two or more (different) measures. And, then, when you have found one, please check to see if multiple measures were used in the related research.

        If a theory makes predictions for one thing, the way to test the theory is to measure that one thing. If a theory makes simultaneous and dependent predictions for two things, the way to test the theory is to measure those two things at the same time in the same subjects. For example, if the theory predicts that the lagged correlation between depression and anxiety should be stronger in one direction for subjects in one condition than for subjects in another condition – an example that I use in my course, showing that your claim that I do not consider this possibility to be erroneous (as well as creepy) – then you must measure those two things at various times in all subjects in order to conduct the appropriate test. But if all the theory predicts is that one group will be more depressed than the other, then all of those extra measures would be a waste of time and effort (and possibly unethical).

        Your last paragraph (in Part 1) included several statements of this sort: “Reaction time is valid.” That is the kind of thing that I’d only expect to see from a first-year undergrad. The word “valid” – when used without a modifier – refers to construct validity. Construct validity is the extent to which a set of empirical measures provides an accurate estimate of a target theoretical construct. As such, no measure is either valid or invalid. Not only is construct validity a continuous variable, but a measure only has construct validity with respect to a given theoretical construct. Mean response time, for example, provides a good estimate of how long it takes a person to perform a task; it’s a highly valid measure of processing duration. At the same time, mean response time provides terrible estimates of anxiety. Thus, the amount of construct validity enjoyed by RT depends on what it is being used for. It’s simplistic to the point of being incredibly insulting and misleading to say that any (well-trained) psychologist would ever say that “response time is valid.”

      • J Toby Mordkoff says:

        As it turns out, having now read the rest of your reply, no more replies from me are warranted. I have no idea how to explain to you why we use multiple subjects when we are interested in the behavior of individuals, especially in light of what you have said about psychology’s various approaches to individual differences. You seem to be criticizing psychology for trying to understand human thought and behavior without giving you a detailed explanation for a specific behavior of a specific human. That’s ridiculous. Do you also complain that classical physics only tries to explain the interactions of all objects with mass, instead of telling you why a specific eight-ball didn’t go into a specific corner pocket when hit by a specific person on a specific evening in February, 2012?

        And then you misrepresent (as well as misspell) Hick/Hyman Law in another lame attempt to make fun of psychology. Again, would you make fun of physics for saying that when you bounce one ball off another, in most cases both move? No? Why not? Is it because there’s a hell of lot more to classical physics than that? Well, guess what? There’s a heck of a lot more to Hick/Hyman Law than “people are slower when they have more options.”

        Do you always make such sweeping pronouncements when you don’t have a detailed enough understanding of the topic? Must be fun, saving all that time.

        More openly and honestly: I’m sorry that you had one of those bitter behaviorists for a professor. Heck, one of the most bitter people in the entire field is one of my academic siblings, so I know what it’s like. But I’m not silly enough to write off the field because of such people, just as I’m not going to spend much if any more time trying to warn you that your closed mind is both denying you access to a wonderful science and making you look like an idiot to those who know the field well.

        cheers

  2. Matt says:

    Perhaps experimental psychologist are inclined to study subtle effects in part because when they study things with a larger effect size they are often accused of wasting their time (and taxpayers’ money) validating that which was already “common sense.”

    • The majority of studies are not externally funded. I would also add that people are quite complex so “common sense” generic rules does not necessarily apply for most interesting things (behaviors) and most people. Nevertheless, I still claim that if you understand a person well, you can predict their behavior/responses quite well. For example, I can predict which movies my wife would like to a marked degree (95% accuracy). But predicting a foreign stranger’s likes (e.g., 80 year old Norwegian woman, or a 21 year old African Bush woman) in movies would be little better than chance.

Leave a Reply

Your email address will not be published. Required fields are marked *