Many normal people have a tendency to take scientific research as the word of god. But often times scientists fall for some of the most basic forms of statistical error. If we want to avoid the [[The Psychology of Questionable Belief|classic human biases]] we have to learn to analyze studies ourselves. Before we can question a study we have to know the claim it's trying to make. ### The three claims studies can make Frequency claim: a claim describing a particular rate or degree of a certain variable. Association claim: a claim that one level of a variable is associated or correlated with another level of a variable. ^0cf539 Casual claim: a claim that one variable is responsible for fluctuations in another. ### How to interrogate the claims with the four big validities The four validities: 1. Construct validity: how well the variables in the study are measured or manipulated. How well are the conceptual variables measured by the [[Operational definition|operational variables]]. 2. External validity: how well does the sample in the study represent the outside population it's trying to represent. Do the results of the study have generalization outside of the study design. 3. Statistical validity: how well do the numbers support the claim—that is, how strong the effect is and the precision of the estimate (the confidence interval). Also takes into account whether the study has been replicated. 4. Internal validity: in a relationship between one variable, A, and another variable, B, the extent to which changes in B can be accounted for by changes in A and not some outside variable C. What three criteria are needed for causation? 1. Internal validity 2. Temporal precedence: the manipulated variable needs to be shown to have been manipulated before the change in the dependent variable. 3. Covariance: the two variables must be correlated **Note how external validity is not necessary to show causation** ### Is there a case for not following all four validities? Most studies simply can't follow all four validities at once. This is okay. For example, external validity is often not possible to get in a study because of the artificial lab environment. But sometimes external validity is not wanted. Mook elaborates on this concept in his article, [[In Defense of External Invalidity]]. ^d11203 The issue is that many believe studies are undergone just to be generalized outside of the lab. So what could be the points of doing experiments if not to predict real-life behavior? First, we may be asking whether something can happen, rather than whether it typically does happen. Second, our prediction may be in the other direction; it may specify something that ought to happen in the lab, and so we go to the lab to see whether it does. Third, we may demonstrate the power of a phenomenon by showing that it happens even under unnatural conditions that ought to preclude it. Fourth, crafting a study in which external validity is present might be too hard. Finally, we may use the lab to produce conditions that have no counterpart in real life at all, so that the concept of "generalizing to the real world" has no meaning. But even where finding cannot possibly generalize and are not supposed to, they can contribute to an understanding of the processes going on. For example, a therapist undergoing a [[Small-N Study Design]] to try and help a individual patient. ##### What validities are most important for Associative claims? In associative claims the most important validities are ==construct== validity and ==statistical validity==. ##### What validities are most important for Causal claims? In casual claims the most important validities are internal, statistical, and construct validity. External validity is nice to have but as Mook explains in his paper, [[In Defense of External Invalidity]], it's not always preferred.