A Dunning-Kruger detection kit

Discussion in 'Science' started by Fortigurn, Mar 5, 2012.

  1. Fortigurn

    Fortigurn Member

    Joined:
    Jun 27, 2001
    Messages:
    13,329
    Location:
    taipei.tw
    In 1999, David Dunning and Justin Kruger published a journal article entitled ‘Unskilled and Unaware of It: How Difficulties in Recognizing One’s Own Incompetence Lead to Inflated Self-Assessments’.[1] Their research indicated that people who are unskilled, or lacking academic or professional qualifications in a particular field, have a tendency to estimate their knowledge and skills in that field unrealistically highly.

    Citing the observation of Charles Darwin that ‘ignorance more frequently begets confidence than does knowledge’,[2] Dunning and Kruger explained the cognitive process by which people over-estimate their competence in fields concerning which they are unskilled or uninformed. In agreement with the popular saying ‘a little learning is a dangerous thing’,[3] Dunning and Kruger noted ‘in order for the incompetent to overestimate themselves, they must satisfy a minimal threshold of knowledge, theory, or experience that suggests to themselves that they can generate correct answers’.[4]

    Accordingly, we ought to refrain from commenting authoritatively on subjects concerning which we are not academically informed or professionally qualified, and should instead seek to understand the subject from the relevant professional literature, instead of from non-professionals and those who are insufficiently qualified.

    Additionally, we should be prepared to accept that our non-professional personal views (and the views of others who are similarly unqualified in the field), are of considerably less value than the existing scholarly literature and consensus, and we should be prepared to accept that the consensus is most likely to be correct, even if it contradicts views or sources which we would prefer to believe are more accurate.

    The following dot points list indicators of the Dunning-Kruger effect, based on the reasons given by Dunning and Kruger as to why individuals succumb to the effect, and why they ‘fail, through life experience, to learn that they are unskilled’.[5] The more indicators are present in a specific case, the more likely it is that the individual in question is experiencing the effect.

    * Skill-boundary transgression: The individual is seeking to operate as an authority or qualified individual, in a field beyond their personal level of academic and professional qualification.[6]

    * Self-identified authority: The individual identifies themselves as sufficiently competent to comment authoritatively on the subject. [7]

    * Unrecognized competence: The individual’s self-assessed competence is not recognized by those who are academically and professional competent.[8]

    * False peers: The individual believes that the favourable commentary of other unskilled and non-professional individuals, indicates they themselves are sufficiently qualified.[9]

    * Scrutiny avoidance: The individual fails to submit their work for professional scrutiny (such as in the relevant scholarly literature), for review by those genuinely qualified.[10]

    * Pioneer complex: The individual self-identifies as a pioneer unconvering previously unknown or unrecognized facts; a Copernicus or Galileo.[11]

    * Conspiracy claims: The individual explains opposition by qualified professionals as a coordinated attempt to suppress truth, in order to defend the existing scholarly consensus.[12]

    * Allocentric claims of bias: The individual explains the difference between their views and those of qualified professionals, as the result of inherent bias on the part of the professionals; accusations of bias are directed at anyone other than themselves, and they claim objectivity.
    __________________________________________________________

    [1] Kruger & Dunning, ‘Unskilled and Unaware of It: How Difficulties in Recognizing One’s Own Incompetence Lead to Inflated Self-Assessments’, Journal of Personality and Social Psychology (77.6.1121-1134), 1999.

    [2] Darwin, ‘The Descent of Man, and Selection in Relation to Sex’ volume 1, p. 4 (1871).

    [3] Commonly misquoted as ‘A little knowledge is a dangerous thing’, this phrase is a line from English poet Alexander Pope’s poem ‘An Essay on Criticism’ (1709).

    [4] Kruger & Dunning, ‘Unskilled and Unaware of It: How Difficulties in Recognizing One’s Own Incompetence Lead to Inflated Self-Assessments’, Journal of Personality and Social Psychology (77.6.1132), 1999.

    [5] Ibid., pp. 1131.

    [6] ‘Incompetent individuals, compared with their more competent peers, will dramatically overestimate their ability and performance relative to objective criteria.’, ibid., p. 1122; the importance of formal academic and professional qualifications is that they constitute objective criteria by which competency can be assessed, so we should place less trust in those lacking such qualifications.

    [7] ‘These findings suggest that unaccomplished individuals do not possess the degree of metacognitive skills necessary for accurate self-assessment that their more accomplished counterparts possess.’, ibid., p. 1122; we cannot rely on those who are not academically and professionally qualified in a particular field, to assess accurately their own authority and competence in that field.

    [8] ‘We propose that those with limited knowledge in a domain suffer a dual burden: Not only do they reach mistaken conclusions and make regrettable errors, but their incompetence robs them of the ability to recognize it.’, p. 1132;. it is far more likely that an unqualified non-professional will be wrong in a given field of specialization, than a qualified professional whose competency has been recognized formally by their equally qualified peers

    [9] ‘Second, the bungled robbery attempt of McArthur Wheeler not withstanding, some tasks and settings preclude people from receiving self-correcting information that would reveal the suboptimal nature of their decisions (Einhorn, 1982).’, ibid., p. 1131; by keeping themselves predominantly in the intellectual company of those who agree with them, individuals experiencing the Dunning-Kruger effect place themselves in a setting which typically prevents their errors being exposed, instead keeping them in a kind of intellectual echo chamber in which their views are reinforced by being repeated back to them with approval by those unqualified to assess them competently.

    [10] ‘One reason is that people seldom receive negative feedback about their skills and abilities from others in everyday life (Blumberg, 1972; Darley & Fazio, 1980; Goffman, 1955; Matlin & Stang, 1978; Tesser & Rosen, 1975)’, ibid., p. 1131; avoidance of scrutiny by professionals enhances this effect, keeping the unqualified away from those who are best able to expose their errors, and preserving their self-delusion that they are correct.

    [11] This is a self-delusional identification since neither Copernicus nor Galileo were ‘gifted amateurs’ opposing a body of professionals (both men were professionals, holding formal teaching positions), and Galileo in particular knew that the subject should be decided by professionals astronomers, placing no value whatsoever on the opinions of the unqualified; writing against the papal edict silencing publications on heliocentrism in the preface of his ‘Dialogue’ (1632), Galileo scorned the unqualified amateur: ‘Complaints were to be heard that advisors who were totally unskilled in astronomical observations ought not to clip the wings of reflective intellects by means of rash prohibitions.’, Galileo, quoted in Næss, ‘Galileo Galilei: When the Earth Stood Still’, p. 131 (2005).

    [12 ‘Third, even if people receive negative feedback, they still must come to an accurate understanding of why that failure has occurred. The problem with failure is that it is subject to more attritional ambiguity to success. For success to occur, many things must go right: The person must be skilled, apply effort, and perhaps be a bit lucky. For failure to occur, the lack of any one of these components is sufficient. Because of this, even if people receive feedback that points to a lack of skill, they may attribute it to some other factor (Snyder, Higgins, & Stucky, 1983; Snyder, Shenkel, & Lowry, 1977).’, ibid., p. 1131; when an unqualified non-professional attributes opposition to or dismissal of their theories by qualified professionals as a conspiracy to maintain the intellectual status quo, the Dunning-Kruger effect is very likely responsible: an example is the Science and Public Policy Institute (a non-profit group in the US which opposes the scientific consensus on global warming), ‘People who are not scientists, or even experts on the subjects they write about often write the SPPI reports, and many convey conspiratorial themes. For example, an SPPI publication by Joanne Nova, who describes herself as a “freelance science presenter, writer, & former TV host”, exemplifies not only the ‘Dunning-Kruger’ effect (Dunning et.al. 2003), but also the inactivist movement’s frustration with mainstream climate science and its inflated sense of victimhood.’, Elshof, ‘Can Education Overcome Climate Change Inactivism?’, Journal for Activism in Science and Technology Education (3.1.25), 2011.
     
  2. mtma

    mtma Member

    Joined:
    Aug 12, 2009
    Messages:
    5,924
    Intriguing reading, but one question to be asked is even if the effect is assessed as likely to be affecting the individual, through what avenues can externs co-erce the subject to recognise their shortcomings of competence?

    (also, is this article from somewhere or is it your original works?)
     
  3. OP
    OP
    Fortigurn

    Fortigurn Member

    Joined:
    Jun 27, 2001
    Messages:
    13,329
    Location:
    taipei.tw
    They need to be confronted by those genuinely competent in the field, or peers who mediate the work of the competent to them, in a persuasive manner.

    It's my own original work.
     
  4. Foliage

    Foliage Member

    Joined:
    Jan 22, 2002
    Messages:
    32,083
    Location:
    Sleepwithyourdadelaide
    This is handy, will be useful to send this to people.

    Perhaps sending it to the gravity engine guy would be a good idea? heh
     
    Last edited: Mar 12, 2012
  5. Dicky

    Dicky (Taking a Break)

    Joined:
    Mar 12, 2005
    Messages:
    691
    Location:
    Sydney
  6. The Wolf

    The Wolf Member

    Joined:
    Sep 9, 2011
    Messages:
    449
    Sorry mate, That is absolute crap! Was Darwin qualified in the origin of species by the then authority? Credentialism, I am qualified, therfore I am correct. Qualification =/= Understanding

    If we subscribed to your ideals, we would still be at the whims of the church. Our personal views on Evolution are incorrect, since the church is the only body qualified (by God) to comment on scienctific matters
    So you belieave that light is not made up of the colour spectrum at all...... Concensus said so....

    Your whole write up is clearly biased, and aimed towards furthering your cause through the use of massive fallacys
     
  7. OP
    OP
    Fortigurn

    Fortigurn Member

    Joined:
    Jun 27, 2001
    Messages:
    13,329
    Location:
    taipei.tw
    Thanks. I have found it useful in many discussions.

    Yes I have. That's my blog. Thanks for the free advertising. If you read the blog properly you'll see my comments under the name 'Fortigurn'.

    Darwin was sufficiently educated in the subject on which he wrote, to write intelligently on it. He was academically qualified to comment intelligently, and did so.

    I have said no such thing.

    This is not in dispute.

    No we wouldn't. Galileo himself made the point that priests who had no training in astronomy were insufficiently competent to comment on the subject, and rightly dismissed their views. I made this point in a footnote; please read it.

    No. The comments on evolution of anyone insufficiently trained in biology can safely be dismissed, no matter what authority they claim for themselves.

    No. On the contrary, the scholarly consensus is that visible light does contain the visible colour spectrum. What are you talking about?

    Can you explain what you mean please? In what way is it biased? What 'cause' am I attempting to further? What 'massive fallacys [sic]' are there? I note you have failed to comment at all on the paper by Dunning and Kruger. Would you care to do so at any point?
     
  8. bjs

    bjs Member

    Joined:
    Feb 9, 2003
    Messages:
    1,878
    Location:
    SA
    Thanks Fortigurn for actually reading the original paper...

    The key point is that these indicators are just that, indicative but not conclusive. The list is by no means exhaustive however and as we know, an appeal to authority/professionalism is by no means a guarantee of accuracy.

    We know that some professional scientific fields, psychiatry and medicine for example have substantial institutionalised biases, as explored by authors such as Ioannidis and the like. Unfortunately, passing peer review in a professional journal is no guarantee of high quality.
     
    Last edited: Mar 21, 2012
  9. FerrisXB9R

    FerrisXB9R Member

    Joined:
    Jan 18, 2005
    Messages:
    3,257
    Location:
    AB, CAN
    I know this will come across as un-necessarily flippant, but it's also a summation of my understanding of the OP;

    Are you saying everyone on the internet should basically STFU?
     
  10. Paronga

    Paronga Member

    Joined:
    Sep 27, 2006
    Messages:
    1,409
    Location:
    Morrabbin, Victoria
    god, sometimes i wish they would!
     
  11. OP
    OP
    Fortigurn

    Fortigurn Member

    Joined:
    Jun 27, 2001
    Messages:
    13,329
    Location:
    taipei.tw
    Welcome.

    True. But the fact is that those who are unqualified in a specific field are typically less competent in the field than those who are both professionally qualified and have the recognition of their equally qualified peers.

    Yes this is true, but peer review is still a whole lot better than free market ignorance.

    No, though it would be lovely if they did. An overwhelming majority of them certainly should. The Dunning-Kruger effect is exhibited strongly on online forums, where people think that just because they're on the internet and have access to Google and Wikipedia, they can pontificate authoritatively on any subject they like.

    Then they get pulled up by someone who is actually educated in a field relevant to the topic under the discussion, and either start whinging that 'This isn't university!', or try and save face. What they should have done was kept their mouth shut in the first place.
     
  12. ikonz0r

    ikonz0r Member

    Joined:
    Dec 23, 2001
    Messages:
    2,551
    Location:
    Brisvegas
    I can't see the part where you acknowledge that experience and success overcomes base qualifications at a certian point?
     
  13. OP
    OP
    Fortigurn

    Fortigurn Member

    Joined:
    Jun 27, 2001
    Messages:
    13,329
    Location:
    taipei.tw
    I didn't use the term 'qualifications' to mean merely academic certification; I actually said 'academic or professional qualifications'. I already addressed the false assumption of a previous poster who thought I was saying 'Credentialism, I am qualified, therfore I am correct'.
     
  14. Hamulus

    Hamulus Member

    Joined:
    Aug 19, 2006
    Messages:
    2,003
    Location:
    Carlingford NSW
    Thanks Fortigurn. This is an excellent and very thorough post. Was aware of the Dunning-Kruger effect, but have never seen it explained before in such detail.

    As an aside, can I ask what qualification or expertise you have in the field of biblical scholarship? Not being a smart-arse here, but I'm genuinely curious. I have a degree in Theology from ACU, so have more than a passing interest in biblical studies myself. I find it remarkable however that anyone who has studied biblical criticism could think that the bible is factually accurate. I haven't read much of your blog yet, but it seems so far that you are defending its literal truth. Is this true?
     
  15. Sankari

    Sankari Member

    Joined:
    Jun 13, 2009
    Messages:
    31,724
    Location:
    Adelaide, SA
    Theology! The queen of sciences! :)

    Currently studying a Bachelor of Theology myself (Tabor College). :Pirate:
     
  16. OP
    OP
    Fortigurn

    Fortigurn Member

    Joined:
    Jun 27, 2001
    Messages:
    13,329
    Location:
    taipei.tw
    You're welcome. I've been interested in the Dunning-Kruger effect for some time, and what I really wanted was a systematic way of applying the study to real life situations, so I thought I would sit down and write a D-K detection kit.

    I have no formal qualifications in biblical scholarship at all. However, I do have a library of academic theological, historical, and archaeological works which would rival some seminaries, which has taken me over ten years to collect and is worth over US$20,000 (yes, that's twenty thousand US$). Add to that an interest in theology and biblical studies, a lot of spare time, and the fact that I'm a professional researcher, and that blog is what you get (along with a couple of books and a few journal articles).

    The phrase 'literal truth' is loaded, as I'm sure you're aware, but I'll save you time and say 'no'. In the Bible I don't find simple literalism, I find poetry, psalmnody, theologically motivated historical didactic discourse, eschatology, typology, ritualized theological instruction, and expressions of dominant and marginal cultic praxis and doctrine. In other words, I read each text in its socio-theological, historical, and literary context.

    If you read my blog in depth you'll see I follow the scholarly consensus on pretty much any issue you can find. I am not a Biblical literalist, I am not a Fundamentalist, and I apply the standard methods of higher criticism (including redaction, genre, literary, and discourse analysis), as well as textual criticism, just as you were taught in seminary.

    You will also find that in my blog I typically don't attempt a defense of the supernatural claims of the Bible. As anyone with a degree in theology knows, however, there is plenty of factual accuracy in the Bible on a range of subjects; even from the point of view of secular higher criticism, it's not all myth, fiction, and supernatural claims (the chronology of the kings of Israel and Judah is a case in point).

    Since I am unqualified myself, you will find that I do not present my own opinions, nor appeal to my own authority or conclusions; I simply systematize and present the majority conclusions of the relevant scholarly literature (not the North American evangelical echo chamber), in a form which is accessible to others. You will find the literature I cite is typically from the highest order academic publishing houses: Oxford, Cambridge, Brill, Eisenbrauns, Walter de Gruyter, Peeters, Mohr Siebeck, Peter Lang, Continuum, Sheffield, etc.
     
  17. irR4tiOn4L

    irR4tiOn4L Member

    Joined:
    Nov 25, 2004
    Messages:
    3,084
    Location:
    Carlingford, Sydney
    Put in context, its not just about self-assessment. A society mostly composed of the less educated assessing themselves highly will, without a frame of reference, gravitate toward recognition and reward of the less educated. Ergo communism's raison d'etre (how could anyone think putting the miners in charge was a good idea?). Thats why our politics is about sound bites, why Phd's do little to improve pay prospects and why intelligent kids often feel isolated and withdrawn until they hit uni.

    Of course, reality is a bit of an equaliser and when it speaks and recognition of what it says is the metric, its easy to tell the difference between the scientist who understands it and the ignoramous who does not. But the more advanced a society and the further a field is from physical phenomena (taken loosely as the split between, for example, the sciences and humanities), the more individuals within the society are isolated from being judged on their ability to tell right from wrong.

    To me this really just sums up why education for everyone matters in a democratic society if its not just the experts making the call. Just look at the split between scientists and the public on global warming to see the wider social implications.
     
    Last edited: Mar 27, 2012
  18. Foliage

    Foliage Member

    Joined:
    Jan 22, 2002
    Messages:
    32,083
    Location:
    Sleepwithyourdadelaide
    A bump for a great thread, I think this goes hand in hand with this topic:

    Four stages of competence

     
  19. [PnP]dredd

    [PnP]dredd Member

    Joined:
    Jul 4, 2001
    Messages:
    478
    Location:
    Canberra
    Interesting, but very focussed on an academic/research world view.

    How does this apply to decisions that are made in the real world? The people that make big decisions (e.g. CEOs / politicians) cannot afford to study primary literature in order to inform their decisions. They also cannot only rely on "experts" because not every decision is technical - there are often other factors that come into play.

    Would it be better if we had a completely technocratic society where experts made all of the decisions without consulting with any non-experts?
     
  20. chip

    chip Member

    Joined:
    Dec 24, 2001
    Messages:
    3,964
    Location:
    Pooraka Maccas drivethrough
    I think you're missing the point somewhat. Dunning-Kruger effect applies to just about anything that requires expertise and knowledge. For example, ask any plumber/sparky about their DIY horror stories of jobs done by amateurs who were completely ignorant about their lack of skill and domain knowledge.
     
    Last edited: Feb 18, 2014

Share This Page

Advertisement: