Stage 1.
Critical Thinking

Thank you for your input! Please, see below the collated data and explore more about critical thinking and digital technology as seen by experts in edu and game fields.

Stage 1.
Critical Thinking

Thank you for your input! Please, see below the collated data and explore more about critical thinking and digital technology as seen by experts in edu and game fields.​

The Delphi Study

This website presents the results of a Delphi study. The current Delphi research is Kantian – “contributory” in its nature (Mitroff & Turoff, 2002, p. 27). It means that participating experts are building a bigger picture of what is known about digital games for the sake of critical thinking.

The whole study consists of 3 iterative stages. Each stage produces a report for you to reflect on – agree, disagree, or offer an alternative. Starting from Stage 2 your remarks are presented in the “Comments” section of the webpage.

This Delphi study counts each idea, and I made every effort to represent the spectrum of your knowledge. While creating each webpage (report), I assigned codes and grouped them into categories (see “Grouped data” block). The text you see below is a condensed essence of your ideas. First, I’m presenting categories and codes with the highest frequency count. This way you may find what the majority of you agreed upon. Also, take your time to explore the ideas mentioned less frequently.

36 experts participated in Stage 1 of this study.

How to proceed?

It is a Delphi study, which means that you are welcome to express your position about the information provided by you or your colleagues.

You may disagree, agree or propose anything else related to what you see on this webpage.

To do so, please, highlight a block of text you are referring to and press CTRL+ENTER to leave your comment.

You may leave as many comments as you wish.

How to proceed?

It is a Delphi study, which means that you are welcome to express your position about the information provided by you or your colleagues.

You may disagree, agree or propose anything else related to what you see on this webpage.

To do so, please, highlight a block of text you are referring to and press CTRL+ENTER to leave your comment.

You may leave as many comments as you wish.

Where we are local:

Albania, Australia, Austria, Belarus, Canada, Denmark, England, Finland, France, Germany, India, Italy, Japan, Kazakhstan, Malaysia, Mexico, Netherlands, Nigeria, Poland, Scotland, Spain, Taiwan, The United Kingdom, Turkey, USA.

Almería, British Columbia, Capital region of Finland, Colorado, Florida, Louisiana, North Texas in the USA, Soviet Union, Victoria (Australia).

Almaty, Almerimar, Amsterdam, Astana, Baltimore, Bangalore, Busto Arsizio, Copenhagen, Denver, Glasgow, Istanbul, Johor Bahru, Lagos, Langley, Madrid, Manchester, Maryland, Melbourne, Minsk, Montreal, Nancy, New Delhi, New Orleans, Paris, Riyadh, Saint Petersburg/Leningrad, Shinjuku, Shkoder, Taipei, Tallahassee, Tampere, Varese, Vienna, Xalapa.

Albania, Australia, Austria, Belarus, Canada, Denmark, England, Finland, France, Germany, India, Italy, Japan, Kazakhstan, Malaysia, Mexico, Netherlands, Nigeria, Poland, Scotland, Spain, Taiwan, The United Kingdom, Turkey, USA.

Almería, British Columbia, Capital region of Finland, Colorado, Florida, Louisiana, North Texas in the USA, Soviet Union, Victoria (Australia).

Almaty, Almerimar, Amsterdam, Astana, Baltimore, Bangalore, Busto Arsizio, Copenhagen, Denver, Glasgow, Istanbul, Johor Bahru, Lagos, Langley, Madrid, Manchester, Maryland, Melbourne, Minsk, Montreal, Nancy, New Delhi, New Orleans, Paris, Riyadh, Saint Petersburg/Leningrad, Shinjuku, Shkoder, Taipei, Tallahassee, Tampere, Varese, Vienna, Xalapa.

Grouped data

In many sections to follow you will see a number in brackets after a word or expression, e.g. “analyze (9)“. The number stands for the number of experts who mentioned this idea or concept – just hover over it to see the data  provided by experts.

You may see a number with the prefix “x”, e.g. “in-game ‘messy problems’ with no fixed answer” (x1). It means that this is the idea of an expert; it is the closest paraphrase or quotation of the original message.

When phrases of several experts are exact (repeated word-for-word) or essentially mean the same, they receive the prefix “(x2)”. This means “analysis of a problem” (x2) is represented twice within the concept of “analyze(9)”.

How we understand critical thinking

CT is an ability to evaluate or assess(12), analyze(9), question(8), understand(4), identify(3), synthesize(3), conclude(2), interpret(2), reflect(2), challenge(1), check(1), classify(1), compare(1), concede(1), create(1), discuss(1), examine(1), imagine(1), infer(1), refute(1), review(1), subdivide(1), summarize(1), test(1) and use information(1). CT elements should be linked through their utility to solve a problem(x1).

CT is used to make an informed/evidence supported decision or conclusion(8), resolve problem or situation(6), attain a holistic understanding of a problem, situation, or subject(5), form a judgment(4), create a meaning(4) or progress a topic towards a most probable future outcome or use(x1). The decision or judgement is ought to be made after examination of facts(5), which are differentiated from or opposed to emotions and opinion(3). CT is also a decision making in a critical situation or crisis, which may appear in any social environment(x1).

CT is reflective(5), unbiased(5), objective(4), reasoned(4), logical(3), rational(3), active(2), out of the box(x2),  clear(1), dialectic(1), high order(1), independent(x1), lateral and convergent(1), skillful(1), strategic(1) and voluntary(1) way of thinking. CT should enable to look for logical fallacy(2) and other faulty reasoning(x1), being aware of non-arguments(x1), consider the ways in which people might react to issue on a non-rational basis(x1), and surpass subjectivity(1). CT means not taking things at their face value(2), to problematize phenomenon(2), and to consider an issue from several perspectives(10). “Critical thinking is the ability to … see the reality from different views and create an own opinion about it” (x1).

CT requires flexibility – ability to recognize your own a priori or beliefs, or that your views can be wrong, and be able to suspend them/alter judgement in the face of new information, facts(3). It is “a reflective stance free of prejudices, having an open mind, being mindful”(x1). CT is “developing a deep understanding of processes and procedures of learning a skill or concept”(x1). “It almost requires certain amount of cynicism and skepticism to [discern statements] based on emotions rather than facts”(x1).  CT is when a child thinks outside the box and progresses from simple tasks [which form his imagination] to more challenging(1).

“[CT] seems more like a way of approaching life, rather than certain heuristic list of things to check to form a judgment”(x1). CT may be used to discern “fake news”(2) and “big claims” which have no foundation(1), “challenge or progress modern thought”(1), learn a language or anything else(1), construct arguments in higher education settings(1), or oppose counterfactual narratives when playing games(1). It is crucial to most workplace settings(1).

“Critical thinking is neither good nor bad when looked from a philanthropic point of view it depends on the individual or a group of individuals as to how they are leveraging the potential powers of critical thinking” (x1).

Critical thinking is also about effective communication(x1) and making others think about problem solution(1).

Additional definitions mentioned:

  • “Critical thinking is the ability to think clearly and rationally about what to do or what to believe” (x1);
  • “Objective analysis methods” (x1);
  • “The term ‘evaluative judgement’ is a neat alternative [for CT]” (x1).

Grouped data

In many sections to follow you will see a number in brackets after a word or expression, e.g. “analyze (9)“. The number stands for the number of experts who mentioned this idea or concept – just hover over it to see the data  provided by experts.

You may see a number with the prefix “x”, e.g. “in-game ‘messy problems’ with no fixed answer” (x1). It means that this is the idea of an expert; it is the closest paraphrase or quotation of the original message.

When phrases of several experts are exact (repeated word-for-word) or essentially mean the same, they receive the prefix “(x2)”. This means “analysis of a problem” (x2) is represented twice within the concept of “analyze(9)”.

My concept of critical thinking is largely a product of:

  My own thinking

  One or more particular theories of CT

  Both options

  Not sure

Theories and conceptual models which shaped our understanding of CT:

Core CT elements

Click twice on “Total Count” to learn more about the most cited CT elements.

You may also click on any of the green pluses to find out excerpts from original responses.

Choose 10 the most relevant to CT concept words

Click twice on “Total Count” to learn more about the most cited CT elements.

Why does an individual need CT?

Note: the names of categories below created to provide you a general outlook of reasons to have CT. Please, look inside to learn more!

To make the best possible(1), judicious(x1), structured(x1), “informed(2) decisions about how to act or make a choice”(x1), decisions based on debate, discussion, and analysis(1), and not dictated by “emotions, blind belief in faith, and falsehoods”(x1) or “feelings and bias”(x1). “Critical thinking is the basis for logic and reasoning”(x1), for reaching conclusions based on extensive questioning(3). CT is needed to deal with problems(2): find new solutions(1), solve real-life(2) or complex(1) problems, respond quickly in a crisis situation(x1). To achieve a goal in a variety of ways by removing standards and frameworks(x1), think out of the box(x1). CT is required for identifying truth from fiction/falsehood(4), attaining a holistic understanding(4), creating a meaning(1) and “learning how to question all assumptions and claims before accepting any of them on face value”(3). It helps us realize “the multi-fold impact of interconnected things which leads to a conclusion/consensus”(x1). It “is the basis of worthwhile actions into the future – projection, forecast, etc.”(x1). “Thinking clearly and systematically can improve the way we communicate or express our ideas”(x1).

Critical thinking is “the basis of all scientific reasoning”(2). The “critical thinking approach is correlated with higher education”(x1), it is “at the heart of the philosophical roots of higher education”(x1). We may apply the concept of critical thinking to a lot of different contexts(x2) or topics, “especially complex ones like language, rhetoric or in computational thinking”(1). CT is needed to “be able to understand the world around”(x1) or “avoid arriving at a faulty understanding”(x1), and “to be able to act on that understanding to ensure human survival”(x1).
“Being able to “critically think” … will lead to more happy people who know what is their place in the world, city, job, family etc.”(x1), who can “navigate”(1) and “succeed in life”(x1), it is a “life-saving skill in the long run”(x1). CT is needed to succeed in education(3) and for adults to “have a great career”(2).

CT “enables a conscious awareness in an individual”(2) “to not take things at their face value”(2), carefully consider information “before forming a part of our personal belief system”(x1), “put aside their own biases”(2) and “free an individual from preconceived beliefs”(x1). “It helps in self-reflection, self-monitoring, and being self-corrective”(x1), including reflection on our own learning(1).  “With critical thinking comes the ability to be able to accept that there are multiple ways one is doing “wrong” or “inefficiently” in their personal lives, or their views on certain topics are subjective. It is fine to have any ideas, feelings, emotions and opinions, but one should know how and why they think or feel the way they do”(x1).

CT is “essential to be an effective human being”(x1). CT is also “the basis for reaching a certain level of maturity”(x1), it “facilitates a personal transformation”(x1) and is needed to build oneself as an individual(x1), to have an opinion(2).

To not be vulnerable to propaganda(1), fake news distributed in news outlets and social media(4), “unfounded political positions”(x1). CT is needed “to not be abused by politicians, media, governments, and companies who drive their own products, ideologies”(8). “If a society has enough people that can’t think critically, it also means poorly informed leaders will be elected and make decisions that impact people negatively (ex: Trump)”(x1), it also “resulted in increasingly decisive and damaging politics [the rise of populist views]”(x1). CT is needed “to not be a mere sheep in our totalitarian system where everyone is being brainwashed to comply”(2).

Critical thinking “is the engine of progress”(x1): social(x2), “cultural and economic”(x1). “It provides the impetus to generate change and innovation”(x1). CT is “required for sustaining and improving any functioning society [to achieve social justice]”(x1). CT is important(x1) and “vital for modern day society”(x1), it is one of the basic skill-sets for modern citizens(3). “If a society has enough people that can’t think critically, it also means poorly informed leaders will be elected and make decisions that impact people negatively (ex: Trump).”(x1).

Do you think there is a difference between "good or effective thinking" and "critical thinking"? ​

22 participants believe that critical thinking and “good or effective thinking” are different concepts.

10 participants describe critical thinking as a more deep or meaningful version of thinking, or/and as one which takes into account multiple perspectives compared to good or effective thinking.

And 1 participant suggested that all types of thinking have a connotation of doing the thinking in a thorough and focused way.

Good and effective thinking were described as: only thinking about a topic (x1), logical thinking (x1), the ability to make decisions based on quick judgements that often involve assumptions (x1), superficial (x1), anything from pondering wild possibilities to imagining different worlds (x1).

Effective thinking was described as: ability to reach a solution for a problem (3), thinking used in situations which do not require CT (x1), effective thinking may be convergent (x1). 

Good thinking was described as: “critical thinking is not necessarily associated with “good” (in value or moral sense)” (2), good thinking has a lot of subjective elements (x1), good thinking can occur in imaginative, creative or other states that aren’t necessarily critical (x1).

Critical thinking was described as: a deep thinking about solutions and creativity (x1), CT reaches the highest levels of thinking to solve problems and analyze them, find appropriate solutions; the ability to understand a problem, explain it and consider different solutions to it and its consequences (x1), the concept of CT refers to the logical processes for evaluating information (x1), critical thinking always involves the rational and multi-sided application of System 2 (x1), CT is more about precise details so that the decisions are based on deductive logic (x1), critical thinking would involve perspective taking and in-depth communication (x1), critical thinking is not necessarily associated with what may be constituted as either “good” (in a value or moral sense) or “effective” (in a productive sense) (x1), CT is by its definition geared towards making a judgment (x1), CT involves more complexity [as opposed to good or effective thinking] (x1), CT is a hybrid of fact-based analytical thinking and subjective thinking but the subjective thought process is the later part of the critical thinking approach (x1), CT is divergent (x1).

Some of you explicitly proposed an option of the hierarchy/relationship/subordination between critical thinking and good and/or effective thinking:

  • Critical thinking can be good/effective or not, and good thinking can be critical or not.
    Good thinking can occur in imaginative, creative or other states that aren’t necessarily critical. However, if you want your thinking to be effective – critical thinking should always be a part of your strategy;
  • Critical thinking should be a subset of effective thinking. While critical thinking can be said to be effective, but it should not be the only form of thinking which are effective. For a very experienced person, intuition (hone through years of experience and critical thinking within the scope) can also be a good form of thinking;
  • Good or effective thinking is a subset of CT. All “critical thinking” is “good/effective” but not necessarily the other way around. Also, what is “good” for one party in a situation might not be the same from someone else’s perspective;
  • Critical thinking is one of the types of effective or creative thinking.

Several experts proposed explicitly confronting positions:

  • 3 participants introduce the dimension of consequences or the “moral” dimension of CT, in other words, thinking not only about how to achieve or solve the problem, but consider consequences. However, 2 experts suggest that the “moral” dimension is not necessarily a part of CT. We may also note that “logical thinking” is differently attributed to critical and effective thinking in these opposing responses.
  • There might be also a difference in whether creative or imaginary thinking should be a part of critical thinking. Here you may find the position of proponents (4) and opponents (2).

2 experts think that there is no difference between critical and effective/good thinking:

    • I think they are the same. Using many disciplines and views of knowledge to solve problems is effective and sums up what critical thinking means. If we want our thinking to be effective/useful we must approach it in a critical way;
    • No [there is no difference].

    4 experts couldn’t conceptualize effective and good thinking or challenged their existence:

      • There is no good or effective thinking whatsoever. Nobody can tell you if there is a good/effective thinking because there is no bad or ineffective thinking. Critical thinking gives you the opportunity to think deeply/thoroughly about the information, it opens the doors to different perspectives;
      • I don’t really know what good or effective thinking refers to, but it sounds like it could be in the same category as critical thinking;
      • I don’t know what is meant by good or effective thinking;
      • Difficult to say depends on the material feed.

      Is there a relationship between culture and CT?

      Is it possible to measure CT?

      Most likely CT is:

        Yes

        No

        Yes

        No

        General skills and abilities - can be taught/used across many disciplinary domains

        Domain and problem area specific

        Another position

      Is there a relationship between culture and CT?

        Yes

        No

      Is it possible to measure CT?

        Yes

        No

      Most likely CT is:

        General skills and abilities - can be taught/used across many disciplinary domains

        Domain and problem area specific

        Another position

      Can CT be taught?

      0

      participants answered "Yes" (0 - "No")

      What kind of digital technologies do you believe support the development of critical thinking?

      Mentioned by 24 participants, including: coding games: Like code Monkey, Frozen in Code.org and Minecraft Education Edition (x1); they [games] would be most effective in co-operative environments (x1); if they can provide access to multiple ways of viewing issues, and show systemic consequences of decisions (x1); in the right game, critical thinking is required to advance – the learning activity is built into the game (x1); I think that digital games can refine critical decision making for specific outcomes (x1); digital games through their experiential interactive nature (x1); Minecraft Creative and Playful Learning – Escape Rooms (x1); To be noted they [games] can also be used negatively (a blog can spread disinformation) (x1); Minecraft (x1); MMOs, match three games/apps, really any digital game with levels/consequences/scenarios (x1); Digital games are promising tools to teach critical thinking as they inherently promote systems thinking, decision making and interactivity. But again design is important, some digital games can be poor tools for teaching critical thinking as well (x1).

      Mentioned by 9 participants. The dominant idea here is that any tech solution may support the development of critical thinking when a proper pedagogy, design, or strategy is applied.

      The responses included: The tech itself has to be designed to give the user the ability/space/narrative moments to question ideas. Otherwise, tech itself won’t drive critical thinking. For example, a person who has mastered Excel Spreadsheet might not question the credibility of political information on the web any more efficiently than another web user (x1); Other [tech] means that foster collaboration and communication, and providing some guidance and feedback (x1); all types of software, applications and technologies. It depends on how the teacher uses these tools (x1); anything that engages people in a debate – but these need strong structuring and facilitation to draw out the areas of criticality. I think initially, interaction is important in developing this skill. Later on, there is definitely opportunity to develop this through writing but needs feedback and support (x1); Any digital technology that encourages problem solving will encourage users to engage in the process of critical thinking. Though one needs to be careful not to overstate the capacity of these systems to produce critical agents (x1); Pretty much everything can support it if it’s created in a proficient manner (x1); Anything that leads to computational thinking: robotics, programming (Scratch), etc as well as technologies that force us to think logically and critically (mainly games and puzzles) (x1); Сombining a variety of areas, for example, I link Minecraft with web 2.0 technologies, google tags, 3D paint, qr-code, working with reduced links, preparing for programming in professional languages and much more (x1); I believe all digital technologies can be designed to deliver pedagogy which promotes critical thinking. All but at the same time none of the digital technologies can be used to teach critical thinking, its all on the design and strategy (x1).

      Mentioned by 9 participants, including: they [WIKIs] would be most effective in co-operative environments (x1); can be used but would require more scaffolding i.e. the development and administration of learning tasks based on the blogs/wikis (x1); to be noted they [WIKIs] can also be used negatively (a blog can spread disinformation) (x1).

      Mentioned by 7 participants, including: can be used but would require more scaffolding i.e. the development and administration of learning tasks based on the blogs/wikis (x1); to be noted they can also be used negatively (a blog can spread disinformation) (x1).

      Mentioned by 4 participants, including: discussion boards that support images can develop the critical eye through human interaction (x1); open feedbacks (x1); Board Games Forums (x1); group discussions (x1).

      Mentioned by 3 participants, including: MOOCs are effective for this [supporting the development of CT], if they can provide access to multiple ways of viewing issues, and show systemic consequences of decisions (x1); To be noted they [MOOCs] can also be used negatively (a blog can spread disinformation) (x1).

      Tech which was mentioned in your responses once: AI, apps (Research and Writing), articles, chat-bots, concept maps, eBooks, Interactive course (they would be most effective in co-operative environments), live-streams, MS Word (even MS Word develops critical thinking as one must consider things, such as font, font size, color, layout, etc. to effectively reach an intended purpose), news websites, online learning platforms, simulations, TED talks, video footage, VR and AR tools.

      Critical thinking can be developed using digital technologies

        Agree

        Somewhat agree

        Neither agree nor disagree

        Somewhat disagree