Revision Report for:
ms 111-v2_debriefing-as-learning
Publication recommendations at this stage for v2
Rev A = 2a. Accept, but require substantial
revision in content and/or form (detailed in my commentary), and send back revised version
to me for re-evaluation.
Rev C =
3a. Accept with minor modifications
to substance and/or form (see my commentary); another review by me not necessary.
Rev D =
2b. Accept on condition that relatively moderate
changes in content and/or form (specified in my commentary) are incorporated into a revised version; and possibly send back to me for a brief review/verification.
Revs: A, C & D
We would like to, again, thank the reviewers for their thorough reviews and, in particular, for their constructive critique and suggestions for improvement. We have processed all comments and worked on improving the paper accordingly.
Besides processing the reviewer comments, the main changes to the third version are:
- The title has been shortened to Introducing the coaching cycle: A coaching by gaming perspective of serious gaming.
- More keywords have been added.
- Clearer focus towards a general approach to serious gaming inspired by current training practices by the military and, thus, softening the claim that this framework is a novel approach.
Reviewer A
Rev A = 2a. Accept, but require substantial
revision in content and/or form (detailed in my commentary), and send back revised version to me for re-evaluation.
222__ A. Importance of topic
11__ B. Aims clearly stated, with logical structure
11__ C: Aims fully achieved
11__ D. Literature review (incl jrnl S&G)
0__ E. Debriefing discussion*
11__ F. Quality of ideas, logic, objectivity
222__ G. Quality of data
0__ H. Quality of method
11__ I. Technical aspects, esp stats
11__ J. Quality of discussion
11__ K. Reliability of results; validity of conclusions
222__ L. Organization
11__ M. Coherence / balance
11__ N. Clarity, concepts, logic
222__ P. Quality of writing (Short, simple sentences, paras, topic sentence, good grammar, etc)
222__ N. Quality & clarity of visuals
11__ P. Significance of contribution to profession/field
General Comment
“I read the paper when it originally came in and was disappointed with it. I spent an hour and a half reviewing the paper again last evening. On the whole, I'm still disappointed that the authors didn't substantively address some of my comments and
comments of Reviewer B.”
|
Reviewer’s comments
|
Authors’ response
|
|
Comment 1
“I have several concerns, but the overarching one is that what is being proposed is not new. The paper presumes that coaching while being an active participant in simulation based (or game based) training is a brand new idea. I have personally
witnessed this approach dozens of times in military training. Just because it might be a less common practice in civilian settings does not justify the claim of discovery of a new approach for instructors. Instructors have changed scenarios and role played
in training (as participants), and even sat in as worker-trainees, for a very long time.“
|
We agree that, within the context of military training (and, to some extent, live role-playing exercises in civilian contexts), the idea of an active instructor or coach is common practice. Our view, however, is that this practice
is an implicit assumption that has not, by and large, been explicitly described and grounded in current theories related to game-based training and cognition/learning. In contrast, there are those (e.g. Klabbers) that assert that the instructor should not
disrupt training, but instead take a step back and only observe how the game evolves.
We have softened our claim that this part of the framework is new and further clarified the contribution of the coaching cycle to the general serious game community (further described in our response to comment 6).
|
Comment 2
“The argument that instructors should play with their students so that they can relate to the trainee experience is also not new, and I would argue that every instructor should go through the training experiences they are going to provide to
students prior to instructing others with it (and I think that this is also common practice).“
|
Even if this is common practice within the military domain, it is largely overlooked in other (civilian) digital game-based training contexts. At least, we have found very few accounts of instructors as game players in the literature. Thus, we
see that the general GBL community could benefit from a theoretically grounded description of this practice.
The clarifications made for some other comment also relate to this comment.
|
Comment 3
“I also do not think that the presumption that playing with students will necessarily increase deliberate practice. Deliberate practice is something that students put into the experience. It is not something that is put into the students by instructional
approaches. It is an age-old idea to employ instructional approaches that provide information, practice, and motivate students to try hard. Everybody knows that a knowledgeable, dedicated, exciting instructor can inspire student motivation. The interest in
using games, gaming approaches or game characteristics is to attempt to crack the motivation nut without requiring an exceptional instructor to do so.”
|
We do not claim that playing with students directly increase deliberate practice. What we claim is that:
1.
Deliberate practice is essential for the development of expertise and coaching is an important part of increasing deliberate practice
2.
Games and simulations are good for deliberate practice because they offer continuous feedback and opportunities for repetition
3.
Only relying on in-game automatic feedback could, however, lead to a pattern of incorrect behaviors that are hard to unlearn
4.
Instructors involved in gameplay can give continuous feedback in a flexible and unobtrusive way (as a complement to automatic/game feedback)
5.
Coaching does not only occur during gameplay, but is also part of the debriefing (as summative feedback and reflective exercises), so that students know what to improve during the next gaming session.
We believe that coaching to increase deliberate practice has more to do with giving feedback and spotting weak spots in trainees’ performance (both on an individual and a group level) than extrinsic motivation. We don’t think that games can replace
the instructors, neither in terms of feedback or motivation – at least not with current level of game AI. A game-based training approach still needs a highly competent instructor to succeed. Otherwise, there is a risk that the students learn how to win the
game instead of picking up the intended learning goals. As serious games researchers we must take into account the situational aspects of serious gaming, i.e. studying not only the game, but also the physical, social, and organizational contexts in which it
is used, including human actors.
We have clarified these points in the section labeled The
power of game-based training.
|
Comment 4
“The goal is to inspire student motivation while reducing the use of costly human instructors, hence ICW, web-based instruction, and training games. The authors acknowledge that cost limits what is possible in training situations (only one instructor
and no trainer operator for civilian fire fighting training) but they do not address the need to improve and/or maintain training effectiveness while reining in cost.”
|
As pointed out, we acknowledge the problem, but consider it outside the scope of this paper to fully examine the cost-effectiveness of the framework. Our focus is training quality.
We have added some text addressing the cost-effectiveness problem in the discussion.
|
Comment 5
“A paper that recommends leveraging standard military approaches to civilian training would be a more reality-grounded approach, but would probably not be worth publishing in Simulation and Gaming.”
|
We concur that the paper would be greatly improved if it was pitched more clearly towards a more general game-based training audience. We have revised the paper accordingly.
It is our hope that the editor of Simulation & Gaming finds it appropriate for this journal.
|
Comment 6
“It is obvious that the authors have read a great deal of literature related to the issues they discuss, but I don't feel that they have clearly articulated what is new and, importantly, what is the value (monetary and training effectiveness
related) of their recommendations. An article with increased clarity of these two issues and data to back them up could be worth publishing.”
|
We have tried to improve the paper according to these issues (see above comments). To sum up, what is “new” about the framework is:
1.
Current training practices are explicitly articulated
2.
Increased focus on debriefing than current practices
3.
Less focus on initial lecture/briefing than current practices (thus giving more time for gaming and debriefing)
4.
Integrated training cycle that combines current training practices with theories from deliberate practice, experiential learning and game-based learning, i.e. a theoretically grounded framework
The value of this framework is related to the development of future serious games. Even though there are promising attempts to create more advanced AI agents in games we are still far from handling the complexity and flexibility that are prevalent
in today’s training – at least to the point where the AI system can replace the instructors’ role during game-based training. However, studying what the instructors are actually doing, we can create requirements for such a system. In fact, this is an important
part of our future work.
|
Reviewer B
Rev C =
3a. Accept with minor modifications
to substance and/or form (see my commentary); another review by me not necessary.
222__ A. Importance of topic
11__ B. Aims clearly stated, with logical structure
11__ C: Aims fully achieved
11__ D. Literature review (incl jrnl S&G)
0__ E. Debriefing discussion*
11__ F. Quality of ideas, logic, objectivity
222__ G. Quality of data
0__ H. Quality of method
11__ I. Technical aspects, esp stats
11__ J. Quality of discussion
11__ K. Reliability of results; validity of conclusions
222__ L. Organization
11__ M. Coherence / balance
11__ N. Clarity, concepts, logic
222__ P. Quality of writing (Short, simple sentences, paras, topic sentence, good grammar, etc)
222__ N. Quality & clarity of visuals
11__ P. Significance of contribution to profession/field
General Comment
“I have completed my review of your second draft and can see that it is much improved from the first version. I have a few comments for you—I apologize to not get these to you earlier. I hope these comments are still helpful.
The manuscript is easy to read and interesting. I really appreciate the addition of several sections. I believe you have addressed the reviewers comments. In general, congratulations on improving your paper considerably.”
|
Reviewer’s comments
|
Authors’ response
|
Comment 1
“Reviewer A indicates that many of the recommendations have been implemented by militaries (but not perhaps public education). I have to agree with this statement. You paper is interesting, please don’t get me wrong, but I believe your paper
would have more impact if you acknowledge prior work or even instances in military training in which instructors are being/have been brought into the gaming experience. You do this a bit in the section called “the many roles of the instructor in game-based
training” and I believe you could mention other work here, primarily work I have done in creating in-game roles for instructors, peers, and training cadre (this is a shameless plug for my own work which I hoped you would have found in your literature research
so I hope you don’t mind if I have attached it directly for your perusal). I have other papers on the topic of multiple roles in-game for instructors and trainees to provide feedback in-game as well as to an automated AAR of logged game events. This approach
has been instantiated in two U.S. military games. We have put instructors in these roles and the 2009 paper discusses data collected with trainees in the roles, not instructors, but you may be interested nonetheless as this approach seems to codify your coaching
cycle notion.”
|
We have changed the paper to be more overtly targeted towards more general game-based training readers, for whom military practices are not widely known.
Your papers are good contributions to our argumentation – thank you!
|
Comment 2
“Given my point above, in the second paragraph, 1st page, you mention that “they (instructors) are reduced to mere observers, or, in the case of distance…” I would suggest that you soften this statement as my own experience with the US military
in designing systems as well as observing training events would suggest that the instructor is always an active part of the process, in one way or another.”
|
The sentence has been rephrased.
|
Comment 3
“Paragraph 4 of the first section describes coaching and deliberate practice—when I read this, I asked myself, how is this scaleable to groups in the methods you describe?”
|
Our work does not get a clear answer to this question, but we agree that it is important. We do not wish to promote a framework that requires one-on-one tutoring, but we also acknowledge that coaching is difficult with too large groups. As we
see it, larger groups have to rely on (1) automatic feedback from the AI system and (2) peer-to-peer feedback.
We have commented on this in the discussion.
|
Comment 4
“The third paragraph of your research approach suggests that you have collected and analyzed empirical data to develop your concepts. You really need to say more about this in detail (what did the data tell you?) or delete this paragraph from
the paper as the reader expects to see a data analysis section in the paper that is never presented.”
|
We have added a paragraph on data analysis in the research approach section.
|
Reviewer D
General Comment
“I have read your revision of the manuscript, Introducing the coaching cycle: Changing the instructor role by facilitating coaching for deliberate practice in game-based training systems, and find it to be an excellent revision job that takes into
account my comments and from my perspective the comments of other reviewers (though they of course will speak for themselves). The ms is much improved from version 1.”
|
Reviewer’s comments
|
Authors’ response
|
N/A
|
N/A
|
|
|
MsID =
Revs, A, C & E
Responses to reviewers
Reviewer A
|
Section of Paper
|
Comments
|
Response
|
|
Abstract
|
|
|
|
Introduction
|
|
|
|
Review of the literature
|
Although games and simulations offer some affordances that other mediated learning do not have, it is easy to get into the trap of viewing “simulation” or “game” as some kind of intervention, much like computer-assisted instruction was once conceived,
resulting in media comparison studies of CAI vs. “traditional” instruction. Richard Clark’s argument is that the media don’t influence learning, rather it is the effective use of instructional strategies and tactics. For example, feedback to learners (correction
of error) can be provided in a variety of contexts for learning, including simulations and games, and feedback is part of effective instruction (e.g., see Merrill’s First Principles of Instruction—especially the ‘application’ principle). The good news here
is that the present study is not another “media comparison” study. But the danger is nonetheless there to treat the idea of simulation or game as if it is a variable itself (or theoretical construct, which is not well-defined). There can be ineffective
simulations and games for a lot of reasons, because they do not employ what is known from instructional and learning theories (again, see Merrill, and also see Clark’s arguments).
|
The authors concur that this study is not a media comparison study.
|
|
Research Questions
|
|
|
|
Methodology
|
Yes, but why would they not also apply what they had learned in the course? And in fact, you later discover that they did. So this basically renders the first research question meaningless.
-----------------------------------------------------
With unequal and relatively small n’s, any kind of statistical inference would be difficult to justify—especially if the within-group variances are not equal.
While Rogers’ theory was one of the main ones, it was not the only theory that the original developers applied in designing the DSG (personal communication from Dr. Molenda).
Note that this was developed to be used in the context of a course in which change management was one of the topics. Furthermore, in this course context, debriefing is included as part of the use of the DSG. There is a debriefing guide that
comes with the licensed version and is recommended. There is no debriefing guide in the free version.
So why was the purple debriefing sheet NOT used that comes with the DSG? It has a list of discussion items that reinforce many of the concepts and relationships from Rogers’ theory.
|
Research questions were rewritten to clarify the questions the researchers investigated.
In rewriting the paper, participants in the study were viewed as one group.
Paper was modified to reflect that additional change models were used in the development of the Diffusion Simulation Game.
The authors were not aware of a debriefing guide for the Diffusion Simulation Game until reviewers mentioned it. The paper now includes this information.
|
|
Results
|
Means should always be accompanied by the N and SD.
Ditto. Note that only 6 students remained in Group A and only 3 in Group B. Missing data could be biasing the resulting means. So the means are rather meaningless. Showing the actual scores, as presented in Table 2 is probably the best thing.
These are all TABLES (not figures, at least in APA parlance).
More than interesting, this makes the first research question meaningless. The groups do not differ on the one dimension that presumably was manipulated (i.e., whether on not a theory was applied in playing the DSG). Given this methodological
flaw from which no recovery is possible, you might as well just talk about everyone as one big group. The A vs. B distinction is irrelevant.
|
N and SD were added to the results table.
Corrected labeling to Tables (not figures).
Revisions to the paper treat all participants as one group.
|
|
Discussion
|
It makes no sense to talk about A vs. B comparisons, since the groups are more alike than different. The theory mismatch argument may be compelling initially, but if students learn from their experience, that should override whatever theory
they might have in mind. What a different theory could provide is dissonance that might interfere initially with learning from the experience.
This is by design in the DSG. Outcomes are stochastic, not deterministic. Sometimes the “correct” strategy just does not work. According to Molenda, the probabilities of success under varying conditions were intended to mimic to some extent
what was likely to work in the real world. E.g. if a strategy under a given condition works 60 percent of the time, this is modeled in the game by 5 outcomes of a move, 3 of which are successful and 2 of which are not. So if a player does not play the DSG
enough times, she or he won’t understand the stochastic nature (it worked before but now it does not, and then it works again the next time???). Repeated play is necessary to get past the problem of jumping to conclusions too quickly based on just one or
two game plays.
Again, I have to ask, why were the debriefing materials provided with the DSG NOT utilized. This debriefing was crucial for the board version of the DSG and is just as important for the online version. Unfortunately, if the DSG is played outside
a course context, there is no obvious way to control the debriefing. People just play the game and draw whatever conclusions they may. If they play the DSG enough, they may learn something that is consistent with theory about diffusion of innovations.
|
Thank you for the comment about the nature of the Diffusion Simulation Game being stochastic. The comment helped the researchers better understand and interpret survey results.
|
|
Limitations of the study
|
|
|
|
Conclusions
|
|
|
Additional comment on the paper from Reviewer A:
In general this is an interesting study and potentially worthy of publication in the Journal of Simulation and Gaming, with revisions indicated and attention to issues raised in below comments.
Response: The paper has been extensively rewritten with attention to issues raised by all reviewers. Reviewer comments helped the authors clarify the research project and better articulate results and interpretation of findings.
Responses to Reviewer comments and suggestions
Reviewer
C
|
Section of Paper
|
Comments
|
Response
|
|
Abstract
|
None
|
|
|
Keywords
|
“I don’t think all of these words work as efficient keywords. Try narrowing down.”
|
Narrowed down to effective keywords
|
|
Introduction
|
1.
Suggested clarifying what students wrote. …“reflection what? Paper Journal?”
2.
Used a instead of
one
…“was there more than one weakness? Maybe change to
a”
|
1.
Changed to “wrote a refection paper about the experience”
2.
Changed to “a weakness of the study was in the area of debriefing”
|
|
Review of the literature
|
End of 2nd paragraph. …Thus, as players learn and experience the rules of a game by assuming a specific role, … “this reads like a run-on sentence. Try breaking up”
|
Broke up to two sentences
|
|
Research Questions
|
None
|
|
|
Methodology
|
1.
Tense, 3rd , 4th, and 7th paragraphs. “since this is your study and this is what you did, shouldn’t this be in past tense?”
2.
Name: The survey consisted of sixteen Likert scale. “technically it should be referred to as a Likert response format”
|
1.
Changed to past tense.
2.
Changed to “The survey consisted of sixteen questions”
|
|
Results
|
None
|
|
|
Discussion
|
“In this section it is not clear where the answers to the research questions are. While I think the author has answered the questions, the research questions should be specifically addressed here followed by
the evidenced based answer (s)”
|
Research question one was addressed in paragraph 1-4 with discussions on the issue of game logs, the use of moves, and adopters gained in both groups. Research question two was addressed from paragraph 5- 10.
|
|
Limitations of the study
|
None
|
|
|
Conclusions
|
Although some negative comments about the game were indicated … “make this active voice”
|
Changed to “While there were some negative comments about the game…”
|
|
Appendix I
|
Likert scale
|
Changed to Likert response format
|
|
Figures
|
Figure 2 Survey Results “although the distribution of scores graphic is a terrific idea, technically for descriptive statistics the standard deviation should be presented along with the mean, even with a small
N.”
|
Standard deviation was included in the table. (all “Figures” were changed to be “Tables” per APA guidelines)
|
Responses to Reviewer comments and suggestions
Reviewer E
|
Section of Paper
|
Comments
|
Response
|
|
Abstract
|
You are talking about a particular simulation, not any simulation, correct?
You referred to it previously as a simulation. The reader may not realize that the “game” you are referring to hear is the same as the “simulation” you mentioned previously. It would be helpful to the reader to make this connection explicit
before using the terms interchangeably.
|
A statement was added to indicate the terms “simulation” and “game” are used interchangeably when used in reference to the Diffusion Simulation Game
|
|
Introduction
|
Is this speculation? Do you have evidence to cite?
This sentence seems to be contradicting itself. How could the majority of business faculty not use simulations if they are used “extensively” in business-related courses?
Though I realize this is true, if you do not provide evidence, then readers must assume you are speculating. Provide evidence for claims.
Again, be specific. It was not any simulation they viewed positively, it was the Diffusion Simulation Game.
|
Literature review has been extensively rewritten to provide citations for statements.
Changes were made throughout the article to refer specifically to the Diffusion Simulation Game
|
|
Review of the literature
|
This is strong language for a generalization about all simulation games. Learning is always probabilistic due to the stochastic nature of humans. I have never seen any simulation, or any other instructional method, that is always effective
in meeting its learning objectives with all learners.
“promoted” may be a better choice of words than “achieved”, or qualifying it with something like “can be” instead of “are”.
Though I have not read Peterson (2010) yet, I wonder if this single piece of “strong evidence” is based on a single game. This may be something you want to expand on.
Without more detail, I would be inclined to discredit a statement like this in the same way that I would discredit a statement that states that “I-Pads are good for learning” while disregarding the instructional method being used with the device.
The effectiveness, or lack thereof, of games and simulations result from the effective (or ineffective) design of the game. For example, I would not expect that a poorly designed game would be effective for learning just because it is a game.
What criteria are you using for this? I would delete “convincingly” and leave it to the reader to make this judgment. The paper should be as unbiased as possible from your personal opinion/beliefs.
Being unspecific makes it feel as though you are generalizing to all simulations, or talking about a different simulation. I would state “DSG” when talking about that particular simulation.
Your lit review, though fairly logical and clear, does not lead to a particular gap in knowledge as far as I can tell. I would like to see the review concluded with a more clear gap in knowledge. Why is there a need for this study?
|
Suggestions for word changes were adopted in the revisions of the literature review.
For example, the word “convincingly” was deleted.
The gap in the literature was defined as the relative low use of simulations in higher education as well as limited information for educators on how to implement a simulation/game
|
|
Research Questions
|
If this is what you are attempting to do, I think your methodology is inappropriate. However, if you are trying to determine if “the DSG” is a valuable addition, then your methodology makes more sense.
The gap in literature (not sure what that is), the goal/purpose (does DSG help students learn content), and the research questions (will students perform better when using a change model to guide their gameplay than students who play the game
with no guidance) are not aligned.
These are significant issues that should be addressed.
Also, if your goal is to assess learning, game performance is not always the best indicator, particularly if the fidelity of the game rules to the theory is low. It is entirely possible that the game was poorly designed and following the change
model would result in a poor game performance.
|
Research questions were rewritten to accurately reflect what the researchers wanted to investigate and learn about.
Fidelity of the Diffusion Simulation Game and the real life change process is addressed in the results and discussion areas of the paper
|
|
Methodology
|
Before or after participation in the study?
Is this the same change model which the game was designed to teach?
Though they were just asked to play the game, did they have knowledge of the change model? Had they learned the content in class prior to participating in the study?
You mentioned the importance of debriefing in your literature review, which is not the same as reflection. If evidence suggests that it is important in learning via games, why was this not part of the intervention? Is debriefing not feasible
in an online course?
Really? Did players even know the gamelog was being saved? Why would they want to delete their gamelog? I wonder if not making an assumption would be better here, and instead just state that 3 of the gamelogs could not found.
Though this is true, the DSG was not solely based on Roger’s theory. It also included other ideas that were known at the time
They are not implementing the strategy… they are convincing the staff to implement it.
This may have been why 3 of the participants logs could not be found… if they started a new game but did not play it.
This answers my previous question.
This addresses a topic I mentioned earlier as well. However, debriefing is not the same as reflection. Whereas reflection allows a player to consider their own gameplay experience, my understanding of debriefing is that others gameplay experiences
are shared as well. In any case, a brief definition of debriefing from the literature will clear up the misunderstanding. You may find that you need to state that you used the reflective journals “in place of” debriefing for whatever reason.
|
The paper describes that participants played the game after nine change models were studied and that the Diffusion of Innovation theory was one of the models studied in the course.
Information and discussion related to debriefing was increased in this second version of the paper. The paper now clearly states that the reflective journal was used in place of a formal debriefing protocol.
The researchers were unaware that a debriefing guide for the Diffusion Simulation Game was/is available.
|
|
Results
|
The average you provide for your participants is based on their “last” gameplay session. I am guessing that the average you are reporting for past DSG players includes all gameplays. If so, this comparison is not valid. At the very least,
this fact should be made known. If the average number of adoptees of past DSG games is based on the “last” gameplay, then state so. If this is so, is the number of gameplays the same? I would expect the more you play the game, the better you get.
Also, I wonder if you are able to get another (better) measure of game performance. Imagine a situation in which one player has 15 people in the trial phase and only 5 adopters. Another player has 3 people in the trial phase and has 8 adopters.
Based on your indicator, the second person performed better. However, if you consider “closeness to adoption”, the former outperformed the latter.
If you have the data available, I would be interested in knowing the game performance based on a closeness to adoption in addition to the total number of adopters.
So they were not asked to use one particular change theory? This is what was stated earlier.
So the treatment, or at least the activity, was not necessarily different from those in group A and those in group B. I wonder if it is important to report that there were two groups then. It may be more beneficial to compare those who actually
used a change theory with those who did not; instead of comparing those who were asked to vs those who were not.
This would have been good to mention earlier so it is made clear to the reader that the treatment varied between participants and that the game performance data being analyzed was of their last gameplay, and I assume you have no way of knowing
how many gameplays led up to that. This makes comparing performance difficult as those who played more games would likely perform better.
|
The results section was rewritten to align results with the research questions.
Closeness to adoption was not investigated in this study. I agree that this is an interesting aspect to investigate and is a good area for future research.
Reviewers were absolutely correct that this research project was an exploratory study.
In the revised paper, participants were treated as one group.
|
|
Discussion
|
. This makes sense because they did not have to play for the whole 2 year calendar (because they won). Is there something else that is being implied here?
In these first two paragraphs you are giving details about what happened during gameplay (findings) but not really discussing the meaning of these findings, which is what I would expect in the discussion. Is there any meaning to these findings?
The small number is a valid reason to not do a statistical analysis (and so is the fact that the treatment was the same for many in both groups due to their prior learning and use of change models). However, groups not having an equal number
is not a great argument for not using a statistical analysis because there are methods for correcting for this.
Particularly since participants from both groups did this.
I wonder if you could provide an example in which following the prescription of one change model does not align with the game. Do these change models differ that dramatically? I would expect that they are all intended to work in the same real-life
situations they are designed for.
This is particularly interesting, given the amount of time it takes to play the game. Do you have any data on the average amount of time the game took to play?
Substitute?
Again, reflection does not allow correction of misunderstandings because there is nobody leading the debriefing and ensuring that misconceptions are addressed.
An online class, correct?
This was an attempt at debriefing, though I still think an expert would need to be involved in the discussion to ensure misconceptions are being addressed (blind not leading the blind).
|
The discussion section was rewritten to align with the research questions and to better clarify the meaning of the findings.
No data is available on the time it took to play the Diffusion Simulation Game.
Weakness in the area of debriefing is discussed in the Discussion section.
|
|
Limitations of the study
|
As far as I can tell, no comparisons were made between group A and group B, and definitely no statistical comparisons. The treatments were not necessarily different as well. I am not sure why you need to mention group A and group B at all.
It may make more sense to note that X participants were explicitly asked to use a change model and the others were not.
|
Limitations section was rewritten. Study participants comprised one group.
|
|
Conclusions
|
You have no evidence that your results are generalizable. This statement would be more appropriate if it were to state “in this particular graduate-level online course”
Not all students… be careful not to imply a generalization. “students in this study”
This is a generalization that cannot be made based on your study. Not only do you not have sufficient evidence to say this about simulations in general (as you were only looking at one particular simulation), you do not have evidence to say
this about online courses (as you were looking at only one online course).
|
Conclusions were rewritten to limit generalizability.
|
n
|