Reaction Paper #1 – What is design based research?
Learning and Design: Why Learning Sciences and Instructional Systems Need Each Other
Hoadley’s article offers an overview of how learning sciences and instructional systems communities are more alike than they are different. I have not thought about the differences or similarities of these two research communities, but after reading this article – I see more areas of potential to overlap than I do separation. One of the biggest areas is in educational technology, where Hoadley states “both learning sciences and he instructional systems design communities encompass educational technologies, although with slightly different purposes” (Hoadley, 2004, p. 8). Working together, these two communities can help practitioners use educational technology effectively. Design based research gives the designers the opportunity to develop technology that can then be researched in the learning sciences. Then with feedback, make the necessary changes to the technology so that it can be more effective. What good is technology that cannot be used in the classroom?
Constructivism is another common link between these two, where Hoadley notes that is has been “highlighted” in instructional design models, and “taken for granted” in the learning sciences. He states that the difference is between cognitive and sociocultural, but is this a problem? These two communities are looking a research through different lenses and will have different points of emphasis. Constructivism is in both places and this can be where more cross pollination can flourish if they will use each other.
I did like the discussion about technology and exactly what that is. It is easy to forget that at one time, pencil and paper were new technologies. Changing this lens a bit to reflect “networked information technology” (p. 9) is the key. Technology has always been around and in many different forms. There is the debate between Clark and Kozma and what influence educational technology really has in the classroom. The difference, to me, is the socially connected aspect of educational technology and affordances that come with this. Someone has to design it and someone has to use it to teach with. Putting these two different aspects together within one community can have a positive impact.
I made a note in the margin of my paper about what Hoadley said on “replicability” (p. 10) with experimenting. With all of the things that occur in a classroom, is it possible to replicate an experiment? If you try to compare data in a classroom from two different years, haven’t the students changed? Are all of the variables going to be the exact same? I am not so sure. There has even been some recently published articles on the inability of researchers to replicated studies – and if you are performing research on a platform that you designed? There is a bias that comes into play and a successful result will pay the bills. However, designing a platform, then handing it off to the educator to use as part of teacher action research can be beneficial. Allowing students and teachers to use the tool and gather data (both quantitative and qualitative) can be so beneficial and more authentic.
The one takeaway from this article that really resonated with me was this “use iterative design of learning environments to explore what works, what’s important, and how it relates to learning theory” (p. 11). Anything to help teachers in the classroom, in my eyes, is one of the most important things researchers can do.
Design Research: What We Learn When We Engage in Design
Edelson got my attention in his opening paragraph “researchers are increasingly choosing to incorporate design into their research activities…to have a direct impact on education” (Edelson, 2002, p. 105). It is my opinion that nothing is more important than bringing the researcher and the educator closer together, the benefits from this can be very positive. By developing a design that is iterative and allows for hypothesis and principles to not “determine every decision” or be “followed so slavishly” (p. 106) that changes can be made that benefit the most important people involved with educational research: the students. Why continue working through research that is showing no benefit or has an unintended consequence or has clear design issues? Make changes in the research, for the students, for the teachers, help develop something that can be of benefit.
The design outcome that Edelson offers is one that has three parts: design procedure, problem analysis, and design solution. I thought it was interesting that the problem analysis was not the first item suggested. If you are not aware of your problem, how can you choose a design that will be the best fit for that research? I don’t have the experience that the author has, but I wonder if I go through a design procedure first, could I be wasting time? If I don’t have a problem analysis, how do I know who needs to be on my research team? Edelson says that “the designers must assemble a team the includes the relevant forms of expertise” (p. 108), without the problem I’m not sure that I can do this.
There were several places that I did like what Edelson was saying and one thing that I really resonated with me was this: “We have found the voice of the teachers in this design process to be particularly valuable” (p. 111). YES! This should be one of the key elements of any educational research that is occurring. It is the voice of the teacher, and the student, that important. Far too often these voices are not heard or are silenced. Teachers and students are valuable assets in any educational research, and from this article in particular, I see several elements of teacher action research.
The iterative process is mentioned again in this article, and I like the example that Edelson draws from Cobb, that an outcome theory was developed through a process of “design and implementation cycles” which led to “the emergence of significant mathematical ideas” (p. 114). Although I did not do any research on flipping my classroom, I followed this process. Designing the structure of my class, evaluating its impact, getting informal feedback from students, updating the class design, getting feedback…The iterative process gives teachers and researchers the opportunity to adjust as they best see fit, this ties into the formative evaluation that Edelson discusses.
I really liked the way this article closes “If the ultimate goal of educational research is the improvement of the educational system, the results that speak directly to the design of activities, materials, and systems will be the most useful result” (p. 119). As a teacher I understand this statement, research based articles were intimidating to me and not practical. I was a teacher that always wanted to walk away with a practical idea or a tweak to that idea to try in my classroom. Design based research has the capability to offer the best of both worlds, theory and practice, that can be truly be beneficial for educators.
Design Experiments: Theoretical and Methodological Challenges in Creating Complex Interventions in Classroom Settings
Where to start with Brown – my only issue is that she referenced herself so much. I know she was writing about the work she had done and how it all tied together, but I thought it was a bit much. She pulled references from all over and I did like that, but could she have used more from others? I don’t know. On the other hand, I really liked this article and see immense value in what she is doing. Working with students and teachers, recognizing that there are myriad variables, using qualitative and quantitative data she is developing a way for students and reading comprehension. Her statement “…an effective intervention should be able to migrate from our experimental classroom to average classrooms operated by and for average students and teachers, supported by realistic technological and personal support” (p. 143) hit home for me. I see a gap between theory and practice, between the researcher and the practitioner, and the work that Brown is doing is closing that gap. She mentioned several times the importance of working directly with the teachers and students and she recognizes the complexity of the classroom. No one variable can be isolated from the thousands of others, confounding issues will plague any truly hypothesis based statistical analysis in the classroom. Her efforts to try and find interventions that are both reliable and repeatable is important, but repeatability seems to be something that is very difficult to do.
As I am reading this, I am thinking to myself how beneficial a paper like this can be for policy creators. Since public education is regulated by both state and federal governments, policy is determined by those not involved in any aspect of the educational process. As such, papers like Brown’s should be required reading. It should not matter what your political values are, whether liberal or conservative, it should matter what can we do to best educate children. Brown shares the challenges of trying to pinpoint specific variables and how difficult that is to accomplish. She talks about Vygotsky’s Zone of Proximal Development, stating that “one should go beyond the current level of competence” (p. 168), not limiting the students to preconceived notions of where the learning should be, but giving the students students some freedom to explore and learn and not “block off certain avenues of learning” (p. 168).
The discussion of discovery learning was good as well, this seems to be all the rage these days. I like discovery learning myself, but turning a class over to the students to “learn” is challenging. Brown addresses the difficult position that teachers are in as they use discovery learning: When should the teacher step in? What if the teacher is not sure of the answer? What if a common misconception is taking over the class thinking? These are hard questions, ones that are not easily answered. Discovery learning seems to be a good fit for design based research and also lines up with teacher action research.
Discovery learning lines up with Brown’s “Reality Principle” about reform in education and the legacy that it has left. The American education system is full of education reform failure and Brown discusses why some of these failures happened. Education reform is difficult at best and is often driven boy those who are not familiar with the complexity of the modern classroom.
Toward a Design Science of Education
Collins references the development of the science of education – I really like this phrase, it makes educational research sound like something that can contribute to the practice of education. He then goes on to say “a design science of education must determine how different designs of learning environments contribute to learning, cooperation, and motivation” (Collins, 1992), implying that there are several factors involved in the education process. He compares this to aeronautics and how the overall goal is to determine what changes impact lift or drag. It was also interesting that Brown referenced aeronautics as well (I thought that Hoadley did as well, but cannot find it).
Collins also points out one of the issues that is plaguing research today in respect to repeatability: who is running the research and what exactly are they reporting. He says that “these experiments are carried out by designers of a technological innovation who have a vested interest in seeing that it works” (Collins, 1992). We discussed in our class last semester about developing a journal (or something similar) for those research projects that are failures or don’t show anything significant. That leads to the Collins list of characteristics that help eliminate some of these biased reporting issues. As with the other articles, one of those characteristics mentioned is to use teachers as co-investigators. Anything that brings the practitioner into the discussion helps to bridge the gap between the researcher and the educator in the field.
Collins talks about setting up an experimental study, where teachers have a control class and a class using the developed design. The issue here would be what if the experimental class is vastly improved by the design, did we deny the control students the opportunity to learn? Typically, there is no extra time in the school year to revisit an entire unit of instruction and reteach it. The data generated from the research would fall into both quantitative and qualitative categories, which can lead to adjustments for follow up studies. This can help students and teachers over time.
Collins also had a theme of technology running all through his article, including factors that affect success of technology. Reading these lists I saw some parallels to my classroom experience, although it was a bit different for me. Most of my students had access to an internet connected wireless device. Our school was a bring your own device (BYOD) school and I encouraged students to do so for me. The one thing from these lists that I recognized as relevant to my practice is from table 2: teacher interest in technology. If I didn’t have this interest and wonder in what technology could do for my practice, I would not be writing this paper right now. One factor that I wasn’t sold on was also from table 2: teaching career enhancement. Collins says that an administrator expecting the use of technology would impact success – I disagree. I know too many teachers who use a minimal amount of technology. They are too married to their tried and true ways of teaching. This is why I am writing this paper – to put myself into a position to reach teachers and give them simple, pedagogically sound, fun lessons using technology. With the goal of inspiring individual teachers to want to know more, to seek more, and to try more.
Brown, A. L. (1992). Design experiments: Theoretical and methodological challenges in creating complex interventions in classroom settings. The Journal of the Learning sciences, 2(2), 141-178.
Collins, A. (1992). Toward a design science of education: Springer.
Edelson, D. C. (2002). Design research: What we learn when we engage in design. The Journal of the Learning sciences, 11(1), 105-121.
Hoadley, C. M. (2004). Learning and design: Why the learning sciences and instructional systems need each other. Educational technology: The magazine for managers of change in education(3), 6-12.