Systematic review proposal


A systematic review differs from a narrative review in that it is formalised and attempts to be objective in answering a pre-determined question. It can be defined as “…a review of the evidence on a clearly formulated question that uses systematic and explicit methods to identify, select and critically appraise relevant primary research, and to extract and analyse data from the studies that are included in the review.”[1]

The first stage of the systematic review will begin with a preliminary investigation of the available literature to determine the scope and basic parameters of the review, as well as to avoid duplication of reviews that have already been conducted in the field (Khan, ter Riet, Glanville, Sowden, Kleijnen, 2001).[2]


Clinical education and practice

A clear definition of “clinical education” is hard to come by, with many variations of exactly what is meant by the phrase leading to confusion among both educators and students (Rothstein, 2002).[3] For the purposes of this review, clinical education describes the learning that takes place within the context of clinical practice while on clinical placements. It includes a gradual decrease in student supervision, which leads to an increase in independence, and the introduction of increasingly complex clinical situations over time. It is also often a time when students are isolated from their peers and placed in environments and scenarios that are new and unfamiliar.

Reflection has been recognised as an important component of clinical practice since Schön (1983,[4] 1987[5]) argued that the positivist approach could not in itself effectively describe practice knowledge within the professional domain. It is defined as “a mental process with purpose and/or outcome in which manipulation of meaning is applied to relatively complicated or unstructured ideas in learning or to problems for which there is no obvious solution” (Moon, 1999, p. 155)[6]. The scientific method has indeed helped the profession move from a “craft tradition” based on experiential knowledge, to one that is based on research and an evidence base. However, it should be recognised that an epistemology of practice recognises several ways of achieving knowledge. Too much emphasis on positivism as the only approach within the health-related professions ignores the personal, complex and multi-factorial nature of clinical reasoning.

A collaborative therapist-client relationship must take into account a variety of factors, including a dynamic environment (Jones & Rivett, 2005).[7] The healthcare practitioner must constantly review and re-prioritise both existing and newly arisen problems in an enterprise of active interpretation (Thornquist, 2001a, 2001b). In addition, the problems are usually not well defined (“messy” as Schön called it) and must constantly be reset while in the context of ongoing patient management. This is achieved by naming and framing the problem in the language of the profession, using the norms that are developed within it's social and institutional frameworks (Mishler, 1986)[8]. Rather than viewing practice knowledge as a representation of the external world that can be defined using universal theories and rules (i.e. the positivist approach), we must consider that it it exists as a relation between questions and answers in a context of meaning that is often intuitive and hidden (Higgs, Richardson & Abrandt Dahlgren, 2004).[9]

This tacit knowledge is difficult to teach and novices in the profession must develop it over a period that usually begins during their undergraduate education at university. Sharing knowledge or experiences within a collegial environment enables external knowledge to be internalized, develops problem-solving strategies and promotes critical reflective thinking by challenging unshared biases and presuppositions (Mason 1998; Hanko 1999). The problem is that institutions of higher learning embrace an approach to education that ignores or places little emphasis on the role of reflection in the learning process. In addition, educators are often challenged in how they encourage learners to engage with a process of reflection (Boud & Walker, 1999)[10] that would facilitate professional development.

It is increasingly evident that today's physiotherapy graduates must not only possess the clinical skills necessary to practice, but also the skills to engage in lifelong learning and personal evaluation. It has been suggested that a careful approach to education within the clinical setting is the best place to develop these extra-curricular skills, and that it should include a professional philosophy or vision.[11] Other essential goals to be achieved within clinical education are also non-technical, and include an awareness of one’s attitudes, values, and responses to health and illness, interpersonal skills and the ability to educate others effectively, good communication skills, the ability to critically evaluate personal and professional practice, and accountability and commitment to the continued development of competence and lifelong learning. In addition, other competencies that are necessary include clinical reasoning, psychomotor skills, examination, treatment and evaluation, integration of theory and practice, and an ability to articulate rationales for treatment.[12] It should be noted that Hayes, Huber, Rogers and Sanders (1999)[13] found that during clinical placements, many incidents involved a lack of non-technical competencies such as communication skills and professional behaviour.

In addition to a poor understanding of how the development of non-technical skills can impact clinical education, there is also the issue of inconsistency between theory and practice, with students and new graduates not making connections between coursework and fieldwork. They fail to identify the difference between academic knowledge and clinical knowledge. In other words there is a separation of theory and practice that may result from the inadequate development of reflective skills and theory development.[14] According to Strohschein, May and Hagler (2002)[15] the following needs, among others, need to be met in response to some of the challenges in clinical education:

  • A need for a common and identifiable philosophy or vision of clinical education that is evident in both the process and product of the clinical experience
  • A need to develop ongoing reflective practice and lifelong learning among students and new graduates.
  • A need to establish mutually beneficial relationships between students and clinical supervisors
  • A need for a consistent quality assurance process to assess and enhance the clinical experience of students
  • A need to place far greater emphasis on the relationship between theory and practice

Strohschein, May and Hagler (2002) go on to identify 10 models of clinical education derived from the literature. They highlight some of the ways in which each model is found lacking in terms of addressing the above needs (see the table below, taken from pg. 164), and suggest that an understanding of the models and their relative strengths and weakness is important to facilitate and enhance clinical education. In addition, no particular model of clinical education has been found to be superior to another, and studies that did make recommendations of one over another, were not methodologically strong.[16]


There may be a role for a blended approach to teaching and learning in clinical education, which might be able to add to the established models, as well as address some of the missing needs.

Blended learning in clinical education

Blended learning refers to a method of integrating online with face-to-face learning opportunities in order to support and enhance meaningful learning experiences (Garrison & Kanuka, 2004).[17] It has been defined by Bliuc, Goodyear and Ellis (2007, pg. 242)[18] as a set of:

“…learning activities that involve a systematic combination of co‐present (face‐to‐face) interactions and technologically‐mediated interactions between students, teachers and learning resources”

It has been suggested that integrating a computer-based teaching model using sound educational principles can to be effective in constructing curricular modules that allow for greater flexibility and responsiveness in teaching and learning. (Lewin, Singh, Bateman & Glover, 2009).[19] In addition, the use of online instruction in clinical education can also help to overcome physical limitations like time and space, support instructional methods that are hard to achieve using textbooks and reach a larger number of students without increasing resource requirements (Gray & Tobin, 2010).[20] The true potential of blending computer-mediated teaching with face-to-face interaction is in enhancing communication, rather than merely transferring content from teacher to learner. This improvement in communication can support “…flexible, engaging and learner-centred teaching, encourage interaction among students and staff and enable them to collaborate and communicate asynchronously” (Ellaway & Masters, 2008, pg. 456)[21]

The following table, taken from Greenhalgh (2001, pg. 40) highlights some of the key reasons that medical educators should be exploring the use of computers and the internet to facilitate improved medical education. greenhalgh_-_why_computer_assisted_learning.jpg

Greenhalgh (2001, pg. 41) also presents the following table (see below left), highlighting some of the ways in which computer-mediated instruction can be used as an adjunct to face-to-face contact. greenhalgh_-_examples_of_blended_learning.jpg

Even though blending instruction has been shown to have potential in higher education in general, there is currently a lack of literature demonstrating that computer-aided instruction can be used to facilitate clinical education.[22][23] Of the literature that is currently available, there is often a lack of methodological quality in the evidence base.[24] With this in mind, it is also important to note that blended learning is highly context-dependent, and that generalisation of concepts is challenging (Harris, Connolly & Feeney, 2009).[25] Thus, applying concepts from other fields to clinical education may be problematic. Finally, blended learning has been described as “old wine in new bottles” (Thorn, 2003, pg. 17)[26],implying that the idea is not new and is merely being dressed up and repackaged. It should be remembered that while the idea of blending face-to-face student contact with alternative forms of teaching e.g. distance learning, has been around for decades, the rise of the internet as a global communication tool is relatively new, and it is this aspect of the blend that is now receiving attention.

Harris, Connolly and Feeney (2009) make the following recommendations when considering the application of blended learning to higher education:

  • The blended learning interventions should be appropriately measured
  • Skills training may be necessary to both design and use the blended learning components
  • Support for all stakeholders should be provided when necessary
  • Time should be set aside for communication with all stakeholders in order to achieve buy in
  • To be effective, emphasis must remain on the pedagogical aspects, rather than the technology, although some technological issues (e.g. accessibility, literacy and support) must also be addressed
  • Evidence based instructional design should underpin the learning activities
  • The needs, motivations and expectations of the stakeholders must be taken into account

Any discussion of the integration of technology into teaching and learning must take into account the fact that “…the medium itself may be less important than the way in which teaching is approached” (Laurillard, cited in Ennew and Fernandez‐Young, 2005, pg. 151).[27]. In addition, no matter how well the blended intervention is implemented, it has been suggested that there needs to be a cultural change in clinical teaching before these methods can be adopted.[28] The authors suggest the following factors be taken into account (pg. 8):

  1. Neither clinical students nor staff will make the change to becoming active in an online community without strong motivations
  2. The investment in working out how to manage an online community is substantial and may be recouped only after several cycles of innovation and improvement
  3. An online community may augment interpersonal communication in clinical settings but does not equate to the stimulus of face-to-face interaction
  4. In order for students to regard using an online community as professionally valuable they must see senior staff modelling a collective approach to its use

Aim of the review

The aim of this systematic review is to determine if blended learning has a role to play in enhancing clinical education.

The aim will be achieved through the following objectives:

  1. To determine the gaps in current clinical education models
  2. To determine if blended learning is an appropriate model to enhance clinical education

A specific, targeted research question was formulated using the PICO acronym (Higgins & Green, 2009)[29] which led to a research question that will attempt to achieve the aim of the review.

The (P)opulation will include allied health, medical or nursing students. The (I)ntervention/s to be considered is the implementation of blended learning in clinical education. The studies selected should include a (C)omparison with traditional methods of teaching, although this need not be requirement as the preliminary search didn't come across any studies with a comparsion. The (O)utcome measure should be a suggestion or recommendation of strategies to be followed should a blended learning approach be considered in clinical education.[30] “Improvement” is specifically not identified as an outcome, as this will bias the review towards studies with positive results.

Research question

Can blended learning enhance clinical education?


The method used to conduct the review will be guided by the following process from the Centre for Reviews and Dissemination (2001), which can be seen in the illustration below, as well as Hemingway and Brereton (2009)[31] and the JBI approach to Evidence based practice, chosen because it takes into account a wide range of what qualifies as legitimate evidence.

Overview of the systematic review process

Inclusion criteria

The search will include peer reviewed publications from electronic databases, print journals (via JSTOR) and published conference proceedings (e.g. AMEE). The databases to be searched include (all via EbscoHost):

  • Academic Search Premier

The Cochrane Library will also be included to ensure that no similar reviews have been conducted. The search will also include a standard web search using both Google and Google Scholar, as well as more recent social research databases like Mendeley. The “Related research” option will be consulted on sites and databases that support this feature. Furthermore, the search will include other sources derived from the researcher's social graph (see Appendix I for a dicussion of the use of the social graph in a systematic review) that will identify serendipitous literature highlighted by established experts in the field.

The review will include full-text articles published in English between 2000 and the present, in order to capture the most recent developments in the field. Studies with both positive and negative outcomes will be included, so as to reduce publication bias. In cases when the titles of the returned articles are insufficient to determine eligibility, the abstracts will be consulted to determine if they should be downloaded. The reference lists of these sources will also be scrutinised for additional sources.

The review will incorporate systematic reviews and observational type studies, including cohort and case studies that used quantitative, qualitative and mixed methods. Search terms were chosen as a result of the preliminary review of relevant literature, and the following key search terms (and their synonyms) were chosen after consultation with an experienced colleague: clinical education OR medical education OR nursing education OR health education, and blended learning OR computer aided instruction OR computer aided learning. The following synonyms for blended learning will also be used: “integrated learning”, “hybrid learning”, “multi-method learning”. Synonyms were obtained from the preliminary search.

Study assessment

The following criteria will be used to assess the methodological quality of the studies:[32]

  • Eligibility against inclusion criteria during initial searches (see previous section)
  • Hierarchy of evidence - It was determined from the preliminary review that many studies in this domain were conducted on cohorts of students via surveys that used quantitative, qualitative and mixed methods, as well as individual case studies. Thus, Level 2 and 3 studies will form the bulk of the data for the review, but Level 4 studies will also be considered.
  • Methodological quality using a critical appraisal framework (see next sections) to exclude poor quality studies
  • Data extraction form/s used to extract results from remaining studies (see next sections)
  • A final list of included studies is then created
  • The assessment with be conducted by two independent reviewers

Critical appraisal tools

  • Joanna Briggs Institute System for the Unified Management, Assessment and Review of Information (JBI-SUMARI) tool (Joanna Briggs Institute Reviewers’ Manual, 2008)[33]
  • A selection of CASP and other critical appraisal tools is available, and which can also be used to evaluate the methodological quality of the cohort and systematic review type studies that use quantitative, qualitative (Joanna Briggs Institute Qualitative Assessment and Review Instrument - JBI-QARI) and mixed methods.

All of the relevant articles will be independently assessed by 2 reviewers. One of the challenges highlighted when using an appraisal tool such as the JBI-SUMARI is the issue of inter-rater reliability. Strategies to reduce the risk of error will include the use of a standard data extraction form and training the reviewers (Joanna Briggs Institute, 2008). Once consensus has been reached, the selected articles will go through the data extraction process.

Data extraction

The review will not make use of self-designed data extraction forms and will rather use generic forms for the specific types of study (i.e. quantitative, qualitative, mixed, cohort, etc.) as specified by JBI. The data extraction should ideally be conducted by 2 independent reviewers, compared and referred to a third reviewer should there be any disagreements.


I conducted the review during the second half of 2010 and have drafted an outline of an article.

Appendix I: The social graph and systematic reviews

Note: of course, none of this section turned out to be relevant as a systematic review doesn't necessarily have to be able to be replicated, only that it's methodology be structured and pre-determined from the outset, so as to avoid selection bias. At the time that I wrote this, I had a poor understanding of systematic reviews. I've left the section here even though it's not applicable because it's still informed my thinking of how we search in a networked world.

This systematic review will draw heavily on established research methodologies in order to gather and select the studies to be included. However, in addition to more traditional approaches (i.e. electronic databases, online and offline journals, etc.), the review will also make use of the author's social graph. The social graph is an abstraction of the model used to describe the relationships between people and the nature of those relationships [34]. In addition, it can be described as a representation of a Network of Practice (NoP), in which informal groups of loosely connected people share knowledge relevant to a set of shared practices [35], using online tools. Networks of Practice are an evolution of Wenger's Communities of Practice (CoP) [36], but are different in that not all characteristics of CoPs are present in a NoP.

With the huge volume of relevant information that is currently available in a networked society, traditional search methodologies cannot be relied on to discover a truly comprehensive set of literature. Use of the social graph allows the net to be thrown wider, and yet paradoxically with a narrower focus, as nodes (users) within the network are able to provide more focused results than an arbitrary search for keywords. One key difference is that there is an inherent, implied context that is present within a network of people who share common ideas and belief systems. This may be a additional, more effective search methodology than mere keyword-based, context-less searches.

The researcher will make use of these relationships within the network to discover additional sources that may not be readily available in pre-determined repositories. However, a dilemma occurs because the use of social networks to facilitate serendipitous literature discovery means that the systematic review cannot be replicated, as every individual's social graph and context will be different. Thus, including the social graph will have the effect of increasing the potential pool of relevant studies to be included in the review, but also means that the review cannot be replicated.

1) Undertaking Systematic Reviews of Research on Effectiveness. CRD's Guidance for those Carrying Out or Commissioning Reviews. CRD Report Number 4 (2nd edition). NHS Centre for Reviews and Dissemination, University of York. March 2001
2) Khan, K.S., ter Riet, G., Glanville, J., Sowden, A.J., Kleijnen, J. (2001). Undertaking Systematic Reviews of Research on Effectiveness: CRD's Guidance for those Carrying Out or Commissioning Reviews
3) Rothstein, J. M. (2002). “Clinical Education” Versus Clinical Education. Physical Therapy, 82(2), 126-127
4) Schön, D.A. (1983). The Reflective Practitioner. How Professionals Think in Action. New York: Basic Books.
5) Schön, D. (1987). Educating the Reflective Practitioner. San Francisco: Jossey Bass
6) Moon, J. A. (1999). Reflection in learning and professional development: Theory and practice. London Sterling, VA: Kogan Page: Stylus Pub
7) Jones, M. & Rivett, D. (2005). Clinical Reasoning for Manual Therapists. Butterworth-Heinemann, London. ISBN: 0750639067
8) Mishler, E.G. (1986). Research interviewing. Cambridge, MA: Harvard University Press.
9) Higgs, J., Richardson, B. & Dahlgren, M.A. (2004). Developing Practice Knowledge for Health Professionals. Butterworth Heinemann, London. ISBN: 0750654295
10) Boud D, Walker D. 1998. Promoting reflection in professional courses: The challenge of context. Studies in Higher Educ 23:191–206
11) Opacich K. Is an educational philosophy missing from the fieldwork solution? American Journal of Occupational Therapy. 1995;49:160–164
12) Higgs J, Glendinning M, Dunsford F, Panter J. Goals and components of clinical education in the allied health professions. In: Proceedings of the 11th International Congress of the World Confederation for Physical Therapy, London. 1991:305–307
13) Hayes, K.W., Huber, G., Rogers, J. & Sanders, B. (1999). Behaviors that cause clinical instructors to question the clinical competence of physical therapist students. Physical Therapy, 79:653–667
14) Steward B. (1996). The theory/practice divide: bridging the gap in occu- pational therapy. British Journal of Occupational Therapy, 59:264–268
15) Strohschein, J., May, L., & Hagler, P. (2002). Assessing the Need for Change in Clinical Education Practices. Physical Therapy, 82(2), 160-172
16) Lekkas, P., Larsen, T., Kumar, S., Grimmer, K., Nyland, L., Jull, G., et al. (2007). No model of clinical education for physiotherapy students is superior to another: a systematic review. Australian Journal Of Physiotherapy, 53, 19-28
17) Garrison, D. R., and Kanuka, H. (2004). Blended learning: Uncovering its transformative potential in higher education. The Internet and Higher Education, 7(2):95‐105
18) Bliuc, A., Goodyear, P., & Ellis, R. A. (2007). Research focus and methodological choices in studies into students' experiences of blended learning in higher education. The Internet and Higher Education, 10(4), 231-244. doi: 10.1016/j.iheduc.2007.08.001
19) Lewin, L. O., Singh, M., Bateman, B. L., & Glover, P. B. (2009). Improving education in primary care: development of an online curriculum using the blended learning model. BMC Medical Education, 9(33), 1-7. doi: 10.1186/1472-6920-9-33
20) Gray, K., & Tobin, J. (2010). Introducing an online community into a clinical education setting: a pilot study of student and staff engagement and outcomes using blended learning. BMC medical education, 10, 6. doi: 10.1186/1472-6920-10-6
21) Ellaway, R, & Masters, K. (2008). AMEE Guide 32: e-Learning in medical education Part 1: Learning, teaching and assessment. Medical Teacher, 20:455-473
22) Berman, N., Fall, L., Maloney, C. & Levine, D. (2008). Computer-assisted instruction in clinical education: A roadmap to increasing CAI implementation. Advances in Health Sciences Education. Theory and Practice, 13:373-383
23) Greenhalgh, T. (2001). Computer assisted learning in undergraduate medical education. British Medical Journal, 322(7277):40-44. doi: 10.1136/bmj.322.7277.40
24) Harris, P., Connolly, J. F., & Feeney, L. (2009). Blended Learning: Overview and Recommendations for Successful Implementation.
25) Harris, P., Connolly, J. F., & Feeney, L. (2009). Blended Learning: Overview and Recommendations for Successful Implementation
26) Thorne, K. (2003). Blended Learning: How to Integrate Online and Traditional Learning. Kogan Page, ISBN: 0749439017
27) Ennew, C. T., and Fernandez‐Young, A. (2006). Weapons of mass instruction? The rhetoric and reality of online learning. Marketing Intelligence and Planning, 24(2), 148‐157
28) Gray, K., & Tobin, J. (2010). Introducing an online community into a clinical education setting: a pilot study of student and staff engagement and outcomes using blended learning. BMC medical education, 10, 6. doi: 10.1186/1472-6920-10-6
29) Higgins, J.P.T. & Green, S. (eds). Cochrane Handbook for Systematic Reviews of Interventions Version 5.0.2 [updated September 2009]. The Cochrane Collaboration, 2009. Available from
30) Hannes, K., Claes, L (2007). Learn to Read and Write Systematic Reviews: The Belgian Campbell Group. Research on Social Work Practice, 17:748-753
31) Hemingway, P. & Brereton, N. (2009). What is a Systematic Review. What is…? series, 2nd edition. Hayward Medical Communications, Hayward Group Ltd. Available at
32) Hemingway, P. & Brereton, N. (2009). What is a Systematic Review. What is…? series, 2nd edition. Hayward Medical Communications, Hayward Group Ltd. Available at
33) Joanna Briggs Institute Reviewers’ Manual (2008). The Joanna Briggs Institute, Royal Adelaide Hospital, North Terrace, Adelaide, South Australia, 5000, Australia
34) Iskold, A. Social graph: Concepts and issues. ReadWriteWeb. Posted 12 September, 2007 at
35) Andersen, T. Edubloggers as a Network of Practice. Virtual Canuck. Posted on 28 February, 2009 at
36) Lave, J. & Wenger, E. (1991). Situated Learning: Legitimate Peripheral Participation. Cambridge: Cambridge University Press. ISBN 0521423740
phd_systematic_review.txt · Last modified: 2011/01/15 22:02 by Michael Rowe