, Canada

T. Bartholomew,

, Australia

J. Bethlehem,

P. Biemer,

F. Bishop,

, Germany

H. Boeije,

J. Burton, England

K. Charmaz,

B. Crabtree,

E. Creamer,

E. De and L. ,

C. Durand, Canada

, Canada

R. Fitzgerald,

F. Fowler,

, Australia

J. Greene,

C. Gringeri,

G. Guest, , vol.360

T. Guetterman,

M. Hadi,

E. Halcomb,

, Carolyn Heinrich

S. Hesse-biber,

, Mieke Heyvaert

J. Hitchcock,

L. Johnson,

M. Lichtman,

P. Lynn,

M. E. Macdonald, Canada

, Canada

S. Merriam,

, Spain

D. Morgan,

P. Nardi, Pitzer College

K. Olson, Canada

A. Papadimitriou,

. Michael-quinn-patton, Independent organizational development and program evaluation consultant

R. Erio and M. Pinto,

C. Vicki-plano,

D. Plowright,

B. Poland, Canada

R. Reynolds,

G. B. Rossman,

E. Ruel,

M. Saini, Canada

J. Saldaña,

, Canada

, Ineke Stoop

S. Thorne, Canada

S. Tracy,

, The authors gratefully acknowledge the sponsorship from the Method Development platform of the Qu ebec SPOR SUP-PORT Unit, the CIHR Doctoral Fellowship Award (#301011), and the FRSQ Senior Investigator Award (#29308)

F. Bunn, D. Trivedi, P. Alderson, L. Hamilton, A. Martin et al., The impact of Cochrane reviews: a mixed-methods evaluation of outputs from Cochrane review groups supported by the National Institute for health research, Health Technol Assess, vol.19, pp.1-100, 2015.

G. V. Glass, Primary, secondary, and meta-analysis of research, Educ Res, vol.5, issue.10, pp.3-8, 1976.

A. L. Cochrane, Effectiveness and efficiency: Random reflections on health services. London: Nuffield Provincial Hospitals Trust, 1972.

A. Harden and D. Gough, Quality and relevance appraisal, An introduction to systematic reviews, vol.2012, pp.153-78

A. Burls, What is critical appraisal, 2009.

J. P. Higgins and D. G. Altman, Cochrane handbook for systematic reviews of interventions, pp.187-242, 2008.

K. Wells and J. H. Littell, Study quality assessment in systematic reviews of research on intervention effects, Res Soc Work Pract, vol.19, issue.1, pp.52-62, 2009.

P. M. Wortman, Judging research quality, The handbook of research synthesis, pp.97-109, 1994.

M. Petticrew and H. Roberts, How to appraise the studies: an introduction to assessing study quality, Systematic reviews in the social sciences: A practical guide, pp.125-63, 2006.

P. Pluye, E. Robert, M. Cargo, G. Bartlett, A. O'cathain et al., Proposal: A Mixed Methods Appraisal Tool for systematic mixed studies reviews, 2011.

A. O'cathain, Assessing the quality of mixed methods research: towards a comprehensive framework, Handbook of mixed methods in social and behavioral research, pp.531-55, 2010.

R. Pace, P. Pluye, G. Bartlett, A. C. Macaulay, J. Salsberg et al., Testing the reliability and efficiency of the pilot Mixed Methods Appraisal Tool (MMAT) for systematic mixed studies review, Int J Nurs Stud, vol.49, issue.1, pp.47-53, 2012.

R. Q. Souto, V. Khanassov, Q. N. Hong, P. L. Bush, I. Vedel et al., Systematic mixed studies reviews: updating results on the reliability and efficiency of the Mixed Methods Appraisal Tool, Int J Nurs Stud, vol.52, issue.1, pp.500-501, 2015.

Q. N. Hong, A. Gonzalez-reyes, and P. Pluye, Improving the usefulness of a tool for appraising the quality of qualitative, quantitative and mixed methods studies, the Mixed Methods Appraisal Tool (MMAT), J Eval Clin Pract, vol.24, pp.459-67, 2018.

S. N. Haynes, D. Richard, and E. S. Kubany, Content validity in psychological assessment: a functional approach to concepts and methods, Psychol Assess, vol.7, issue.3, pp.238-285, 1995.

Q. N. Hong and P. Pluye, A conceptual framework for critical appraisal in systematic mixed studies reviews, J Mix Methods Res, 2018.
DOI : 10.1177/1558689818770058
URL : http://digitool.Library.McGill.CA:80/webclient/DeliveryManager?pid=155636&custom_att_2=direct

M. Heyvaert, K. Hannes, B. Maes, and P. Onghena, Critical appraisal of mixed methods studies, J Mix Methods Res, vol.7, issue.4, pp.302-329, 2013.
DOI : 10.1177/1558689813479449

D. Walsh and S. Downe, Appraising the quality of qualitative research, Midwifery, vol.22, issue.2, pp.108-127, 2006.

M. Santiago-delefosse, A. Gavin, C. Bruchez, P. Roux, and S. Stephen, Quality of qualitative research in the health sciences: analysis of the common criteria present in 58 assessment guidelines by expert users, Soc Sci Med, vol.148, pp.142-51, 2016.
URL : https://hal.archives-ouvertes.fr/halshs-01429299

S. Keeney, F. Hasson, and H. Mckenna, The Delphi technique in nursing and health research, 2011.
DOI : 10.1002/9781444392029

F. Hasson, S. Keeney, and H. Mckenna, Research guidelines for the Delphi survey technique, J Adv Nurs, vol.32, pp.1008-1023, 2000.
DOI : 10.1046/j.1365-2648.2000.t01-1-01567.x

Q. N. Hong, P. Pluye, M. Bujold, and M. Wassef, Convergent and sequential synthesis designs: implications for conducting and reporting systematic reviews of qualitative and quantitative evidence, Syst Rev, vol.6, pp.1-14, 2017.

J. Baker, K. Lovell, and N. Harris, How expert are the experts? An exploration of the concept of 'expert' within Delphi panel techniques, Nurse Res, vol.14, issue.1, pp.59-70, 2006.

V. Der-gracht and H. A. , Consensus measurement in Delphi studies: review and implications for future quality assurance, Technol Forecast Soc Change, vol.79, issue.8, pp.1525-1561, 2012.

S. F-abregues and J. F. Molina-azor-in, Addressing quality in mixed methods research: a review and recommendations for a future agenda, Qual Quant, vol.51, issue.6, pp.2847-63, 2017.

C. Carroll and A. Booth, Quality assessment of qualitative evidence for systematic review and synthesis: is it meaningful, and if so, how should it be performed?, Res Synth Methods, vol.6, issue.2, pp.149-54, 2015.

, Measuring and reporting sources of error in surveys. Washington DC: Statistical Policy Office, Office of Information and Regulatory Affairs, Office of Management and Budget, Federal Committee on Statistical Methodology, 2001.

R. M. Groves and E. Peytcheva, The impact of nonresponse rates on nonresponse bias: a meta-analysis, Public Opin Q, vol.72, issue.2, pp.167-89, 2008.

S. F-abregues, . Par-e-m-h, and J. Meneses, Operationalizing and conceptualizing quality in mixed methods research: a multiple case study of the disciplines of education, nursing, psychology, and sociology, J Mix Methods Res, 2018.

P. Pluye, G. Bengoechea, E. Granikov, V. Kaur, N. Tang et al., A world of possibilities in mixed methods: review of the combinations of strategies used to integrate the phases, results, and qualitative and quantitative data, Int J Mult Res Approaches, vol.10, issue.1, pp.41-56, 2018.

M. D. Fetters, L. A. Curry, and J. W. Creswell, Achieving integration in mixed methods designs -Principles and practices, Health Serv Res, vol.48, pp.2134-56, 2013.

P. Whiting, R. Wolff, S. Mallett, I. Simera, S. et al., A proposed framework for developing quality assessment tools, Syst Rev, vol.6, issue.204, pp.1-9, 2017.

M. Viswanathan, M. T. Ansari, N. D. Berkman, S. Chang, L. Hartling et al., Agency for Healthcare Research and Quality (AHRQ) Methods Guide for Comparative Effectiveness Reviews, 2012.

P. Herbison, J. Hay-smith, and W. J. Gillespie, Adjustment of meta-analyses on the basis of quality scores should be abandoned, J Clin Epidemiol, vol.59, pp.1249-56, 2006.

J. P. Higgins and S. Green, Cochrane handbook for systematic reviews of interventions, 2008.

M. Crowe and L. Sheppard, A review of critical appraisal tools show they lack rigor: Alternative tool structure is proposed, J Clin Epidemiol, vol.64, pp.79-89, 2011.

F. Colle, F. Rannou, M. Revel, J. Fermanian, and S. Poiraudeau, Impact of quality scales on levels of evidence inferred from a systematic review of exercise therapy and low back pain, Arch Phys Med Rehabil, vol.83, pp.1745-52, 2002.

G. Guyatt, A. D. Oxman, E. A. Akl, R. Kunz, G. Vist et al., GRADE guidelines: 1. Introduction -GRADE evidence profiles and summary of findings tables, J Clin Epidemiol, vol.64, pp.383-94, 2011.

S. Lewin, C. Glenton, H. Munthe-kaas, B. Carlsen, C. J. Colvin et al., Using qualitative evidence in decision making for health and social interventions: an approach to assess confidence in findings from qualitative evidence syntheses

, PLoS Med, vol.12, issue.10, p.1001895, 2015.

M. Davern, Nonresponse rates are a problematic indicator of nonresponse bias in survey research, Health Serv Res, vol.48, pp.905-917, 2013.

D. A. Dillman, G. Phelps, R. Tortora, K. Swift, J. Kohrell et al., Response rate and measurement differences in mixed-mode surveys using mail, telephone, interactive voice response (IVR) and the Internet, Soc Sci Res, vol.38, issue.1, pp.1-18, 2009.

A. Bryman, Integrating quantitative and qualitative research: how is it done?, Qual Res, vol.6, issue.1, pp.97-113, 2006.

C. Carroll, A. Booth, L. , and M. , Should we exclude inadequately reported studies from qualitative systematic reviews? An evaluation of sensitivity analyses in two case study reviews, Qual Health Res, vol.22, pp.1425-1459, 2012.

P. Pluye, Critical appraisal tools for assessing the methodological quality of qualitative, quantitative and mixed methods studies included in systematic mixed studies reviews, J Eval Clin Pract, vol.19, p.722, 2013.

R. Sirriyeh, R. Lawton, P. Gardner, and G. Armitage, Reviewing studies with diverse designs: the development and evaluation of a new tool, J Eval Clin Pract, vol.18, pp.746-52, 2012.

J. Higgins, D. G. Altman, P. C. Gøtzsche, P. J?-uni, D. Moher et al., The Cochrane Collaboration's tool for assessing risk of bias in randomised trials, Br Med J, vol.343, issue.d5928, pp.1-9, 2011.

U. Majid and M. Vanstone, Appraising qualitative research for evidence syntheses: a compendium of quality appraisal tools, Qual Health Res, vol.28, pp.2115-2146, 2018.

M. R. Lynn, Determination and quantification of content validity, Nurs Res, vol.35, issue.6, pp.382-388, 1986.

D. F. Polit, C. T. Beck, and S. V. Owen, Is the CVI an acceptable indicator of content validity? Appraisal and recommendations, Res Nurs Health, vol.30, pp.459-67, 2007.

S. Jamieson, Likert scales: how to (ab)use them, Med Educ, vol.38, issue.12, pp.1217-1225, 2004.