https://doi.org/10.4081/ripppo.2023.687
Validation of the client satisfaction questionnaire: a pilot psychometric analysis in Portuguese routine practice
Submitted: February 8, 2023
Accepted: June 4, 2023
Published: July 27, 2023
Accepted: June 4, 2023
Abstract Views: 1802
PDF: 465
HTML: 56
HTML: 56
Publisher's note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.
Authors
Treatment satisfaction has been found to have good therapeu- tic results in psychotherapy, and the 18-item version of the client satisfaction questionnaire (CSQ-18) is one of the most widely used measures to evaluate it. This study sought to carry out a pilot analysis of the psychometric’s properties and factorial structure, as well as validate the CSQ-18’s applicability to the Portuguese population within the context of routine clinical practice. It also sought to explore the association that treatment satisfaction has with general symptoms, therapeutic alliance, and expectations at an early stage of psychotherapy. The sample comprised 98 clients who were undergoing psychotherapy in a routine practice context. All these clients completed self-reported measures for symptom assessment and therapeutic relationship, namely the clinical out- come routine evaluation-outcome measure, working alliance in- ventory, and credibility/expectancy questionnaire, in addition to the CSQ-18. The semi-confirmatory factorial analysis demon- strated that the CSQ-18 has good psychometric properties and re- vealed an association between treatment satisfaction and therapeutic alliance. The results corroborate the findings of other versions of the measure and present a good adjustment model for the semi-confirmatory factorial analysis.
Attkisson, C. C., & Zwick, R. (1982). The client satisfaction questionnaire. Psychometric properties and correlations with service utilization and psychotherapy outcome. Evaluation and Program Planning, 5(3), 233-237. doi: 10.1016/0149-7189(82)90074-X.
DOI: https://doi.org/10.1016/0149-7189(82)90074-X
Baglin, J. (2014). Improving your exploratory factor analysis for ordinal data: a demonstration using FACTOR. Practical Assessment, Research and Evaluation, 19(5). doi: 10.7275/dsep-4220.
Barkham, M., Bewick, B., Mullin, T., Gilbody, S., Connell, J., Cahill, J., Mellor-Clark, J., Richards, D., Unsworth, G., & Evans, C. (2013). The CORE-10: a short measure of psychological distress for routine use in the psychological therapies. Counselling and Psychotherapy Research, 13(1), 3-13. doi: 10.1080/14733145.2012.729069.
DOI: https://doi.org/10.1080/14733145.2012.729069
Blenkiron P., Hammill C.A. (2003). What determines patients' satisfaction with their mental health care and quality of life? Postgraduate Medical Journal, 79(932):337-40. doi: 10.1136/pmj.79.932.337. PMID: 12840123; PMCID: PMC1742729.
DOI: https://doi.org/10.1136/pmj.79.932.337
Buffini, K. B., & Gordon, M. (2015). One-to-one support for crisis intervention using online synchronous instant messaging: evaluating working alliance and client satisfaction. British Journal of Guidance and Counselling, 43(1), 105-116. doi: 10.1080/03069885.2014.987723.
DOI: https://doi.org/10.1080/03069885.2014.987723
Brown, T. (2015). Confirmatory factor analysis for applied research. New York, Guilford Publications.
Carlson, M. J., & Gabriel, R. M. (2001). Patient satisfaction, use of services, and one-year outcomes in publicly funded substance abuse treatment. Psychiatric Services, 52(9), 1230-1236. doi: 10.1176/appi.ps.52.9.1230.
DOI: https://doi.org/10.1176/appi.ps.52.9.1230
Connell, J. & Barkham, M. (2007). CORE-10 user manual, version 1.1. Rugby, CORE System Trust & CORE Information Management Systems Ltd.
Constantino, M. J., Arnkoff, D. B., Glass, C. R., Ametrano, R. M., & Smith, J. Z. (2011). Expectations. Journal of Clinical Psychology, 67(2), 184-192. doi: 10.1002/jclp.20754.
DOI: https://doi.org/10.1002/jclp.20754
Costello, A. B., & Osborne, J. W. (2005). Best practices in exploratory factor analysis: four recommendations for getting the most from your analysis. Practical Assessment, Research and Evaluation, 10. doi: 10.7275/jyj1-4868.
Devilly, G. J., & Borkovec, T. D. (2000). Psychometric properties of the credibility/expectancy questionnaire. Journal of Behavior Therapy and Experimental Psychiatry, 31(2), 73-86. doi: 10.1016/S0005-7916(00)00012-4.
DOI: https://doi.org/10.1016/S0005-7916(00)00012-4
Donker, T., Bennett, K., Bennett, A., Mackinnon, A., Van Straten, A., Cuijpers, P., Christensen, H., & Griffiths, K. M. (2013). Internet-delivered interpersonal psychotherapy versus internet-delivered cognitive behavioral therapy for adults with depressive symptoms: Randomized controlled noninferiority trial. Journal of Medical Internet Research, 15(5), e82. doi: 10.2196/jmir.2307.
DOI: https://doi.org/10.2196/jmir.2307
Donovan, D. M., Kadden, R. M., DiClemente, C. C., & Carroll, K. M. (2002). Client satisfaction with three therapies in the treatment of alcohol dependence: results from Project MATCH. The American Journal of Addiction, 11, 291-307. doi: 10.1080/1055049029008809.
DOI: https://doi.org/10.1080/10550490290088090
Draper, M., Cohen, P., & Buchan, H. (2001). Seeking consumer views: what use are results of hospital patient satisfaction surveys? International journal for quality in health care, 13, 463-468.
DOI: https://doi.org/10.1093/intqhc/13.6.463
Egede, L.E., Acierno, R., Knapp, R.G., Walker, R.J., Payne, E.H., & Frueh, B.C.(2016) Psychotherapy for Depression in Older Veterans Via Telemedicine: Effect on Quality of Life, Satisfaction, Treatment Credibility, and Service Delivery Perception. Journal of Clinical Psychiatry, 77(12):1704-1711. doi: 10.4088/JCP.16m10951. PMID: 27835713.
DOI: https://doi.org/10.4088/JCP.16m10951
European Foundation for Quality Management. (2013). EFQM Excelence Model 2013. Brussels: EFQM.
Falkenström, F., Hatcher, R. L., Skjulsvik, T., Larsson, M. H., & Holmqvist, R. (2015). Development and validation of a 6-item working alliance questionnaire for repeated administrations during psychotherapy. Psychological Assessment, 27(1), 169-183. doi: 10.1037/pas0000038.
DOI: https://doi.org/10.1037/pas0000038
Field, A. (2017). Discovering Statistics Using IBM SPSS Statistics (5ºed). London: SAGE Publications.
Ferrando, P. J., & Lorenzo-Seva, U. (2017). Program FACTOR at 10: origins, development and future directions. Psicothema, 29(2), 236-240. doi: 10.7334/psicothema2016.304.
Hanson, K., Yip, W. C. & Hsiao, W. (2004) The impact of quality on the demand for outpatient services in Cyprus. Health Economics, 13 (12), 1167-1180. doi: 10.1002/hec.898.
DOI: https://doi.org/10.1002/hec.898
Horvath, A. O. and Greenberg, L. S. 1989. Development and validation of the Working Alliance Inventory. Journal of Counseling Psychology, 36: 223–233.
DOI: https://doi.org/10.1037/0022-0167.36.2.223
Howard, P.B., Rayens, M.K., El-Mallakh, P., & Clark, J.J. (2007). Predictors of satisfaction among adult recipients of medicaid mental health services. Archives of psychiatric nursing 21(5):257-69. doi: 10.1016/j.apnu.2007.06.004. PMID: 17904483.
DOI: https://doi.org/10.1016/j.apnu.2007.06.004
Hundt, Na., Armento, M. E. A., Porter, B., Cully, J., Kunik, M. E., & Stanley, M. (2013). Predictors of treatment satisfaction among older adults with anxiety in a primary care psychology program. Evaluation and Program Planning, 37, 58-63. doi: 10.1016/j.evalprogplan.2013.01.003.
DOI: https://doi.org/10.1016/j.evalprogplan.2013.01.003
Hutchison, M., Russell, B. S., Gans, K. M., & Starkweather, A. R. (2022). Online administration of a pilot mindfulness-based intervention for adolescents: feasibility, treatment perception and satisfaction. Current Psychology, 1-13. doi: 10.1007/s12144-022-03025-x.
DOI: https://doi.org/10.1007/s12144-022-03025-x
Jenkins-Guarnieri, M. A., Pruitt, L. D., Luxton, D. D., & Johnson, K. (2015). Patient perceptions of telemental health: systematic review of direct comparisons to in-person psychotherapeutic treatments. Telemedicine and E-Health, 21(8), 652-660. doi: 10.1089/tmj.2014.0165.
DOI: https://doi.org/10.1089/tmj.2014.0165
Kaiser, H. F. (1960). The application of electronic computers to factor analysis. Educational and Psychological Measurement, 20, 141–151.
Keleher, B., Oakman, J. M., Capobianco, K., & Mittelstaedt, W. H. (2019). Basic psychological needs satisfaction, working alliance, and early termination in psychotherapy. Counselling Psychology Quarterly, 32(1), 64-80. doi: 10.1080/09515070.2017.1367271.
DOI: https://doi.org/10.1080/09515070.2017.1367271
Kooistra, L. C., Ruwaard, J., Wiersma, J. E., van Oppen, P., van der Vaart, R., van Gemert-Pijnen, J. E. W. C., & Riper, H. (2016). Development and initial evaluation of blended cognitive behavioural treatment for major depression in routine specialized mental health care. Internet Interventions, 4, 61-71. doi: 10.1016/j.invent.2016.01.003.
DOI: https://doi.org/10.1016/j.invent.2016.01.003
Larsen, D. L., Attkisson, C. C., Hargreaves, W. A., & Nguyen, T. D. (1979). Assessment of client/patient satisfaction: development of a general scale. Evaluation and Program Planning, 2(3), 197-207. doi: 10.1016/0149-7189(79)90094-6.
DOI: https://doi.org/10.1016/0149-7189(79)90094-6
Lorenzo-Seva, U., & Ferrando, P. J. (2006). FACTOR: a computer program to fit the exploratory factor analysis model. Behavior Research Methods, 38(1), 88-91. doi: 10.3758/bf03192753.
DOI: https://doi.org/10.3758/BF03192753
Machado, P., & Horvath, A. (1999). Inventário de aliança terapêutica – WAI. In M. R. Simões, M. M. Gonçalves & L. S. Almeida (Eds.), Testes e Provas Psicológicas em Portugal (Vol. 2, pp. 87-94). Braga: APPORT/SHO.
Matsubara, C., Green, J., Astorga, L. T., Daya, E. L., Jervoso, H. C., Gonzaga, E. M., & Jimba, M. (2013). Reliability tests and validation tests of the client satisfaction questionnaire (CSQ-8) as an index of satisfaction with childbirth-related care among Filipino women. BMC Pregnancy and Childbirth, 13, 235. doi: 10.1186/1471-2393-13-235.
DOI: https://doi.org/10.1186/1471-2393-13-235
Miller, L., & Weissman, M. (2002). Interpersonal psychotherapy delivered over the telephone to recurrent depressives: a pilot study. Depression and Anxiety, 16(3), 114-117. doi: 10.1002/da.10047.
DOI: https://doi.org/10.1002/da.10047
Palacios, J. E., Richards, D., Palmer, R., Coudray, C., Hofmann, S. G., Palmieri, P. A., & Frazier, P. (2018). Supported internet-delivered cognitive behavioral therapy programs for depression, anxiety, and stress in university students: open, non-randomised trial of acceptability, effectiveness, and satisfaction. JMIR Mental Health, 5(4), e11467. doi: 10.2196/11467.
DOI: https://doi.org/10.2196/11467
Priebe S, Golden E, McCabe R, Reininghaus U. (2012). Patient-reported outcome data generated in a clinical intervention in community mental health care – Psychometric properties. Biomed Central Psychiatr, 12, 113. doi: 10.1186/1471-244X-12-113.
DOI: https://doi.org/10.1186/1471-244X-12-113
Roberts, R. E., Atrkisson, C. C., & Mendias, R. M. (1984). Assessing the client satisfaction questionnaire in English and Spanish. Hispanic Journal of Behavioral Sciences, 6(4), 385-396. doi: 10.1177/07399863840064004.
DOI: https://doi.org/10.1177/07399863840064004
Roe, D., Dekel, R., Harel, G., Fennig, S., & Fennig, S. (2006). Clients’ feelings during termination of psychodynamically oriented psychotherapy. Bulletin of the Menninger Clinic, 70(1), 68-81. doi: 10.1521/bumc.2006.70.1.68.
DOI: https://doi.org/10.1521/bumc.2006.70.1.68
Rosenvinge, J. H., & Klusmeier, A. K. (2000). Treatment for eating disorders from a patient satisfaction perspective: a Norwegian replication of a British study. European Eating Disorders Review, 8(4), 293-300. doi: 10.1002/1099-0968(200008)8:4<293::AID-ERV346>3.0.CO;2-4.
DOI: https://doi.org/10.1002/1099-0968(200008)8:4<293::AID-ERV346>3.0.CO;2-4
Rumpold, G., Doering, S., Smrekar, U., Schubert, C., Koza, R., Schatz, D. S., Bertl-Schuessler, A., Janecke, N., Lampe, A., & Schuessler, G. (2005). Changes in motivation and the therapeutic alliance during a pretherapy diagnostic and motivation-enhancing phase among psychotherapy outpatients. Psychotherapy Research, 15(1–2), 117-127. doi: 10.1080/10503300512331327092.
DOI: https://doi.org/10.1080/10503300512331327092
Sabourin, S., Pérusse, D., & Gendreau, P. (1989). Psychometric properties of the French-Canadian version of the client of Mental Health Services Satisfaction Questionnaire (QSC-8 et QSC-18B). Canadian Journal of Behavioural Science, 21(2), 147-159. doi: 10.1037/h0079853. [Article in French].
DOI: https://doi.org/10.1037/h0079853
Säilä, T., Mattila, E., Kaila, M., Aalto, P., & Kaunonen, M. (2008). Measuring patient assessments of the quality of outpatient care: a systematic review. Journal of Evaluation in Clinical Practice, 14(1), 148-154. doi: 10.1111/j.1365-2753.2007.00824.x.
DOI: https://doi.org/10.1111/j.1365-2753.2007.00824.x
Sales, C. M. D., Moleiro, C. M. de M., Evans, C., & Alves, P. C. G. (2012). The Portuguese version of the CORE-OM: translation, adaptation, and preliminary study of psychometric properties. Revista de Psiquiatria Clínica, 39(2), 54-59. doi: 10.1590/S0101-60832012000200003. [Article in Portuguese].
DOI: https://doi.org/10.1590/S0101-60832012000200003
Schulte, S. J., Meier, P. S., & Stirling, J. (2011). Dual diagnosis clients’ treatment satisfaction - a systematic review. BMC Psychiatry, 11. doi: 10.1186/1471-244X-11-64.
DOI: https://doi.org/10.1186/1471-244X-11-64
Smeets, R. J. E. M., Beelen, S., Goossens, M. E. J. B., Schouten, E. G. W., Knottnerus, J. A., & Vlaeyen, J. W. S. (2008). Treatment expectancy and credibility are associated with the outcome of both physical and cognitive-behavioral treatment in chronic low back pain. Clinical Journal of Pain, 24(4), 305-315. doi: 10.1097/AJP.0b013e318164aa75.
DOI: https://doi.org/10.1097/AJP.0b013e318164aa75
Smith, D., Roche, E., O’Loughlin, K., Brennan, D., Madigan, K., Lyne, J., Feeney, L., & O’Donoghue, B. (2014). Satisfaction with services following voluntary and involuntary admission. Journal of Mental Health, 23(1), 38-45. doi: 10.3109/09638237.2013.841864.
DOI: https://doi.org/10.3109/09638237.2013.841864
Silva, S., Barbosa, E., Salgado, J., & Cunha, C. (2021). Portuguese validation of the credibility/expectancy questionnaire in routine practice. Research in Psychotherapy: Psychopathology, Process and Outcome, 24(1). https://doi.org/10.4081/ripppo.2021.495
DOI: https://doi.org/10.4081/ripppo.2021.495
Timmerman, M. E., & Lorenzo-Seva, U. (2011). Dimensionality assessment of ordered polytomous items with parallel analysis. Psychological Methods, 16(2), 209-220.
DOI: https://doi.org/10.1037/a0023353
Vázquez, F. L., Torres, Á., Otero, P., Blanco, V., & Clifford Attkisson, C. (2019). Psychometric properties of the Castilian Spanish version of the client satisfaction questionnaire (CSQ-8). Current Psychology, 38(3), 829-835. doi: 10.1007/s12144-017-9659-8.
DOI: https://doi.org/10.1007/s12144-017-9659-8
Warnecke, I., Reiss, N., Werner, A., Luka-Krausgrill, U., & Rohrmann, S. (2020). Treatment satisfaction and therapeutic alliance in three different treatment groups for test anxiety. Journal of Cognitive Psychotherapy, 34(2), 131-147. doi: 10.1891/JCPSY-D-18-00027.
DOI: https://doi.org/10.1891/JCPSY-D-18-00027
World Health Organization (2007). People-Centred Health Care: A policy framework. Geneva, Switzerland: WHO Document Production Services.
How to Cite
Silva, S., Basto, I., Salgado, J., & Cunha, C. (2023). Validation of the client satisfaction questionnaire: a pilot psychometric analysis in Portuguese routine practice. Research in Psychotherapy: Psychopathology, Process and Outcome, 26(2). https://doi.org/10.4081/ripppo.2023.687
Copyright (c) 2023 the Author(s)
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
PAGEPress has chosen to apply the Creative Commons Attribution NonCommercial 4.0 International License (CC BY-NC 4.0) to all manuscripts to be published.
Similar Articles
- Linnea Nissling, Sandra Weineland, Kristofer Vernmark, Ella Radvogin, Anna-Karin Engström, Sara Schmidt, Eva Nieto Granberg, Elin Larsson, Timo Hursti, Effectiveness of and processes related to internet-delivered acceptance and commitment therapy for adolescents with anxiety disorders: a randomized controlled trial , Research in Psychotherapy: Psychopathology, Process and Outcome: Vol. 26 No. 2 (2023)
- Osmano Oasi, Laura Buonarrivo, Alberto Codazzi, Margherita Passalacqua, Gio Matteo Risso Ricci, Francesca Straccamore, Roberto Bezzi, Assessing personality change with Blatt's anaclitic and introjective configurations and Shedler-Westen Assessment Procedure profiles: two case studies in psychodynamic treatment , Research in Psychotherapy: Psychopathology, Process and Outcome: Vol. 20 No. 1 (2017)
- Marjolein M. W. Koementas-de Vos, Lucas C. Colleye, Bea Tiemens, Fabiana Engelsbel, Kim de Jong, Cilia L. M. Witteman, M. Annet Nugter, The feasibility, acceptability and effectiveness of a feedback-informed group treatment (FIGT) tool for patients with anxiety or depressive disorders , Research in Psychotherapy: Psychopathology, Process and Outcome: Vol. 25 No. 3 (2022): SPECIAL ISSUE: "Group psychotherapy: between settled benchmarks and new horizons"
- Cheri L. Marmarosh, Steven Sandage, Nathaniel Wade, Laura E. Captari, Sarah Crabtree, New horizons in group psychotherapy research and practice from third wave positive psychology: a practice-friendly review , Research in Psychotherapy: Psychopathology, Process and Outcome: Vol. 25 No. 3 (2022): SPECIAL ISSUE: "Group psychotherapy: between settled benchmarks and new horizons"
- Antonino La Tona, Sofia Tagini, Agostino Brugnera, Barbara Poletti, Edoardo Nicolò Aiello, Gianluca Lo Coco, Lidia Del Piccolo, Angelo Compare, Italian validation of the Clinical Outcomes in Routine Evaluation-10 (CORE-10): a short measure for routine outcome monitoring in clinical practice , Research in Psychotherapy: Psychopathology, Process and Outcome: Vol. 26 No. 1 (2023)
- Diego Rocco, Silvia Salcuni, Elena Antonelli, A pilot study of the Italian adaptation of the Session Evaluation Questionnaire fourth version , Research in Psychotherapy: Psychopathology, Process and Outcome: Vol. 20 No. 2 (2017)
- Chiara Rossi, Francesca De Salve, Martina Agliati, Osmano Oasi , Coping strategies and mental health: A web-based survey among the Italian population dealing with COVID-19 , Research in Psychotherapy: Psychopathology, Process and Outcome: Vol. 25 No. 2 (2022)
- Margarida Pedroso de Lima, Isabel Albuquerque, Paulo Jorge Martins, António-José Gonzalez, Personal Projects Analysis as an idiographic approach in psychotherapy: an exploratory study , Research in Psychotherapy: Psychopathology, Process and Outcome: Vol. 26 No. 1 (2023)
- Rachele Mariani, Silvia Monaco, Christopher Christian , Michela Di Trani, Dreaming in quarantine: linguistic analysis of referential process of dreams during COVID-19 pandemic lockdown , Research in Psychotherapy: Psychopathology, Process and Outcome: Vol. 24 No. 2 (2021): SPECIAL ISSUE "Working on dreams, from psychotherapy to neuroscience"
You may also start an advanced similarity search for this article.