English | Français  


Project Results

What is a Quality Measure?

Who is CEQM?

National Consensus

National Consensus Summary

Top 30 Quality Measures

Quality Measures Database

Priority Domains

Data Infrastructure

Measurement Implementation

Knowledge Transfer / Communication

Project Activities



Staff/Partner log-in
National Consensus Subproject Results








National Consensus Subproject

The purpose of the National Consensus subproject was to develop a set of health measures for Canadian primary care mental health services, reflecting a multi-stakeholder perspective, and suitable for facilitating quality improvement.

Project Overview and National Consensus Model

Formation of a Multi-regional steering committee

The scope of this project was determined in part by the availability of human resources, by our commitment to a consensus based model and our determination to secure national and multi-regional input into designing a framework for the work of the project.

In January, 2004 each of our project partner sites (AB, BC, SK, ON, PQ) were surveyed regarding time and availability as well as their ideas regarding revised project goals and objectives. By March 2004, a Steering Committee and an Advisory Group were formed. The core Steering Committee members included expert representatives from the Universities of British Columbia, Alberta, Saskatchewan, the University of Toronto and the University of Western Ontario and from l’Institut national de santé publique due Québec.

We began with a preliminary “setting the stage” process

The Steering Committee, through a consensus process, defined in further detail, tasks and timelines for this initiative. This “setting the stage” component allowed for the delineation of three subsequent stages (see diagram above), each driven by the establishment of national consensus processes related to primary mental health care and quality measurement. During this preliminary stage, the project also grappled with a fundamental challenge: limiting of the scope of the project while maintaining continued inter-regional “buy-in.”

Highlights - “setting the stage” process

  • Established project partners at six sites in five provinces (AB, BC, SK, ON, PQ)
  • Created four sub-project teams – National Consensus, Knowledge Transfer, Data Analysis, and Research & Evaluation
  • Created a consensus based process with feedback loops amongst partners to maintain consistency with PHCTF objectives
  • Formalized internal consensus and knowledge transfer and exchange processes to further collaboration toward the development of quality measures for primary mental health care. Ensured inclusion of ‘state of the art’ best practices, methodology and evidence informed literature regarding quality measurement based on work by Stephen Campbell (UK) 

Overview of Stage 1 – populating and defining a framework

A flashlight in a dark room

The current challenge of quality measurement in health care has been compared to attempting to navigate a large, dark room with only a flashlight to illuminate one’s way. One must chose where to focus the light in order to navigate the room effectively and safely. Similarly, with health quality measurement, no current set of ideal measures exist that will allow us to have a full understanding of the whole system.

Use of consensus to develop areas of focus

In Stage 1, in order to address the challenge of a potentially limitless number of quality measures, our Steering Committee directed the project to develop a consensus process to determine key areas of focus ( that is, where the project would point our “flashlight in the dark room”). As part of this process the project developed a framework regarding various aspects of the primary mental health system (domains). This framework was adapted from both the Canadian Institute of Health Information (CIHI) Health Indicators Framework and Richard Hermann’s (2002) mental health framework. Additional domains were added to the above frameworks to increase the relevance to primary mental health care. These additional domains were generated through a consensus based process that involved a wide range of stakeholders with additional reference to current research literature.

Highlights - Stage 1

  • Developed an inclusive framework of domains based on existing literature; as well as multi-stakeholder feedback

  • Regional sites recruited participants within the identified stakeholder groups to facilitate knowledge transfer and exchange about the domains and the work of the project

  • Initiated linkages with the Canadian Collaborative Mental Health Care project

  • 145 clinicians, decision-makers and mental health user representatives in 5 regions (AB, BC, SK, ON, PQ) participated Survey 1

  • National consensus was achieved on 22 priority domains out of a possible 81.

  • Initiated knowledge transfer and exchange linkages with federal/provincial/territorial decision-making bodies

  • The National KT team began creating a roster of national opinion leaders within our four stakeholder groups by retrieving and gathering background information on these individuals

Overview of Stage 2 - expert review regarding evidence informed best practices and literature

In Stage 2 of the initiative, we conducted a second survey with a national and international group of experts to research best practices and existing quality measures associated with the domains or areas of focus identified in Stage 1 of the project.

Highlights - Stage 2

  • Conducted a Best Practices and Quality Measures literature review by administering a consultation survey to identified academics, clinicians, decision-makers, and mental health user representatives knowledgeable about evidence-based practices in primary mental health care in each of the 22 national consensus domains. Respondents were asked to suggest best practices and measures of quality where the literature is inadequate or did not exist.

  • The Quality Measures Database was created, containing functionalities (i.e. tracking edits, as well as running queries) to assist in assessing the results from Survey 2 and to facilitate the decision making about the content of the project’s 3rd and final survey. All identified best practices and quality measures (approximately 3,000) are compiled into this searchable database.

  • Data group members investigated a method or rating system for examining data feasibility for quality measures. These preliminary results fed into the Stage 2 literature review process by assisting in the design for the review forms/evidence extraction sheets for best practices and quality measures.

  • Expansion and continuation of the project’s national consensus and knowledge transfer and exchange processes:
    • Initial formation of a primary mental health care Knowledge Transfer & Exchange Network with up to 300 multi-stakeholder group respondents from across Canada, including representation from every province and territory. A future CEQM KTE Network should allow for enriched collaboration and knowledge exchange, furthering quality improvement in PMHC across the country. [ on KTE page, add link to this section]
    • Dissemination about the project via linkages with other projects, with leading research institutions and attendance at a range of national and regional events, fully described in the dissemination activities section of this report.

Overview of Stage 3 – results and final measures 

In Stage 3 of the initiative, we conducted our third and final survey with respondents from every province and territory in Canada and within those regions, with a range of stakeholders.

Highlights – Stage 3

  • A two round modified Delphi postal survey
  • Each round consisted of a 90+minute survey
  • Rating of relevance, actionability, overall importance of 160 measures by 212 final respondents with a response rate of 80%
  • Consensus methodology used amongst regions and stakeholder groups to narrow down measures to a final set of 30 measures, with significant inter-stakeholder and regional variation
  • Consumer/advocates, clinicians, academics and government decision makers from regional, provincial and federal levels participated
  • A small group of people with expertise in First Nations and rural-setting health issues were also included
  • A convenience sampling strategy was used to ensure a balanced representation
  • Respondents emphasized actionability, which lead to a “pragmatist” orientation in results; rigour of consensus methodology and an evidence-informed process gives a “green light” for action in primary mental health care reform


Team lead

Paul Waraich
Assistant Professor, Department Psychiatry, UBC
Adjunct Professor, CARMHA, Faculty of Health Sciences, SFU

Other team members

Denise Aubé
Donald Addington
Ulrieke Birner
Ruchi Bhalla
Jana Carson
Lydia Cartar
John Conway
Alex Craciunescu

Martha Donnelley
Jennifer Hermann
David Haslam
Wayne Jones
Pamela Joshi
Hamish Khamisa
Tania Kyle
Elizabeth Lin

Eric MacNaughton
Nawaf Madi
Luljeta Pallaveshi
Radha Puri
Mélanie St-Onge
Ellisa Wee
Natasha Zaslavska


v 3 Sep 28, 2006


Copyright © 2006 CEQM and CARMHA • infoceqm-acmq.com

The views expressed herein do not necessarily represent the official policies of Health Canada