Development And Preliminary Psychometric Testing Of An Adult Chronic Kidney Disease Self-Management (CKD-SM) Questionnaire
Mar 09, 2022
Introduction
Chronic kidney disease (CKD) affects approximately 10% of the adult population in Canada and leads to significant morbidity, mortality, and health care resource utilization.1 Chronic kidney disease is a chronic condition requiring continuous management; the number of people requiring kidney replacement therapy for kidney failure (dialysis or transplant) is steadily increasing, resulting in poor health outcomes for patients and an unsustainable cost to the health care system.2,3 In an effort to reduce the risk of kidney failure, CKD management has shifted toward early identification and initiation of proven preventive activities, such as pharmacological and lifestyle management.4,5 By intervening early, often before patients experience many symptoms, the risk of developing end-stage kidney disease (ESKD) can be reduced significantly.6,7. Patients with CKD who are not yet on kidney replacement therapy (KRT) or preparing to start KRT often have few symptoms, and consequently, they are less inclined to engage in prevention activities, such as dietary modification.1 As a result, interventions aiming to promote self-management soon after CKD diagnosis are increasingly common.8 Despite these efforts, sustained behavior change can be challenging, and most interventions have not been rigorously evaluated; therefore, it is challenging to determine which interventions may be effective and why.8,9 Compounding this is a lack of valid self-report instruments to evaluate self-management interventions targeted toward patients with CKD who are not on KRT. Most available instruments evaluate interventions targeted for “chronic disease” in general or focus on later stages of CKD when patients are nearing or already receiving dialysis and their self-management needs are very different (such as dialysis-related fistula management and closely monitoring serum potassium and phosphorus levels).10-18 Moreover, clinicians and content experts developed many of these existing questionnaires with little, or no, input from patients and caregivers, and so the questionnaires may not reflect patients’ self-management needs and priorities. Four related questionnaires developed to address CKD awareness and/or self-management,14,15,19,20 have been used to assess self-management in CKD. However, these questionnaires were developed and validated in China and Taiwan, which introduces some cultural context and translation challenges that may limit the “conceptual equivalence” of the questionnaires in a Canadian context.21 Moreover, several important self-management domains identified by patients and caregivers in Canada22 are not addressed. As a result, there is a need to develop a patient-informed questionnaire that specifically evaluates CKD self-management behavior in a Canadian context; such an instrument would assist in the evaluation of future self-management interventions and educational initiatives targeted to this population. We, therefore, aimed to develop and test a patient-informed questionnaire to assess CKD-specific self-management behavior in adults with CKD (CKD categories G2-G5 not on KRT).
Keywords: chronic kidney disease, self-management, patient-reported outcomes, questionnaire development, pre-dialysis
CISTANCHE WILL IMPROVE KIDNEY/RENAL DIALYSIS
Methods The CKD self-management (CKD-SM) questionnaire was developed in 4 phases (Figure 1): (1) item identification; (2) expert panel review; (3) pilot test; and (4) test within a CKD self-management intervention study. Phase 1: Item Identification The purpose of this phase was to review the self-management literature and draft potential questionnaire items for the first round of expert panel review in phase 2. Self-management is broadly defined as being an active participant in one’s medical care and treatment.23,24 Self-management also incorporates aspects of self-efficacy (the belief in one’s ability to take action) and self-care (the ability and confidence to carry out activities that are needed to attain optimal health).25,26 We used self-management theoretical frameworks (each incorporating aspects of knowledge, skills, and confidence needed to optimally manage chronic illness; see Box 1), patient-identified self-management priorities identified by Donald et al22,27 (understanding CKD, diet, symptoms, medications, physical and mental wellbeing, finances, travel, work, and education), and reviewed relevant existing self-management questionnaires (Table S1) to guide questionnaire development. Existing self-management/self-efficacy questionnaires were identified using a multi-pronged approach: a MEDLINE and National Institutes of Health National Library of Medicine Health Services and Sciences Research Resources Instrument search (using the search terms self-management, self-care, self-efficacy, kidney) in January 2020.
Three research team members (M.D.S., M.D., B.R.H.) iteratively reviewed the potential items using a content coverage matrix (Figure 2) to assess content coverage and representativeness and to ensure congruence with self-management theoretical framework components and patient-identified priorities. We received preliminary feedback regarding content coverage and representativeness from content experts (3 nephrologists and a health services researcher specializing in CKD self-management who subsequently joined the expert panel), prior to the expert panel review in phase 2 of the study. Once the potential CKD-SM questionnaire items were identified, we selected a scaling response strategy for the questionnaire. As attitudes and behaviors, such as knowledge and confidence to manage CKD, lie on a continuum,21 we selected a continuous direct estimation scaling method to collect participant responses along a 7-point Likert scale ranging from “strongly disagree” to “strongly agree” and “extremely not confident” to “extremely confident.” A 7-point scale was chosen to optimize response reliability while minimizing respondent burden.21
Phase 2: Expert Panel Review and Item Revision The purpose of this phase was to reach a consensus among experts about items and finalize the draft questionnaire for pilot testing with patients. An expert panel, comprising 8 members (including 4 nephrologists, 2 CKD nurses, 1 patient partner, and 1 CKD self-management researcher), was purposively selected to ensure representation from a variety of clinical roles and prior experience with CKD self-management concepts. The panel was provided with a description of the construct (CKD self-management), the purpose of the tool (to detect whether patients experience a change in self-management behavior following an intervention), and a draft of the CKD-SM questionnaire items. Each round of review was conducted via an electronic questionnaire using Qualtrics software (Qualtrics, Provo, Utah). Panel members were asked to rate each item on a 4-point Likert scale ranging from “not relevant” to “highly relevant” for self-management in early-stage CKD (defined as patients with CKD categories G2-G5 not on KRT or preparing for KRT). If a panel member selected “not relevant” or “somewhat relevant” for an item, they were prompted to record a rationale. An open-ended question was also included for panel members to suggest additional items they felt were an important aspect of CKD self-management. The questionnaire was modified following feedback, including deletion of items scored “not relevant” or “somewhat relevant” by a majority (>50%) of the panel and addition of newly suggested items. The panel independently reviewed the modified questionnaire items iteratively until the content validity ratio (CVR) was ≥0.75 (recommended for panels with 7-9 raters), indicating the majority of reviewers believed the item was essential.21,31 Items with a CVR <0.75 were considered for retention if their removal compromised an important content domain (content validity) outlined in the content coverage matrix (Figure 2).
Phase 3: Pilot Test The purpose of the pilot test was to evaluate reliability and validity with patients and collect information on understandability, phrasing, and potential self-management content gaps that may lead to the identification of additional questionnaire items. The CKD-SM questionnaire was pilot tested with a convenience sample of Canadian, English-speaking adults with CKD (non-dialysis) via an electronic questionnaire advertised by the Interdisciplinary Chronic Disease Collaboration (ICDC; https://cumming.ucalgary.ca/research/icdc) in a single tweet on Twitter in April 2020. Respondents who self-identified as having been diagnosed with CKD and not on dialysis were eligible to participate. Questionnaires were collected anonymously using Qualtrics survey software (Qualtrics). We sought to include 30 participants in the pilot test; this sample size was chosen based on similar studies suggesting that a sample size of 30 to 50 is adequate for a “pretest” of this nature.32 In addition to completing the CKD-SM questionnaire, participants were asked open-ended questions relating to clarity of the questions, comprehension, spelling/word familiarity, ease of response, and whether there were any content/questions missing that they felt were important to self-management in CKD. Questionnaire response reliability was evaluated using the Cronbach α test for internal consistency (to determine whether the items consistently measured the same construct). A Cronbach α value greater than 0.70 was considered acceptable.33 Responses to open-ended questions were categorized using a descriptive, conventional content analysis approach.34
CISTANCHE WILL IMPROVE KIDNEY/RENAL INFECTION
Phase 4: Embed in a CKD-SM Intervention. The purpose of this phase was to pilot test the CKD-SM questionnaire within a self-management intervention study and collect psychometric data relating to reliability, stability, and responsiveness. As the CKD-SM questionnaire was under development, we did not seek to evaluate the self-management intervention with this questionnaire (a separate study to evaluate the feasibility of the self-management intervention was underway and is published elsewhere35); rather, the intervention allowed an opportunity to collect additional psychometric data on the instrument itself. During the self-management intervention, participants had self-directed access to an electronic self-management tool, the My Kidneys My Health website (www. mykidneysmyhealth.com), for 8 weeks.35 The website was co-designed with patients, researchers, and clinicians with an aim to provide CKD self-management support through education and interactive tools (such as kidney-friendly food lists, nonprescription medication guidance, and depression screening).36
The CKD-SM questionnaire was administered twice in the preintervention period, at baseline and 1 week later (test-retest), and postintervention approximately 8 weeks after participants obtained access to the Web-based support tool. The study included a convenience sample of Canadian, English-speaking adults with CKD (non-dialysis) recruited online via Twitter and organizational Web sites (ICDC, Can-SOLVE CKD Network, and the Kidney Foundation of Canada) and through advertisements in 2 nephrology clinics in Alberta from June 2020 to December 2020. Questionnaires were emailed to participants at each time point and completed electronically using Research Electronic Data Capture (RedCap) 10 data capture software (Vanderbilt University Medical Center). The analysis included internal consistency reliability (Cronbach α) and test-retest reliability using a 2-way mixed, absolute effects model (intraclass correlation coefficient [ICC]) to determine whether respondents’ scores were stable between administrations prior to the self-management intervention and pre-post test scores (Wilcoxon sign-rank test). Cronbach α and ICC values greater than 0.7 were considered acceptable.33,37 Responsiveness was considered acceptable if preintervention test-retest responses were stable (ICC >0.7) but paired pre-post intervention responses were significantly different (P ≤ .05), suggesting a measurable change in self-management following the intervention. This study was reviewed and approved by the University of Calgary Conjoint Research Ethics Board (CHREB), REB20-0153. Consent was collected and documented for all study participants (implied consent was collected as a component of the anonymous electronic questionnaire for pilot test participants in phase 3, and oral consent was collected and documented for CKD self-management intervention study participants in phase 4).
Results
Phases 1 and 2: Item Identification and Expert Panel Review Following a review of existing chronic disease self-efficacy questionnaires and patient-identified priorities, 22 potential questionnaire items were drafted for the first round of expert panel review (Table S2). A panel of 8 content experts (including 4 nephrologists, 2 CKD nurses, 1 patient partner, and 1 CKD self-management researcher) reviewed the draft CKD-SM questionnaire. Two rounds of the review were completed; of the 22 items reviewed in the first round, 13 items were retained. In the second round, 11 items were reviewed and, initially, 10 were retained; following review of the content coverage matrix (Figure 2), the item relating to travel and work was retained to preserve content validity overall. Of the 24 items retained following expert panel review of the questionnaire, 21 had >85% acceptance (CVR ≥0.75) and 3 had 75% acceptance (CVR ≥0.50). The 3 items with CVR <0.75 were retained to preserve content validity. Following expert panel review, the finalized questionnaire included content related to knowledge of CKD, skills/confidence to engage in CKD self-management, blood pressure targets, laboratory measurements, nonprescription medications, symptoms, diet, lifestyle factors, communicating needs, and information-seeking behavior.
CISTANCHE WILL IMPROVE KIDNEY/RENAL PAIN
Phase 3: Pilot Test
Thirty patients with CKD from across Canada participated in the pilot test from April 2020 to May 2020 (Table 1). Approximately half were men (14/30; 46%) and between 50 and 64 years of age (13/30; 43%). Approximately 40% (12/30) had an estimated glomerular filtration rate (eGFR) of <30 mL/min/1.73 m2 ; the remainder had an eGFR of ≥30 mL/min/1.73 m2 or did not know their eGFR. Internal consistency was high (Cronbach α = 0.921). Participants indicated they were satisfied with the content, wording, and design; one participant indicated the questionnaire was “Very quick and easy” (pt 105) and another stated they “thought it was really good. It is progressive, one question leads to the next in terms of relevance” (pt 102) (Table S3). A common comment was the need for a question that more explicitly addressed mental health; for example, one participant responded, “There are no questions about mental health, about family and support systems as CKD affects the whole family” (pt 110) and another asked, “Why is everyone afraid to deal with the mental health issues we face?” (pt 129). Based on this feedback and review with the expert panel, we included an additional item relating to mental health in the final questionnaire: “I feel I am able to manage my mental health concerns, should they arise” (Figure 2) prior to the intervention study in phase 4 (bringing the total number of questions in the final questionnaire to 25 items); see Figure S1 for a hard copy version of the final questionnaire.
Phase 4: Test Embedded in a CKD-SM Intervention Study Twenty-nine patients from across Canada participated in the CKD self-management intervention study from June 2020 to February 2021 (Table 1). Approximately half were men (15/29; 52%) and between 50 and 64 years of age (11/29; 38%). Approximately 40% (11/29) had an eGFR of <30 mL/ min/1.73 m2 and 60% (18/29) had an eGFR of ≥30 mL/ min/1.73 m2 or did not know their eGFR. Internal consistency reliability (Cronbach α) was 0.912 (Table 2). Test-retest reliability, measured approximately 1 week apart (preintervention) using ICC, was 0.732 (95% confidence interval, 0.686-0.77) (P < .001). Twenty-two paired pre/postintervention responses, measured approximately 2 months apart using Wilcoxon sign-rank test, demonstrated significant improvements (P < .05) in self-management for 8 items (Table 3) despite stable preintervention test-retest scores; no significant change was observed for the remaining 17 items. Seven participants lost to follow-up did not provide a reason for nonresponse. Two reminder emails were sent to these participants and then they were marked as lost to follow-up.
Discussion
We developed a CKD-SM questionnaire for adults with CKD categories G2-G5 not on KRT and conducted preliminary psychometric testing (including validity, reliability, stability, and responsiveness). The CKD-SM questionnaire was developed through expert consensus, pilot-tested with patients, and administered in a pre-post CKD self-management intervention study. Overall, the results suggest the CKD-SM questionnaire performed reliably across multiple administrations, indicating that the CKD-SM questions consistently measured dimensions of the self-management construct. The questionnaire also appeared to be relatively stable across repeated administrations (in the absence of self-management
intervention), but was also responsive to potential changes in perceived self-management ability following participation in an electronic self-management intervention. While this study did not evaluate the self-management intervention directly, preliminary results from a feasibility study conducted concurrently with this questionnaire development study suggest the intervention supported patients to manage CKD, particularly with respect to finding and understanding information about CKD, kidney-friendly foods, and choosing nonprescription medications.35 The CKD self-management questionnaire items that demonstrated improvement following this intervention reflect improvement in these specific domains, suggesting that the questionnaire was sensitive to the improvements noted by participants in the feasibility study of the self-management intervention Studies suggest that effective self-management may slow CKD progression8,9; however, there are few measures available to evaluate patient self-management in CKD, particularly for individuals experiencing early stages of CKD, making it challenging to identify effective self-management interventions targeted at this population.8,9 Existing measures tend to be nonspecific, evaluating chronic disease broadly, or include questions relating to symptoms and therapies that are common only in later-stage CKD, such as dialysis-related fatigue, fistula management, and monitoring blood phosphorus levels.10-18 We identified 3 existing questionnaires that were relevant to CKD self-management for patients not on KRT.14,15,19 These questionnaires served as excellent references for item identification for the first iteration of our CKD-SM questionnaire. Although the questionnaires addressed many of the content areas used to develop the CKD-SM questionnaire, there were some gaps across the questionnaires primarily relating to patient-identified self-management domains (such as confidence choosing nonprescription medications, traveling, and managing mental/ emotional health) and lack of “conceptual equivalence” for a Canadian context (these questionnaires were developed and validated in China and Taiwan, and some of the translated questions, such as “Do you know how to evaluate your curative effect”15 may be unclear to this audience). Our study builds on these earlier questionnaires by attempting to address these gaps and providing a comprehensive self-management questionnaire for patients with CKD who are not on KRT in a Canadian context.
The CKD-SM questionnaire was developed by integrating self-management behavior theories (relating to tasks, skills, processes, and integration25,28-30) and by consulting with a panel of clinicians and patients to ensure the question nature's content aligns with what is currently known about patients’ self-management needs in early-stage CKD (ie, understanding CKD, diet, symptoms, medications, physical and mental well-being, finances, travel, work, and education)22,27 As a result, the CKD-SM questionnaire provides a patient-informed measure that incorporates aspects of the physical, emotional, and social needs of patients, in addition to the more typically considered clinical aspects related to
CKD self-management. As reported by others,8,38,39 clinical teams are increasingly acknowledging the importance of incorporating the needs of the “whole person” in self-management interventions. The CKD-SM questionnaire content is a reflection of these broader self-management concepts, including questions relating to social and emotional support, for example, in addition to the more typical clinical outcomes used to assess self-management, such as monitoring blood pressure and laboratory tests.
Limitations There are limitations that should be considered when interpreting the results of this study. First, although the questionnaire development was guided by patient priorities and finalized through expert consensus, the sample sizes were small for each component of the analysis and the sampling overall was consecutive/convenience-based. With that in mind, the consistently high Cronbach α results across multiple administrations provide assurance that the internal consistency of the questionnaire is acceptable. Second, our literature review of existing self-management questionnaires was not systematic. Finally, while the study included patients from across Canada, participants had a range of CKD severity (none of the participants were on KRT, however). We attempted to mitigate this in the pilot study by asking patients to reflect on their needs and experiences when they were first diagnosed. As the questionnaire is intended to assess CKD self-management in patients not on KRT, psychometric studies encompassing larger samples sizes and from multiple contexts are needed.
CISTANCHE WILL IMPROVE KIDNEY/RENAL FAILURE
Conclusion The CKD-SM questionnaire performed well in preliminary psychometric testing and has the potential to contribute to our understanding of self-management among those with CKD. The questionnaire could be particularly helpful in clinical settings as a baseline measure of perceived ability to manage aspects of CKD that have been identified as important for successful CKD self-management, to monitor and proactively address gaps in knowledge, skills, and confidence to manage CKD, and/or to evaluate self-management interventions. Additional psychometric testing in larger studies will strengthen our understanding of the effectiveness of the questionnaire for measuring CKD self-management; the questionnaire will be further evaluated in self-management intervention studies conducted by our team in the future.
