Evidence map›Paper›PMID 37713198›Full record

ArticleJAMA network open2023

Visit and Between-Visit Interaction Frequency Before and After COVID-19 Telehealth Implementation.

Sarah Nouri, Courtney R Lyles, Elizabeth B Sherwin, Magdalene Kuznia, Anna D Rubinsky, Kathryn E Kemper, Oanh K Nguyen, Urmimala Sarkar, Dean Schillinger, Elaine C Khoong

Abstract read
In one paragraph

Article in JAMA network open, 2023. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 13 papers, 1 of them a synthesis that pooled it.

0numbers the graph read from it
0cells of the map it votes in
13citing papers in PubMed, 1 pooled it
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

13 citing papers in PubMed, 1 synthesis or guideline pooled it.

  1. Pooled it
  2. Trial
  3. Article
  4. Article
  5. Article
  6. Article
  7. Article
  8. Factors impacting electronic patient-generated data use in safety-net systems: a qualitative study.Journal of the American Medical Informatics Association : JAMIA · 2025
    Article
  9. Article
  10. Article
  11. Article
  12. Applying Machine Learning Techniques to Implementation Science.Online journal of public health informatics · 2024
    Article
  13. Appointments aren't Access: The Hidden Work of Getting Care.Journal of primary care & community health
    Article
4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

10 authors.

Sarah NouriDivision of Palliative Medicine, Department of Medicine, University of California San Francisco.
Courtney R LylesDivision of General Internal Medicine at Zuckerberg San Francisco General Hospital, Department of Medicine, University of California San Francisco.
Elizabeth B SherwinDepartment of Epidemiology and Biostatistics, University of California San Francisco.
Magdalene KuzniaUniversity of California San Francisco School of Nursing.
Anna D RubinskyDepartment of Epidemiology and Biostatistics, University of California San Francisco.
Kathryn E KemperUCSF Center for Vulnerable Populations, University of California San Francisco.
Oanh K NguyenUCSF Center for Vulnerable Populations, University of California San Francisco.
Urmimala SarkarDivision of General Internal Medicine at Zuckerberg San Francisco General Hospital, Department of Medicine, University of California San Francisco.
Dean SchillingerDivision of General Internal Medicine at Zuckerberg San Francisco General Hospital, Department of Medicine, University of California San Francisco.
Elaine C KhoongDivision of General Internal Medicine at Zuckerberg San Francisco General Hospital, Department of Medicine, University of California San Francisco.

Funding

VARC CoreP30AG044281 · NIA · UNIVERSITY OF CALIFORNIA, SAN FRANCISCO · PI REBECCA L SUDORE · 2013 to 2026
$19.9M
Institutional Career Development Core SupplementKL2TR001870 · NCATS · UNIVERSITY OF CALIFORNIA, SAN FRANCISCO · PI KANAYA, ALKA M., SARKAR, URMIMALA · 2016 to 2025
$19.7M
Translational Research Core - Health Engagement & Action Translational (HEAT)P30DK092924 · NIDDK · KAISER FOUNDATION RESEARCH INSTITUTE · PI Alyce Sophia Adams, HILARY Kessler SELIGMAN · 2011 to 2026
$9.2M
Testing combinations of population interventions to encourage healthy eatingR01DK116852 · NIDDK · UNIVERSITY OF CALIFORNIA, SAN FRANCISCO · PI SCHILLINGER, DEAN · 2018 to 2022
$3.7M
Evaluation of Natural Experiments of Nutrition Assistance to Prevent and Control Diabetes among Low-Income CommunitiesU18DP006526 · DP · UNIVERSITY OF CALIFORNIA, SAN FRANCISCO · PI SCHILLINGER, DEAN · 2020 to 2024
$1.9M
Addressing Social Vulnerabilities to Prevent Hospital Readmissions in Adults with Cardiovascular DiseaseK23HL133441 · NHLBI · UT SOUTHWESTERN MEDICAL CENTER · PI NGUYEN, OANH KIEU · 2017 to 2021
$932k
Designing a Remote Monitoring Intervention for Value & Equity in Hypertension (DRIVE-HTN)K23HL157750 · NHLBI · UNIVERSITY OF CALIFORNIA, SAN FRANCISCO · PI KHOONG, ELAINE · 2021 to 2025
$921k
Understanding the Impact of Technology Access and Digital Literacy on Post-Discharge Care and Hospital Readmissions in Adults with Cardiovascular DiseaseR03HL157931 · NHLBI · UNIVERSITY OF CALIFORNIA, SAN FRANCISCO · PI NGUYEN, OANH KIEU · 2021 to 2022
$162k
NCATS NIH HHS KL2 TR001870NCCDPHP CDC HHS U18 DP006526NHLBI NIH HHS K23 HL133441NHLBI NIH HHS K23 HL157750NHLBI NIH HHS R03 HL157931NIA NIH HHS P30 AG044281NIDDK NIH HHS P30 DK092924NIDDK NIH HHS R01 DK116852
6 · The paper itself

Abstract

Importance: Telehealth implementation associated with the COVID-19 public health emergency (PHE) affected patient-clinical team interactions in numerous ways. Yet, studies have narrowly examined billed patient-clinician visits rather than including visits with other team members (eg, pharmacists) or between-visit interactions. Objective: To evaluate rates of change over time in visits (in-person, telehealth) and between-visit interactions (telephone calls, patient portal messages) overall and by key patient characteristics. Design, Setting, and Participants: This retrospective cohort study included adults with diabetes receiving primary care at urban academic (University of California San Francisco [UCSF]) and safety-net (San Francisco Health Network [SFHN]) health care systems. Encounters from April 2019 to March 2021 were analyzed. Exposure: Telehealth implementation over 3 periods: pre-PHE (April 2019 to March 2020), strict shelter-in-place (April to June 2020), and hybrid-PHE (July 2020 to March 2021). Main Outcomes and Measures: The main outcomes were rates of change in monthly mean number of total encounters, visits with any health care team member, visits with billing clinicians, and between-visit interactions. Key patient-level characteristics were age, race and ethnicity, language, and neighborhood socioeconomic status (nSES). Results: Of 15 148 patients (4976 UCSF; 8975 SFHN) included, 2464 (16%) were 75 years or older, 7734 (51%) were female patients, 9823 (65%) self-identified as racially or ethnically minoritized, 6223 (41%) had a non-English language preference, and 4618 (31%) lived in the lowest nSES quintile. After accounting for changes to care delivery through an interrupted time-series analysis, total encounters increased in the hybrid-PHE period (UCSF: 2.3% per patient/mo; 95% CI, 1.6%-2.9% per patient/mo; SFHN: 1.8% per patient/mo, 95% CI, 1.3%-2.2% per patient/mo), associated primarily with growth in between-visit interactions (UCSF: 3.1% per patient/mo, 95% CI, 2.3%-3.8% per patient/mo; SFHN: 2.9% per patient/mo, 95% CI, 2.3%-3.4% per patient/mo). In contrast, rates of visits were stable during the hybrid-PHE period. Although there were fewer differences in visit use by key patient-level characteristics during the hybrid-PHE period, pre-PHE differences in between-visit interactions persisted during the hybrid-PHE period at SFHN. Asian and Chinese-speaking patients at SFHN had fewer monthly mean between-visit interactions compared with White patients (0.46 [95% CI, 0.42-0.50] vs 0.59 [95% CI, 0.53-0.66] between-visit interactions/patient/mo; P < .001) and English-speaking patients (0.52 [95% CI, 0.47-0.58] vs 0.61 [95% CI, 0.56-0.66] between-visit interactions/patient/mo; P = .03). Conclusions and Relevance: In this study, pre-PHE growth in overall patient-clinician encounters persisted after PHE-related telehealth implementation, driven in both periods by between-visit interactions. Differential utilization based on patient characteristics was observed, which may indicate disparities. The implications for health care team workload and patient outcomes are unknown, particularly regarding between-visit interactions. Therefore, to comprehensively understand care utilization for patients with chronic diseases, research should expand beyond billed visits.

Indexed as

COVID-19Diabetes MellitusTelemedicineAdultDelivery of Health CareFemaleHumansMalePrimary Health CareRetrospective Studies

Identifiers

PMID37713198
PMCPMC10504619

What OpenQuestion holds

Textmetadata
LicenceCC BY
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.