Using text analysis to assess the mental health impacts of COVID-19 on rural healthcare providers

COVID-19 exacerbated the lack of health care access in rural communities. We used Linguistic Inquiry and Word Count to analyze the impact of COVID-19 by comparing the use of emotion words in interviews conducted before (n = 12) and after (n = 13) the statewide lockdowns due to the 2020 coronavirus pandemic. In a qualitative text analysis of 25 rural health care providers, there was evidence of reduced positive affect after the onset of COVID-19, but no evidence of increased negative emotions. The rewarding aspects of their work was reflected in the number of positive words they used, and was more evident after COVID than prior to COVID. The providers expressed concern about their patients well-being and access to care during the pandemic. They were also aware of the heightened risk of mental health disorders and their ability to treat them. Policy makers need to address this health care inequity because challenges in rural communities are anticipated to worsen due to climate change, an aging population, the on-going COVID-19 pandemic, and future pandemics.


INTRODUCTION
The Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-COV-2) pandemic spread to rural areas in the United States (U.S.) later than urban areas (Healy, Tavernise, Gebeloff, & Cai, 2020). Although the population density is lower in rural areas, which is likely to reduce transmission rates of the virus (Wang et al., 2021), the rural population has greater challenges in combating coronavirus disease 2019 (COVID-19) . Rural residents are more vulnerable to adverse outcomes due to a limited, under-resourced health care infrastructure (e.g., lack of ventilators, ICU beds, medications, medical expertise), increased travel times to health care facilities able to treat COVID-19, and population characteristics (e.g., older age, obesity, chronic health conditions, low income, lack of trust in health care systems).
Reports of increased stress responses in frontline healthcare providers have been studied in previous disease outbreaks and epidemics (for reviews see Imo, 2017;Magill et al., 2020, Woo et al., 2020. Symptoms of distress in these workers include anxiety, insomnia, burnout and post-traumatic stress disorder (Luo et al., 2021;Sultana et al., 2020). Health care worker stress is an international experience. Many reports have been written on the topic in Chinese health care providers since they were the first to experience the challenge of coping with COVID-19. Despite the hundreds of reports from around the globe on the mental health challenges frontline health care workers were facing and continue to endure, there are few reports on the experiences of rural health care providers working in heavily agricultural areas in the U.S. To understand the lived experiences of this group of frontline medical providers during the pandemic, qualitative data is needed.
Theoretically, the increased use of emotion-related words in participants' interview data could be indicative of the stressors they are experiencing (Pennebaker, 1999), especially during a pandemic. Text analysis technology has improved over the past decade, allowing researchers to employ the use of extensive word databases within text analysis software for rapid assessment of words used in participant responses. Linguistic Inquiry and Word Count (LIWC) was developed by having groups of content experts evaluate the degree to which many English words or word stems fit into categories such as positive emotion words and negative emotion words (Pennebaker, Francis, & Booth, 2001). LIWC software has yielded 80 "psychologically meaningful" categories (Tausczik & Pennebaker, 2010). In a study using LIWC, Pennebaker and Chung (2007) speculated that using too few negative emotion words is representative of inhibiting emotions or alexithymia, leading to poor coping. Conversely, a high use of negative emotion words may be characteristic of those with high levels of negative affect, excessively ruminating on negative events without gaining closure by resolving negative emotions related to traumatic events.
The theory behind coding emotion words is that linguistically assigning a label to an event and the emotions attached to the event coerces a structure on the experience (Pennebaker, 1993;Pennebaker & Francis, 1996). This theory has two models that continue to be studied (Vine, Boyd, & Pennebaker, 2020). First is the differential emotion model which posits that the use of more negative words and the use of fewer positive words predicts better health outcomes (Pennebaker, 1993). Second is the summed emotion model which holds that it is only the use of positive words that predicts better health outcomes (Pennebaker & Francis, 1996). This cognitive construction of the experience can result from writing about the event or during verbal sharing of the event. The act of translating stressful events and accompanying emotions into language results in encoding of understanding, which is stored in a more organized, coherent, and simplified manner in the cognitive architecture.
LIWC has generally been used to analyze domains within written text, such as journal entries or reflective writing. In a 1997 study, Pennebaker and colleagues used LIWC in a longitudinal study of coping with grief to analyze transcribed interview data of 30 individuals who had lost a partner to AIDS and who had been the primary caregiver (Pennebaker, Mayne, & Francis, 1997). LIWC was used to count words hypothesized to be related to grief including insight, cause, death, past tense words, negative emotion and positive emotion. The two brief interviews two weeks apart were able to predict mental health well-being one year later. Specifically, the summed emotion model was correlated with the Centers for Epidemiological Studies Depression measure (CES-D; Radloff, 1977) of cognitive, affective, and vegetative symptoms of depression and the Positive States of Mind Scale, six items that measure pleasure, productivity, focused activity, and connection to others (Horowitz, Adler, & Kegeles, 1988). This suggests utility in using text analysis software to better understand someone's mental state without asking them directly. An indirect measure may have greater reliability because the subconscious choice of words provides insights that the subject might not articulate.
At the time the SARS coronavirus arrived in the U.S., we were conducting a qualitative study of the perceptions of stress in 25 rural community health care providers. The study was initiated prior to the lockdowns resulting from the SARS-COV-2 pandemic. This interruption in data collection by a pandemic provided an opportunity to assess whether changes in mental health status due to the pandemic could be seen in this group of rural providers. Twelve interviews were completed before the pandemic-related restrictions and thirteen interviews occurred after the start of the pandemic (March 11, 2020). This cutoff date is somewhat uncertain in rural areas due to the different pattern of impacts that occurred in rural communities (Wang et al., 2021), however, since the counties were in a single U.S. state, state mandates applied across all counties. Rural areas were affected later but with similar incidence rates as urban areas as the pandemic progressed in the U.S. (Wang et al., 2021). However, when the rural areas of the U.S. began feeling the impacts of COVID-19, the effects were devastating to an already stretched health care system. In this report, we investigated the self-reported effects of COVID-19 in rural regions and used a novel approach to dig deeper into the data to analyze the emotional language that might be informative about mental health effects in rural providers.
We used LIWC to assess differences in the use of positive and negative emotion words and the psychological drive of rewards to detect differences between pre-COVID-19 interviews and post-COVID-19 interviews. We hypothesized that: (1) post-COVID-19 interviews would reveal a higher percentage of negative words than pre-COVID-19 interviews; (2) the use of positive words would be higher in pre-COVID-19 interviews compared to post-COVID-19 interviews; and (3) the pressures of the work would lead to changes in the rewards providers might experience in serving their rural patients, although we were not sure what direction that change might show. In addition, we hypothesized that positive emotion words would be positively associated with reward words. Rural health care providers feel strongly about the importance of their work and helping their rural community members. In a pandemic, they may feel more strongly about the work they do, or they may feel despondent over conditions they face. Resiliency has been shown to be an important factor in health care provider responses to pandemics (Blanc et al., 2021;Croghan, et al., 2021;Huffman et al., 2020) and rural health care workers are likely to have more of this trait than their urban counterparts (Walters et al., 2015). We were unsure whether the differential emotion model or the summed emotion model would hold, but wanted to assess evidence for each model. Lastly, we wanted to examine the words of providers impacted by COVID-19 directly to assess their perceptions of their situation.

Participants and measures
Rural health care clinics were identified from medical care directories and on-line resources. Rural was defined using The Centers for Medicare and Medicaid Services as those that are located in areas designated by the Bureau of the Census as rural and by the Secretary of the Department of Health and Human Services (DHHS) or the State as medically underserved. In a rural health care clinic professionals other than medical doctors can provide services including nurse practitioners, physician's assistants, certified nurse midwifes, clinical psychologists or clinical social workers. Clinics were contacted by phone and e-mail, and interviews were scheduled. Providers expressed a willingness to be interviewed, but finding the available time caused delay between initial contact and the scheduled interview due to demands on provider time. The co-author, being a registered nurse with experience working in rural communities, facilitated a positive response to the interview request.
Sixteen medical doctors, four physician assistants and five nurse practitioners residing in six large rural counties in a western state consented to be interviewed for our study. Contact was made by telephone to rural clinics and appointments for the interviews were scheduled with providers. Interviews were conducted both in person and by telephone until the pandemic began and then strictly by telephone after. Interviews lasting from 30 to 75 minutes were tape recorded and transcribed. The data were cleaned before analysis to correct misspellings and add missing punctuation, as needed for the qualitative software. The study was approved by the Colorado State University Human Subjects Research Institutional Review Board.
The semi-structured interview questions focused on the challenges in rural clinics. The 10 questions were identical for interviews occurring before and after the start of the pandemic period. No questions were asked of participants related to the pandemic, but all providers talked about the SARS-CoV-2 pandemic unprompted. The distribution of provider type was the same before and after the start of the lockdown. Each question contained two prompts and the prompts were used in each interview to elaborate on the question being asked. We were interested in whether we could identify cultural differences in healthcare providers and rural residents and how well rural healthcare providers understood rural culture. Topics covered perceptions of mental health; how providers recognize poor mental health in their patients; what people share with providers about their physical and mental health conditions; where people go in the community for assistance with depression and behavioral health challenges; level of support for depression, anxiety, and suicide ideation or attempt, and how it is treated in the community; ideas for improving access to care; and challenges to improving health care and helping those who need it. Due to the ongoing rural health care professional shortages, lack of resources, and lack of support, it is likely that the situation was likely to worsen after COVID-19.

Statistical analysis
We used the LIWC text analysis program to examine the set of psychological variables using the word count feature to classify certain types of words based on extensive dictionary databases (Pennebaker, Booth, Boyd, & Francis, 2015;Pennebaker, Boyd, Jordan, & Blackburn, 2015). Word frequencies are based on a set of psychometrically validated words that correlate with the big five personality traits. Our focus was on the use of general affect words, which included positive emotion words and negative emotion words. The negative emotion words included subcategories for anxiety, anger and sadness. The reward category was also of interest and is classified as a psychological drive and defined as references to incentives, positive goals and positive approaches. The word count measures are calculated as the amount of content that reflects a certain word category. The percentages are generally low because of the nature of interview data covering a broad range of topics, but the measure incorporates the total word counts in each interview for the denominators. The measure is interpreted as the contribution of a certain word category to the total content.
After running the software program on a set of interview responses for pre-and post-COVID interviews separately, codes were applied to each of the two groups to use for classification based on the March 11 cut-off date. The variables were tested for normality and with the exception of the anger variable, all were normallydistributed by the Wilks-Shapiro test at p < 0.05. We calculated t-tests on the normally-distributed measures and used a non-parametric Kruskal-Wallis test on the anger variable. We did not expect to find significant differences with groups of size 12 and 13, so we also calculated a Cohen's D statistic for effect size and its non-parametric counterpart for the anger effect size. 1 The two sample tests were conduced in SAS 9.4 (The SAS Institute, Cary, NC) and considered significant at alpha = 0.05.  Observing that positive emotion word frequency and reward word frequency were borderline significant with medium effect sizes, we pursued an analysis of covariance (ANCOVA) model. The dependent variable was percentage of reward-related words. The primary independent variable of interest was the pre-and post-COVID indicator. We used the proportion of positive emotion words as a covariate and tested for an interaction between COVID group and positive emotion words. We calculated the proportion of variance explained by COVID and positive emotion words on reward. The data were plotted in R.
After examining the results of the text analysis, we sought to demonstrate some of the experiences health care providers were reporting and selected relevant themes based on the coding of the qualitative data related to COVID-19. In every interview post-COVID, health care providers talked about their challenges, some more than others.

RESULTS
The interviews conducted prior to COVID-19 had a mean total number of words per interview of 622.5, a standard deviation of 311.6 and a range of 271-912. After COVID-19, the mean total number of words was 566.0, standard deviation of 222.3 and a range of 228-1119. These were not statistically significantly different (Table 1). 2 Comparing the emotion variables in the pre-COVID group to the post-COVID group showed that overall affect words was reduced as an overall proportion of the content (Table 1). After COVID, positive emotion words decreased, as did negative emotion words. Within the negative emotion domain, anxiety content remained unchanged, anger words increased slightly and sadness words decreased. COVID may have influenced positive affect to a greater extent than the negative emotion domain. Interestingly, reward words increased. Although none of these differences reached statistical significance at alpha = 0.05, positive emotion came close with a pvalue of 0.06. The reward words were significant at an alpha = 0.10. Despite the lack of statistical significance, the effect sizes showed moderate effects for overall affect words, positive emotion words and reward-related words. The mean word counts increased slightly post-COVID but the effect size was small. The overall ANCOVA model was statistically significant (F [2, 22] = 13.8, p = 0.0001). We report the sequential sums of squares because we were most interested in accounting for variation due to the presence of COVID. COVID was significantly associated with the rewards outcome (F = 5.71, p = 0.026) and the proportion of positive emotion words showed a greater positive association with reward words (F = 21.9, p = 0.0001). The R 2 for the model was 0.56. The proportion of variance explained by COVID on reward was 11.5% and by positive emotion words was 44.2%. The least squares adjusted means for the pre-COVID group was 0.73 and for the post-COVID group 1.51. This means that adjusting the means of the reward variable for the proportion of positive affect words resulted in a larger difference between the two groups than was observed in the twosample t-test. The interaction between COVID group and positive emotion words was not significant, which can be seen in the parallel lines shown in Figure 1. Table 2 shows statements made by rural healthcare providers who were struggling with COVID in their clinics. The major issues for these providers was the lack of access to care because clinics were overwhelmed with COVID-19 patients and people were afraid to seek care for fear of being infected. There were also concerns related to an increase in sadness, depression and suicide in Figure 1. Associations between positive emotion words and reward-related words in 12 pre-COVID rural health care providers compared to 13 rural post-COVID health care providers. the community and that those experiencing distress were unable to access treatment. Providers seem to think that because of COVID-19, more people were talking about mental health and about their sadness, depression and feelings of loss. Providers also expressed the possibility that COVID will reduce the stigma around mental and behavioral health issues in rural communities. Reading the statements suggests that rural providers know how important they are in their communities. Many providers emphasized that they were the only health care clinic within an hour or more of driving time to the next closest clinic.

DISCUSSION
Our data showed a better fit to the summed emotion model with a decreased use of positive words after COVID compared to before COVID. In contrast to our first hypothesis, the use of negative emotion words also decreased, but with a small effect. Our finding is similar to the study of grief in 30 men who had lost their partners to AIDS (Pennebaker, Mayne, & Francis, 1997). In that study positive emotion words showed changes in counts, but negative emotion words did not. In this sample of rural health care providers, the use of negative words decreased and there was no evidence of an increase in anxiety, anger or sadness. Although there was a moderate change in affect overall, it was primarily due to the decrease in positive affect indicators. With only 12 pre-COVID interviews and 13 post-COVID interviews we were able to test specific hypotheses using the LIWC software, which allowed for an objective assessment across the two groups based on a well-designed set of word databases. Qualitative data was coded into quantitative data using this approach. Although the results indicate trends in the impacts of the pandemic, it is not clear whether this sample of 25 providers is representative of rural health care providers in other rural areas. 3 The most unexpected finding was the increased use of reward-related words after the pandemic began. This result was not expected to show a difference by COVID group, and not expected to be strongly related to positive emotion words. We examined rewards as a word category because the interview data showed that rural health care providers were committed to making a difference in their communities and we were interested in what effect COVID might have on that psychological drive. Given the extreme pressure rural providers experience, adding an even greater burden to their workload Table 2. Descriptive statistics for percentage of words in interview data indicating positive and negative emotion and reward in 25 rural health care providers prior to and following the COVID-19 pandemic, 2020.

Theme
Interviewee statement Access to care during COVID What we see now is affected dramatically by COVID. Social problems now present as crisis when we finally are able to see them. Same is true for chronic conditions which are now acute due to a fear of access to care. Our help is even more limited due to remote mandates and many of our patients have no internet access. Right now it is the pandemic! We are in a serious crisis mode. I do not know where people can get help if they cannot even get it from us. Impact of COVID on health care The severity of illness brought on by COVID has since overwhelmed surrounding hospitals with very sick patients. We have even fewer options during this pandemic, turning what was already a crisis into a catastrophe. That is the question: how DO I help these people? COVID impacts on depression A number of patients come in with depressive symptoms post covid infection. I feel that in addition to aspects that may contribute to depression, the pandemic and those directly impacted by a covid infection or death are doubly at risk. COVID and suicide risk I do not know if there is enough support for suicidal people in the community. Certainly, when they arrive to the clinic they are supported. But in the community I do not know if they are supported enough. I do not know where this is best handled. I do know it is not handled well presently because of the pandemic and people are terribly isolated. It is a crisis for the mentally ill. Very common, completed suicide is more common in the COVID pandemic.
COVID has reduced stigma I find that people are talking about it [sadness] more with COVID because it is at a crisis level and there are more negative outcomes. Before COVID, I think it was less common [talking about sadness] and I think less people admitted it. I think the COVID crisis has brought things to a boiling point. The isolation and loss has been overwhelming for many. These days it is not only normal [to talk about sadness], I think it is much more common. One good thing about the COVID crisis is that it has us talking more about mental health because of the serious impact it has had on access to care. was a factor that was thought to reduce work satisfaction and the sense of reward. The R 2 value of 0.56 suggests that the number of positive emotion words is associated with feelings of reward, and COVID contributed to the association.
The plot of reward words versus positive emotion words by COVID group showed some intriguing results ( Figure 1). First, the range of x-values are truncated and shifted towards the higher percentage of positive words in the pre-COVID group compared to the post-COVID group in relation to rewards. Second, the post-COVID line is relatively parallel to the pre-COVID line, but is shifted upwards, and appears steeper than the pre-COVID line. There are only two pre-COVID points in the post-COVID 95% confidence interval band, but there are four post-COVID points in the pre-COVID band; only six of 25 points cannot be classified perfectly by COVID group in the model. The information provided in this graph appears to be that the use of positive emotion words are more revealing about work rewards during a crisis such as COVID than in less turbulent times. The association between the domain of positive affect and the cognitive domain of reward are stronger when the health care workers were being tested by a pandemic. They may have felt that what they did was more important in this situation. It is possible that these health care providers went into the pandemic with higher levels of resiliency due to having endured years of feeling underserved and under-resourced. The study of rural practitioners sheds light on this possibility by identifying factors that practitioners used to cope with the lack of resources including optimism, self-reflection and metacognition (Walters et al., 2015). Future work should explore this possibility and assess the ways in which they may be different from that of their more urban counterparts in ways that influence mental health.
A review of the interview statements related to COVID is a display of the frustration and tension the providers were feeling. Providers were pleading for help for their communities. They wanted to see policy changes to increase funding for health care. They worried about their patients being able to access services. They understood that they were not equipped to handle the unmet need and did not see a way to remedy the problem. There is also the observation that they are vigilant about the effects of COVID in the community and observing that there might be a positive side if rural residents can start talking about their feelings rather than being shamed by those feelings.
The positive affect words are based on positive affect in psychiatry. A reduction in positive affect is associated with depression, as measured by the CES-D scale. It includes happiness, optimism, hope, and extraversion. Studies in the field of positive psychology have shown that positive mood or affect directly reduces stress levels and can interact with resilience to improve mental health (Fredrickson, 2001). Improving positive affect involves developing personal physical, psychological, and social resources (Garland et al., 2020). There is no doubt that COVD-19 reduced the availability of these resources during a time of exacerbated stress.
This study has implications for the mental health of rural providers that should be explored in a larger study. The sense that a provider is having a positive impact in the community was demonstrated by the use of positive language. As it is unlikely that greater resources will be directed to rural communities and as rural hospitals continue to see closures, it is important to retention that they have opportunities to make an impact in their communities. This is one of the lessons learned from the pandemic in general, but with limits. These same providers are at greater risk of occupational burnout, so identifying ways to increase psychological occupational rewards is critical to maintaining rural health care access.
This study has limitations. Our sample size of 25 was small, but because of the type of data analyzed we were able to do statistical tests and modeling of this qualitative data. We did not make adjustment for multiple comparisons because results were not statistically significant at an alpha of 0.05 in the t-tests or Kruskal-Wallis test. It was surprising that nearly all of the variables were normally distributed given the sample size. There are also limitations with the software used. LIWC is one of the best text analysis programs available, but as the software developer states, word counts alone do not capture the nuances contained in sentences and sentence structure. Using single words that reflect an emotion is likely missing other important aspects in the texts. However, this would hold for both pre-and post-COVID groups equally and is not likely to bias the results. Additionally, the interviews varied in length depending on how busy the provider was, which might create bias in the samples. Although it might be expected that post-COVID interviews would be shorter due to time pressures, that was not the case. It is probably true that some providers wanted to talk about what they were seeing and said more because of COVID.

CONCLUSION
Rural health care providers showed evidence of reduced positive affect after the onset of COVID-19, but did not show evidence of increased negative emotions. The rewarding aspects of their work was reflected in the number of positive words they used, and was more evident after COVID than prior to COVID. The providers expressed concern about their patients well-being and access to care during the pandemic. They were also aware of the heightened risk of mental health disorders and their ability to treat them. Policy makers need to address this health care inequity because challenges in rural communities are anticipated to worsen due to climate change, an aging population, the on-going COVID-19 pandemic, and future pandemics.
Cheryl Beseler holds a PhD in epidemiology, and master's degrees in biochemistry and statistics. She has been working on mental health and occupational injury in the agricultural community for the past 20 years. She is currently a co-investigator with the Central States Center for Agricultural Safety and Health at the University of Nebraska Medical Center.

Stacia
McNeely is an RN with 20 years of experience in public health. She currently has an MSN, Ed. and previously held an Affiliate Faculty position at Regis University for seven years. She is passionate about helping others and finding solutions to the most challenging problems facing underserved communities. Currently, she resides in a rural community in Colorado, where she continues her work in public health.