Estimating Telecommuting Rates in the US Using Twitter Sentiment Analysis

doi:10.21203/rs.3.rs-3879832/v1

Download PDF

Research Article

Estimating Telecommuting Rates in the US Using Twitter Sentiment Analysis

https://doi.org/10.21203/rs.3.rs-3879832/v1

This work is licensed under a CC BY 4.0 License

Journal Publication

published 29 Oct, 2024

Read the published version in Data Science for Transportation →

You are reading this latest preprint version

The COVID-19 pandemic had a significant impact on virtually every human activity. Millions of workers around the globe from eligible professions stayed at home working as part of the measures taken to contain the virus’ spread. The change in transportation demand associated to this phenomenon poses a challenge for cities, especially regarding public transportation, where the decrease in demand arose critical questions on how to assess decreased ridership and potential rebound effects. With this in mind, we ask: can we obtain real-time demand change estimates using social media data? Hence, the aim of this work is to take social media unstructured information and transform it into structured insights that can offer almost real-time estimates on demand trends associated with telecommuting. To achieve this, we obtained around 50,000 geo-tagged tweets relevant to telecommuting in the US. With that, we leveraged transformers Machine Learning methods to fine-tune a language model capable of automatically assigning a sentiment to tweets on this topic. We used the time evolution of the obtained sentiments as covariates in time series forecasting models to estimate telecommuting rates at both the national and state levels, observing a drastic improvement over the estimates without such covariates. Our major finding indicates that it is possible to structure social media data in order to use it to obtain demand change estimates, and that the accuracy of such estimates is going to depend heavily on how much people discuss the topic in question in a determined geography. This finding is in line with others that have found alternative ways of obtaining insights on transportation data, and hence, is a relevant contribution towards real-time data-driven approaches for transportation demand assessment.

Telecommuting

Work from home

COVID-19

Sentiment Analysis

Time Series Analysis

On March 11, 2020, the World Health Organization (WHO) declared the beginning of a global COVID-19 pandemic after 118,000 cases in 114 countries were reported, with a death toll of 4,291 (Cucinotta and Vanelli 2020). This event marked the beginning of a series of worldwide attempts to contain the spread of the virus, which in many places resulted in strict lockdowns and mobility restrictions. One of the most implemented measures in both private and public sectors was a rapid switch to working from home. This means millions of people around the globe who were used to daily commutes to work and back home were now facing a different dynamic in which their home also became their office space. As of this writing, after more than three years since the beginning of the pandemic, facing several contagion waves, virus strains, and the availability of COVID-19 vaccines, it is still unclear what the future of the traditional workspace is going to be. Several companies are still working under a working-from-home scheme, while some others have been attempting a gradual return to their office spaces. At the company level, management teams have been trying to gather data regarding the plans and preferences of their workers, as well as developing metrics to estimate the impact telecommuting has had on productivity. While this certainly gives the companies reliable data for decision-making at the management level, the overall picture of public policy is more difficult to grasp.

In an effort to understand the nationwide impact of the pandemic on daily activities, The COVID Future Survey (CFS) was developed as a panel survey that collected information on different stages of the pandemic, aiming to gain a more profound knowledge of travel attitudes and mode shift choices (Chauhan, Bhagat-Conway, et al. 2021). The first version of the survey took place between April and June 2020, and a second version was released from July to October of the same year. The collected data are publicly available with the goal of helping academics and policymakers understand the current and future impacts of the pandemic.

Using CFS data, Mohammadi et al. (2022) conducted a study on telecommuting that evaluated the risk perception associated with exposure to COVID-19 and the perceived productivity at home. The findings suggest that the population segment more likely to telecommute in the post-pandemic era are the millennials, people with long-distance commutes and high income, and the highly educated. The overall picture suggests that telecommuting will increase at different rates among socioeconomic groups.

CFS data offer a valuable resource because they contain demographic information and people’s perception in two different moments of the pandemic. However, tracking people's perceptions continuously during different contagion waves and measures being imposed and lifted is impossible. One way to track people’s perception of key issues like telecommuting is through social media. Twitter, specifically, is one of the most popular social media platforms, with around 68 million active users per month in the US. It also has the presence of politicians, organizations, and influential people not only in the entertainment space but also academics and policy. For this reason, we consider Twitter a potential data source for tracking people’s perceptions of telecommuting and productivity.

Even though the current Twitter platform supports different types of media content such as video, images, recordings, and gifs, the primary information source is still encoded into text. Therefore, text mining techniques are applicable in the data extraction from Twitter. Attempting to understand using opinions and feelings from the written text is usually known as sentiment analysis. Sentiment analysis for Twitter data has been around since 2010 (Celikyilmaz, Hakkani-Tur, and Feng 2010) for a wide range of opinion analysis, from making predictions (Bollen, Mao, and Zeng 2011) to detecting irony in tweets and estimating user emotions (Giachanou and Crestani 2017). These algorithms rely mainly on Machine Learning (ML) classifiers (Severyn and Moschitti 2015), in which a set of labeled tweets is passed to a classifier which performs a feature extraction on the text and then passes the feature vector to a classifier to train the algorithm. Several types of classifiers have been used to for this task, including Support Vector Machines, K-Nearest Neighbors, and Deep Learning.

In this study, we propose an implementation of word embeddings using Transformers models and sentiment classification using Gated Recurrent Unit (GRU) to determine the sentiment of tweets from US users regarding their perception of telecommuting during the pandemic. We use this model to obtain insights into the perception of telecommuting and its temporal and geographical variability in the US. Additionally, we use Census data to study the actual prevalence of telecommuting in the country and develop a forecasting model able to estimate changes in this prevalence employing past values of the Census data and future values of Twitter data.

In the next section, we review the literature on three different topics: telecommuting, sentiment analysis of Twitter data, and forecasting models using twitter data as input. After, we describe the Census and Twitter datasets used in this study. Next, we go over the development of the sentiment analysis model and its results. After this, we show how the sentiment data can be leveraged to improve the forecasting models instead of models relying on just past observations. Finally, we discuss the results, limitations, and possible solutions.

2.1. Telecommuting

Telecommuting, also known as remote work or telework, is a work arrangement in which employees can work from a location other than the traditional office environment, usually their home or another remote location using digital tools to communicate and collaborate with colleagues and complete their work tasks (Nilles 1988). Telecommuting has become increasingly popular in recent years, driven by technological advancements, changing work cultures, and the need for flexibility and work-life balance. In particular, the COVID-19 pandemic caused an overnight change in work culture by making telecommuting a necessity for social distancing, and many acknowledge that the pandemic accelerated the shift toward telecommuting for the long term (Guyot and Sawhill 2020; Parker, Horowitz, and Minkin 2022). However, the sudden shift from in-person collaboration to remote teamwork caused many employers and employees to need help with telecommuting adoption. As a result, the literature on telecommuting has only recently started to grow significantly.

To gain a deeper understanding of how to improve workplace culture when many employees telecommute, it is necessary to examine both employers’ and employees’ attitudes toward telecommuting. However, due to the difficulty of collecting data from managers, the literature has focused on studying whether employees perceive telecommuting as a desirable option and what factors influence this attitude. Employees’ preferences for telecommuting have been shown to be influenced by constraints, opportunities, and socio-demographic factors. Previous studies referred constraints to factors that might limit an employee's ability to work remotely, such as the suitability of their home environment or the nature of their job tasks. On the other hand, opportunities refer to factors that make telecommuting a desirable option, such as saving commute time that helps maintain better work-life balance (Balbontin, Hensher, and Beck 2022; Mohammadi et al. 2022; Nguyen 2021; Nguyen and Armoogum 2021; Salon et al. 2022)

Nguyen and Armoogum (2021) conducted a study during the pandemic to investigate the effects of constraints and opportunities on attitudes and perceptions towards telecommuting and how these differ between male and female employees. The research revealed that females generally have a more favorable attitude towards telecommuting than males, but this is negatively affected by household chores such as childcare. In contrast, male employees' preferences are influenced by prior telecommuting experience, perceived productivity at home, and income. In another study using the same dataset, Nguyen (2021) investigated attitudes toward full-time and part-time telecommuting. This study found that demographic factors such as age, gender, education, and income, as well as prior telecommuting experience and home environment constraints, influence telecommuting preferences. In addition, job-related factors such as employer policy, job type, data accessibility, commute distance, and other home environment attributes such as employee productivity at home, were identified as significant factors.

The successful implementation of telecommuting depends on various factors, such as the differences in profiles between telecommuters and non-telecommuters, the importance of employer telecommuting policies, employee productivity, remote collaboration, and work-life balance challenges. This concept has been emphasized in the literature on telecommuting since the COVID-19 pandemic (Balbontin, Hensher, and Beck 2022; Beck and Hensher 2022; Mohammadi et al. 2022; Nguyen 2021; Nguyen and Armoogum 2021; Tahlyan et al. 2022). For example, Beck and Hensher (2022) conducted a study in Australia to investigate the positive and negative impacts of working from home on employees, employers, and society. The study revealed that the rapid adoption of telecommuting caused many employees to face challenges related to remote collaboration and technology, such as reduced collaboration and innovation, poor internet connectivity, inadequate home office setups, and difficulties accessing company systems remotely. They also highlighted the difficulty of maintaining a boundary between work and life and the feeling of social isolation as consequences of full-time telecommuting.

Tahlyan et al. (2022) conducted a study on the factors that impact employee satisfaction with telecommuting, using a dataset that included 318 employees in the US. The study revealed that certain groups, including those with children at home, disabilities, and lower income and education levels, had less success with telecommuting. Conversely, the authors found that satisfaction with telecommuting improved when employees had job autonomy, such as flexibility in their work schedule, workplace, and how they performed their tasks, task variety, and connectivity with co-workers.

Some authors of this article were part of a team that conducted a 3-phase nationwide survey in the US from early 2020 to late 2021 to explore modifications in household activities, such as work behavior, due to the COVID-19 outbreak (Capasso Da Silva et al. 2021; Chauhan et al. 2022; Chauhan, Bhagat-Conway, et al. 2021; Chauhan, Capasso Da Silva, et al. 2021; Javadinasr et al. 2022; Mirtich et al. 2021; Mohammadi et al. 2022; Nafakh et al. 2022a; 2022b; Salon et al. 2022; 2021). Based on the first wave of the survey conducted throughout 2020, Salon et al. (2022) examined what factors influence the ability to telecommute and the frequency of it. They found that higher educational attainment and income, together with specific job categories, largely determine whether workers have the option to telecommute.

Mohammadi et al. (2022) utilized data from the first and second waves of the same survey to investigate employees' preferences for telecommuting, considering the impact of unobserved behaviors such as productivity at home and COVID-19 risk perception. The study found that risk perception and productivity at home positively influence preferences for telecommuting. Factors such as home environment attributes (e.g., childcare, distractions), job-related factors (e.g., job type, lack of required technology), and work-life balance opportunities (e.g., saving commute time) also impacted job productivity. The study also found that preferences for telecommuting are affected by education, age, income, commute trip features, and prior telecommuting experience.

This study contributes to the existing literature in four ways. First, comprehensively examines of US employees' thoughts and opinions on telecommuting by analyzing their social media posts. This approach results in a larger sample size due to the high activity of people on social media, leading to a more accurate representation of employees. Second, the study explores sentiments toward telecommuting from both temporal and spatial dimensions. Third, the study analyzes telecommuting data collected in all three waves of the COVID Future, a recent dataset on activity-travel behavior collected across the US. Fourth, the study compares the sentiments toward telecommuting shared on social media with those obtained through surveys to determine the similarities and differences between the information collected by social media and surveys.

2.2. Sentiment Analysis on Twitter data

Natural Language Processing (NLP) is a field of Machine Learning (ML) and linguistics that aims to model language using computational methods. NLP offers much promise given its ability to process large amounts of text automatically in a short time compared to the time it would take for a human to complete this sort of task. One of the most prominent applications of NLP is sentiment analysis, which manages opinions and subjective text mainly for classification purposes. This can be used to process a large number of user reviews, public opinions, and social media posts, which later help assess product performance, elections, and major public events (Tul et al. 2017). The main benefit of sentiment analysis is that it can offer an interface between large amounts of unstructured text data and structured data mined from these text sources. For this reason, sentiment analysis methods have gained popularity among researchers who can now incorporate insights from different text sources in qualitative and quantitative research. One of the most prominent sources of text data is Twitter. Twitter posts contain real-time access to public plans and perceptions on all sorts of topics, not only from individuals but also from corporations, government organizations and figures, NGOs, and all kinds of public figures. This has made Twitter a perfect candidate to mine data from the text for various applications like public policy evaluation, stock prediction, and advertisement.

The first attempts to extract sentiment from tweets did so by using Naïve Bayes (NB), Maximum Entropy (ME), and Support Vector Machines (SVM) to classify tweets as either positive or negative and were able to reach 83.0% accuracy (Go, Bhayani, and Huang 2019). Pak and Paroubek (2010) extracted sentiment from tweets using NB and ME algorithms, with the addition of a neutral class that boosted the problem from binary to multi-class classification. Other approaches tackled the problem by considering the relevance of the tweets. For example, Barbosa and Feng (2010) introduced a first layer with a model of binary classification to determine whether the tweet itself reflected an opinion that could be labeled as positive or negative. Then, a second binary classification model was used to determine its sentiment. The study achieved a maximum accuracy of 81.9% with SVM. A substantially different approach was taken by Davidov et al. (2010), which included emoticons, hashtags, and punctuation as essential features in both the data collection process and the binary sentiment classification. This approach achieved an F1 score of 86.0%.

To date, SVM played a significant role in developing the first SA approaches for Twitter data. The approach by Bakliwal et al. (2012) trained an SVM classifier on 11 Twitter-specific features and features that arose from NLP pre-processing techniques (e.g., stemming and spelling correction). Other authors were also able to obtain good results using combining SVM with a proper feature selection (CBalabantaray, mohd, and Sharma 2012; Gokulakrishnan et al. 2012; Kiritchenko, Zhu, and Mohammad 2014). On the importance of the feature definition and selection, several authors contributed with their research on the impact of such parameters (Agarwal and Sabharwal 2012; Aisopos, Papadakis, and Varvarigou 2011; Aston, Liddle, and Hu 2014; Hamdan, Béchet, and Bellot 2013; Kouloumpis, Wilson, and Moore 2011; Saif, He, and Alani 2012).

The second half of the 2010s decade saw a rapid increase in the capabilities and applications of neural networks and the boom of deep learning. With this, the ability to extract abstract features using neural networks enabled a whole new level of approaches leveraging this technology. With this, Convolutional Neural Networks (CNN) were introduced for the sentiment analysis on tweets (Severyn and Moschitti 2015). The study introduced an architecture containing a single convolutional layer to extract features from individual sentences inside the tweets and individually classify them. Another approach using complex architectures of neural networks was the one developed by Tang et al. (2014) in which three neural networks were employed to learn word embeddings specific for sentiment analysis and further used as features.

An adaptive Recursive Neural Network (RNN) was proposed for Twitter SA by Dong et al. (2014) in which a dependency tree was used to track and propagate sentiment across words and their corresponding syntactically related targets, in addition to introducing their own manually labeled dataset for this task. This dataset was later used by Vo and Zhang (2015) whose proposed approach consisted of modeling separately the context before and after a specific target in each tweet. Additionally, rich features were extracted automatically.

Different combinations and architectures of these deep learning methods were also implemented to extract sentiment from tweets. By 2017, a seminal work on Transformers was published by Vaswani et al. (2017). Transformers models arose mainly to overcome the limitations of previous language models to capture context variations in word representations. For instance, even though the word “left” can have completely different meanings depending on the context, previous models only offered single-word embedding. Transformers consist of two main modules: encoder and decoder. These leverage the concept of the attention mechanism, which assigns a score to every word in a sentence to determine their relevance in building representations of words in the sentence using their context. The encoder module generates embeddings for each word which are then improved with the aggregation of information from the context words. Similarly, the decoder generates sequential outputs by attending to previously generated outputs and the encoded embeddings.

One of the most important applications of Twitter text mining and sentiment analysis focuses on prediction. For instance, Yao and Qian (2021) analyzed people's work and rest patterns to predict next-day morning traffic congestion, showing promising signs of improved prediction accuracy compared to traditional methods. Another study employs a deep Bi-directional Long Short-Term Memory (LSTM) stacked autoencoder model using Twitter, traffic, and weather data for short-term urban traffic prediction, showing improved accuracy over classical and machine learning models (Essien et al. 2021). Ratnani and Kumar (2021) investigated the correlation between Twitter data related to sporting events to predict passenger flow, aiming to improve metro transit system management and traffic control. Tweetluenza, a linear regression model utilizing cross-lingual Twitter data, effectively predicts Influenza prevalence and hospital visits in the UAE with improved accuracy when combining English and Arabic tweets (Alkouz, Aghbari, and Abawajy 2019). Valencia et al. (2019) explored the use of neural networks (NN), SVM, and random forest (RF) with Twitter and market data to predict cryptocurrency market movements, finding that neural networks outperform other models. Another study develops an algorithm for analyzing flood-related disaster tweets, categorizing them by priority and predicting user locations using the Markov model, achieving 81% classification accuracy and 87% location prediction accuracy (Singh et al. 2019). Lastly, using Twitter data and sentiment analysis, Yavari et al., (2022) predicted election results based on positive-to-negative message ratios, achieving high accuracy in predicting the 2020 US presidential election. More recently, Sun et al. (2023) showed Twitter sentiment can be used to explain variations in mobility and activity participation by using Kyoto, Japan as case study.

3.1. Data

3.1.1. Current Population Survey (CPS)

The Current Population Survey (CPS) is administered monthly (on the week of the 19th ) by the Census Bureau on a sample of 60,000 households and asks respondents about activities during the prior week. The survey includes all 50 US states and the District of Columbia. The labor force data included in the survey applies only to individuals aged 16 and over. In addition, people in institutions such as prisons and nursing homes are not eligible for the survey. Starting in May 2020, a series of questions related to the Covid-19 pandemic and its effect on individuals’ labor force were added. For this study, we consider COVID-19 question #1 (PTCOVID1), which asked: At any time in the last four weeks, did (you/name) telework or work at home for pay because of the coronavirus pandemic? The number of respondents who answered yes was tracked from May 2020 to May 2022 at national and state levels using the state representative weights provided by CPS.

3.1.2. Twitter

To retrieve tweets from Twitter, we used the Academic Researcher access on the Twitter developer API (Twitter 2012). Given that we must only use tweets that are geo-tagged, we used the GTdownloader library (Acosta-Sequeda and Derrible 2023) and its TweetDownloader class to write the query with the required tweet download criteria shown in Table 1.

Table 1

Download parameters used to retrieve the data according to GTdownloader options
Parameter	Value
query	(telecommuting) OR (telework) OR (remote work) OR (remote job) OR (working remotely) OR (teleworking) OR (telecommute) OR (work from home) OR (home office) OR (work at home) OR (working from home) OR (working at home)
language	English
start_date	October 30th 2019 at 00:00:00
end_date	October 29th 2022 at 23:56:59
place	US
Include re-tweets	NO

Table 1 shows that the text query included the most common terms used to refer to telecommuting in English. A two-year time frame was used to incorporate a period before the pandemic and all different stages of the pandemic and post-pandemic until the download query was run. The tweets were geographically constrained to the US, and we added a conditional to include only original tweets and not tweets that were just the re-posting of another tweet.

The number of tweets downloaded fulfilling our conditions is 49,997. The tweets’ temporal distribution is displayed in Fig. 1. There is an evident spike when telecommuting was a hot topic in social media corresponding to the period when the WHO officially declared the pandemic in March 2020. After that, we can observe a rapid decrease between June and July of the same year, although not to pre-pandemic levels.

To complement the temporal distribution, we obtained the spatial distribution of the tweets at the county level in the continental US (see Fig. 2). The most remarkable feature of this map is its closeness to the US density population map, which suggests the number of tweets in each county is positively correlated to the population in that county.

For all subsequent analysis, only the tweets in the continental US were considered.

3.2. Model

To implement the Transformers model architecture, we took advantage of a Bidirectional Encoder Representations from Transformers (BERT) pre-trained model (Devlin et al. 2019). Pre-training is a common and effective practice in natural language (and Machine Learning in general) tasks in which models trained for another task can be employed in a new one by means of the extraction of one or more of their modules or the refined training of the model with data for the new task (Ruder et al. 2019).

Language models usually employ an embedding layer to generate the appropriate vector representations for each word. However, in this case, that is going to be replaced by the pre-trained BERT model embeddings that are going to be fed into a Gated Recurrent Unit (GRU) model (Cho et al. 2014), which is a type of RNN with update and reset gates in charge of managing the information and decide what to be considered for the output, with the important feature that it is capable of retaining long-term information. The main task of the GRU is to predict the sentiment of each tweet. The input data consists of 11,825 manually labeled tweets classified into positive and negative. Then, 80% of the tweets were used for the model training, 10% for validation, and the remaining 10% for testing the final selected model. The selected model is the one with the highest validation accuracy. Here it is worth mentioning that a single model was trained using data for all the US, but the results are examined both at the national and the state level.

4.1. Model Performance

The overall training accuracy of the model was 87.05%, while the test accuracy was 80.11%, with 3,213 positive tweets and 1,472 negative in the train dataset, and 372 positive and 151 negative in the test dataset. Given the importance of both the temporal and geographical attributes of each tweet, we also obtained the evolution of the accuracy in time and the spatial distribution of the accuracy at the state level. Figure 3 shows that the moving train accuracy oscillates around the overall train accuracy with no apparent disruption in time. However, the moving test accuracy presents a steep decrease between August and September 2020. This is also the period when the peak in Fig. 2 ended, marking a steady number of daily tweets on the topic. This decrease in accuracy could be due to a limitation of the model at capturing the change in the amount of data, and a possible sentiment shift during that time. Figure 4 shows the geographical distribution of the train and test accuracies at the state level. We can see in these maps that the training accuracy is satisfying among all the states, although it falls below 50% in some states. Specifically, the states with poor train accuracy are states with low data availability, as shown in Fig. 2, so the lack of sufficient training observations is a potential explanation of the problematic generalization capabilities of the model in such states.

4.2. Sentiment Analysis

After obtaining the sentiment of all the tweets, which includes the labeled ones and the unlabeled ones, we obtained the percentage of negative tweets as a function of time. In addition to this, and for the sake of having a time reference, we overlaid some pandemic milestones obtained from the CDC COVID-19 pandemic milestones archive (CDC 2023). Figure 5 shows that the proportion of negative tweets on telecommuting rapidly increased after the pandemic was declared in March 2020. This negativity remained steady for around nine months between April and December of the same year, which then started to decrease at a moment that coincided with the FDA approval of the first COVID-19 vaccine. From then on, the negativity showed a steady decline until March 2022, when was a slight increase in negativity that went back down a few days later.

The same metric time evolution can be observed in a geographical fashion if we disaggregate the tweets to the state level and in determined time intervals, as shown in Fig. 6. We can see that before the pandemic, there was no clear pattern in the distribution of states with high or low negativity percentages, but as the pandemic evolves, almost every state sees an increase in the proportion of negative tweets. States marked in white represent states with no sufficient data to be displayed. The second and third-time intervals, which enclose the pandemic from its beginning to the time the first CDC-approved vaccine was announced, show an increase in negativity that is especially prominent in southern and coastal states. After this period, southern and northeastern states go back to the pre-pandemic levels of negativity, but interestingly, the west coast states do not go back to these levels and maintain the proportion of negative tweets. Even though we consider these to be relevant insights, a closer look must be taken to understand them better. This requires us to disaggregate the data further to the county scale. However, as shown in Fig. 2, there is a high imbalance between counties with a high number of tweets and counties with few or no tweets on the subject. For this reason, in this phase, we will focus our analysis on highly populated counties, which will also enable us to draw some parallels with existing literature on the topic.

To estimate the prevalence of telecommuting, we model the percentage of individuals working from home according to the CPS survey as a time series with Twitter extracted features as future covariates. This means that given past values of the percentage of WFH workers, we will use past and current values of Twitter data to estimate the current percentage of WFH workers. The selected Twitter features are the average number of tweets in the past 30 days, the percentage of tweets with positive sentiment in the past 30 days, the average number of likes in the past 30 days, and the percentage of likes that belong to positive tweets in the past 30 days.

5.1. Model evaluation

In order to evaluate the time series estimations, we make use of the following metrics:

Coefficient of variation (CV). It is a normalized measure of the variation between the two time series expressed as a percentage. It is calculated as follows:

$$CV=100*RMSE({y}_{t}, {\widehat{y}}_{t})/{\stackrel{-}{y}}_{t}$$

where ${y}_{t}$ are the actual telecommuting values, ${\widehat{y}}_{t}$ are the predicted values, ${\stackrel{-}{y}}_{t}$is the average of ${y}_{t}$, and RMSE is the root mean squared error.

Mean Squared Error (MSE):

$$MSE= \frac{1}{T}\sum _{t=1}^{T}{\left({y}_{t}-{\widehat{y}}_{t}\right)}^{2}$$

for T observations.

Mean Absolute Error (MAE):

$$MAE= \frac{1}{T}\sum _{t=1}^{T}\left|{y}_{t}-{\widehat{y}}_{t}\right|$$

Overall Percentage Error (OPE):

$$OPE=100*\left|\frac{\sum {y}_{t}-\sum {\widehat{y}}_{t}}{\sum {y}_{t}}\right|$$

Mean Absolute Ranged Relative Error (MARRE):

$$MARRE=100*\frac{1}{T}\sum _{t=1}^{T}\left|\frac{{y}_{t}-{\widehat{y}}_{t}}{{\text{max}}_{t}{y}_{t}-{\text{min}}_{t}{y}_{t}}\right|$$

Dynamic Time Warping (DTW). Applies DTW to the time series before and then passes them to the OPE metric.

5.2. National prevalence

We start by modeling the national prevalence of telecommuting among US workers between May 2020 and May 2022, according to CPS data availability. We fit the model using the initial 40% of observations and then proceed to evaluate the models according to the unobserved 60%. This procedure is done with four different models: Linear Regression (LR), Kalman Filter Forecaster (KFF), simple RNN, and GRU. All the models were obtained by implementing the Darts Python library (Herzen et al. 2022). Each model was fitted with and without covariates to better visualize the effects of covariates and to establish a baseline model for each case. Figure 7 shows the fitted curves for each of the four models. In addition to this, Table 2 shows various error metrics to compare the performance of the models and the effects of the covariates.

Table 2

Metrics of fitted models.
Model	Approach	CV	MSE	MAE	OPE	MARRE	DTW
LR	No covariates	56.457	0.010	0.092	70.015	65.507	19.797
LR	With covariates	11.871	0.001	0.018	6.440	13.113	6.230
KFF	No covariates	14.583	0.001	0.024	15.692	16.868	4.846
KFF	With covariates	9.586	0.000	0.013	8.370	9.003	4.547
RNN	No covariates	29.815	0.003	0.047	34.562	33.154	5.232
RNN	With covariates	11.805	0.000	0.016	5.536	11.544	3.548
GRU	No covariates	21.444	0.002	0.033	24.157	23.598	6.513
GRU	With covariates	7.856	0.000	0.011	1.456	7.585	5.561

From Table 2 and Fig. 7, we can see that for all four models, the forecast using Twitter covariates outperforms the forecast that only relies on past observations. Particularly, the models using covariates can accurately forecast subsequent not captured by the models without covariates. The KFF model is the one that performs the best for both approaches, capturing well the overall trend without covariates and successfully forecasting the peaks with the covariates approach.

5.3. State prevalence

Once we approach the state-level data in the same way we did at the national level, we encounter some limitations that hurt the predicting capabilities of the models. At the state level, we observe that some states need more data to produce models that are accurate enough. In some other cases, even when the number of observations seems sufficient, the models still display poor predicting capabilities, as shown in Fig. 8.

This problem could be linked to limitations arising from state-specific SA accuracies. To better illustrate this, Fig. 9 shows the relation between the number of available observations per state, the error on the forecasted values, and the accuracy of the test dataset on the sentiment analysis model at the given state.

What we present in this research is divided into two major sections: a SA model and time series forecasting using Twitter features. The SA model served the purpose of classifying the tweets depending on whether the author was displaying a positive or negative attitude towards working from home. This last part is crucial because it differentiates this model from general binary sentiment classifiers. These general classifiers can determine if a sentence is positive or negative but not concerning a specific topic. For instance, the sentence: “I am excited about finally being able to go back to the office tomorrow” will be classified as positive under these models, but in reality it should be labeled as negative because the author shows excitement about not telecommuting. This is what we intended to capture with our SA model.

Here it is worth discussing the second part of this research. We used the output of the SA model and other tweet features like the number of likes and tweets to estimate current values of the number of telecommuters in the US. As we saw in Fig. 7, including these features as covariates in the time series models outperforms the models that rely only on past observation to predict current values. The value behind this result lies in the possibilities that such a workflow would enable in this and other fields. Even though social media data has been shown not to be representative of the general population, the people that take part in it are still part of the population, and the ideas they discuss, the meaning their posts have, and the sentiment their words carry are all the product of these people’s interaction with the real world. Hence, it would not be surprising if an individual user brings social media insights into users with different demographics or geographies. It would be almost impossible to find direct ways to account for this. However, we do know that Machine Learning models are capable of feature extraction to the extent that it becomes a black box to humans, but it could be useful to obtain insights on at least part of their complex social relationships. Our intention with the presented models is to contribute with an additional step on this by combining two sources of data to show that we can reconstruct the behavior of representative data through non-representative data and some feature extraction.

As we showed, there are still at least two limitations. One of them has to do with the data availability. Figure 9 shows that, in general, the state-level models in which more data was available perform better. This challenge has to do with privacy restrictions because even though there are millions of tweets on telecommuting, we had to limit ourselves to tweets in which the authors disclosed their location, which left us with tens of thousands. Yet, we are confident that in the future, this challenge can be overcome by integrating other sources of data, not only by using other social media platforms but also by being able to extract features from images (e.g., an uploaded picture from a user showing a street could immediately offer information about the traffic congestion in the area). Another limitation has to do with the model’s accuracy. Part of this must also do with data availability, especially in some specific geographies, but part of it also has to do with the limitations of the used models. We are confident that this can be vastly improved soon, especially with the rapid improvements in NLP and the possibility of leveraging techniques such as transfer learning.

In this study we downloaded around 50,000 tweets related to telecommuting between 2020 and 2022. Using these, we developed a binary SA classifier able to determine whether a Twitter post was positive or negative towards the idea of telecommuting. We applied this classifier on tweets written during the Covid-19 pandemic to be able to model the sentiment of users towards telecommuting. Using survey data on the prevalence of WFH during the pandemic, we showed that incorporating the sentiment analysis evolution as well as other Twitter metrics significantly improves the estimations of telecommuting in the US and states with sufficient data.

Despite its limitations, this study reinforces the fact that content in social media posts is underutilized. Future work could study the potential of using social media posts to capture and model interrelationships between interdependencies between civil infrastructure. For instance, telecommuting has an important impact on electricity, water, and gas consumption (Derrible 2019; Movahedi et al. 2023; Movahedi and Derrible 2021), which might be better modeled using social media posts.

Conflict of interest

all authors declare that they have no competing interests.

Ethics Approval and Consent to Participate

Not Applicable

Funding

Not Applicable.

Author Contribution

The authors confirm contribution to the paper as follows: study conception and design: J.A.S, development and implementation: J.A.S, M.M., and S.P; analysis and interpretation of results: J.A.S, M.M, A.M and S.D. ; draft manuscript J.A.S, M.M., S.P, A.M and S.D.

Acosta-Sequeda, Juan, and Sybil Derrible. 2023. “GTdownloader: A Python Package to Download, Visualize, and Export Georeferenced Tweets from the Twitter API.” Journal of Open Research Software.
Agarwal, Apoorv, and Jasneet Sabharwal. 2012. “End-to-End Sentiment Analysis of Twitter Data.” In Proceedings of the Workshop on Information Extraction and Entity Analytics on Social Media Data, 39–44. Mumbai, India: The COLING 2012 Organizing Committee. https://aclanthology.org/W12-5504.
Aisopos, Fotis, George Papadakis, and Theodora Varvarigou. 2011. “Sentiment Analysis of Social Media Content Using N-Gram Graphs.” In Proceedings of the 3rd ACM SIGMM International Workshop on Social Media, 9–14. WSM ’11. New York, NY, USA: Association for Computing Machinery. https://doi.org/10.1145/2072609.2072614.
Alkouz, Balsam, Zaher Al Aghbari, and Jemal Hussien Abawajy. 2019. “Tweetluenza: Predicting Flu Trends from Twitter Data.” Big Data Mining and Analytics 2 (4): 273–87. https://doi.org/10.26599/BDMA.2019.9020012.
Aston, Nathan, Jacob Liddle, and Wei Hu. 2014. “Twitter Sentiment in Data Streams with Perceptron.” Journal of Computer and Communications 2014 (February). https://doi.org/10.4236/jcc.2014.23002.
Bakliwal, Akshat, Piyush Arora, Senthil Madhappan, Nikhil Kapre, Mukesh Singh, and Vasudeva Varma. 2012. Mining Sentiments from Tweets. Proc. of the 3rd Workshop In Computational Approaches to Subjectivity and Sentiment Analysis.
Balbontin, Camila, David A. Hensher, and Matthew J. Beck. 2022. “Advanced Modelling of Commuter Choice Model and Work from Home during COVID-19 Restrictions in Australia.” Transportation Research Part E: Logistics and Transportation Review 162 (June): 102718. https://doi.org/10.1016/j.tre.2022.102718.
Barbosa, Luciano, and Junlan Feng. 2010. “Robust Sentiment Detection on Twitter from Biased and Noisy Data.” In Coling 2010: Posters, 36–44. Beijing, China: Coling 2010 Organizing Committee. https://aclanthology.org/C10-2005.
Beck, Matthew J., and David A. Hensher. 2022. “Working from Home in Australia in 2020: Positives, Negatives and the Potential for Future Benefits to Transport and Society.” Transportation Research Part A: Policy and Practice 158 (April): 271–84. https://doi.org/10.1016/j.tra.2022.03.016.
Bollen, Johan, Huina Mao, and Xiaojun Zeng. 2011. “Twitter Mood Predicts the Stock Market.” Journal of Computational Science 2 (1): 1–8. https://doi.org/10.1016/j.jocs.2010.12.007.
Capasso Da Silva, Denise, Sara Khoeini, Deborah Salon, Matthew W. Conway, Rishabh S. Chauhan, Ram M. Pendyala, Ali Shamshiripour, et al. 2021. “How Are Attitudes Toward COVID-19 Associated with Traveler Behavior During the Pandemic?” Findings, June. https://doi.org/10.32866/001c.24389.
CBalabantaray, R, Mudasir mohd, and Nibha Sharma. 2012. “Multi-Class Twitter Emotion Classification: A New Approach.” International Journal of Applied Information Systems 4 (September): 48–53. https://doi.org/10.5120/ijais12-450651.
CDC. 2023. “CDC Museum COVID-19 Timeline.” Centers for Disease Control and Prevention. March 15, 2023. https://www.cdc.gov/museum/timeline/covid19.html.
Celikyilmaz, Asli, Dilek Hakkani-Tur, and Junlan Feng. 2010. “Probabilistic Model-Based Sentiment Analysis of Twitter Messages.” In 2010 IEEE Spoken Language Technology Workshop, 79–84. Berkeley, CA, USA: IEEE. https://doi.org/10.1109/SLT.2010.5700826.
Chauhan, Rishabh Singh, Matthew Wigginton Bhagat-Conway, Denise Capasso da Silva, Deborah Salon, Ali Shamshiripour, Ehsan Rahimi, Sara Khoeini, Abolfazl Mohammadian, Sybil Derrible, and Ram Pendyala. 2021. “A Database of Travel-Related Behaviors and Attitudes before, during, and after COVID-19 in the United States.” Scientific Data 8 (1): 245. https://doi.org/10.1038/s41597-021-01020-8.
Chauhan, Rishabh Singh, Matthew Wigginton Bhagat-Conway, Tassio Magassy, Nicole Corcoran, Ehsan Rahimi, Abbie Dirks, Ram Pendyala, Abolfazl Mohammadian, Sybil Derrible, and Deborah Salon. 2022. “COVID Future Panel Survey: A Unique Public Dataset Documenting How U.S. Residents’ Travel Related Choices Changed During the COVID-19 Pandemic.” https://doi.org/10.48550/ARXIV.2208.12618.
Chauhan, Rishabh Singh, Denise Capasso Da Silva, Deborah Salon, Ali Shamshiripour, Ehsan Rahimi, Uttara Sutradhar, Sara Khoeini, Abolfazl (Kouros) Mohammadian, Sybil Derrible, and Ram Pendyala. 2021. “COVID-19 Related Attitudes and Risk Perceptions across Urban, Rural, and Suburban Areas in the United States.” Findings, June. https://doi.org/10.32866/001c.23714.
Cho, Kyunghyun, Bart van Merrienboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014. “Learning Phrase Representations Using RNN Encoder-Decoder for Statistical Machine Translation.” arXiv. http://arxiv.org/abs/1406.1078.
Cucinotta, Domenico, and Maurizio Vanelli. 2020. “WHO Declares COVID-19 a Pandemic.” Acta Bio-Medica: Atenei Parmensis 91 (1): 157–60. https://doi.org/10.23750/abm.v91i1.9397.
Davidov, Dmitry, Oren Tsur, and Ari Rappoport. 2010. “Enhanced Sentiment Learning Using Twitter Hashtags and Smileys.” In Coling 2010: Posters, 241–49. Beijing, China: Coling 2010 Organizing Committee. https://aclanthology.org/C10-2028.
Derrible, Sybil. 2019. “Urban Engineering for Sustainability.” MIT Press. November 19, 2019. https://mitpress.mit.edu/9780262043441/urban-engineering-for-sustainability/.
Devlin, Jacob, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. “BERT: Pre-Training of Deep Bidirectional Transformers for Language Understanding.” arXiv. http://arxiv.org/abs/1810.04805.
Dong, Li, Furu Wei, Chuanqi Tan, Duyu Tang, Ming Zhou, and Ke Xu. 2014. “Adaptive Recursive Neural Network for Target-Dependent Twitter Sentiment Classification.” In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), 49–54. Baltimore, Maryland: Association for Computational Linguistics. https://doi.org/10.3115/v1/P14-2009.
Essien, Aniekan, Ilias Petrounias, Pedro Sampaio, and Sandra Sampaio. 2021. “A Deep-Learning Model for Urban Traffic Flow Prediction with Traffic Events Mined from Twitter.” World Wide Web 24 (4): 1345–68. https://doi.org/10.1007/s11280-020-00800-3.
Giachanou, Anastasia, and Fabio Crestani. 2017. “Like It or Not: A Survey of Twitter Sentiment Analysis Methods.” ACM Computing Surveys 49 (2): 1–41. https://doi.org/10.1145/2938640.
Go, Alec, Richa Bhayani, and Lei Huang. 2019. “Twitter Sentiment Classiﬁcation Using Distant Supervision.”
Gokulakrishnan, Balakrishnan, Pavalanathan Priyanthan, Thiruchittampalam Ragavan, Nadarajah Prasath, and AShehan Perera. 2012. “Opinion Mining and Sentiment Analysis on a Twitter Data Stream.” In International Conference on Advances in ICT for Emerging Regions (ICTer2012), 182–88. https://doi.org/10.1109/ICTer.2012.6423033.
Guyot, Katherine, and Isabel V. Sawhill. 2020. “Telecommuting Will Likely Continue Long after the Pandemic.” Brooking Institution, 2020. https://www.brookings.edu/blog/up-front/2020/04/06/telecommuting-will-likely-continue-long-after-the-pandemic/.
Hamdan, Hussam, Frederic Béchet, and Patrice Bellot. 2013. “Experiments with DBpedia, WordNet and SentiWordNet as Resources for Sentiment Analysis in Micro-Blogging.” In Second Joint Conference on Lexical and Computational Semantics (*SEM), Volume 2: Proceedings of the Seventh International Workshop on Semantic Evaluation (SemEval 2013), 455–59. Atlanta, Georgia, USA: Association for Computational Linguistics. https://aclanthology.org/S13-2075.
Javadinasr, Mohammadjavad, Tassio Maggasy, Motahare Mohammadi, Kouros Mohammadain, Ehsan Rahimi, Deborah Salon, Matthew W. Conway, Ram Pendyala, and Sybil Derrible. 2022. “The Long-Term Effects of COVID-19 on Travel Behavior in the United States: A Panel Study on Work from Home, Mode Choice, Online Shopping, and Air Travel.” Transportation Research Part F: Traffic Psychology and Behaviour 90 (October): 466–84. https://doi.org/10.1016/j.trf.2022.09.019.
Kiritchenko, S., X. Zhu, and S. M. Mohammad. 2014. “Sentiment Analysis of Short Informal Texts.” Journal of Artificial Intelligence Research 50 (August): 723–62. https://doi.org/10.1613/jair.4272.
Kouloumpis, Efthymios, Theresa Wilson, and Johanna Moore. 2011. “Twitter Sentiment Analysis: The Good the Bad and the OMG!” Proceedings of the International AAAI Conference on Web and Social Media 5 (1): 538–41. https://doi.org/10.1609/icwsm.v5i1.14185.
Mirtich, Laura, Matthew Wigginton Conway, Deborah Salon, Peter Kedron, Rishabh Singh Chauhan, Sybil Derrible, Sara Khoeini, Abolfazl (Kouros) Mohammadian, Ehsan Rahimi, and Ram Pendyala. 2021. “How Stable Are Transport-Related Attitudes over Time?” Findings, June. https://doi.org/10.32866/001c.24556.
Mohammadi, Motahare (Yalda), Ehsan Rahimi, Amir Davatgari, Mohammadjavad Javadinasr, Abolfazl (Kouros) Mohammadian, Matthew Wigginton Bhagat-Conway, Deborah Salon, Sybil Derrible, Ram M. Pendyala, and Sara Khoeini. 2022. “Examining the Persistence of Telecommuting after the COVID-19 Pandemic.” Transportation Letters, May, 1–14. https://doi.org/10.1080/19427867.2022.2077582.
Movahedi, Ali, and Sybil Derrible. 2021. “Interrelationships between Electricity, Gas, and Water Consumption in Large-Scale Buildings.” Journal of Industrial Ecology 25 (4): 932–47. https://doi.org/10.1111/jiec.13097.
Movahedi, Ali, Amir Bahador Parsa, Anton Rozhkov, Dongwoo Lee, Abolfazl Kouros Mohammadian, and Sybil Derrible. 2023. “Interrelationships between Urban Travel Demand and Electricity Consumption: A Deep Learning Approach.” Scientific Reports 13 (1): 6223. https://doi.org/10.1038/s41598-023-33133-y.
Nafakh, Ahmad Jalal, Abolfazl Kouros Mohammadian, Sybil Derrible, Ram M. Pendyala, and Deborah Salon. 2022a. “Examining the Impacts of the Pandemic on Employment for Women.” Open Journal of Social Sciences 10 (13): 318–34. https://doi.org/10.4236/jss.2022.1013025.
———. 2022b. “Examining the Impacts of the Pandemic on Work-from-Home for Individuals with Disabilities.” Open Journal of Social Sciences 10 (13): 167–83. https://doi.org/10.4236/jss.2022.1013014.
Nguyen, Minh Hieu. 2021. “Factors Influencing Home-Based Telework in Hanoi (Vietnam) during and after the COVID-19 Era.” Transportation 48 (6): 3207–38. https://doi.org/10.1007/s11116-021-10169-5.
Nguyen, Minh Hieu, and Jimmy Armoogum. 2021. “Perception and Preference for Home-Based Telework in the COVID-19 Era: A Gender-Based Analysis in Hanoi, Vietnam.” Sustainability 13 (6): 3179. https://doi.org/10.3390/su13063179.
Nilles, Jack M. 1988. “Traffic Reduction by Telecommuting: A Status Review and Selected Bibliography.” Transportation Research Part A: General 22 (4): 301–17. https://doi.org/10.1016/0191-2607(88)90008-8.
Pak, Alexander, and Patrick Paroubek. 2010. “Twitter as a Corpus for Sentiment Analysis and Opinion Mining.” In Proceedings of the Seventh International Conference on Language Resources and Evaluation (LREC’10). Valletta, Malta: European Language Resources Association (ELRA). http://www.lrec-conf.org/proceedings/lrec2010/pdf/385_Paper.pdf.
Parker, Kim, Juliana Menasce Horowitz, and Rachel Minkin. 2022. “COVID-19 Pandemic Continues To Reshape Work in America.” Pew Research Center, February 16, 2022. https://www.pewresearch.org/social-trends/2022/02/16/covid-19-pandemic-continues-to-reshape-work-in-america/.
Ratnani, Harsha S, and Siddharth Kumar. 2021. “TWITTER BASED TRAFFIC ANALYSIS AND PREDICTION MODEL FOR PLANNED EVENTS” 20 (10).
Ruder, Sebastian, Matthew E. Peters, Swabha Swayamdipta, and Thomas Wolf. 2019. “Transfer Learning in Natural Language Processing.” In Proceedings of the 2019 Conference of the North, 15–18. Minneapolis, Minnesota: Association for Computational Linguistics. https://doi.org/10.18653/v1/N19-5004.
Saif, Hassan, Yulan He, and Harith Alani. 2012. “Semantic Sentiment Analysis of Twitter.” In The Semantic Web – ISWC 2012, edited by Philippe Cudré-Mauroux, Jeff Heflin, Evren Sirin, Tania Tudorache, Jérôme Euzenat, Manfred Hauswirth, Josiane Xavier Parreira, et al., 508–24. Lecture Notes in Computer Science. Berlin, Heidelberg: Springer. https://doi.org/10.1007/978-3-642-35176-1_32.
Salon, Deborah, Matthew Wigginton Conway, Denise Capasso Da Silva, Rishabh Singh Chauhan, Sybil Derrible, Abolfazl (Kouros) Mohammadian, Sara Khoeini, et al. 2021. “The Potential Stickiness of Pandemic-Induced Behavior Changes in the United States.” Proceedings of the National Academy of Sciences 118 (27): e2106499118. https://doi.org/10.1073/pnas.2106499118.
Salon, Deborah, Laura Mirtich, Matthew Wigginton Bhagat-Conway, Adam Costello, Ehsan Rahimi, Abolfazl (Kouros) Mohammadian, Rishabh Singh Chauhan, Sybil Derrible, Denise Da Silva Baker, and Ram M. Pendyala. 2022. “The COVID-19 Pandemic and the Future of Telecommuting in the United States.” Transportation Research Part D: Transport and Environment 112 (November): 103473. https://doi.org/10.1016/j.trd.2022.103473.
Severyn, Aliaksei, and Alessandro Moschitti. 2015. “Twitter Sentiment Analysis with Deep Convolutional Neural Networks.” In Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval, 959–62. Santiago Chile: ACM. https://doi.org/10.1145/2766462.2767830.
Singh, Jyoti Prakash, Yogesh K. Dwivedi, Nripendra P. Rana, Abhinav Kumar, and Kawaljeet Kaur Kapoor. 2019. “Event Classification and Location Prediction from Tweets during Disasters.” Annals of Operations Research 283 (1): 737–57. https://doi.org/10.1007/s10479-017-2522-3.
Sun, Wenzhe, Hironori Kobayashi, Satoshi Nakao, and Jan-Dirk Schmöcker. 2023. “On the Relationship Between Crowdsourced Sentiments and Mobility Trends During COVID-19: A Case Study of Kyoto.” Data Science for Transportation 5 (3): 17. https://doi.org/10.1007/s42421-023-00080-z.
Tahlyan, Divyakant, Maher Said, Hani Mahmassani, Amanda Stathopoulos, Joan Walker, and Susan Shaheen. 2022. “For Whom Did Telework Not Work during the Pandemic? Understanding the Factors Impacting Telework Satisfaction in the US Using a Multiple Indicator Multiple Cause (MIMIC) Model.” Transportation Research Part A: Policy and Practice 155 (January): 387–402. https://doi.org/10.1016/j.tra.2021.11.025.
Tang, Duyu, Furu Wei, Nan Yang, Ming Zhou, Ting Liu, and Bing Qin. 2014. “Learning Sentiment-Specific Word Embedding for Twitter Sentiment Classification,” June. https://doi.org/10.3115/v1/p14-1146.
Tul, Qurat, Mubashir Ali, Amna Riaz, Amna Noureen, Muhammad Kamranz, Babar Hayat, and Aziz Ur Rehman. 2017. “Sentiment Analysis Using Deep Learning Techniques: A Review.” International Journal of Advanced Computer Science and Applications 8 (January). https://doi.org/10.14569/IJACSA.2017.080657.
Twitter. 2012. “Twitter API.” 2012. https://developer.twitter.com/en/docs/twitter-api.
Valencia, Franco, Alfonso Gómez-Espinosa, and Benjamín Valdés-Aguirre. 2019. “Price Movement Prediction of Cryptocurrencies Using Sentiment Analysis and Machine Learning.” Entropy 21 (6): 589. https://doi.org/10.3390/e21060589.
Vaswani, Ashish, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. “Attention Is All You Need.” arXiv. https://doi.org/10.48550/arXiv.1706.03762.
Vo, Duy-Tin, and Yue Zhang. 2015. “Target-Dependent Twitter Sentiment Classification with Rich Automatic Features.” In Proceedings of the 24th International Conference on Artificial Intelligence, 1347–53. IJCAI’15. Buenos Aires, Argentina: AAAI Press.
Yao, Weiran, and Sean Qian. 2021. “From Twitter to Traffic Predictor: Next-Day Morning Traffic Prediction Using Social Media Data.” Transportation Research Part C: Emerging Technologies 124 (March): 102938. https://doi.org/10.1016/j.trc.2020.102938.
Yavari, A, H Hassanpour, B Rahimpour, and M Mahdavi. 2022. “Election Prediction Based on Sentiment Analysis Using Twitter Data.” International Journal of Engineering 35 (2): 372–79. https://doi.org/10.5829/IJE.2022.35.02B.13.

No competing interests reported.

Download PDF

Journal Publication

published 29 Oct, 2024

Read the published version in Data Science for Transportation →

Editorial decision: Revision requested
02 Aug, 2024
Reviews received at journal
02 Aug, 2024
Reviewers agreed at journal
20 Jul, 2024
Reviews received at journal
23 Apr, 2024
Reviewers agreed at journal
15 Feb, 2024
Reviewers invited by journal
13 Feb, 2024
Editor assigned by journal
12 Feb, 2024
Submission checks completed at journal
12 Feb, 2024
First submitted to journal
19 Jan, 2024

You are reading this latest preprint version

Estimating Telecommuting Rates in the US Using Twitter Sentiment Analysis

Status:

Journal Publication

Version 1

Abstract

Figures

1. Introduction

2. Literature Review

2.1. Telecommuting

2.2. Sentiment Analysis on Twitter data

3. Methodology

3.1. Data

3.1.1. Current Population Survey (CPS)

3.1.2. Twitter

3.2. Model

4. Sentiment Analysis

4.1. Model Performance

4.2. Sentiment Analysis

5. Estimating the telecommuting prevalence

5.1. Model evaluation

5.2. National prevalence

5.3. State prevalence

6. Discussion

7. Conclusions

Declarations

Funding

Author Contribution

References

Additional Declarations

Status:

Journal Publication

Version 1