Dataset for "Investigation of Differences in Tobacco Use Language Between Groups: Corpus-Assisted Analysis"

Data collection method:

The search parameters for the tobacco industry corpus (TIC) were: web pages and documents published by British American Tobacco (BAT) and Philip Morris International (PMI). We used the following search query to capture contents containing terms related to smoking status and smoking behaviour: smok* OR quit* OR cessation OR switch* OR vap* OR experiment* OR relaps* OR "us* NRT*" OR "dual" OR addict*. We carried out three separate searches: 1) Google site search, 2) using the search function within the two tobacco company websites, and 3) searching web archive tools – The Internet Archive and archive.today – to collect back dated content not captured in searches 1 or 2.

Data processing and preparation activities:

The list of returned results were saved as structured CSV files containing metadata such as URLs, headings, publication dates. This list was then manually filtered to exclude job adverts, pages spotlighting named individuals, and pages or documents in which search terms only appeared in cautionary statements or legal disclosures. We also excluded region-specific/geographically local content pages.

Technical details and requirements:

We used Python scripts to process the CSV files. The scripts included routines for downloading linked PDF documents, extracting and cleaning text from PDF documents using the pdfminer library (https://pypi.org/project/pdfminer/), and exporting web-based content as individual plain text (.txt) files.