A Demonstration of Using MS Access & VBA to Link to or Scrape Data from the Web - Wikipedia Example

Описание: Dive into the world of automation with our step-by-step tutorial on how to write VBA code in Microsoft Access to seamlessly link to and scrape data from tables on Wikipedia pages. Whether you're looking to enrich your database with dynamic web content or streamline your data collection process, this video is your go-to guide. For those of you unfamiliar with web-scraping or simply "scraping", this means an automated extraction of data from the web via a script. In our case, we are linking to and scraping data from a Wikipedia page.

This is one of those things I haven't seen before, but I figured was possible in MS Access. Sure enough, it was possible. Keep in mind this video is more on a demonstration or proof of concept more than anything.

In this comprehensive tutorial, we'll cover:

An introduction to web scraping and linking to data on the web (like a table).
A detailed walkthrough on setting up your MS Access database for web scraping and linking to web data.
Step-by-step instructions on writing VBA code to connect to Wikipedia, target specific tables, and import data directly into your Access database.

Perfect for database administrators, developers, and anyone interested in automating data collection processes, this video will empower you with the skills to bring the vast world of web data into your MS Access projects.

Because the video was on the long side, I did not get into the permissions and ethics of scraping data from the web. Web scraping can be incredibly powerful, but it's essential to do it the right way. As we dive into the technical side of scraping data from websites like Wikipedia, we should also talk about permissions and why they matter:

How to Check for Permission: The best place to start is by looking at the website's robots.txt file. You can find this by adding /robots.txt to the end of the website URL (for example, wikipedia.org/robots.txt). This file tells you what parts of the site are off-limits for scraping. Note: If you are having trouble understanding the robots.txt file, just copy and paste it into ChatGPT or BingAI for guidance.

Ethical Scraping Practices: Always scrape data responsibly. This means checking permissions, not overloading servers, and using the data you collect in a way that respects privacy and copyright laws.

For more information about robots.txt files and how to read them, see: https://www.seerinteractive.com/insig...

Link to VBA Code: https://controlc.com/1d8e9eda

VIDEO TIMELINE

0:00 - Intro
0:04 - Overview
0:51 - How to add in VBA code for modules
2:09 - Create button to trigger web extraction from modules
4:03 - Review of Module 1 (get log of scraped table)
7:42 - Review of Module 2 (obtain and organize data from log table)
9:30 - Review of Module 3 (Regular Expressions to remove brackets)
10:48 - Running it one more time
11:43 - Outro