Certificate Errors in urllib

#Web #Web Scraping

Dealing with errors when scraping data

# Import modules
import urllib.request, urllib.parse, urllib.error, ssl

# Ignore SSL certificate errors
ctx = ssl.create_default_context()
ctx.check_hostname = False
ctx.verify_mode = ssl.CERT_NONE

# Retrieve data
# The context = ctx will ignore the errors from certificates
url = 'https://www.google.com'
html = urllib.request.urlopen(url, context=ctx).read()

References: Worked Example: BeautifulSoup (Chapter 12)

Published: by ;

Authors: Lei Ma

Table of Contents

Current Ref:

  • til/data/python-urllib-ssl.md