Certificate Errors in urllib

#Web #Web Scraping

Dealing with errors when scraping data

# Import modules
import urllib.request, urllib.parse, urllib.error, ssl

# Ignore SSL certificate errors
ctx = ssl.create_default_context()
ctx.check_hostname = False
ctx.verify_mode = ssl.CERT_NONE

# Retrieve data
# The context = ctx will ignore the errors from certificates
url = 'https://www.google.com'
html = urllib.request.urlopen(url, context=ctx).read()

References: Worked Example: BeautifulSoup (Chapter 12)

Published: by ;

Lei Ma (2018). 'Certificate Errors in urllib', Datumorphism, 06 April. Available at: https://datumorphism.leima.is/til/data/python-urllib-ssl/.

Table of Contents

Current Ref:

  • til/data/python-urllib-ssl.md