HTTPS certificate verification problem in Python 2.7.9

Over the last few days I was working through some AWS security best practices as outlined in a video presentation from AWS re:Invent 2013 called “Intrusion Detection in the Cloud (SEC402)” (video linkPDF presentation link).

One of the interesting ideas that the authors were promoting was this particular Python script (script linkJSON template for IAM policy) which would use the AWS Python SDK to obtain textual descriptions of various critical security settings that would be part of an AWS account (such as IAM users, groups and policies, S3 bucket policies, EC2 security groups and so on). These settings could all be exported and saved in a text file, the script could be scheduled to run at various intervals and alerts could be raised each time differences in the file would be detected.

I’m currently in the middle of setting up the AWS environment for a client project so I liked the idea of having a script such as the one mentioned above which I could use to script various critical settings and monitor how they change over time. Besides … Python is not really a language that I use often (though I do plan to become more familiar with it) so I welcomed the chance to try something new – how hard could it really be? 😉

I initially proceeded to download and install the latest version of Python – 3.4.2. It didn’t take too long to realize that the script authors actually wrote it for the Python 2.x branch. I was getting all sorts of errors trying to run it under 3.4.2 – as soon as I managed to get one of them fixed, another one came up. I didn’t really care much which version of Python I’d use so since 3.4.2 was giving me enough trouble I decided to switch to the 2.x branch and installed 2.7.9.

Success … or so it seemed. The script finally started running until half-way through its execution it died with this strange looking error while trying to connect to some AWS API endpoint:

ERROR: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:581)

I was pretty sure that Python was not happy about some discrepancy it found in a HTTPS certificate that it received while trying to make a HTTPS call to AWS. The problem was that the HTTPS site it was reaching out to was controlled by AWS and used deep in the AWS Python SDK code – not a resource for which I could do anything about the configuration of its HTTPS certificate.

So what exactly was Python complaining about? After some research I came across this link:

https://www.python.org/dev/peps/pep-0476/

It appears that a fairly recent change was made in the 2.7.x branch in regards to the default behavior for certificate verification. I ended up using the following monkey patch to globally disable verification for all HTTPS calls made by the script:

import ssl

ssl._create_default_https_context = ssl._create_unverified_context

The script finally ran successfully to the end and I had my scripted security settings. All in all it was an interesting first adventure in the land of Python.