Verify SSL Expiration with Python3
If you’ve ever had a need to verify multiple SSL certificates for expiration times in a batch and wanted to script it in Python, you’ll find this article interesting.
When I try to solve a problem, programatically, I usually start with the “what makes sense” question. In this case, the SSL cert main problem was not knowing when a site expired. With that said, there are several other things that are important to know:
- HTTP Response Codes
- Known text on the URI
- Which Cipher Suites are Used on the SSL?
- What version of SSL/TLS
- What is the key strength of the cipher in use
- SSL Serials
- Domain Matching
- Other tests such as Beast vulnerability
- Is TLS compression in use? (CRIME vulnerability)
- Do I have Session Ticket Support?
- Do I have Ephemeral Key Support?
And the list goes on! As new vulnerabilities come up you need to know things about the certs you’ve chosen to use on your servers, and the clients you connect with.
Filter On Important/Wanted Info
As you can see there are many reasons you should review your SSL certs. For this script, I started with just a few:
TLS Version
Reason: Vulnerability attacks such as Beast on version 1.0
SSl x.509 Version
Reason: 1, 2, or 3 with 3 being the preferred
Expiration Date
Reason: For simple SSL cert management I need to know when to renew my certs!
Cipher Suite
Reason: Currently RSA 256*8 (2048) length is probably ok, but knowing the length and cipher suite is important to know what needs to be upgraded when the next attack of the day proves a cipher is no longer strong enough. There are several weak ciphers and both the client and server must agree on one. This shows my python client at least.
Tools such as SSLyze, sslcaudit and tlssled (all in the Kali toolset and available on git), show this type of information too. However, I wanted to send warning emails for certificate renewals and provide reports across an enterprise environment (and learn more about SSL/TLS in general).
Sample Data to Scan SSL
First, we need sample data and a list of servers to check. This could easily be made much larger (thousands of sites), but for our demo/tutorial purposes we will use the following 4 sites. This type of data structure is a dictionary {} of lists []. YOu can see that I also added a key (1000-1003) as I intend to eventually track these records in mysql using pymsql. The “200” you see is the expected http status code field, but I didn’t actually do anything with it in the script.
{
"1000": ["howsmyssl.com", 443, 200, "https://", "/", "N/A"],
"1001": ["www.google.com", 443, 200, "https://", "/", "N/A"],
"1002": ["wellsfargo.com", 443, 200, "https://", "/", "N/A"],
"1003": ["somesite.com", 443, 200, "https://", "/", "N/A"]
}
Script to Test SSL Expirations
The final version of this script will send email to alert, store periodic checks for DNS load times, TCP handshake times, and page load times as well as scanning the page for specific text to make sure the page is up. In this way we can monitor thousands of sites. Being proactive when issues appear, before they are issues is always the best way!
You will also notice that this python script does the SSL scans fast. The first version was single threaded. We had to wait on DNS look ups, TCP Handshake, etc 1 at a time. I rewrote it to use threading and now it’s very fast. It will scan hundreds of sites in just a few seconds.
Anyway, here is the script as-is. It is simple enough to learn from, but is missing some of the features that I will be putting into it over the next few days/weeks:
- Add Email Alert Function
- Add Text on URI verification
- Store in MySQL
- Finally - run from Cron
'''
ssl tester 2/08/20
Main purpose was to verify my cert bot expiry times so I could
have plenty of time to renew them. I'm now thinking of what I want
to look at across the board on all of my servers.
Ideas welcome!
todo:
- add email notification warning if too close to expire
- look at poodle or other known vulns
- add in tcp handshake time
- add in dns time to monitor dns requests
- add in total page load time
- check page text for known and/or fingerprint
- warn on status code alerts like 500 or !200
- pull and id server version?
- fix up the ctrl c thing. It works but not like I intended
- put entire thing in tkinter so others can more easily use it
- store into over time graph on a sql type database
topics learned/refreshed:
modules
ctrl+c without error
threading
benchmarking
dictionary of list data structure
loops
definitions
time formatting
time deltas
ssl and https basic checks
json read/write
'''
#https://docs.python.org/3/library/ssl.html
from urllib.request import Request, urlopen, ssl, socket
from urllib.error import URLError, HTTPError
# for dumping our sites dictionary to json
import json
# for getting response codes from header
import urllib.request
# for date calcs
from datetime import datetime
# for proper ctrl + c captures
from signal import signal, SIGINT
from sys import exit
# actual code start time
import time
startTime = time.time()
import threading # we want to multi thread this
from queue import Queue # and have queue management
'''
Sample output (single threaded 40~ sites)
Running. Press CTRL+C to exit.
...
Run Time: 42.77 seconds
Sample output (multi threaded 40~ sites)
Running. Press CTRL+C to exit.
...
Run Time: 10.23 seconds (DNS fresh)
Run Time: 4.03 seconds (DNS cached)
sites = {
"1000": ["howsmyssl.com",443,200,"https://","/","N/A"],
"1001": ["www.google.com",443,200,"https://","/","N/A"],
"1002": ["wellsfargo.com",443,200,"https://","/","N/A"],
"1003": ["somesite.com",443,200,"https://","/","N/A" ]
}
# write it out to json so I can just run it from a json load later
print("Writing 'sites.json' file...")
with open ('sites.json', 'w') as outfile:
json.dump(sites, outfile, indent=2)
# after I write out my sample, I can write the rest or manually add
# todo: create function to add sites to data easily
'''
'''
# threader thread pulls worker from queue and processes
'''
def threader():
while True:
# gets worker from queue
worker = q.get()
# run job with available worker in queue (thread)
getSSLInfo(worker)
# complete with the job, shut down thread
q.task_done()
'''
This was supposed to do a clean exit on ctrl+c but instead
It requires a few ctrl+c to work. It doesnt' error now at least
when ctrl+c is pressed.
'''
def handler(signal_received, frame):
# Handle any cleanup here
print('SIGINT or CTRL-C detected. Exiting gracefully')
exit(0)
'''
Just an http status code return from uri input
todo: strip off the new lines
'''
def getResponseCode(uri):
conn = urllib.request.urlopen(uri)
return conn.getcode()
'''
Pulls some info from ssl:
expiry, serial, ssl version, bits, cipher, Expire time, domain given (not verified), port checked, response code
expiry is a negative countdown, or in the case of http, just a 1 (see below)
Running. Press CTRL+C to exit.
1000 TLSv1.2 3 -82 May 4 02:04:04 2020 GMT 256 ECDHE-RSA-CHACHA20-POLY1305 howsmyssl.com:443 200
1001 TLSv1.2 3 -62 Apr 14 08:16:35 2020 GMT 256 ECDHE-ECDSA-CHACHA20-POLY1305 www.google.com:443 200
1002 TLSv1.2 3 -58 Apr 9 12:00:00 2020 GMT 256 ECDHE-RSA-AES256-GCM-SHA384 wellsfargo.com:443 200
1003 TLSv1.2 3 -103 May 24 23:59:59 2020 GMT 256 ECDHE-RSA-AES256-GCM-SHA384 somesite.com:443 200
'''
def getSSLInfo(kid):
host = str(sites[kid][0]) # somesite.com
port = str(sites[kid][1]) # 80
proto = str(sites[kid][3]) # http:// or https://
path = str(sites[kid][4]) # / or /somepath.php
text = str(sites[kid][5]) # text to validate on the site
# It's either going to be http or https
# if it's http, we just put place holders for now
checkthis = proto + host + ":" + port + path
if (proto == 'http://'):
nd[kid] = [1,0,0,0,0,0,host + ":" + str(port),str(getResponseCode(checkthis))]
else:
try:
# if it's https, this will work, or should work
context = ssl.create_default_context()
with socket.create_connection((host, port)) as sock:
with context.wrap_socket(sock, server_hostname=host) as ssock:
# get the ssl/tls info:
d = ssock.getpeercert()
# dumps a 3 part list: encryption, version, bits
cipherinfo = ssock.cipher()
# time from input
dt1 = datetime.strptime(d['notAfter'], '%b %d %H:%M:%S %Y GMT')
# time difference
timediff = dt2 - dt1
#print(str(timediff.days) + ",", end='')
#print(d['serialNumber'] + ",", end='') # 56E41941EC60E545555
#print(str(d['version']) + ",", end='') # 3
#print(ssock.version() + ",", end='') # TLSv1.2 (or SSLv2, SSLv3, TLSv1, TLSv1.1)
#print(str(cipherinfo[2]) + ",", end='') # 256 (x8 for 2048 RSA key)
#print(cipherinfo[0] + ",", end='') # ECDHE-RSA-AES256-GCM-SHA384
#print(d['notAfter'] + ",", end='') # May 24 23:59:59 2020 GMT
#print(host + ":" + str(port) + ",", end='') # somedomain:443
#print(getResponseCode(checkthis))
nd[kid] = [ssock.version(),d['version'],str(timediff.days),d['notAfter'],cipherinfo[2],cipherinfo[0],host + ":" + str(port),str(getResponseCode(checkthis))]
except:
# whoops, it's probably not https. Do something better here
nd[kid] = [1,0,0,0,0,0,host + ":" + str(port),str(getResponseCode(checkthis))]
# get the current time for expiry time calculations
# current time is only needed once at start
now = datetime.now()
dt2 = datetime.strptime(now.strftime("%Y-%m-%d %H:%M:%S"), '%Y-%m-%d %H:%M:%S')
# loop over our dictionary of lists
# Tell Python to run the handler() function when SIGINT is recieved
signal(SIGINT, handler)
# preload from sites.json file instead of data in script
with open('sites.json', 'r') as infile:
# put it into a dictionary called "data"
sites = json.load(infile)
# we have to load this guy up quickly then print out when it's complete
# the with print_lock slows things down too much
nd = {}
# run the program inside a ctrl + c check
print('Running. Press CTRL+C to exit.')
while True:
# create queue and threader
q = Queue()
for x in range(200):
# thread id
t = threading.Thread(target = threader)
# classifying as a daemon, so they will die when the main dies
t.daemon = True
# begins, must come after daemon definition
t.start()
# this is the range or variable passed to the worker pool
# we are loading up the thread pool here
# work is done in the threader, not here
for worker in (sites.keys()):
q.put(worker)
# wait until thrad terminates, then reassemble
q.join()
# now that we built this dictionary threaded, spit it out in single thread
# this allows a sort and seemed the fastest way instead of print locking
# todo: I'm sure this isn't the pythonic way to make a CSV!
for kid in sorted (nd.keys()):
print(str(kid) + "\t" + str(nd[kid][0]) + "\t" + str(nd[kid][1]) + "\t" + str(nd[kid][2]) + "\t" + str(nd[kid][3]) + "\t" + str(nd[kid][4]) + "\t" + str(nd[kid][5]) + "\t" + str(nd[kid][6]) + "\t" + str(nd[kid][7]))
# ok, give us a final time report
runtime = float("%0.2f" % (time.time() - startTime))
print("Run Time: ", runtime, "seconds")
# end of ctrl + c check too
exit(0)
Sample Output
