Script: Simple HTTP Status Checker

Sure, you could fire up Screaming Frog, wait for the Java environment to load, configure the include and exclude settings, run the crawl, and export the CSV. But that’s like firing up a V8 engine just to drive to the mailbox.

Sometimes, you just need a pocket knife. You need to check 50 URLs right now to see if the dev team actually set up those redirects they promised.

Here is how to build your own lightweight, lightning-fast HTTP Status Checker.

Why Build This?

Let’s address the elephant in the room. Why write code when tools exist?

1. Speed – This script runs instantly. No UI, no loading screens.
2. Portability – You can put this on a server and run it automatically every morning at 6 AM (Cron Job). Screaming Frog requires a desktop environment.
3. Cost – This is free. It scales to 1,000,000 URLs, and it still costs $0.
4. The Flex – There is a visceral satisfaction in watching your code ping the server and report back.

Prerequisites

You need Python installed and the requests library. If you haven’t done this yet:

pip install requests

The Code

Open VS Code, create a file named quick_check.py, and paste this in. We aren’t building a rocketship here, but only a probe.

import requests
import time

# Target List
# (In the future, we will pull this from a CSV. For now, we list them here.)
urls = [
    "https://seo-automata.com",
    "https://seo-automata.com/broken-page",
    "https://google.com"
]

# 2. Configuration (Look like a human)
headers = {
    'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36'
}

print(f" Starting Scan on {len(urls)} URLs...\n")

# 3. Loop
for url in urls:
    try:
        # Request
        # allow_redirects=False is CRITICAL for SEO. 
        # We want to see the 301, not follow it to the 200.
        response = requests.get(url, headers=headers, allow_redirects=False, timeout=5)
        
        # Output Logic
        code = response.status_code
        
        # Color-coding the output (just for style)
        if code == 200:
            status = "200 OK"
        elif 300 <= code < 400:
            status = f"{code} REDIRECT"
        elif code == 404:
            status = "404 NOT FOUND"
        elif code >= 500:
            status = "{code} SERVER ERROR"
        else:
            status = f"{code}"

        print(f"{status} | {url}")

    except requests.exceptions.RequestException as e:
        # This handles DNS failures or connection timeouts
        print(f" {url} - Error: {e}")
    
    # Be polite to the server
    time.sleep(0.5)

print("\n Scan Complete.")

Breakdown

Let’s decode the mechanics so you can modify it later.

1. allow_redirects=False

This is the most important line for an SEO.

By default, the requests library acts like a browser – if it hits a 301, it follows it to the destination.

• Default – Pings old-page, then Follows 301, then Lands on new-page, and then Reports 200 OK.
• SEO Mode (False) – Pings old-page, then Sees 301, then Stops, and then Reports 301 Moved Permanently.

You can’t audit a redirect chain if your tool automatically skips to the end.

2. The headers Dictionary

Servers are paranoid. If you knock on the door identifying as python-requests/2.26, many security firewalls (Cloudflare, Akamai) will block you instantly (403 Forbidden).

We spoof the User-Agent to look like a standard Chrome browser. It’s a digital disguise.

3. The try...except Block

The internet is messy. Sometimes a server doesn’t respond with a status code. It just times out or refuses the connection.

If you don’t wrap your request in a try block, one bad URL will crash your entire script. The except block catches the crash, prints the error, and moves to the next URL.

Your Next Goal

Run the script. Watch the terminal populate.

You just built a tool that does exactly what you need, nothing more, nothing less.

Here is a version 2.0 idea – Replace the urls = [...] list with pandas.read_csv('urls.csv') to check 10,000 links at once. Or even try to find a way to export the results into a .csv