How to Build a Cars.com Scraper Using Python and Selenium

Cars.com is one of the largest online automotive marketplaces in the United States—making it a rich source for competitive pricing analysis, market trend research, and inventory monitoring. However, collecting this data manually is slow and impractical at any useful scale. A cars.com scraper can pull structured listing data from the site automatically, giving you a clean dataset ready for analysis.

This tutorial shows how to scrape cars.com using Selenium and BeautifulSoup.

Car.com Scraper: The Environment

Start by installing:

• Selenium: pip install selenium
• BeautifulSoup and lxml: pip install beautifulsoup4 lxml

Selenium 4.6+ includes a built-in driver manager, so you no longer need to download ChromeDriver separately. The correct driver version is resolved automatically at runtime.

Analyzing the Cars.com Search Results Page

Before writing any code for cars.com data scraping, understand how Cars.com structures its search results. Open your browser, navigate to cars.com, and run a search. For example, used Toyota vehicles.

The resulting URL follows this pattern:

https://www.cars.com/shopping/results/?makes[]=toyota&maximum_distance=all&models[]=&page=1&stock_type=used&zip=

The key query parameters are:

Parameter	Description
makes[]	The car manufacturer (e.g., toyota, honda)
stock_type	used, new, or cpo (certified pre-owned)
page	The page number for pagination
zip	ZIP code for location-based results
maximum_distance	Search radius (all, 10, 25, 50, etc.)

Now, right-click on a vehicle listing card and select Inspect to open Chrome DevTools. You will notice that each listing is wrapped inside a custom <fuse-card> web component. The key data points inside each card are:

• Vehicle Name: Inside an <h2> tag, which contains an <a> link to the detail page
• Price: Inside an element with the class fuse-body-larger
• Mileage: Inside an element with the class mileage
• Dealer Name: Inside a <span> with the class fuse-body-small
• Dealer Rating: Inside an element with the class review-star
• Location: Adjacent to a <fuse-svg> element with name=”map-marker-outline”

Cars.com also embeds a data-vehicle-details JSON attribute directly on each <fuse-card> element. This attribute contains structured metadata (price, mileage, stock type) that we can use as a fallback when the visible HTML elements are missing or formatted inconsistently.

Cars.com Scraper The Code

 

Cars.com uses Cloudflare Turnstile to detect and block automated traffic. A standard HTTP request with the requests library will return a 403 Forbidden response. To get past this, use a headless browser session.

Selenium launches an actual Chrome window that behaves like a regular user’s browser. However, Chrome exposes a navigator.webdriver property that Cloudflare checks to identify automation.

Here is how we configure Selenium to launch Chrome without the automation markers:

from selenium import webdriver
from selenium.webdriver.chrome.options import Options

options = Options()
options.add_argument('--disable-blink-features=AutomationControlled')
options.add_experimental_option("excludeSwitches", ["enable-automation"])
options.add_experimental_option('useAutomationExtension', False)

driver = webdriver.Chrome(options=options)

# Remove the navigator.webdriver flag via Chrome DevTools Protocol
driver.execute_cdp_cmd("Page.addScriptToEvaluateOnNewDocument", {
    "source": "Object.defineProperty(navigator, 'webdriver', {get: () => undefined})"
})

The –disable-blink-features=AutomationControlled argument prevents Chrome from setting the navigator.webdriver flag. The two experimental_option calls remove the “Chrome is being controlled by automated test software” infobar and disable the automation extension. Finally, the CDP command overrides navigator.webdriver at the JavaScript level, returning undefined instead of true.

Next, navigate to the Cars.com search URL. Since the page may present a Cloudflare challenge on first visit, we pause execution and wait for the user to solve it manually before continuing:

url = "https://www.cars.com/shopping/results/?makes[]=toyota&maximum_distance=all&models[]=&page=1&stock_type=used&zip="

driver.get(url)

# Pause to allow manual Cloudflare challenge completion
input("Press ENTER once the page is fully loaded and any captcha is solved...")

After you press Enter, the scraper reads the fully rendered page source and hands it off to BeautifulSoup for parsing.

Once you get the page HTML, parse it with BeautifulSoup and locate all <fuse-card> elements. Not every <fuse-card> on the page is a vehicle listing, so filter for cards that contain an <h2> heading with a link to a /vehicledetail/ URL:

from bs4 import BeautifulSoup
import json

html_content = driver.page_source
soup = BeautifulSoup(html_content, 'lxml')

cards = soup.find_all('fuse-card')
vehicle_cards = [
    c for c in cards
    if c.find('h2') and any('/vehicledetail/' in a.get('href', '') for a in c.find_all('a'))
]

For each vehicle card, extract the data points using CSS class selectors and the embedded JSON metadata:

for car in vehicle_cards:
    # Parse the embedded JSON metadata
    details_json = {}
    details_attr = car.get('data-vehicle-details')
    if details_attr:
        details_json = json.loads(details_attr)

    # Vehicle Name
    h2 = car.find('h2')
    name = h2.text.strip() if h2 else ""

    # Listing URL
    a_tag = h2.find('a') if h2 else None
    raw_href = a_tag['href'] if a_tag else ""
    url = raw_href if 'https' in raw_href else 'https://www.cars.com' + raw_href

    # Price (visible element, with JSON fallback)
    price_elem = car.find(class_='fuse-body-larger')
    price = price_elem.text.strip() if price_elem else None
    if not price and 'price' in details_json:
        price = f"${int(details_json['price']):,}"

    # Mileage (visible element, with JSON fallback)
    mileage_elem = car.find(class_='mileage')
    mileage = mileage_elem.text.strip() if mileage_elem else None
    if not mileage and 'mileage' in details_json:
        mileage = f"{int(details_json['mileage']):,} mi."

    # Stock Type
    stock_type = details_json.get('stockType', 'Unknown')

    # Dealer Name
    dealer_elem = car.find('span', class_='fuse-body-small')
    dealer = dealer_elem.text.strip() if dealer_elem else None

    # Dealer Rating
    rating_elem = car.find(class_='review-star')
    rating = rating_elem.text.strip() if rating_elem else None

    # Dealer Location
    location = None
    loc_svg = car.find('fuse-svg', attrs={'name': 'map-marker-outline'})
    if loc_svg and loc_svg.parent:
        location = loc_svg.parent.text.strip()

The embedded data-vehicle-details JSON acts as a reliable fallback. Some listings omit the visible mileage or price elements depending on the listing type (e.g., “Contact for Price” listings), but the JSON attribute almost always contains the raw numeric values.

Don’t want to code your own scraper? Contact us to build custom web scraping APIs for you.

Saving the Data

After extracting all listings, write the structured data to a JSON file:

with open('toyota_cars.json', 'w', encoding='utf-8') as f:
    json.dump(data, f, indent=4, ensure_ascii=False)

Here is the complete, runnable cars.com scraper. It accepts three command-line arguments: the car make, the number of pages to scrape, and the stock type.

import argparse
import json
from bs4 import BeautifulSoup
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC


def scrape_cars(search_term, max_pages, stock_type):
    url = (
        f"https://www.cars.com/shopping/results/?makes[]={search_term}"
        f"&maximum_distance=all&models[]=&page=1&stock_type={stock_type}&zip="
    )

    print("Launching Chrome...", flush=True)
    options = Options()
    options.add_argument('--disable-blink-features=AutomationControlled')
    options.add_experimental_option("excludeSwitches", ["enable-automation"])
    options.add_experimental_option('useAutomationExtension', False)

    try:
        driver = webdriver.Chrome(options=options)
        driver.execute_cdp_cmd("Page.addScriptToEvaluateOnNewDocument", {
            "source": "Object.defineProperty(navigator, 'webdriver', {get: () => undefined})"
        })

        print(f"Navigating to: {url}", flush=True)
        driver.get(url)

        input("\nPress ENTER once the page is loaded and any captcha is solved... ")

        print("Parsing listing cards...", flush=True)
        try:
            WebDriverWait(driver, 10).until(
                EC.presence_of_element_located((By.TAG_NAME, "fuse-card"))
            )
        except Exception:
            print("Warning: listing cards not found on the page.", flush=True)

        html_content = driver.page_source
        soup = BeautifulSoup(html_content, 'lxml')

        cards = soup.find_all('fuse-card')
        vehicle_cards = [
            c for c in cards
            if c.find('h2') and any(
                '/vehicledetail/' in a.get('href', '') for a in c.find_all('a')
            )
        ]
        print(f"Found {len(vehicle_cards)} vehicle listings.", flush=True)

        data = []
        for car in vehicle_cards:
            details_json = {}
            try:
                details_attr = car.get('data-vehicle-details')
                if details_attr:
                    details_json = json.loads(details_attr)
            except Exception:
                pass

            h2 = car.find('h2')
            name = h2.text.strip() if h2 else ""

            a_tag = h2.find('a') if h2 else None
            raw_href = a_tag['href'] if a_tag else ""
            href = raw_href if 'https' in raw_href else 'https://www.cars.com' + raw_href

            price_elem = car.find(class_='fuse-body-larger')
            price = price_elem.text.strip() if price_elem else None
            if not price and 'price' in details_json:
                price = (
                    f"${int(details_json['price']):,}"
                    if details_json['price'] != "0" else None
                )

            mileage = None
            mileage_elem = car.find(class_='mileage')
            if mileage_elem:
                mileage = mileage_elem.text.strip()
            elif 'mileage' in details_json:
                mileage = f"{int(details_json['mileage']):,} mi."

            stock_type_val = details_json.get('stockType')
            if not stock_type_val:
                if name.startswith('Used'):
                    stock_type_val = 'Used'
                elif name.startswith('New'):
                    stock_type_val = 'New'
                elif name.startswith('Certified'):
                    stock_type_val = 'Certified'
                else:
                    stock_type_val = 'Unknown'

            dealer_elem = car.find('span', class_='fuse-body-small')
            dealer = dealer_elem.text.strip() if dealer_elem else None

            rating = None
            rating_elem = car.find(class_='review-star')
            if rating_elem:
                rating = rating_elem.text.strip()

            location = None
            loc_svg = car.find('fuse-svg', attrs={'name': 'map-marker-outline'})
            if loc_svg and loc_svg.parent:
                location = loc_svg.parent.text.strip()

            data.append({
                "Name": name,
                "Price": price,
                "URL": href,
                "Mileage": mileage,
                "Stock Type": stock_type_val,
                "Dealer": {
                    "Name": dealer,
                    "Rating": rating,
                    "Location": location
                }
            })

        driver.quit()
        return data
    except Exception as e:
        print(f"Error: {e}", flush=True)
        return []

def main():
    parser = argparse.ArgumentParser()
    parser.add_argument('search', help='Car make (e.g., toyota, honda)')
    parser.add_argument('max_pages', help='Number of pages to scrape')
    parser.add_argument('stock_type', help='used, new, or cpo')
    args = parser.parse_args()

    car_data = scrape_cars(args.search, args.max_pages, args.stock_type)

    if car_data:
        filename = f"{args.search}_cars.json"
        print(f"Scraped {len(car_data)} listings.", flush=True)
        with open(filename, 'w', encoding='utf-8') as f:
            json.dump(car_data, f, indent=4, ensure_ascii=False)
        print(f"Saved to {filename}", flush=True)
    else:
        print("No data scraped.", flush=True)

if __name__ == "__main__":
    main()

Run the scraper from your terminal:

python scraper.py toyota 1 used

Chrome will open and navigate to the Cars.com search results page. If a Cloudflare challenge appears, solve it manually. Once the listings are visible, switch back to your terminal and press Enter. The scraper will parse all visible listings and save them to toyota_cars.json.

Sample Output

Here is what a typical entry in the output JSON looks like:

{
    "Name": "Used 2022 Toyota Camry SE",
    "Price": "$22,998",
    "URL": "https://www.cars.com/vehicledetail/abcd1234-5678-efgh/",
    "Mileage": "34,521 mi.",
    "Stock Type": "Used",
    "Dealer": {
        "Name": "AutoNation Toyota",
        "Rating": "4.7",
        "Location": "Libertyville, IL (31 mi)"
    }
}

Code Limitations

This cars.com scraper is a starting point. However, there are several limitations of this to keep in mind before using it for any production workload:

• Cloudflare Turnstile: Cars.com uses Cloudflare’s bot detection, which requires a manual captcha solution on the first page load. This makes fully automated, headless scraping unreliable without additional tooling.
• Selector fragility: Cars.com uses custom web components (<fuse-card>, <fuse-rating>, <fuse-svg>) that may change without notice. When the frontend is updated, the CSS selectors in this script will break and need manual adjustment.
• Single-page scope: This script only scrapes the first page of results. Adding pagination requires looping through the page query parameter and handling potential rate limiting.
• No detail page data: The scraper only collects data visible on the search results page. Vehicle specifications, full descriptions, dealer reviews, and photos require navigating to each listing’s detail page, which triggers additional Cloudflare challenges.
• Rate limiting and IP blocking: The code doesn’t use techniques, such as rotating proxies and request throttling to prevent getting blacklisted.

Why Use a Web Scraping Service

You can build a cars.com scraper that extracts vehicle listing data from search results using Python. Simply get the HTML source code using Selenium. Parse and extract data points using BeautifulSoup. And save the data to a JSON file.

This approach of automotive data scraping works on a small scale, but maintaining CSS selectors, handling captchas, and managing IP rotation adds significant overhead as your scraping needs grow. 

If you need Cars.com data without writing or maintaining code, contact ScrapeHero. Our web scraping service can build you high quality scrapers that handle anti-bot measures, pagination, and data formatting automatically. Get the data you need without any setup required.

Frequently Asked Questions