How to Scrape Zillow Real Estate Listings using Python and LXML

Excellent tutorial! Is it possible to have the code search individual properties? For example if I had a list of 600 addresses in excel, could we perform the scrape in this article on each of these properties? Can it be modified to use the specific address as a dynamic user input into the website and then automatically return each property in order?

Thank you in advance for your time.

Great script! And very easy to use. However, it only reaches the first page of the Zillow results — in other words, 25 houses. Is there a way to get it to pull all houses available in the zipcode searched?

Nice article, it seems zillow will only show 500 (25/page * 20 pages) data, in order to fetch all datas in some super hot zipcodes. We might need to add filters to the search which I think price range (0-10k, 10k-20k) might be a useful attribute to narrow down the data size. The problem with price range is that it hugely depend on the area, by which I mean if some area only have houses price from 3m – 4m we will go through a bunch of unnecessary requests and this will increase the probability to get blocked. What do you think is the best attribute to limit results within 500?

Ran the script, but it gets detected and Zillow is passing back a captcha now. “Please verify you’re a human to continue”

Awesome article. When attempting to run it on the agents page it returns empty string using multiple classes. In example this page: https://www.zillow.com/malibu-ca/real-estate-agent-reviews/?sortBy=None&page=3&showAdvancedItems=False&regionID=12520&locationText=Malibu%20CA

I see there is another head and body loading at the end of the page is this a trick to prevent it from being scraped? Do you have any work around to this?

Nice scraping logic. A few tweaks and you can use the code on other real estate sites. Thanks!

If scraping Zillow page, does output CSV capture the property feature codes (eg a RESO compliant set of listing features and description)

just dont get it. why my csv was empty but fieldnames.

I’m also getting empty csv

hi, same here, can’t get any csv output beyond the defined headers!

what to do?

thanks,
kurt

Would anyone be able to assist me? I am experiencing a ‘invalid syntax’ error. I have the script saved in the script folder in Python. Using Windows 10, Python 3.7.0

You can just use Zillow APIs – they are free

i tried this code … am not getting any error message but output is not get…
when am running in cmd Its showing only path..

C:\Users\Dell\Desktop\Python1>zillow.py 20005 newest

C:\Users\Dell\Desktop\Python1>

again am running:::

C:\Users\Dell\Desktop\Python1>zillow.py 20005 newest

C:\Users\Dell\Desktop\Python1>

And how legal is it to scrape from Zillow that sources data from other MLSs? Also to what scale can one do this? Nationwide?

Will i be able to use this code to gather First and Last name as well as phone numbers and Emails?

I am receiving this error:

File “C:\Users\smith\Zillow.py”, line 2, in
from lxml import html
File “C:\ProgramData\Anaconda3\lib\site-packages\lxml\html__init__.py”, line 54, in
from .. import etree
ImportError: DLL load failed: The specified module could not be found.

I installed the lxml. Any ideas on why this is happening?

Thanks!

Nice job putting this together.

I was able to modify your code to do a few more cool things. I used regex to extract the zillow property ID’s from the URL. Then I registered for a ZWSID Zillow API Key and used that in conjunction with the property ID to run API queries against properties. I am able to get things like the Zestimate and rent estimate.

I’m running the program, but the csv file returns blank and the command line indicates that only 200 requests were made. Do you know of a solution to this issue?

Thanks,
Vish

excellent code …

i got some issue
AFTER RUNNING MULTIPLE TIMES “python zillow.py 20850 newest”

first try :

python zillow.py 20850 newest
Fetching data for 20850
https://www.zillow.com/homes/for_sale/20850/0_singlestory/days_sort
status code received: 200
Traceback (most recent call last):
File “zillow.py”, line 187, in
scraped_data = parse(zipcode, sort)
File “zillow.py”, line 118, in parse
response = get_response(url)
File “zillow.py”, line 69, in get_response
save_to_file(response)
File “zillow.py”, line 44, in save_to_file
fp.write(response.text)
File “C:\Program Files\Python37\lib\encodings\cp1252.py”, line 19, in encode
return codecs.charmap_encode(input,self.errors,encoding_table)[0]
UnicodeEncodeError: ‘charmap’ codec can’t encode character ‘\u0100’ in position 7647: character maps to

second try :

Fetching data for 20850
https://www.zillow.com/homes/for_sale/20850/0_singlestory/days_sort
status code received: 200
parsing from html page
Writing data to output file

and it happens randomly !!!

i know its because of coding and encoding but i dono how to solve it…
and where is the problem…

Is it normal to get sent to a Captcha when running this code?

Anyone else having this issue when running the code?

status code received: 200

Traceback (most recent call last):
File “C:/Users/matto/PycharmProjects/Real_Estate_Scraping/zillow.py”, line 185, in
scraped_data = parse(zipcode, sort)
File “C:/Users/matto/PycharmProjects/Real_Estate_Scraping/zillow.py”, line 129, in parse
return get_data_from_json(raw_json_data)
File “C:/Users/matto/PycharmProjects/Real_Estate_Scraping/zillow.py”, line 74, in get_data_from_json
cleaned_data = clean(raw_json_data).replace(‘“, “”)
AttributeError: ‘NoneType’ object has no attribute ‘replace’

I ended up installing tesseract to handle Captcha’s and reran zillow.py. Still no luck

Did anyone figure out how to do this?

Follow the advice of “Xiyu-1 commented on Mar 7” from the git site “https://gist.github.com/scrapehero/5f51f344d68cf2c022eb2d23a2f1cf95”

Here Xiyu describes how the script needs to be modified to return the results and complete creation of the csv file.

Cheers,

-diytechy