Home -> Projects -> Craigslist Bot

---

Craigslist Bot

Have you ever searched for an apartment on Craigslist? Did you love constantly refreshing your six different search pages to see if there are new entries and if the new entries are actually new apartments? Me neither, but it’s the best way to find an apartment before anyone else.

…or instead of checking you could just write a script to check for you! Here are the specs I came up with for my script:

• input: a list of search URLs to check
• the bot should check them regularly (I chose once per hour and later bumped it to once every 6 hours)
• the bot should filter out duplicate listings
  • I started by filtering out titles that matched exactly and then expanded it to also check to see if the images matched another posting. If either check found a duplicate I classified the listing as a duplicate
• notify me by email of each new listing
  • I ran this script on a server at MIT, so this only involved connecting to a local SMTP server. If you don’t have a local SMTP server this may not work for you

Results

I found an apartment! More notably, though, I found that my bot produced better results than all the real estate agents I met in the process. The only apartments agents told me about that I hadn’t already seen from my bot were outside my search parameters (normally too expensive). Bots >> agents.

Code on Github

https://github.com/dakaufma/craigslist_bot