forked from AnthonyBloomer/daftlistings
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge pull request AnthonyBloomer#92 from AnthonyBloomer/remove-v1-te…
…sts-examples Remove v1 tests & examples
- Loading branch information
Showing
28 changed files
with
1 addition
and
1,141 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,5 +1,6 @@ | ||
# Daftlistings | ||
|
||
|
||
[![Build Status](https://travis-ci.org/AnthonyBloomer/daftlistings.svg?branch=dev)](https://travis-ci.org/AnthonyBloomer/daftlistings) | ||
[![codecov](https://codecov.io/gh/AnthonyBloomer/daftlistings/branch/master/graph/badge.svg)](https://codecov.io/gh/AnthonyBloomer/daftlistings) | ||
|
||
|
@@ -60,215 +61,6 @@ for listing in listings: | |
print(listing.price) | ||
``` | ||
|
||
By default, the Daft `search` function iterates over each page of results and appends each Listing object to the array that is returned. If you wish to disable this feature, you can set `fetch_all` to `False`: | ||
|
||
``` python | ||
daft.search(fetch_all=False) | ||
``` | ||
|
||
|
||
### Examples | ||
|
||
Get apartments to let in Dublin City that are between €1000 and €1500 and contact the advertiser of each listing. | ||
|
||
``` python | ||
from daftlistings import Daft, RentType | ||
|
||
daft = Daft() | ||
|
||
daft.set_county("Dublin City") | ||
daft.set_listing_type(RentType.APARTMENTS) | ||
daft.set_min_price(1000) | ||
daft.set_max_price(1500) | ||
|
||
listings = daft.search() | ||
|
||
for listing in listings: | ||
|
||
contact = listing.contact_advertiser( | ||
name="Jane Doe", | ||
contact_number="019202222", | ||
email="[email protected]", | ||
message="Hi, I seen your listing on daft.ie and I would like to schedule a viewing." | ||
) | ||
|
||
if contact: | ||
print("Advertiser contacted") | ||
``` | ||
|
||
You can sort the listings by price, distance, upcoming viewing or date using the SortType object. The SortOrder object allows you to sort the listings descending or ascending. | ||
|
||
``` python | ||
|
||
from daftlistings import Daft, SortOrder, SortType, RentType | ||
|
||
daft = Daft() | ||
|
||
daft.set_county("Dublin City") | ||
daft.set_listing_type(RentType.ANY) | ||
daft.set_sort_order(SortOrder.ASCENDING) | ||
daft.set_sort_by(SortType.PRICE) | ||
daft.set_max_price(2500) | ||
|
||
listings = daft.search() | ||
|
||
for listing in listings: | ||
print(listing.formalised_address) | ||
print(listing.daft_link) | ||
print(listing.price) | ||
features = listing.features | ||
if features is not None: | ||
print('Features: ') | ||
for feature in features: | ||
print(feature) | ||
print("") | ||
|
||
``` | ||
|
||
Parse listing data from a given search result url. | ||
|
||
``` python | ||
|
||
from daftlistings import Daft | ||
|
||
daft = Daft() | ||
daft.set_result_url("https://www.daft.ie/dublin/apartments-for-rent?") | ||
listings = daft.search() | ||
|
||
for listing in listings: | ||
print(listing.formalised_address) | ||
print(listing.price) | ||
print(' ') | ||
|
||
|
||
``` | ||
|
||
Find student accommodation near UCD that is between 850 and 1000 per month | ||
|
||
``` python | ||
from daftlistings import Daft, SortOrder, SortType, RentType, University, StudentAccommodationType | ||
|
||
daft = Daft() | ||
daft.set_listing_type(RentType.STUDENT_ACCOMMODATION) | ||
daft.set_university(University.UCD) | ||
daft.set_student_accommodation_type(StudentAccommodationType.ROOMS_TO_SHARE) | ||
daft.set_min_price(850) | ||
daft.set_max_price(1000) | ||
daft.set_sort_by(SortType.PRICE) | ||
daft.set_sort_order(SortOrder.ASCENDING) | ||
daft.set_offset(offset) | ||
listings = daft.search() | ||
|
||
for listing in listings: | ||
print(listing.price) | ||
print(listing.formalised_address) | ||
print(listing.daft_link) | ||
|
||
``` | ||
|
||
|
||
|
||
Map the 2-bed rentling properties in Dublin and color code them wrt to prices. | ||
Save the map in a html file. | ||
|
||
``` python | ||
from daftlistings import Daft, SortOrder, SortType, RentType, MapVisualization | ||
import pandas as pd | ||
|
||
daft = Daft() | ||
daft.set_county("Dublin City") | ||
daft.set_listing_type(RentType.ANY) | ||
daft.set_sort_order(SortOrder.ASCENDING) | ||
daft.set_sort_by(SortType.PRICE) | ||
# must sort by price in asending order, MapVisualization class will take care of the weekly/monthly value mess | ||
daft.set_max_price(2400) | ||
daft.set_min_beds(2) | ||
daft.set_max_beds(2) | ||
|
||
listings = daft.search() | ||
properties = [] | ||
print("Translating {} listing object into json, it will take a few minutes".format(str(len(listings)))) | ||
print("Ignore the error message") | ||
for listing in listings: | ||
try: | ||
if listing.search_type != 'rental': | ||
continue | ||
properties.append(listing.as_dict_for_mapping()) | ||
except: | ||
continue | ||
|
||
|
||
df = pd.DataFrame(properties) | ||
print(df) | ||
|
||
dublin_map = MapVisualization(df) | ||
dublin_map.add_markers() | ||
dublin_map.add_colorbar() | ||
dublin_map.save("dublin_apartment_to_rent_2_bed_price_map.html") | ||
print("Done, please checkout the html file") | ||
|
||
``` | ||
|
||
|
||
|
||
|
||
For more examples, check the [Examples folder](https://github.com/AnthonyBloomer/daftlistings/tree/dev/examples) | ||
|
||
### Parallel as_dict() | ||
|
||
lisitng.as_dict() is relatively slow for large volume of listings. Below is an exmple script using threading and joblib library technique to speedup this process | ||
|
||
``` python | ||
from daftlistings import Daft, RentType | ||
from joblib import Parallel, delayed | ||
import time | ||
|
||
def translate_listing_to_json(listing): | ||
try: | ||
if listing.search_type != 'rental': | ||
return None | ||
return listing.as_dict_for_mapping() | ||
except: | ||
return None | ||
|
||
daft = Daft() | ||
daft.set_county("Dublin City") | ||
daft.set_listing_type(RentType.ANY) | ||
daft.set_max_price(2000) | ||
daft.set_min_beds(2) | ||
daft.set_max_beds(2) | ||
|
||
listings = daft.search() | ||
properties = [] | ||
print("Translating {} listing object into json, it will take a few minutes".format(str(len(listings)))) | ||
print("Ignore the error message") | ||
|
||
# time the translation | ||
start = time.time() | ||
properties = Parallel(n_jobs=6, prefer="threads")(delayed(translate_listing_to_json)(listing) for listing in listings) | ||
properties = [p for p in properties if p is not None] # remove the None | ||
end = time.time() | ||
print("Time for json translations {}s".format(end-start)) | ||
|
||
``` | ||
|
||
Table of perfomance speedup for 501 listings | ||
Threads | Time (s) | Speedup | ||
------------ | ------------- | ------------- | ||
1 | 178 | 1.0 | ||
2 | 101 | 1.8 | ||
3 | 72 | 2.5 | ||
4 | 61 | 2.9 | ||
6 | 54 | 3.3 | ||
|
||
## Tests | ||
|
||
The Python unittest module contains its own test discovery function, which you can run from the command line: | ||
|
||
``` | ||
python -m unittest discover tests/ | ||
``` | ||
|
||
## Contributing | ||
|
||
- Fork the project and clone locally. | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,4 +1,3 @@ | ||
from .daft import * | ||
from .enums import * | ||
from .exceptions import DaftException | ||
from .map_visualization import * |
This file was deleted.
Oops, something went wrong.
This file was deleted.
Oops, something went wrong.
This file was deleted.
Oops, something went wrong.
This file was deleted.
Oops, something went wrong.
Oops, something went wrong.