This client had a need to get their booking payout data out of Airbnb on a recurring basis and then be able to access and query it freely using any particular method that the might decide to use in the future. The crux with this project is that Airbnb does not make their API available to all hosts. Why? …I really don’t know. As an Airbnb host myself, I find it frustrating as did the client. To complicate matters further, Airbnb actively monitors for logins from multiple locations from the same user and also automated logins (bots). All this to prevent programatic access – at least that’s the opinion we and the client formed. The client has many many short term rental properties which made manual gathering and import untenable.
The Ask
The ask was to create and automate a sort-of covert and randomized process that would login as a user, navigate the airbnb admin console, download the the needed data by selecting date ranges, etc and then land that data into a suitable data architecture in Snowflake. Following that, use dbt or or some other data modeling tool to demonstrate how they can work with the data in Snowflake in the near future.
What we did …in a nut shell
Using the Python Selenium driver and various techniques to randomize timing and more we were able to reliably pull all the data without ever getting blocked by Airbnb. From there we took advantage of Snowflakes native AW S3 integration to very securely and easily move data directly from S3 straight into Snowflake tables. dbt was used to orchestrate everything from the import into Snowflake from S3 to basic demo model building.
