avatar
alexstarr.bsky.social @alexstarr.bsky.social

Both websites have a closed API that needs payment to access. Which makes it difficult to collect their properties' info. They also both have google maps UI for their listings which dynamically load/reload the page, making UI automation interaction very inconsistent to do. It's doable, but not easy😞

jan 14, 2025, 12:47 am • 4 0

Replies

avatar
Dayvie @dayvie.bsky.social

you can use something like ScrappingBee to load as a real device to scrape the URLs from the DOM after render. Lots of options these days!

jan 14, 2025, 4:22 pm • 1 0 • view
avatar
giles @giles.fun

closed api = we want web scrapers please hurt us

jan 14, 2025, 1:49 am • 5 0 • view