This is really weird because I just scraped like 1800 URLs that found 20X that number of internal links. All of the links are in the TXT file
Then I try to scrape ONE URL, https://www.zipcodestogo.com/ZIP-Codes-by-State.htm, and it finds 70 internal links but when I open the TXT file, there’s nothing there.
I also disabled my proxies (they were getting a “socket error # 10054”) and then I tried to scrape the homepage of The Onion as a test.
Same problem, it says that 33 internal links were found but nothing saved to the file.
And I tried using my three ProxyMesh proxies, my StormProxies that I just bought today, and no proxies. Every time it says it’s saved but the file is empty.
Also, is there a way to save scraped data with a more descriptive name? It’s really difficult to find files when the file name is just a long string of numbers.