Are you interested in scraping the internet? Is Proxy Crawl a good option? Read our full review of the service and see if it is the right service for your scraping needs.
Living in a digital age with tons of data does not mean that it is all junk. Even though most of the data on the internet today is either unusable or obsolete, there is a small part that can be used. Considering how much data there is, that little part is quite a lot, but that is an entirely different topic.
There are tons of applications for data scraping, just as there are tons and tons of scrapers out there. Even though all of them have the same function, most of them are unique. The uniqueness lies in the fine details when you use them for grabbing data from somewhere. On today’s review list, we have Proxy Crawl, where we are going to dive-in into its features, pricing, and usability.
** Official website : https://proxycrawl.com
Our Expert Rating and Test Results
Overall
9.1/10 9.1/10- Scraping Performance - 9.1/10 9.1/10
- Anonymity - 8.7/10 8.7/10
- Locations - 9/10 9/10
- Success Rate - 9.7/10 9.7/10
Proxy Crawl History
One thing that is somewhat confusing about this company is the fact that there is very little information on the company, but it has a lot of customers. Regarding the company, the only information available on the internet is that it was founded in 2017 in Spain. Their website says that the entire company is remote, and the team members work from all over the world, and there is no information on the team whatsoever.
On the other hand, they claim that they have over 27000 paying customers that crawled over 800 billion unique pages – these are some serious numbers. That means that there are 27000 individuals or companies that decided to pay despite the lack of information on the company or the team.
Products and Pricing
There may be very limited information on this company, but they make it up with the available services. A lot of ProxyCrawl’s competitors will offer one or two services leaving you with the need to look for other services elsewhere. These guys designed the service so that you have everything you need in one place. Good for them because they can earn more and good for you because you do not need to bother looking for additional providers, it is a win-win situation.
Another advantage of going to ProxyCrawl is the fact that all of these products are individual. It means that you can purchase only one product, or two or all of them, depending on what you need.
Earlier, we mentioned the uniqueness of data crawlers and scrapers and ProxyCrawl’s uniqueness is that it is an all in one service for anything data-related that you might need. Let us look at the products that they have to offer.
-
Crawling API
The first product in ProxyCrawl’s lineup is its crawling API. Similar to other APIs out there that can crawl websites, this one is easy to set up and use in your preferred language and framework.
One thing that makes it stand out is that unlike other APIs where you would have to set up quite a lot of things before the software starts to crawl, with ProxyCrawl’s crawling API, you can achieve that in a matter of minutes. All you need to do is call their API and destination website and let the magic happen.
During the crawling process, ProxyCrawl will use its own proxies so that you do not have to scour the internet for good proxies that you can use. On top of that, their artificial intelligence system can bypass any kinds of blocks and captchas so that your crawling process goes smoothly. What about dynamic pages?
No problem, you can crawl those as well. Their API can crawl through real browsers, which means that it can get data from sites that are not HTML. Javascript, Angular, Vue, React, and others will not be an obstacle.
Having all of this usually means high prices for the services. Typically, yes, but in this case, no. ProxyCrawl’s pricing structure is relatively simple – you pay for each successful request. The more requests you make, the cheaper each request is.
Due to the complexities, prices for regular pages are not the same as the ones with javascript. The costs for regular pages start from 0.3 cents per request if you make 1000 or less successful requests, and you will be charged 0.002 cents after you make over 1 billion successful requests.
There are a few other prices per request in between. If you need to crawl from javascript pages, then one request will cost you 0.6 cents if you make 1000 or less successful ones. After 1 billion successful requests, it will cost you 0.004 cents, and, same as the regular pages, there are a few other prices in between.
Regardless of how many requests you intend to make, there is no monthly commitment. When you sign up, the first 1000 requests are free, which means that you are basically getting a free trial and you can cancel the subscription without paying if you do not find the service usable.
-
Crawler
Do not let the name full you; this is not a crawler as you would expect. You might be thinking about a service where you paste in a URL, and the crawler does its job. This is not the job of this crawler. ProxyCrawl’s crawler is tasked with the push system, which works with callbacks.
When you crawl using the Crawling API, you will be sending the URLs to the Crawler, and it will return the page to your server. This process can be achieved by setting up a webhook URL on your server, and with that, you will be able to get the crawled data.
Since this service works alongside the crawling API, the pricing is the same as for the Crawling API. But do not be confused, you will not be paying double, for each successful request you will be charged just as their website says.
-
Backconnect Proxy
This product is something that will confuse most people. ProxyCrawl also sells proxies, which is what makes it an excellent service for multiple needs. Some might find that their crawlers are not working per your requirements, so you can still be a customer by buying proxies that you can use with another crawler or scraper.
Unlike other dedicated proxy providers, ProxyCrawl does not disclose the number of available proxies or their locations. From what we can tell, there are over 1 million proxies in over 100 locations. They just say that they offer worldwide proxies, and you have a datacenter, residential, and mobile proxies. By default, all proxies are set up to rotate on each request, making them excellent for scraping with minimal input on your end. All you would need to do is whitelist your local IP or the one on your server and start getting data. Alternatively, you can also set things up to use a static IP address.
For this product, you have the option of choosing between three pricing plans and a custom one. All plans come with unlimited bandwidth, and the difference between is the number of threads, unique proxies, IP rotation, locations, types of proxies, and a few other features. For the cheapest one, you would get the least amount of proxies, IP rotation, and threads with some features unavailable, while if you go for the most expensive one, you will get everything loaded to the max. Alternatively, you can reach their sales department and ask for a custom plan made specifically for your needs.
There is no trial here, but you can get a refund if you ask for one in less than 24 hours of making the payment. It is not a lot of time, but it should be enough for you to test out the proxies.
-
Cloud Storage
Scraping od crawling data always means that you will need a place to store it. In most cases, people would have it saved locally on their computers or on the server where the crawler was running, and usually, that is fine, but not always. On a small scale, you should not run into any problems, but once you start getting more and more data, you will have more considerable difficulties maintaining it or sifting through it if you need something specific.
ProxyCrawl’s cloud storage is an excellent solution for that. First of all, it is an excellent solution if you are using their crawler so you can automatically save the crawled data to their cloud storage. Even if you do not use their crawler, you can still pay to have your crawled data stored on their servers.
The pricing for this product is weird. They do not have a pricing plan like most other products. Instead, they claim that the service is free to use for developers, and companies will pay a small fee. Additionally, if you need a lot of storage space for over 10 thousand pages monthly, you will need to reach out to support to see if that would be possible.
-
Scraping API
As we mentioned at the beginning of this article, crawling and scraping are not the same thing. With that in mind, ProxyCrawl is offering its scraping API as a separate product from its crawling API. We believe that this is the product that shines.
A scraping API is not something that we have not seen before, but this one is unique. The API works just as any other would, it will parse, scrape, and store. The advantage is that this scraper has scrapers dedicated to specific sites. The list of sites is Amazon, Google, Facebook, Twitter, Instagram, Linkedin, Quora, eBay, and Aliexpress. Apart from these, you can use their generic scraper for all other websites.
For their scraping API, you can choose from three pricing plans. The bandwidth is unlimited for all of them, but they differ in the available credit, geolocation, JS rendering type of support, and a few other features. You can give the scraping API a test run with 1000 requests which should be more than enough to see if you can make it work. What is weird is the fact that they do not mention anything about bigger or custom packages.
-
Screenshots API
Sometimes the data you scrape in a textual form might not be enough, so you might need to take screenshots. ProxyCrawl has got you covered. Their screenshots API will let you track visual changes on various websites with ease.
Some might think that taking screenshots from websites is not worth paying for, and that is true if you need to take a couple of screenshots. Of you need several hundred or thousand, then you will have a difficult time doing it by hand. ProxyCrawl’s screenshot API will be able to take screenshots from pages just as a human would, but a lot faster.
It can take a screenshot from a page with various resolutions, covering the mobile sizes as well, it will also scroll down and take a full shot of an entire page, and it can even avoid any kind of blocks such as captchas. All images are saved as JPEG, and depending on the package, you can have them saved directly to their cloud storage.
Package-wise, you get three options, and they all work well for what people might need. The cheapest package is relatively limited; it will not take mobile screenshots or scroll an entire page, nor will you be able to save them to their cloud storage. If you need all of these features, you can opt-in for the other two packages, where the most expensive one will provide you access to ProxyCrawl’s cloud storage and premium support.
As with almost all other products, you get 50 free screenshots when you signup.
-
Leads API
The last product on the list is the leads API. It is a bit confusing to have a product for scraping leads, but keep in mind that data scraping is often used for generating leads. The great thing about using the leads API is the fact that you will not be getting email addresses from a database.
Their API will get you fresh and updated email addresses, so you avoid the need to verify if they work. All you will need to do is add the URL of the website where you want to get the emails from. In addition to that, their artificial intelligence will try to guess which is the best person to reach out to.
As with the other products, you have three pricing packages to choose from. All are limited by the number of credits that you get each month. The cheapest package is the most limited one, where you do not get extended lookups or premium support. For extended lookups, you need to get the middle package, and if you want premium support, then you need the most expensive one. For the leads API, you also get a trial where the first 100 leads are free. They do not mention anything about a custom plan if you need more leads than what the most expensive package offers.
Ease of Use
The ease of use is a double-edged sword. It is easy and straightforward, but a bit technical at the same time. When you log in to the dashboard in front of you, all products are available, and you navigate through them as with any other simple menu. Within them is where things get technical. Unlike some crawlers or scrapers where you can get the data with the point and click technique in a user interface when using ProxyCrawl’s APIs, you will need to do it in a terminal. This is expected because, after all, they offer APIs.
To put it simply, if you want to use their products and services, you will need to have some technical knowledge to be able to get the most out of them. For the beginners out there, it will be a lot more complicated.
Customer Support
This is where ProxyCrawl excels compared to some of its competitors. In a lot of cases, we say that not having phone support is negative, and the same applies to this case. The excel part is in regards to their documentation and email support.
Let us start with the documentation. When you set up your account and log in to your dashboard, the first thing you do is choose the product that you want to use. When you do, the service will instantly redirect you to the documentation page for that product.
One we go to that part, we found that it is probably the most extensive documentation of a product that we have seen with most of their competitors. On top of that, if you have any problems or questions about something not outlined in the documentation, you can reach out through the contact form. This is indeed a much slower process than calling them on the phone, but based on a lot of feedback from users, their support is superb.
Our verdict – Is ProxyCrawl recommended?
The internet is crawling with tons of scrapers and crawlers. Having that big of choice will often lead to choosing the wrong one or the one that does not work as advertised. That is not the case with ProxyCrawl.
For starters, they offer a wide variety of services related to scraping and crawling. On top of that, the prices are well below average, making them a “cheap” choice. Cheap usually means that there is something wrong, but in the case of ProxyCrawl, that is not true.
During our testing, we did not find anything not working or having difficulties with their services. Speaking of difficulties, if you decide to get some of the products from them, the documentation section will be more than enough to get you started. If that is not enough, the support is there to provide additional assistance.
In general, ProxyCrawl is an excellent choice for individuals or companies that need scraping or crawling on any scale. There are plenty of packages which will not cost you an arm and a leg. Bear in mind, though, if you want to purchase something from them, you will need to have the technical know-how to be able to use it as intended.