{"id":37302,"date":"2019-10-31T22:16:52","date_gmt":"2019-10-31T19:16:52","guid":{"rendered":"https:\/\/prohoster.info\/blog\/kak-reshit-problemy-sajtov-agregatorov-s-pomoshhyu-rezidentnyh-proksi\/"},"modified":"2019-10-31T22:16:52","modified_gmt":"2019-10-31T19:16:52","slug":"kak-reshit-problemy-sajtov-agregatorov-s-pomoshhyu-rezidentnyh-proksi","status":"publish","type":"post","link":"https:\/\/prohoster.info\/en\/blog\/administrirovanie\/kak-reshit-problemy-sajtov-agregatorov-s-pomoshhyu-rezidentnyh-proksi","title":{"rendered":"How to solve aggregator site problems using residential proxies","gt_translate_keys":[{"key":"rendered","format":"text"}]},"content":{"rendered":"

\"How<\/a><\/noindex><\/p>\n

Image: Pexels<\/a><\/noindex><\/i><\/p>\n

For aggregator sites in the e-commerce sector, it is crucial to maintain up-to-date information. Otherwise, their main advantage \u2013 the ability to see the most relevant data in one place \u2013 disappears. <\/p>\n

To address this issue, it is necessary to use web scraping techniques. The idea is to create special software \u2013 a crawler \u2013 that navigates through a list of required websites, parses information from them, and uploads it to the aggregator site.<\/p>\n

The problem is that often the website owners from whom aggregators collect data do not want to provide access so easily. This is understandable \u2013 if pricing information from an online store appears on an aggregator site and is higher than that of the competitors also listed there, the business will lose customers. <\/a><\/noindex><\/p>\n

Methods to combat scraping<\/h2>\n

\nTherefore, website owners often combat scraping \u2013 that is, the downloading of their data. They can identify requests sent by bot crawlers by IP address. Typically, such software uses so-called server IPs, which are easy to detect and block.<\/p>\n

Moreover, instead of blocking requests, another method is often employed \u2013 irrelevant information is shown to identified bots. For example, they may inflate or deflate product prices or alter their descriptions.<\/p>\n

A common example cited in this context is airline ticket prices. Indeed, airlines and travel agencies often display different results for the same flights depending on the IP address. A real case: searching for a flight from Miami to London on the same date using an IP address from Eastern Europe returns different results than an IP address from Asia. <\/p>\n

For the IP address from Eastern Europe, the price looks like this:<\/p>\n

\"How
\n
\nAnd for the IP address from Asia, it looks like this:<\/p>\n

\"How
\n
\nAs seen, the price for the same flight differs significantly \u2013 the difference is $76, which is quite substantial. For an aggregator site, nothing is worse than this \u2013 if incorrect information is presented, users will not utilize it. Additionally, if a specific product has one price on the aggregator and changes upon visiting the seller's site, this also negatively impacts the project's reputation.<\/p>\n

Solution: using residential proxies<\/h2>\n

\nAvoiding issues when scraping data for aggregation needs can be achieved by using residential proxies. Server IPs are provided by hosting providers. Identifying the ownership of an address by a specific provider is fairly simple \u2013 each IP has an ASN number that contains this information.<\/p>\n

There are numerous services for analyzing ASN numbers. Often, they are integrated with anti-bot systems that block access for crawlers or manipulate the responses to their requests. <\/p>\n

Residential IP addresses help bypass such systems. These IPs are assigned by internet providers to homeowners, with relevant markings in all associated databases. There are special residential proxy services that allow you to use residential addresses. Infatica<\/a><\/noindex> \u2013 this is exactly such a service.<\/p>\n

Requests sent by crawlers from residential IPs appear as if they are coming from regular users in a specific region. Regular visitors are not blocked \u2013 in the case of online stores, these are potential customers.<\/p>\n

As a result, using rotating proxies from Infatica allows aggregator websites to obtain guaranteed accurate data while avoiding blocks and parsing difficulties. <\/p>\n

Other articles on the topic of using residential proxies for business:
\n<\/h2>\n

<\/p>\n