Components of total protection

About web-scraping

Unique information is a target for competitors, intermediaries and intruders. The databases of online services for cartography and navigation systems, product characteristics in internet shops as well as advertisements on digital trading platforms are particularly popular. However, manual copying of content is costly, exhausting and unproductive. The automated download (exfiltration) systems of content are developed and used to save time and resources. The specifically configured software performs website crawling – it automatically downloads the web pages, analyses the content, finds the links to other sections and recursively copies the entire content of the website.

Problems generated by web-scraping :

• Attacked websites are deprived of unique content and lose their positions in the organic search engine ranking;
• Automatic correction of stolen texts makes it difficult to find duplicates even with the help of search engines;
• Automatic downloading of data creates a serious parasitic load; it disrupts the stable operation of the website and may lead to a denial of service to legitimate visitors;
• An automatically created copy of the website can be used for phishing attacks when by using authorization forms, intruders steal en masse users’ account data.

In its turn, legal methods of protection against web-scraping have limited abilities.

Copying of information from public sources is not as legally protected as copying and use of personal data by the Directive 95/46/EC is. For that reason, technological methods of protection of information from copying are also in demand.

With the help of Active Bot Protection, the Systron platform can filter out relevant requests, preventing unauthorized copying of content and reducing the load on the client’s infrastructure.

Smart migration of web scrappers on pages of goods in e-store exfiltrating current price list

Active Bot protection as a fraud preventing solution

About click fraud

Click fraud refers to one of the types of online scams. Interested parties, such as competitors or intruders, make false clicks on advertising links (Pay Per Click) and spend the advertising budget. There are manual and automatic click frauds, the latter occurs, when actions of clickers are imitated through special programs.

Examples of click fraud can be as follows :

• Technical clicks – the search engines follow the links for indexing the website’s content;
• Clicks from advertisers – the advertisers click on their own contextual ads to increase their click-through-rate (CTR);
• Clicks from competitors – the competitors follow the links from contextual ads for reducing the advertising budget;
• Clicks from unscrupulous webmasters – the users and automated programs follow the links to commercial ads to increase the profitability of their own advertising platforms;
• Fraudulent clicks – network fraudsters alter the results of online voting systems and increase the counters on the website.

Protection from click fraud is required for :

• Advertising and marketing agencies – the filtering of false clicks will help to collect reliable marketing information and reduce the reputational risks;
• Advertising companies – the protection of ads from clickers saves the budget in the context-based advertising;
• Website owners – the protection from automatic increase of votes guarantees exact results of voting systems on the website.

Bot traffic may reach 80-90% out of total infrastructure capacity or/and application perfomance :

Clickfraud e-commerce adds :

Prevention from periodical web-scraping of client’s media resource :

API definition

According to the Wikipedia: “In computer programming, an application programming interface (API) is a set of subroutine definitions, protocols, and tools for building application software. In general terms, it is a set of clearly defined methods of communication between various software components.”

As we see rapid development of mobile Internet access networks and the growth of the number of smartphones, many companies whose business is connected to the Internet, are developing and supporting their own mobile applications.

This allows them to make their services accessible to as many users as possible. A home computer is no longer required to purchase services or to consume content.

Most mobile applications use a client-server architecture where the application is installed on the user’s device and the service content is delivered via a centralized API.

Major threats to APIs :

As all mobile services are interested in reaching their maximum audience, users should be able to use the application around the world. An API is often a great resource and therefore attracts hackers as well as legitimate users. A weakness of APIs is that they can be subject to service failure attacks. This type of attack makes it impossible for the user to access the mobile application and faced by the fact that the service is unavailable, forces them to turn to alternative options. There are many ways to address these types of attacks. However, blocking the offending addresses is usually not the best method. The historical approach of blocking IP addresses could lead to restricting access for legitimate users of the application on the same shared WIFI or public access point at a hotel, airport or train. This leads to a negative impact on brand and revenue.

Another type of abuse of mobile and public APIs is automated scraping and/or publishing of data to the service by bot accounts.

An optimal solution could be systron’s Active Bot Protection (ABP), which is able to filter-out non-legitimate requests. The application backend will receive web-traffic that is cleaned from undesirable requests. ABP is based on a multifactorial analysis of each Internet session. The method includes both statistical and technical data analysis. Apart from these, Systron uses behavioral analysis which enables to distinguish bots and humans with a high level of confidence.

What Systron recommends :

API protection service extends systron’s Active Bot Protection to all applications including mobile, mitigating low-frequency bot attacks that are normally missed.

With systron’s Always-On API protection, companies notice a decrease in load and increase in capacity due to filtering unwanted bot traffic.

Detection of password bruteforce attacks Human traffic is variable / Bot traffic is constant

Smart detection and mitigation of DDoS attacks not only on infrastructure or web but also on mobile application APIs