{"id":53531,"date":"2019-12-04T00:00:00","date_gmt":"2019-12-03T21:00:00","guid":{"rendered":"https:\/\/prohoster.info\/blog\/blog_prohoster\/ishhem-anomalii-i-predskazyvaem-sboi-s-pomoshhyu-nejrosetej"},"modified":"2020-02-18T14:01:26","modified_gmt":"2020-02-18T11:01:26","slug":"ishhem-anomalii-i-predskazyvaem-sboi-s-pomoshhyu-nejrosetej","status":"publish","type":"post","link":"https:\/\/prohoster.info\/en\/blog\/administrirovanie\/ishhem-anomalii-i-predskazyvaem-sboi-s-pomoshhyu-nejrosetej","title":{"rendered":"Searching for anomalies and predicting failures using neural networks","gt_translate_keys":[{"key":"rendered","format":"text"}]},"content":{"rendered":"

\"Searching<\/p>\n

<\/p>\n

Industrial software system development requires great attention to the reliability of the final product, as well as rapid response to failures and breakdowns when they occur. Monitoring certainly helps in responding to failures and crashes more effectively and quickly, but it's not enough. Firstly, it's very difficult to keep track of a large number of servers \u2013 it requires a significant number of people. Secondly, one needs to have a good understanding of how the application is structured to predict its status. Consequently, a lot of knowledgeable people are needed who understand the systems we are developing, their metrics, and peculiarities. Even if we can find enough people willing to do this, it still takes considerable time to train them.<\/p>\n

<\/p>\n

What should we do? Here, artificial intelligence comes to our aid. This article will discuss predictive maintenance<\/a><\/noindex> (predictive maintenance). This approach is gaining popularity. A large number of articles have been written about it, including on Habr. Large companies actively use this approach to support the functionality of their servers. After studying a significant number of articles, we decided to try applying this approach. What came of it? <\/p>\n

<\/a><\/noindex><\/p>\n

Introduction<\/h2>\n

<\/p>\n

A software system, sooner or later, goes into operation. It's important for the user that the system functions without failures. If an exceptional situation does occur, it should be resolved with minimal delays. <\/p>\n

<\/p>\n

To simplify the technical support of a software system, especially when there are many servers, monitoring programs are typically used. These programs collect metrics from the running software system, allow diagnosing its state, and help identify what caused the failure. This process is called software system monitoring.<\/p>\n

\n\"Searching<\/p>\n

Figure 1. Grafana monitoring interface<\/em><\/p>\n

Metrics are various indicators of a software system, the environment in which it runs, or the physical computing machine under which the system is running, marked with a timestamp indicating when the metrics were obtained. In static analysis, the metric data is referred to as time series. To observe the state of the software system, metrics are displayed in the form of graphs: time on the X-axis and values on the Y-axis (Figure 1). A functioning software system can generate several thousand metrics (from each node). They form a space of metrics (multidimensional time series). <\/p>\n

<\/p>\n

Since complex software systems generate a large number of metrics, manual monitoring becomes a challenging task. To reduce the volume of data analyzed by administrators, monitoring tools include features for automatically identifying potential issues. For instance, a trigger can be set to activate when free disk space decreases below a specified threshold. It is also possible to automatically diagnose server stoppage or critical slowdowns in service speed. In practice, monitoring tools perform well in detecting already occurred failures or identifying simple symptoms of future failures, but overall, predicting potential failures remains a tough nut to crack. Prediction through manual analysis of metrics requires the involvement of qualified specialists and is low-productivity. Most potential failures may go unnoticed.<\/p>\n

<\/p>\n

Recently, among large IT companies in software development, the so-called predictive maintenance of software systems has gained increasing popularity. The essence of this approach lies in identifying faults that lead to system degradation at early stages, before its failure, using artificial intelligence. This approach does not completely eliminate manual system monitoring; rather, it serves as an auxiliary process for monitoring as a whole. <\/p>\n

<\/p>\n

The main tool for implementing predictive maintenance is the task of anomaly detection in time series, as when an anomaly arises<\/strong> in the data, there is a high probability that after some time a failure or breakdown may occur<\/strong>. An anomaly is a deviation in the metrics of a software system, such as a degradation in the response time of a particular type of request or a decrease in the average number of handled requests while the number of client sessions remains constant.<\/p>\n

<\/p>\n

The task of anomaly detection in software systems has its own specifics. Ideally, for each software system, it is necessary to develop or refine existing methods, as the detection of anomalies heavily depends on the data in which it is performed, and the data of software systems varies significantly depending on the tools used to implement the system, right down to the computing machinery it runs on.<\/p>\n

<\/p>\n

Methods of anomaly detection in predicting failures of software systems<\/h2>\n

<\/p>\n

First of all, it should be noted that the idea of predicting failures was inspired by the article \u201cMachine Learning in IT Monitoring\u201d<\/a><\/noindex>. To test the effectiveness of the automated anomaly detection approach, the software system 'Web Consolidation' was chosen, which is one of the projects of NPO 'Krista'. Previously, manual monitoring was performed based on obtained metrics. Since the system is quite complex, a large number of metrics are collected for it: JVM metrics (garbage collector load), metrics of the operating system under which the code is executed (virtual memory, CPU load %), network metrics (network load), server metrics (CPU load, memory), WildFly metrics, and the application's own metrics across all critical subsystems. <\/p>\n

<\/p>\n

All metrics are collected from the system using Graphite. Initially, a Whisper database was used as the standard solution for Grafana, but with the growth of the client base, Graphite stopped coping, having exhausted the throughput capacity of the data center's disk subsystem. After this, the decision was made to search for a more efficient solution. The choice was made in favor of Graphite+ClickHouse<\/a><\/noindex>, which significantly reduced the load on the disk subsystem and cut the disk volume occupied by five to six times. Below is a diagram of the metric collection mechanism using Graphite+ClickHouse (Figure 2).<\/p>\n

\n\"Searching
\n<\/p>\n

Figure 2. Metric Collection Diagram<\/em><\/p>\n

The diagram is taken from internal documentation. It illustrates the data exchange between Grafana (the user interface for monitoring that we use) and Graphite. Metrics from the application are collected by a separate software \u2013 jmxtrans<\/a><\/noindex>. It also stores them in Graphite.
\nThe 'Web Consolidation' system has several features that create challenges for failure prediction:<\/p>\n

<\/p>\n

    \n
  1. trends often change. Various versions of this software system are released, each carrying changes in the system's programming. Consequently, developers directly influence the metrics of this system, which may lead to a shift in trends; <\/li>\n
  2. implementation features, as well as the goals for which clients use this system, often invoke anomalies without prior degradation; <\/li>\n
  3. the percentage of anomalies relative to the entire dataset is low (< 5%); <\/li>\n
  4. gaps may occur in receiving indicators from the system. For short periods, the monitoring system may fail to receive metrics, for instance, if the server is overloaded. This is critical for training the neural network. There is a need to fill these gaps synthetically;<\/li>\n
  5. Cases of anomalies are often relevant only for a specific date\/month\/time (seasonality). This system has a clear usage policy for its users. Consequently, metrics are only relevant for specific times. The system may not be used continuously but only during certain months: selectively depending on the year. There are situations where the same metric behavior may lead to the failure of the software system in one case and not in another.
    \nInitially, methods for detecting anomalies in monitoring data of software systems were analyzed. In the literature on this topic, when there are low percentages of anomalies relative to the rest of the dataset, neural networks are most often recommended. <\/li>\n<\/ol>\n

    <\/p>\n

    The main logic for finding anomalies using neural network data is depicted in Figure 3:<\/p>\n

    \n\"Searching
    \n<\/p>\n

    Figure 3. Finding anomalies using neural networks<\/em><\/p>\n

    The result of the forecast or recovery of the current stream of metrics calculates the deviation from the measurements obtained from the running software system. If there is a significant difference between the metrics obtained from the software system and the neural network, one can conclude that the current segment of data is anomalous. This raises a series of problems for using neural networks:<\/p>\n

    <\/p>\n

      \n
    1. For correct operation in streaming mode, the data for training neural network models must include only 'normal' data; <\/li>\n
    2. It is necessary to have a current model for accurate detection. Changes in trends and seasonality in metrics can cause a large number of false positives for the model. To update it, one must clearly define the time when the model is considered outdated. If the model is updated too late or too early, it is likely that there will be a high number of false positives.
      \nOne must also remember to search for and prevent the frequent occurrence of false positives. It is expected that they will most often occur in non-standard situations. However, they can also result from errors in the neural network due to insufficient training. It is essential to minimize the number of false positives produced by the model. Otherwise, false forecasts will waste a lot of the administrator's time that is intended for system checks. Sooner or later, this will lead to the administrator simply stopping responding to the 'paranoid' monitoring system.<\/li>\n<\/ol>\n

      <\/p>\n

      Recurrent Neural Network<\/h2>\n

      <\/p>\n

      To detect anomalies in time series, one can use recurrent neural networks <\/a><\/noindex>with LSTM memory. The only issue is that it can only be applied to predictable time series. In our case, not all metrics are predictable. An attempt to apply RNN LSTM to a time series is presented in Figure 4.<\/p>\n

      \n\"Searching
      \n<\/p>\n

      Figure 4. Example of a recurrent neural network with LSTM memory cells<\/em><\/p>\n

      As seen in Figure 4, the RNN LSTM was able to detect anomalies during this time period. Where the result has a high prediction error (mean error), an anomaly indeed occurred in the metrics. Using a single RNN LSTM will clearly not be sufficient, as it is applicable to a limited number of metrics. It can be used as a supplementary method for anomaly detection. <\/p>\n

      <\/p>\n

      Autoencoder for failure prediction<\/h2>\n

      <\/p>\n

      Autoencoder<\/a><\/noindex> \u2013 essentially an artificial neural network. The input layer is the encoder, and the output layer is the decoder. The drawback of all neural networks of this type is their poor localization of anomalies. A synchronous autoencoder architecture was chosen.<\/p>\n

      \n\"Searching
      \n<\/p>\n

      Figure 5. Example of an autoencoder's operation<\/em><\/p>\n

      Autoencoders are trained on normal data and then identify anomalies in the model's input data. This is precisely what is needed for this task. It remains only to choose which of the autoencoders is suitable for this task. The simplest architectural form of an autoencoder is a feedforward, non-recurrent neural network that closely resembles multilayer perceptron<\/a><\/noindex> (multilayer perceptron, MLP), with an input layer, an output layer, and one or more hidden layers connecting them.
      \nHowever, the differences between autoencoders and MLPs lie in the fact that in an autoencoder, the output layer has the same number of nodes as the input layer, and instead of learning to predict the target value Y given the input X, the autoencoder learns to reconstruct its own X. Thus, autoencoders are unsupervised learning models. <\/p>\n

      <\/p>\n

      The task of the autoencoder is to find temporal indexes r0 \u2026 rn corresponding to the anomalous elements in the input vector X. This effect is achieved by minimizing the quadratic error.<\/p>\n

      \n\"Searching <\/p>\n

      Figure 6. Synchronous autoencoder<\/em><\/p>\n

      A synchronous architecture<\/a><\/noindex>was chosen for the autoencoder. Its advantages include the ability to use a streaming processing mode and a relatively smaller number of neural network parameters compared to other architectures.<\/p>\n

      <\/p>\n

      False positive minimization mechanism<\/h2>\n

      <\/p>\n

      Due to various abnormal situations arising and the potential for insufficient training of the neural network, a decision was made to develop a mechanism to minimize false positives for the anomaly detection model being developed. This mechanism is based on a template database, which is classified by an administrator. <\/p>\n

      <\/p>\n

      Dynamic Time Warping Algorithm<\/a><\/noindex> (DTW algorithm, from English dynamic time warping) allows finding the optimal alignment between time sequences. It was first applied in speech recognition: used to determine how two speech signals represent the same original spoken phrase. Subsequently, it has found applications in other fields as well.<\/p>\n

      <\/p>\n

      The main principle of minimizing false positives is the collection of a reference database using an operator who classifies suspicious cases detected by neural networks. Next, the classified reference is compared with the case identified by the system, and a conclusion is made about whether the case is false or leads to a failure. The DTW algorithm is specifically used for comparing two time series. However, classification remains the primary tool for minimization. It is assumed that after collecting a large number of reference cases, the system will ask the operator less frequently due to the similarity of most cases and the occurrence of similar ones.<\/p>\n

      <\/p>\n

      As a result of the above methods of neural networks, an experimental program for predicting system failures in the 'Web-Consolidation' was developed. The aim of this program was to evaluate the effectiveness of this approach for our software systems by using the existing archive of monitoring data and information about previously occurred failures. The working scheme of the program is presented below in Figure 7.<\/p>\n

      \n\"Searching <\/p>\n

      Figure 7. Scheme for predicting failures based on metric space analysis<\/em><\/p>\n

      The diagram highlights two main blocks: the search for anomalous time segments in the data monitoring flow (metrics) and the mechanism for minimizing false positives. Note: for experimental purposes, the data is retrieved via JDBC connection from the database where it is saved by Graphite.
      \nBelow is the interface of the monitoring system developed as a result of the project (Figure 8).<\/p>\n

      \n\"Searching<\/p>\n

      Figure 8. Interface of the Experimental Monitoring System<\/em><\/p>\n

      The interface displays the percentage of anomaly based on the received metrics. In our case, data acquisition is being simulated. We already have several weeks of data and are gradually loading it to test the case of an anomaly leading to a failure. The bottom status bar shows the overall percentage of data anomaly at the current time, determined using the autoencoder. Additionally, a separate percentage for forecasted metrics is calculated by the RNN LSTM.<\/p>\n

      <\/p>\n

      An example of anomaly detection based on CPU metrics using the RNN LSTM neural network (Figure 9).<\/p>\n

      \n\"Searching<\/p>\n

      Figure 9. RNN LSTM Detection<\/em><\/p>\n

      A fairly simple case, essentially a typical outlier, but leading to system failure, was successfully computed using the RNN LSTM. The anomaly metric during this time frame is 85 \u2013 95%, anything above 80% (threshold determined experimentally) is considered an anomaly.
      \nAn example of anomaly detection when the system failed to boot after an update. This situation is detected by the autoencoder (Figure 10).<\/p>\n

      \n\"Searching<\/p>\n

      Figure 10. Example of Detection by Autoencoder<\/em><\/p>\n

      As seen in the figure, PermGen remained at a constant level. The autoencoder found this strange, as it had not encountered anything like it before. Here, the anomaly persists at 100% until the system returns to operational status. Anomaly is displayed across all metrics. As mentioned earlier, the autoencoder cannot localize anomalies. The operator is expected to perform this function in such situations.<\/p>\n

      <\/p>\n

      Conclusion<\/h2>\n

      <\/p>\n

      The \"Web-Consolidation\" PC has been under development for several years. The system is in a fairly stable state, and the number of recorded incidents is low. Nevertheless, it was possible to identify anomalies leading to failures 5 \u2013 10 minutes before the occurrence of a failure. In a number of cases, early warning of a failure could have saved the scheduled time allocated for maintenance work.<\/p>\n

      <\/p>\n

      It is still too early to draw final conclusions based on the experiments conducted so far. Currently, the results are contradictory. On one hand, it is evident that algorithms based on neural networks can identify \"useful\" anomalies. On the other hand, there is still a significant percentage of false positives, and not all anomalies that a qualified specialist can identify are detectable by the neural network. A downside is that the neural network currently requires supervised learning for effective operation.<\/p>\n

      <\/p>\n

      To further develop the failure prediction system and bring it to a satisfactory state, several paths can be considered. This includes a more detailed analysis of the cases with anomalies that lead to failures by complementing the list of important metrics that significantly affect the system's condition and filtering out the irrelevant ones. Additionally, moving in this direction, attempts can be made to specialize the algorithms specifically for our cases of anomalies that cause failures. There is also another approach: improving the architectures of neural networks to enhance detection accuracy while reducing training time.<\/p>\n

      <\/p>\n

      I express my gratitude to my colleagues who assisted me in writing and keeping this article up to date: Viktor Verbitsky<\/a><\/noindex> and Sergey Finogenov.<\/p>\n

      Source: habr.com<\/a><\/p>","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"excerpt":{"rendered":"

      \u041f\u0440\u043e\u043c\u044b\u0448\u043b\u0435\u043d\u043d\u0430\u044f \u0440\u0430\u0437\u0440\u0430\u0431\u043e\u0442\u043a\u0430 \u043f\u0440\u043e\u0433\u0440\u0430\u043c\u043c\u043d\u044b\u0445 \u0441\u0438\u0441\u0442\u0435\u043c \u0442\u0440\u0435\u0431\u0443\u0435\u0442 \u0431\u043e\u043b\u044c\u0448\u043e\u0433\u043e \u0432\u043d\u0438\u043c\u0430\u043d\u0438\u044f \u043a \u043e\u0442\u043a\u0430\u0437\u043e\u0443\u0441\u0442\u043e\u0439\u0447\u0438\u0432\u043e\u0441\u0442\u0438 \u043a\u043e\u043d\u0435\u0447\u043d\u043e\u0433\u043e \u043f\u0440\u043e\u0434\u0443\u043a\u0442\u0430, \u0430 \u0442\u0430\u043a\u0436\u0435 \u0431\u044b\u0441\u0442\u0440\u043e\u0433\u043e \u0440\u0435\u0430\u0433\u0438\u0440\u043e\u0432\u0430\u043d\u0438\u044f \u043d\u0430 \u043e\u0442\u043a\u0430\u0437\u044b \u0438 \u0441\u0431\u043e\u0438, \u0435\u0441\u043b\u0438 \u043e\u043d\u0438 \u0432\u0441\u0435-\u0442\u0430\u043a\u0438 \u0441\u043b\u0443\u0447\u0430\u044e\u0442\u0441\u044f. \u041c\u043e\u043d\u0438\u0442\u043e\u0440\u0438\u043d\u0433, \u043a\u043e\u043d\u0435\u0447\u043d\u043e \u0436\u0435, \u043f\u043e\u043c\u043e\u0433\u0430\u0435\u0442 \u0440\u0435\u0430\u0433\u0438\u0440\u043e\u0432\u0430\u0442\u044c \u043d\u0430 \u043e\u0442\u043a\u0430\u0437\u044b \u0438 \u0441\u0431\u043e\u0438 \u044d\u0444\u0444\u0435\u043a\u0442\u0438\u0432\u043d\u0435\u0435 \u0438 \u0431\u044b\u0441\u0442\u0440\u0435\u0435, \u043d\u043e \u043d\u0435\u0434\u043e\u0441\u0442\u0430\u0442\u043e\u0447\u043d\u043e. \u0412\u043e-\u043f\u0435\u0440\u0432\u044b\u0445, \u043e\u0447\u0435\u043d\u044c \u0441\u043b\u043e\u0436\u043d\u043e \u0443\u0441\u043b\u0435\u0434\u0438\u0442\u044c \u0437\u0430 \u0431\u043e\u043b\u044c\u0448\u0438\u043c \u043a\u043e\u043b\u0438\u0447\u0435\u0441\u0442\u0432\u043e\u043c \u0441\u0435\u0440\u0432\u0435\u0440\u043e\u0432 \u2013 \u043d\u0435\u043e\u0431\u0445\u043e\u0434\u0438\u043c\u043e \u0431\u043e\u043b\u044c\u0448\u043e\u0435 \u043a\u043e\u043b\u0438\u0447\u0435\u0441\u0442\u0432\u043e \u043b\u044e\u0434\u0435\u0439. \u0412\u043e-\u0432\u0442\u043e\u0440\u044b\u0445, \u043d\u0443\u0436\u043d\u043e \u0445\u043e\u0440\u043e\u0448\u043e \u043f\u043e\u043d\u0438\u043c\u0430\u0442\u044c, \u043a\u0430\u043a […]<\/p>\n","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[688],"tags":[],"class_list":["post-53531","post","type-post","status-publish","format-standard","hentry","category-administrirovanie"],"aioseo_notices":[],"aioseo_head":"\n\t\t\n\t\n\t\n\t\n\t\n\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\n","aioseo_head_json":{"title":"\ud83e\udd47We are searching for anomalies and predicting failures using neural networks | ProHoster","description":"Industrial development.","canonical_url":"https:\/\/prohoster.info\/en\/blog\/administrirovanie\/ishhem-anomalii-i-predskazyvaem-sboi-s-pomoshhyu-nejrosetej","robots":"max-image-preview:large","keywords":"","webmasterTools":{"miscellaneous":""},"schema":null,"og:locale":"en_US","og:site_name":"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b","og:type":"article","og:title":"\ud83e\udd47\u0418\u0449\u0435\u043c \u0430\u043d\u043e\u043c\u0430\u043b\u0438\u0438 \u0438 \u043f\u0440\u0435\u0434\u0441\u043a\u0430\u0437\u044b\u0432\u0430\u0435\u043c \u0441\u0431\u043e\u0438 \u0441 \u043f\u043e\u043c\u043e\u0449\u044c\u044e \u043d\u0435\u0439\u0440\u043e\u0441\u0435\u0442\u0435\u0439 | ProHoster","og:description":"\u041f\u0440\u043e\u043c\u044b\u0448\u043b\u0435\u043d\u043d\u0430\u044f \u0440\u0430\u0437\u0440\u0430\u0431\u043e\u0442\u043a\u0430.","og:url":"https:\/\/prohoster.info\/en\/blog\/administrirovanie\/ishhem-anomalii-i-predskazyvaem-sboi-s-pomoshhyu-nejrosetej","og:image":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:secure_url":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:width":350,"og:image:height":350,"article:published_time":"2019-12-03T21:00:00+00:00","article:modified_time":"2020-02-18T11:01:26+00:00","article:publisher":"https:\/\/www.facebook.com\/prohoster","article:author":"https:\/\/www.facebook.com\/prohoster"},"aioseo_meta_data":{"post_id":"53531","title":null,"description":null,"keywords":null,"keyphrases":null,"primary_term":null,"canonical_url":null,"og_title":null,"og_description":null,"og_object_type":"default","og_image_type":"default","og_image_url":null,"og_image_width":null,"og_image_height":null,"og_image_custom_url":null,"og_image_custom_fields":null,"og_video":null,"og_custom_url":null,"og_article_section":null,"og_article_tags":null,"twitter_use_og":false,"twitter_card":"default","twitter_image_type":"default","twitter_image_url":null,"twitter_image_custom_url":null,"twitter_image_custom_fields":null,"twitter_title":null,"twitter_description":null,"schema":{"blockGraphs":[],"customGraphs":[],"default":{"data":{"Article":[],"Course":[],"Dataset":[],"FAQPage":[],"Movie":[],"Person":[],"Product":[],"ProductReview":[],"Car":[],"Recipe":[],"Service":[],"SoftwareApplication":[],"WebPage":[]},"graphName":"","isEnabled":true},"graphs":[]},"schema_type":null,"schema_type_options":null,"pillar_content":false,"robots_default":true,"robots_noindex":false,"robots_noarchive":false,"robots_nosnippet":false,"robots_nofollow":false,"robots_noimageindex":false,"robots_noodp":false,"robots_notranslate":false,"robots_max_snippet":null,"robots_max_videopreview":null,"robots_max_imagepreview":"large","priority":null,"frequency":null,"local_seo":null,"seo_analyzer_scan_date":"2026-01-24 07:46:24","breadcrumb_settings":null,"limit_modified_date":false,"reviewed_by":null,"ai":null,"created":"2021-02-28 20:23:55","updated":"2026-01-24 07:46:24","focus_keyword":null,"additional_keywords":null,"truseo_locale":null},"gt_translate_keys":[{"key":"link","format":"url"}],"_links":{"self":[{"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/posts\/53531","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/comments?post=53531"}],"version-history":[{"count":0,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/posts\/53531\/revisions"}],"wp:attachment":[{"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/media?parent=53531"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/categories?post=53531"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/tags?post=53531"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}