Limit
There is such a restriction on LinkedIn — . It is very likely that you, like me until recently, have never encountered or heard of it.
The essence of the limit is that if you search for people outside your contacts too frequently (there are no exact metrics, it's determined by an algorithm based on your actions — how often and how much you searched for and added people), then the search results will be limited to three profiles instead of 1000 (by default, 100 pages with 10 profiles per page). The limit resets at the beginning of each month. Naturally, .
But not long ago, for a side project, I started experimenting a lot with search on LinkedIn and suddenly encountered this restriction. Naturally, I wasn't very pleased, as I wasn't using it for any commercial purposes, so my first thought was to study the limitation and try to bypass it.
[Important clarification — the materials in the article are presented solely for informational and educational purposes. The author does not encourage their use for commercial purposes.]
Identifying the problem
We have: instead of ten profiles with pagination, the search only returns three, after which there is a block with a 'premium account recommendation' and below are blurred and non-clickable profiles.
Immediately, my hand reaches for the developer console to see these hidden profiles — perhaps we can remove some styles that apply blur or extract information from the block in the markup. But, quite predictably, these profiles are merely and do not store any information.
Alright, now let's check the Network tab to see if the alternative search results are really triggered, returning only three profiles. We find the request for 'api/search/blended' and look at the response.
Profiles come in the array `included`, but there are actually 15 entities in it. In this case, the first three of them are objects with additional information, each object contains information about a specific profile (for example, whether the profile is premium).
The subsequent 12 are real profiles — search results, of which only three will be shown to us. As you might guess, it only displays those for which additional information is received (the first three items). For example, if we take a response from a profile without a limit, we would receive 28 entities — 10 objects with additional information and 18 profiles.
Response for a profile without a limit
Why do more than 10 profiles come through when only 10 are requested, and they do not participate in the display at all, not even on the next page? I still don't know. If we analyze the request URL, we can see that count=10 (how many profiles to return in the response, a maximum of 49).
I would appreciate any comments on this matter.
We are experimenting
Well, the most important thing we now know for sure is that more profiles come in the response than are shown to us. This means we can extract more data, despite the limit. Let's try to call the API ourselves, directly from the console, using fetch.
As expected, we get an error, 403. This is due to security; here we do not send a CSRF token (. In short — a unique token is added to each request, which is checked for authenticity on the server).
It can be copied from any other successful request or from cookies, where it is stored in the ‘JSESSIONID’ field.
Where to find the tokenHeader of another request:
Or from cookies, directly through the console:
Let's try again, this time we are passing settings in fetch, in which we specify the csrf-token parameter in the header.
Success, we receive all 10 profiles. :tada:
Due to the difference in headers, the response structure is slightly different from what comes in the original request. We can achieve the same structure by adding ‘Accept: 'application/vnd.linkedin.normalized+json+2.1' to our object, alongside the csrf token.
Example response with the added header
What's next?
Next, you can edit (manually or automate) the parameter `start`, which indicates the index from which we will receive 10 profiles (by default = 0) from the entire search result. In other words, by incrementing it by 10 after each request, we obtain a regular paginated output, 10 profiles at a time.
At this stage, I had enough data and freedom to continue working on my pet project. However, it would be a shame not to try to display this data right away since I had it on hand. We won’t dig into Ember, which is used on the front end. jQuery was included on the site, and by recalling the basic syntax, I could create the following in just a couple of minutes.
Code in jQuery
/* рендер блока, принимаем данные профиля и вставляем блок в список профилей используя эти данные */
const createProfileBlock = ({ headline, publicIdentifier, subline, title }) => {
$('.search-results__list').append(
`<li class="search-result search-result__occluded-item ember-view">
<div class="search-entity search-result search-result--person search-result--occlusion-enabled ember-view">
<div class="search-result__wrapper">
<div class="search-result__image-wrapper">
<a class="search-result__result-link ember-view" href="/en/in/${publicIdentifier}/">
<figure class="search-result__image">
<div class="ivm-image-view-model ember-view">
<img class="lazy-image ivm-view-attr__img--centered EntityPhoto-circle-4 presence-entity__image EntityPhoto-circle-4 loaded" src="http://www.userlogos.org/files/logos/give/Habrahabr3.png" />
</div>
</figure>
</a>
</div>
<div class="search-result__info pt3 pb4 ph0">
<a class="search-result__result-link ember-view" href="/en/in/${publicIdentifier}/">
<h3 class="actor-name-with-distance search-result__title single-line-truncate ember-view">
${title.text}
</h3>
</a>
<p class="subline-level-1 t-14 t-black t-normal search-result__truncate">${headline.text}</p>
<p class="subline-level-2 t-12 t-black--light t-normal search-result__truncate">${subline.text}</p>
</div>
</div>
</div>
<li>`
);
};
// Fetching API, getting data and rendering profiles
const fetchProfiles = () => {
// Token
const csrf = 'ajax:9082932176494192209';
// Object with request settings, passing the token
const settings = { headers: { 'csrf-token': csrf } }
// Request URL, with dynamic start index at the end
const url = `https://www.linkedin.com/voyager/api/search/blended?count=10&filters=List(geoRegion-jp0,network-S,resultType-PEOPLE)&origin=FACETED_SEARCH&q=all&queryContext=List(spellCorrectionEnabled-true,relatedSearchesEnabled-true)&start=${nextItemIndex}`;
/* Making the request, calling render block for each profile in response, and incrementing the start index by 10 */
fetch(url, settings).then(response => response.json()).then(data => {
data.elements[0].elements.forEach(createProfileBlock);
nextItemIndex += 10;
});
};
// Removing all profiles from the list
$('.search-results__list').find('li').remove();
// Inserting the button to load profiles
$('.search-results__list').after('<button id="load-more">Load More</button>');
// Adding functionality to the button
$('#load-more').addClass('artdeco-button').on('click', fetchProfiles);
// Setting default profile index for the request
window.nextItemIndex = 0;
If you execute this directly in the console on the search page, it will add a button that loads 10 new profiles each time it’s clicked, rendering them as a list. Of course, change the token and URL to what you need beforehand. The profile block will contain the name, position, location, profile link, and a placeholder image.
Conclusion
Thus, with minimal effort, we managed to find a vulnerability and restore unrestricted search for ourselves. It was enough to analyze the data and its pathway, taking a look at the request itself.
I can’t say this poses a serious problem for LinkedIn since it doesn’t present any real threat. At most, it results in lost revenue due to such 'workarounds' that allow users to avoid paying for premium services. It’s possible that this server response is necessary for the proper functioning of other parts of the site, or it might just be laziness from the developers due to a lack of resources to implement it properly. (The limitation was introduced in January 2015; there were no limits before this.)
P.S.
Naturally, the jQuery code is a rather primitive example of the possibilities. Currently, I’ve created a browser extension for my own needs. It adds control buttons and renders complete profiles with images, an invite button, and shared connections. Plus, it dynamically gathers filters for locations, companies, and more, extracting the token from cookies. So there’s no need to hard-code anything anymore. It also adds additional settings fields, like 'how many profiles to request at a time, up to 49.'
I'm still working on this extension and plan to release it to the public. Feel free to reach out if you’re interested.
Source: habr.com
