Global Parameter Values

Below is the list of global parameter values which work the same way in all data sources. On the right side menu you can see the full list of parameters and click on any of them to jump to its section for more information.

Parameter	Required/Optional
source	Required
url or query	Required
parse	Optional
render	Optional
user_agent_type	Optional
callback_url	Optional
client_notes	Optional

Source

source parameter is one of the most important parameters. It sets the scraper that we will use to process your request (e.g. google_search, amazon_product, etc.).

Parameter values

The section below outlines the product-target-source relationships.

E-Commerce Scraper API

Target	Sources
Amazon	amazon, amazon_bestsellers, amazon_pricing, amazon_product, amazon_questions, amazon_reviews, amazon_search, amazon_sellers
Google Shopping	google_shopping, google_shopping_search, google_shopping_product, google_shopping_pricing
Wayfair	wayfair, wayfair_search
Other targets	universal_ecommerce

SERP Scraper API

Target	Sources
Google	google, google_search, google_ads, google_images, google_lens, google_maps, google_travel_hotels, google_suggest, google_trends_explore
Yandex	yandex, yandex_search
Bing	bing, bing_search
Baidu	baidu, baidu_search

Web Scraper API

Target	Sources
All targets (Including: Booking.com, Expedia, Hotels.com, Priceline, Trivago, Zillow, Redfin, Zoopla, Reddit and many more)	universal

URL or Query

When submitting a request, include either the URL or query parameter, depending on the chosen source parameter. For example, if using universal source to extract data from a target website, include the URL parameter with the URL you wish to scrape. Alternatively, if using google_search source, include a query parameter with UTF-encoded keyword. Refer to the respective target sub-pages in the documentation for detailed guidance on when to use which parameter.

Parse

For some sources of SERP Scraper API and E-Commerce Scraper API , we have developed dedicated parsers. If you want to get structured data, add the parse parameter and set it to true. Find more details in the respective target pages.

Target	Sources	Page types
Google	google, google_search, google_ads, google_images	Web search, images search, news search, reverse image search pages.
Bing	bing, bing_search	Search pages.
Google Shopping	google_shopping, google_shopping_search, google_shopping_product, google_shopping_pricing	Search, product and offer listing pages.
Amazon	amazon, amazon_search, amazon_product, amazon_pricing, amazon_reviews, amazon_questions, amazon_bestsellers, amazon_sellers	Search, product, offer listing, reviews, questions and answers, best sellers and sellers pages.
Best Buy	universal_ecommerce	Product pages.
Etsy	universal_ecommerce	Product pages.
Target	universal_ecommerce	Search, product and vendor pages.
Walmart	universal_ecommerce	Search and product pages

Render

The render parameter lets you enable JavaScript execution. Use it when the target website requires JavaScript to load the necessary content or when the site uses an anti-bot tool that utilizes JavaScript to fingerprint their visitors.

JavaScript rendering takes more time to scrape the page. Please set timeout on the client side to 180 seconds if using Realtime or Proxy Endpoint integration methods.

Parameter values

There are two available values for this parameter: html (get raw output) and png (get a Base64-encoded screenshot).

Value	Description
html	The output will include an HTML result.
png	The output will include a PNG screenshot of the result.

User_agent_type

The user_agent_type parameter lets you specify what kind of a User-Agent header value we should use to fulfil your request.

You can find a complete list of available values in the table below.

Parameter values

Value	Description
desktop	A User-Agent of a desktop browser.
desktop_chrome	A User-Agent of one of the latest versions of a desktop Chrome browser.
desktop_edge	A User-Agent of one of the latest versions of a desktop Edge browser.
desktop_firefox	A User-Agent of one of the latest versions of a desktop Firefox browser.
desktop_opera	A User-Agent of one of the latest versions of a desktop Opera browser.
desktop_safari	A User-Agent of one of the latest versions of a desktop Safari browser.
mobile	A User-Agent of a mobile browser.
mobile_android	A User-Agent of one of the latest versions of an Android mobile browser.
mobile_ios	A User-Agent of one of the latest versions of an iOS browser.
tablet	A User-Agent of a tablet browser.
tablet_android	A User-Agent of one of the latest versions of an Android tablet browser.
tablet_ios	A User-Agent of one of the latest versions of an iPad tablet browser.

Callback_url

A callback is a POST request that we send to your machine, informing you that the data extraction task is completed and the result is ready to be downloaded. A callback URL, in turn, is the URL we will send the POST request to.

Parameter values

Use the callback_url parameter to specify the URL where we should POST our notification to.

Client notes

You can add client_notes parameter when submitting a job via v1/queries endpoint. These client notes are then saved to database and to s3 storage along other job information.

Payload example

{
    "source": "universal",
    "url": "https://sandbox.oxylabs.io",
    "client_notes": "123456789"
}

Response example

{
    "_links": [
        {
            "href": "http://data.oxylabs.io/v1/queries/7099386176976459777",
            "method": "GET",
            "rel": "self"
        },
        {
            "href": "http://data.oxylabs.io/v1/queries/7099386176976459777/results",
            "method": "GET",
            "rel": "results"
        }
    ],
    "callback_url": null,
    "client_id": 1234,
    "client_notes": "123456789",
....