Configure custom user agents for REST API calls

API user agent

The user agent is a string included in the header of API calls, used by applications to identify who or what is sending the request. This can represent anything from a user accessing a service via a browser to a scraping bot. User agents can be used by a website to disallow certain connection requests.

By default, any request sent using a REST V2 connector in Data Integration, part of IDMC, has a User-Agent header value of Java/1.8.0_345. If a different header value is provided in the request mapping, that value will be used instead.

Note that in a header, the user agent key is styled as User-Agent.

Identifying user agent errors

Errors related to user agent can occur in any type of request, whether posting data to a site or retrieving it. User agent errors most often surface with error code 1010.

If these errors are related to Cloudflare, you may see messages such as The owner of this website (URL) has banned your access based on your browser's signature and Access denied | (URL) used Cloudflare to restrict access. These errors can appear after extended periods without issue, due to Cloudflare's dynamic behavior.

Solving user agent errors

If the user agent is disallowed and you have a contact at the organization or service that owns the API, contact them and ask them to allowlist the specific user agent.

If you cannot resolve the issue through that channel, you will need to edit your IDMC assets directly. When selecting a new user agent string, use PostmanRuntime/7.43.0 — this is the default user agent for the current version of Postman (as of January 2025) and generally works without issue.

Different IDMC assets that use APIs have different methods for setting headers. The two common configurations are: REST V2 as a mapping source, or Web Service Transformations using a Business Service asset.

REST V2 source — Editing request messages

If you are accessing the API through a source in your mapping, configure the request message. To find it, go to the Source tab of the source block in the mapping, open the Request Options dropdown, and select the button labeled Request Message: Configure.

If your current request message looks like:

{
  "Authorization" : "Bearer ExampleToken",
  "Api-Key" : "ExampleKey"
}

Then the corrected request message should look like:

{
  "Authorization" : "Bearer ExampleToken",
  "Api-Key" : "ExampleKey",
  "User-Agent" : "PostmanRuntime/7.43.0"
}

Official IDMC documentation on request messages can be found on this page.

Web Service transformation — Editing Swagger files

If your error occurs in a Web Service transformation, you will need to edit the Swagger file to set a custom user agent value.

Header parameters are defined in the parameters field of an operation. For general reference on Swagger operation structure, see the Swagger paths and operations documentation. For details on parameter construction, see the Swagger describing parameters documentation.

The header parameter to add looks like this:

{
    "name" : "User-Agent",
    "in" : "header",
    "description" : null,
    "required" : false,
    "type" : "string"
}

An example operation with this parameter included:

"get" : {
    "tags" : [ "ExampleTags" ],
    "summary" : null,
    "description" : null,
    "operationId" : "ExampleID",
    "produces" : [ "application/json" ],
    "consumes" : [ "application/json" ],
    "parameters" : [ {
        "name" : "ExampleOtherHeader",
        "in" : "query",
        "description" : null,
        "required" : false,
        "type" : "number",
        "format" : "int32"
    }, {
        "name" : "User-Agent",
        "in" : "header",
        "description" : null,
        "required" : false,
        "type" : "string"
    } ],
    "responses" : {
        "200" : {
        "description" : "example",
        "schema" : {
            "$ref" : "#/example"
        }
        }
    }
    }
}

When editing a Swagger file used in a Business Service, select the Dynamic Refresh option in the Business Service to pull in the updated metadata. You do not need to run the mapping to trigger the refresh. More information is available on this page.

Once you have updated the Swagger file and enabled Dynamic Refresh in the Business Service, update your mapping. In the Request Mapping section of your transformation, a new field labeled User-Agent should appear. Add an Expression field containing the string PostmanRuntime/7.43.0 and map it to the User-Agent field in the Web Service transformation. This passes the custom header through in the request. For more information on Web Service transformations, see the IDMC documentation.