---
title: "REST Services Source"
slug: "rest-services-source"
updated: 2026-08-18T13:40:06Z
published: 2026-08-18T13:40:06Z
canonical: "docs.plainid.io/rest-services-source"
---

> ## Documentation Index
> Fetch the complete documentation index at: https://docs.plainid.io/llms.txt
> Use this file to discover all available pages before exploring further.

# REST Services Source

Policy Information Point (PIP) Settings allow you to configure Data Sources that provide essential information for Policy decisions. In many cases, the Data Source you need to integrate with provides the data through a REST service that exposes different endpoints serving subsets of the data. In some cases, these services are your internal application’s self-developed services, where you have more control over the integration pattern and the available endpoints. In other cases, there are third-party REST services solutions you are using in your organization that manage some of your data. This guide focuses on setting up REST Service Data Sources, enabling you to integrate external REST APIs into your Policy Authorization process.

## Configuring the REST Service Data Source

### Prerequisites

Before configuring a REST Service Data Source, ensure that you have:

- Appropriate permissions in your tenant
- Basic understanding of REST APIs and JSON
- Knowledge of SQL syntax for defining data structures

### General

The configuration of a REST data source includes the settings required to make a valid REST request to your service. It also includes a guide on managing the response from your service and transforming it from JSON to a relational data structure. The overall setup includes:

- A declaration of a virtual table
- A definition of Data Attributes fetched from the REST service response
- Optional `OPTIONS` clause for each Data Attribute
- The Base URL of the service
- Authentication to the service
- The service endpoint’s URI
- A REST Method
- The location of the relevant data in the service response
- `OPTIONS` clause with additional configurations influencing the REST request and response
- Additional translator properties configuring the translator behavior

### To Configure a Data Source:

1. Open the Tenant or Environment Settings screen and select the PIP Settings tile.
2. In the Policy Authorization Group drop-down menu, select the relevant PAA. A list of Data Sources and Views are displayed.
3. In the Data Source List, click **New Data Source**.
4. In the Data Source Details section:
  - Enter a Display Name for the new Data Source.
  - In the Connection Adapter dropdown, select **REST**.
  - In the Connection Settings section under Policy Authorization Agent, select a PAA.
  - Input your REST service's base URL in the designated End Point field. For example: `https://domain.com/data`.
  - You also have the option to specify a mock server's base URL, which can assist in designing and testing an integration when developing a new use case.
  - In the Models section under the Model Name field, input a unique Model name.
  - Under the DDL section, write a valid DDL to fetch REST attributes according to the service response. Refer to the section on [Configuring a DDL](/v1/docs/rest-services-source#configuring-a-ddl-for-the-rest-service) for more information.

*Note: If using OAuth 2 as a Security Type in the Connection Settings, select None from the dropdown, and see the full [OAuth 2](/docs/oauth2-authentication-support) instructions on setting up this authentication method.*

## Configuring a DDL for the REST Service

The structure of a DDL includes the declaration of the table, declaration of all attributes, and an optional `OPTION` property. The DDL defines the data structure that the REST Service creates. It details Attributes and their options, and allows users to configure the REST request options. This configuration determines how a request is generated and executed in addition to how to handle and parse the response.

When inputting a DDL property, ensure that you specify the name of the attribute and the type. You can add a `nameinsource` option in the `OPTIONS` clause if indicating a specific location in the response structure using a JSONPath. You can configure the path by utilizing the relative path to the general Data Source path when you have a general path set and you want to specify the property relative to this base path. This is helpful when the base path can change, but the relative path to the property remains consistent.

E.g. If the general base path is `$.data.user`, then simply specify the name to get the user’s name.

#### Example Usage

```
{
  "data": {
    "user": {
      "id": 123,
      "name": "John Doe"
    }
  }
}
```

- **Relative Path** (with general path `$.data.user`): `name`

### DDL Structure

The basic structure of a DDL for the REST Service is as follows:

#### OPTIONS Clause Structure

See available options [here](/v1/docs/rest-services-source#available-options-in-the-ddl).

```
OPTIONS (nameinsource "<option>"<'option value'>, "<option>"<'option value'>)
```

#### Overall Structure

```
CREATE FOREIGN TABLE <Virtual_Table_Name>(
    <Attribute_Name> <Data_Type>,
    <Attribute_Name> <Data_Type> OPTIONS (nameinsource '<relative JsonPath to the location of the field in the response>')
) OPTIONS (nameinsource '<JsonPath to the root location in the response>', "plainid:method" 'GET', "plainid:URI" '<ENDPOINT_URI_AFTER_BASE_URL>', "plainid:unwrap" '<True/False>');
```

JSONPath Note

The JSONPath used in the Attribute is a concatenation of the root JSONPath from the general option and the relative JSONPath with the Attribute `nameinsource`. See [Available OPTIONS in the DDL](/docs/rest-services-source#available-options-in-the-ddl) for more information.

### Available OPTIONS in the DDL

Below is a list of OPTIONS that can be used in the DDL. Refer to the table below for examples.

- **plainid:URI**: The URI of the service endpoint that is used for the REST request. This parameter is added to the base URI defined in the data source. In addition, users can add a variable to this relative URI, which is used as a dynamic parameter based on values from the WHERE clause of a query sent to the REST Source: **Example**: `/users/all` It is possible to add a variable to the URI. In this case, the query will be translated based on the WHERE clause. **Example**: `/users/$(id)` `Select * from groups where id = 101` will be translated to `/users/101`.
- **plainid:method** is used for the REST request and supports GET and POST (with parameters/payload) methods.
- **plainid:bodyJsonContent** property includes the JSON content used as the POST request body. Dynamic parameters can be employed within this payload, which dynamically adjust the JSON content for POST requests. In the DDL, JSON should be converted into a string format. When utilizing this property, a standard Content-Type header is applied to the request sent to the REST service with the value `application/json`.
- **nameinsource** property is used both at the Attribute and general level:
  - At the Attribute level, this property is optional and should be used when a different Attribute name than the name in the response is needed, or when a specific JSONPath for that Attribute in the response needs to be configured.
  - This property is required and must contain a JSONPath that specifies the location from which to extract data from the response.
  - The value `($)` specifies the root location’s JSONPath in the response where all Attributes are retrieved from. Other variations can include `$.results[*]`, `groups[*]`, or any other valid JSONPath locations that guide where to parse the response from and fetch the relevant data retrieved from the service. Note that this JSONPath is the base for each Attribute’s JSONPath.
- **plainid:unwrap** property supports unwrapping complex REST responses while ensuring that the relationship between attributes remains intact.

The JSON returned needs to be converted into a relational database format for complex Views. The unwrap feature allows for JSON multi-values to "unwrap" into an array, creating multiple records from each value in the array.

If working with complex nested structures and/or large data sets and arrays that may create a very large set of permutations, expect that the unwrap calculation will take a longer time and may affect performance. Ensure that you reduce complex structures, sets, or arrays where possible and only map required Attributes to reduce permutations. Contact the PlainID Global Services team for more details.

### Property Descriptions and Examples

| Property | Description | Example |
| --- | --- | --- |
| **plainid:URI** | Relative URI for the REST request. Variables can be added to this URI, which is used as a dynamic parameter based on values from the WHERE clause. See [URI Encoding Rules](/v1/docs/rest-services-source#uri-encoding-rules) for more information. | `/users/all`, `/users/$(id)` |
| **plainid:method** | Supported methods are GET (default) and POST (with parameters/payload). | `OPTIONS (... "plainid:method" 'GET', ..., )` |
| **plainid:bodyJsonContent** | It is added to the endpoint defined at the Data Source level. | `{"param a":"aaa","param b":1234,"param c":$(attributeC)}, "param d": true}` |
| **nameinsource** | Variations can look like `$.results[*]`, `groups[*]`, or any other valid JSONPath locations that guide where to parse the response from and fetch the relevant data retrieved from the service. | `OPTIONS (nameinsource '$', ... );` |
| **plainid:unwrap** | Default: `false`. This option is related to handling multi-value attributes. If set to `true`, the translator creates all permutations of values between array attributes. In complex structures like nested objects, the relationships of data properties are still intact while unwrapping. | If a response contains an Attribute with three values, the PIP will create three records. If it contains two multi-value Attributes, one with two values and the second with four values, PIP will create eight records. |

### URI Encoding Rules

When setting the `plainid:URI` property, as a static URI and especially when using Dynamic Parameters injected at Runtime, some characters in the path, query, or dynamically injected values may require special handling to comply with URI Encoding rules. The encoding is done automatically by the PIP Rest Translator, unless the URI is manually encoded. In such cases, users are responsible for encoding all parts, as the automatic encoding will be avoided.

If users do not perform manual encoding, the system automatically applies encoding based on these rules:

|  | **URI Path** | **URI Query Param** | **Explanation** |
| --- | --- | --- | --- |
| **Auto Encoded** | \ [ ] { } # % | : $ @ ! / \ [ ] { } # + ? % | These characters are automatically encoded. |
| **Allowed Unencoded** | : $ @ ! ; + & |  | These characters are allowed to remain unencoded |
| **Limitation** | ? / | & ; | These characters cannot be used, except when used for separating URI parts |

Example: If you include an `&amp;` in a parameter value, encode it as `%26`.

## Translator Properties

Translator properties direct the REST translator's behavior and functionalities.

Check out our [Translator and Model Properties article](/v1/docs/jdbc-data-sources) for more information about Translator Properties.

  

Refer to the supported translator properties for detailed explanations and examples.

### Support Dynamic Filters

When using a dynamic parameter in the form of an `$(Attribute)`, in the URI, in a dynamic header, or in a JSON option for POST payload, the `supportsfilter` property must be added with the value `true`. If not defined or set to `false`, the `$(Attribute)` is treated as text and is not replaced dynamically with a value.

- `true`: Filters are sent to the REST Data Source according to the dynamic parameters in the REST definition. When the data is considered as authorized, it is sent back to the requesting service (i.e., the PDP). The advantage of the `true` setting is that users can leverage the Data Source filtering capabilities by applying dynamic parameters and reducing the overall performance and payload with a smaller and filtered dataset. The drawbacks of this setting is that filtering is limited to the REST endpoint functionality, so only supported filters are applied on the query. The REST Data Source needs to support all required filters to avoid fetching unauthorized data. Refer to the [Partial Filtering](/v1/docs/rest-services-source#partial-filtering) section for more information.
- `false`: Filters are not transmitted to the REST Data Source. Instead, the data is fetched according to the REST definition without any dynamic filters. After the data is retrieved, the PIP settings apply filters to the dataset based on the incoming SQL query, and only then is the filtered data sent back to the requesting service (i.e., the PDP). The advantage of the `false` setting is that filters can vary and can be updated dynamically as the filtering takes place at the SQL level in the PIP service. The drawbacks of the `false` setting is that it fetches larger datasets (potentially all datasets) from the REST Data Source to the PIP service, which can cause big payloads and performance impact.

*Note: If this property is not sent, the default is `false`.*

### Multi-value Parameters

When a multi-value Attribute is passed in the query, all the values used in the dynamic parameters are injected into the REST request.

**Examples:**

```
POST {
  "paramA": [
    "a1",
    "a2",
    "a3"
  ]
}
```

```
GET ../../..?paramA=a1,a2,a3
```

To partially filter the REST Data, refer to the section on [Partial Filtering](/v1/docs/rest-services-source#partial-filtering). You can also check out the [Data Source Filtering Parameters](/v1/docs/rest-services-source#data-source-filtering-parameters) section for more information.

### Redis-Based Request Distributed Caching

The **Redis-based Request Distributed Cache** feature optimizes data retrieval for **REST** and **Entra ID** translators by introducing a shared caching mechanism between **PIP instances**. This feature significantly reduces redundant data service calls and improves performance, especially when working with **large data sets** or when **full data sets** are fetched into the PIP cache.

**Ensure that you use this feature solely with a PIP Caching Materialized View. For more information, refer to** [**Caching Views**](/docs/caching-views).

When multiple PIP instances request the same data from a source, such as **Entra ID**, the caching mechanism ensures that:

- Only **one PIP instance** sends the data request to the source.
- This instance stores the response in a shared **Redis cache** as a distribution location.
- All other PIP instances retrieve the cached response directly from Redis instead of making new requests to the external service.

This process reduces load on the data source, minimizes redundant network calls, and improves overall efficiency when loading large data sets into PIP Cache.

---

          Resource Considerations

          

Enabling Redis caching requires additional Redis resources for the duration of data distribution to all PIP instances. Before enabling, consult with **Professional Services** to evaluate the appropriate Redis capacity based on your data set size and data source characteristics.

---

**Configuration**

To enable this feature, set the following translator property:

| Property | Description | Default |
| --- | --- | --- |
| `redisCacheEnabled` | Enables the Redis caching mechanism. | `false` |

Once enabled, you can fine-tune cache behavior using the following property:

| Property | Description | Default |
| --- | --- | --- |
| `valueTtlMs` | Defines how long cached values are retained in Redis (in milliseconds) available for distribution to PIP instances. Higher values retain data longer but increase cache memory usage for a longer duration. | `300000` |

---

**Example Scenario**

**Three PIP instances** (`PIP1`, `PIP2`, and `PIP3`) in a PAA are deployed in the same cluster, configured to use Redis caching.

Each instance processes identical data requests, such as fetching a **User Materialized View** from the Data Source Without Redis caching, all three PIPs would **independently query the Data Source**, creating unnecessary load and duplication.

With **Redis-Based Requests Distributed Cache** enabled:

1. **PIP1** sends the initial data request to the Data Source.
2. **PIP1** stores the retrieved data in the **Redis cache** and loads it into its local memory.
3. When **PIP2** and **PIP3** issue the same request, they find the response already cached in Redis.
4. Instead of re-querying Azure, **PIP2** and **PIP3** pull the data directly from Redis.

As a result:

- The data source (like **Entra ID** or **REST**) receive **only one request** (per page).
- All PIP instances share the **same consistent data**.
- Overall system performance improves, particularly for **large data sets** or **high-frequency REST calls**.

### Headers

In cases when your REST services require headers, whether fixed or custom headers like content-type, x-request, tokens, etc., you can use the headers property to inject custom headers into the REST request. The value is a JSON containing a list of headers, each containing the header name and an array of values in this form:

```
{"x-header1": ["value1", "value2"] ,"x-header2": ["value1", "value2"]}
```

Note that if you need to use X-Auth-Token to authenticate your REST endpoint, you can add the property `headers` with the value `{"X-Auth-Token":["&lt;token&gt;"]}`.

To define this token as a Secret store key, refer to [Secret Management for REST Headers](/v1/docs/rest-services-source#secret-management-for-rest-headers).

---

#### Dynamic Headers

When configuring headers in the translator property, users can optionally adjust dynamic header values denoted by `$(&lt;Attribute_name&gt;)`. For example, the structure of a header setup can be similar to the following existing configuration: `{"x-header-accountID":["$(accID)"]}`. In this setup, `$(accID)` is replaced with the value sent by the Runtime during the preparation of the REST request.

It is important to note that if the value of the dynamic parameter `$(&lt;Attribute_name&gt;)` is null or empty, meaning it is not sent in the Runtime request, the corresponding header is skipped and not sent as an empty value.

**Note:** Make sure the `supportsfilter` is set to `true` when setting up dynamic headers.

---

**Dynamic Parameters in Structured Header Values**

Dynamic parameters can be used inside structured header values, including values that represent arrays, like JSON arrays encoded as strings. Multi-value dynamic parameters should be used only when supported by the target REST service, like when working with structured array headers.

These parameters can appear anywhere within a header value, including as part of a longer string. This approach is useful when the downstream REST service expects a specific value structure rather than a plain string.

**Quote Behavior for Multi-Value Dynamic Parameters**

When a dynamic parameter resolves to multiple values, the REST Translator preserves the formatting defined in the header value template.

- If the dynamic parameter appears **inside quotes**, each injected value is quoted.
  - This header:`{"headers": {"X-Accounts": ["$(accID)"]}}`
  - Results in:  "X-Accounts": "acc1", "acc2", ...
- If the dynamic parameter appears **without quotes**, values are injected as a comma-separated list.
  - This header:`{"headers": {"X-Accounts": [$(accID)]}}`
  - Results in: "X-Accounts": acc1, acc2, ...

---

**Example: Azure Cosmos DB Partition Key Header**

Azure Cosmos DB REST APIs require the partition key header value to be formatted as a JSON array string.

```
{
  "headers": {
    "x-ms-documentdb-partitionkey": ["[\"$(id)\"]"]
  }
}
```

With `WHERE id = '1232458ZXCA'`:

`x-ms-documentdb-partitionkey: ["1232458ZXCA"]`

With `WHERE id IN ('a', 'b', 'c')`:

`x-ms-documentdb-partitionkey: ["a", "b", "c"]`

#### Authorization Headers

You can also set the Access Token header name and prefix.

| Header | Default | Description |
| --- | --- | --- |
| `accessTokenHeaderName` | `Authorization` | Specifies the header name for the Access Token. |
| `authScheme` | *(None)* | Defines a custom scheme prefix for the Access Token. If not set and `overrideAuthScheme = true`, the token will be used without a scheme prefix. If set, it will override the default `Bearer` scheme, even if `overrideAuthScheme` is `false`. |
| `overrideAuthScheme` | `false` | Determines whether to override the default `Bearer` scheme behavior. When set to `true`, the token is used without any scheme prefix unless `authScheme` is explicitly defined. |

#### Traceability

To support end-to-end traceability when requesting data from a REST-based data source, the REST integration can propagate the **PDP Request ID** to the downstream REST service. Based on the **Teiid driver** capability, each SQL query requested by the PDP includes the Authorization Request ID, and the PIP REST Translator uses it for logging and passing through to the underlying data source. This enables correlation and full traceability for logging and troubleshooting purposes. The Request ID received from the PDP as part of the query context is logged in PIP logging

For full traceability with the customer REST service used as a data source, a new optional **Translator Property: `traceabilityHeaderName` (Optional)** can be used. Use this property to define the HTTP header name that includes the PDP request ID when making a request to the REST service.

- If `traceabilityHeaderName` is **not set**, the request ID is logged only in connector logs at **INFO** level.
- If `traceabilityHeaderName` **is set**, the Translator adds a header with the specified name to the generated REST request and uses the extracted PDP request ID as the header value.
- If the same header is also configured in the `headers` Translator property, it is **overridden** by the value defined in `traceabilityHeaderName`.

### Secret Management for REST Headers

The REST Translator supports integration with **Secrets Management (SM)**. This integration allows sensitive header values, such as API keys or tokens, to be securely referenced instead of being defined in plain text.

When configuring the `headers` Translator property, you can reference a secret store key using the Secrets Management syntax. At Runtime, the REST Translator resolves the secret value and injects it into the generated REST request.

Refer to the [Secret Management](/v1/docs/secret-management-configuration-1) documentation for more information.

#### Secret Reference Syntax

Secret values must be referenced using the Secrets Management syntax:

```
{{store=<store_name>, key=<secret_key>, ...}}
```

The REST Translator uses the Secrets Management client to fetch the secret value from the configured store.

#### Header Configuration Example

```
{
  "X-API-Key": ["{{store=vault, key=finance-api-key}}"]
}
```

At Runtime:

- The secret is fetched from the specified store.
- The resolved secret value is used as the header value in the REST request.

#### Caching Behavior

Secrets fetched by the Secrets Management client may be **cached and reused** across REST requests to avoid fetching the secret for every query. Caching behavior is controlled by the secret configuration itself, for example, by using the secret `interval` property.

#### Error Handling

- If a referenced secret cannot be found or fetched during header generation, the REST request **fails**.
- The PIP returns an error indicating that the required secret store key is missing or unavailable.
- Detailed error information is logged to assist with troubleshooting.

#### Logging and Security

- If REST requests or headers are logged by the PIP, injected secret values are **masked**.
- Sensitive values are never written to logs in clear text.

---

### Partial Filtering

The Partial Filtering function in the REST translator enhances data filtering from a REST data source. It enables filtering even when the REST service does not support the dynamic filters required for data integration and policy evaluation. This function leverages the `supportsFilter` property to push supported filters down to the REST service, while additional filtering is simultaneously applied to the retrieved subset of data. This approach balances performance with flexibility, accommodating a wide range of use cases.

To set up partial filtering on your Attributes:

1. Set the `supportsFilter` and `supportsPartialFiltering` properties to `true`. Both parameters are `false` by default.
2. Configure the Dynamic Parameters supported by the REST service so they can be passed down in the service call.
3. Add the option `"teiid_rel:partial_filter"` `'true'` for each Attribute you want to filter further after receiving the response from the REST Data Source.

#### Partial Filter and Full Configuration Examples

**Partial Filter Configuration Example:**

```
attributeA string OPTIONS ("teiid_rel:partial_filter" 'true'),
attributeB string OPTIONS ("teiid_rel:partial_filter" 'true'),
attributeC string
```

In this example:

- `AttributeA` and `AttributeB` are set up for partial filtering, meaning they can be further filtered.
- `AttributeC` does not have partial filtering enabled. Depending on the data, filtering with or for `AttributeC` may result in either a full payload response or no response at all.

**Full DDL examples with Partial Filtering:** Example 1:

```
CREATE FOREIGN TABLE REST_PIP_Table( 
attributeA string OPTIONS ("teiid_rel:partial_filter" 'true'),
attributeB string OPTIONS ("teiid_rel:partial_filter" 'true'),
attributeC string
) OPTIONS (nameinsource '$.data[*]', "plainid:unwrap" 'true', "plainid:method" 'GET', "plainid:URI" '/byAttributeA/$(AttributeA)')
```

Example 2: x

```
CREATE FOREIGN TABLE accountData_byUser(
    userId string OPTIONS (nameinsource 'user', "teiid_rel:partial_filter" 'true'),
    accountId string OPTIONS (nameinsource 'accountsData[*].accountName', "teiid_rel:partial_filter" 'true'),
    balance string OPTIONS (nameinsource 'accountsData[*].accountBalance', "teiid_rel:partial_filter" 'true')
) OPTIONS (nameinsource '$', "plainid:method" 'GET', "plainid:unwrap" 'true', "plainid:URI" '/accountsData/$(userId)');
```

### Data Source Filtering Parameters

| Parameter | Description | Values |
| --- | --- | --- |
| **supportsfilter** | Indicates whether the Data Source supports filtering and expects a dynamic value sent as part of the REST request. | `true`, `false` (default) |
| **supportsPartialFiltering** | Enables partial filtering for specified attributes. When this parameter is set, it indicates that additional filtering is necessary because not all filtering criteria were applied before the data was retrieved from the REST Data Source. | `true`, `false` (default) |

Set the `supportsPartialFiltering` property to `true` to use the optional property added to attributes.

### REST Error Handling

Due to request misconfigurations, temporary network issues, or service issues, REST services might return errors in some cases. If an error occurs, ensure that the data source was created properly. You can also use the following translator properties to configure how errors are handled in the response:

| Property | Description | Value |
| --- | --- | --- |
| **throwHttpErrors** | This property allows users to control if an error is thrown or an empty response is sent instead. | `true` (default), `false` |
| **handledHttpErrorCodes** | This property specifies which HTTP error codes to handle - for these error codes an empty response will be sent from PIP instead of an error. To use this property, ensure that the `throwHttpErrors` property is set to `true`. | A list of comma-separated error codes. Use `xx` as wildcards if you wish to cover a pattern of error codes. E.g. `400, 401, 501`, etc. |