PIP Operator Service
    • 30 Mar 2025
    • 3 Minutes to read
    • Dark
      Light
    • PDF

    PIP Operator Service

    • Dark
      Light
    • PDF

    Article summary

    The PIP Operator Service is used for managing, deploying, and mapping data sources.

    Installation Guide

    • Setup environment variables:
      • REDIS_HOST - Redis host
      • REDIS_PORT - Redis port
      • REDIS_PASS - Redis password (default is with no password)

    For environments like staging and production it is recommended to use an external and managed Redis instance.

    Notice: Redis must enable notify-keyspace-events so either allow configuration changes to redis from the app, or change them yourself manually.
    Additional settings can be found under Environmental Variables below.

    For Users with Custom Deployments

    When upgrading to the pip-operator image with the tag 5.2448.4 and up, ensure that you configure the PIP service by adding the JAVA_OPTS environment variable with the new Java system property:
    Dorg.apache.cxf.transport.http.forceVersion=1.1

    Environment Variables

    Management

    • SERVER_PORT - Server Port
    • MANAGEMENT_PORT - Port for the info and health checks
    • APP_AUTH_ENABLED - Enable/disable jwt authentication (default is false)
    • APP_AUTH_JWT - JWT for authentication
    • APP_AUTH_SECRET - JWT secret key
    • APP_AUTH_ISS - JWT Issuer
    • TRANSACTION_MANAGER_ENABLED - Enable/disable transaction manager
    • TRANSACTION_MANAGER_COORDINATOR_ENVIRONMENT_BEAN_DEFAULT_TIMEOUT - Transaction Manager time out (default is 300 seconds)
    • REDIS_HOST - Redis host
    • REDIS_PORT - Redis port
    • REDIS_PASS - Redis password (default is with no password)
    • REDIS_SSL_ENABLED - Enable/disable SSL (default is false)
    • REDIS_TIMEOUT_MIL - Redis connection timeout in milliseconds (default value is 60000)
    • REDIS_POOL_MAX_TOTAL - This setting controls the max number of connections that can be created at a given time.
    • REDIS_POOL_MAX_IDLE - This is the max number of connections that can be idle in the pool without being immediately evicted (closed).
    • REDIS_POOL_MIN_IDLE- This is the number of "warm" connections (e.g. ready for immediate use) that remain in the pool even when load has reduced
    • REDIS_POOL_MIN_EVIC_MI - This specifies the minimum amount of time a connection may sit idle in the pool before it is eligible for eviction due to idle time.
    • REDIS_POOL_TIME_BETWEEN_EVIC_MIL - The number of milliseconds to sleep between runs of the idle connection validation, abandoned cleaner and idle pool resizing.
    • MV_STATUS_LOGGING_FREQUENCY - Determines whether Mat View status is retrieved and logged, and specifies the logging frequency. Default is 60 minutes.
    • MV_STATUS_LOGGING_EXTERNAL_STATUS_TABLES - Specifies the name(s) of external materialized view status table(s) as defined in the materialized view property teiid_rel:MATVIEW_STATUS_TABLE, in the form of <model_name>.<status_table_name>. Use the MV_STATUS_LOGGING_FREQUENCY configuration key to set how often the status is fetched and logged.
    • APP_MAX_THREADS (default: 64) - Sets the number of threads for query engine work (plans, transactions, source queries). Increase for high-concurrency workloads or systems with many processors.
    • APP_MAX_ACTIVE_PLANS (default: 20) - Sets the number of active query plans. Increase for long-running queries or high-concurrency workloads. Adjust memory settings if issues arise. Note: This should be smaller than max-threads.
    • APP_USER_REQUEST_SOURCE_CONCURRENCY (default: 0) - Sets concurrent source queries per request. 0 uses the default (2 * (max-threads / max-active-plans)), 1 forces serial execution, and values greater than a 1 limit concurrency. Adjust for workloads with many long-running queries. Note: This should be smaller than max-threads.
    • QUERY_QUEUE_STATUS_LOGGING_FREQUENCY(default: 1 minute) - Controls how often PIP queries and logs the VDB query queue status. Set to 0 to disable or a value greater than 0 (in minutes) to log at the INFO level.
    • ENABLE_VDB_HEALTH_METRICS: Enables the collection and exposure of 3 new Prometheus metric counters for PIP Request Queue and better VDB health visibility:
      • requests_view_requests_count
      • requests_view_requests_count_is_source_true
      • requests_view_requests_count_is_source_false

    VDB

    • VDB_USERNAME - Set the username for the jdbc connection (default is pip)
    • VDB_PASSWORD - Set the username for the jdbc connection (default is pa33word)
    • TRANSPORT_PROTOCOL - VDB transport protocol (default is teiid)
    • TRANSPORT_BIND_ADDRESS - Transport bind address (default is 0.0.0.0)

    Logging

    The available log levels are: error, warn, info, trace, and debug

    • LOGGING_FORMAT - Logging format - JSON or text (default is JSON)
    • LOGGING_ROOT_LEVEL - Root logging level (default is error)
    • LOGGING_TOMCAT_LEVEL - Tomcat logging level (default is error)
    • LOGGING_WEB_LEVEL - Web logging level (default is error)
    • LOGGING_PLAINID_LEVEL - PlainID components logging level (default is debug)
    • LOGGING_TEIID_LEVEL - Teiid logging level (default is info)
    • LOGGING_TEIID_RUNTIME_LEVEL - Teiid runtime logging level (default is info)
    • LOGGING_TEIID_CONNECTOR_LEVEL - Teiid connector logging level (default is info)
    • LOGGING_TEIID_COMMAND_LOG_LEVEL - Teiid command log level (default is info)

    Proxy

    • PROXY_SOCKET_TIMEOUT - How long should the proxy socket remain open after initial handshake (default is 2 hours)
    • PROXY_SOCKET_BUFFER_SIZE - Buffer size of the proxy input stream (default is 8192 bytes)
    • PROXY_SERVER_PORT - Which port the Tiied proxy server should listen to (default is 31350)

    Agent

    • AGENT_URL - URL of the agent on the client side (default is http://localhost:8761)
       

    For users using their own deployment procedure (instead of Helm or Standalone), ensure that you add the PAA Version Number to the Agent Deployment.
    Environment Variable Example: PAA_PACKAGE_VERSION: PAA VERSION NUMBER

    Health-check

    URL

    http://<host>:<port>/actuator/health
    

    OK Response

    {
       "status": "UP"
    }
    

    Java

    The PlainID Agent runs with Java 17 and above.

    Command Line

        java -jar pip-operator-1.0.jar
    

    Was this article helpful?