Configuration Guide

CompleteConfiguration Guide

v1.4.0 configuration guide: πŸ†• Enterprise multi-tenancy, index access control, query caching, cross-cluster search, hot reload configuration

Hot Reload Configuration
πŸ†• Index Access Control
πŸ†• Query Result Caching
πŸ†• Enterprise Multi-Tenancy

Configuration Categories

Detailed configuration instructions organized by functionality

Basic Configuration

Elasticsearch connection and basic service settings

elasticsearch.urlRequired

Elasticsearch base URL (required)

elasticsearch.url: http://localhost:9200
elasticsearch.username

ES basic authentication username

elasticsearch.username: elastic
elasticsearch.password

ES basic authentication password

elasticsearch.password: your_password
server.port

HTTP service port

server.port: 8000
elasticsearch.requestTimeoutSeconds

Request timeout in seconds

elasticsearch.requestTimeoutSeconds: 30

Cross-Cluster Search Configuration

CCS alias routing configuration for multi-cluster management

elasticsearch.clusters.logs.url

Log cluster address

elasticsearch.clusters.logs.url: http://es-logs:9200
elasticsearch.clusters.metrics.url

Metrics cluster address

elasticsearch.clusters.metrics.url: http://es-metrics:9200
elasticsearch.clusters.{alias}.username

Cluster authentication username

elasticsearch.clusters.{alias}.username: elastic

Security Configuration

IP access control, query limits, and protection policies

proxy.allowList

IP whitelist, supports CIDR format

proxy.allowList: ["127.0.0.1", "10.0.0.0/24"]
proxy.blockList

IP blacklist (higher priority than whitelist)

proxy.blockList: []
proxy.limits.rateLimit

Access rate limit

proxy.limits.rateLimit: 100/minute
proxy.limits.maxQueryDepth

Maximum query nesting depth

proxy.limits.maxQueryDepth: 5
proxy.limits.maxResultWindow

Maximum pagination window

proxy.limits.maxResultWindow: 10000
proxy.limits.allowPrefixWildcard

Allow prefix wildcards

proxy.limits.allowPrefixWildcard: false
proxy.limits.strictQueryValidation

Strict query validation

proxy.limits.strictQueryValidation: false

Multi-Tenant Configuration

πŸ†• v1.4.0 Enterprise multi-tenant isolation and tenant management

proxy.multi-tenant.enabled

Enable multi-tenant functionality (πŸ†• v1.4.0 core feature)

proxy.multi-tenant.enabled: true
proxy.multi-tenant.defaultTenant

Default tenant ID (backward compatibility)

proxy.multi-tenant.defaultTenant: default
proxy.multi-tenant.strictTenantIsolation

Strict tenant isolation mode

proxy.multi-tenant.strictTenantIsolation: true
proxy.multi-tenant.allowAnonymousAccess

Allow anonymous access (development environment)

proxy.multi-tenant.allowAnonymousAccess: false
proxy.multi-tenant.tenants.{tenant-id}.name

Tenant display name

proxy.multi-tenant.tenants.{tenant-id}.name: Log Analysis Tenant
proxy.multi-tenant.tenants.{tenant-id}.basicAuth.usernameRequired

HTTP Basic authentication username

proxy.multi-tenant.tenants.{tenant-id}.basicAuth.username: tenant-user
proxy.multi-tenant.tenants.{tenant-id}.basicAuth.passwordRequired

HTTP Basic authentication password

proxy.multi-tenant.tenants.{tenant-id}.basicAuth.password: tenant-password
proxy.multi-tenant.tenants.{tenant-id}.allowedIps

Tenant IP access control (array)

proxy.multi-tenant.tenants.{tenant-id}.allowedIps: ["10.1.0.0/24", "192.168.1.0/24"]
proxy.multi-tenant.tenants.{tenant-id}.elasticsearch.cluster

Tenant ES cluster alias

proxy.multi-tenant.tenants.{tenant-id}.elasticsearch.cluster: logs
proxy.multi-tenant.tenants.{tenant-id}.elasticsearch.username

Tenant ES credentials username

proxy.multi-tenant.tenants.{tenant-id}.elasticsearch.username: es_tenant_user
proxy.multi-tenant.tenants.{tenant-id}.limits.allowedIndices

Tenant allowed index patterns

proxy.multi-tenant.tenants.{tenant-id}.limits.allowedIndices: ["logs-*", "app-logs-*"]
proxy.multi-tenant.tenants.{tenant-id}.limits.forbiddenIndices

Tenant forbidden index patterns

proxy.multi-tenant.tenants.{tenant-id}.limits.forbiddenIndices: ["system-*", ".security*"]
proxy.multi-tenant.tenants.{tenant-id}.monitoring.enabled

Tenant independent monitoring

proxy.multi-tenant.tenants.{tenant-id}.monitoring.enabled: true

Performance Optimization

Connection pool, network, and performance-related configuration

elasticsearch.maxConnections

Maximum concurrent connections

elasticsearch.maxConnections: 200
elasticsearch.pendingAcquireMaxCount

Connection waiting queue size

elasticsearch.pendingAcquireMaxCount: 10000
elasticsearch.connectTimeoutMillis

TCP connection timeout in milliseconds

elasticsearch.connectTimeoutMillis: 3000
elasticsearch.maxIdleTimeSeconds

Maximum connection idle time in seconds

elasticsearch.maxIdleTimeSeconds: 30
elasticsearch.maxLifeTimeSeconds

Maximum connection lifetime in seconds

elasticsearch.maxLifeTimeSeconds: 300

Monitoring Configuration

Logging, metrics monitoring, and health checks

proxy.useElasticsearchMonitoring

Enable ES monitoring logs (daily indexed)

proxy.useElasticsearchMonitoring: true
proxy.logLevel

Proxy log level

proxy.logLevel: INFO
management.endpoints.web.exposure.include

Exposed Actuator endpoints

management.endpoints.web.exposure.include: health,info,prometheus,refresh
management.endpoint.health.probes.enabled

Enable liveness/readiness probes

management.endpoint.health.probes.enabled: true
spring.elasticsearch.uris

Actuator health check ES address

spring.elasticsearch.uris: http://localhost:9200

Configuration Examples

Complete configuration examples for different environments

Development Environment Complete Configuration

Complete development and testing environment configuration with all major configuration items

application-dev.yml
# ===================================
# Development Environment Complete Configuration
# ===================================

# HTTP Server Configuration
server:
  port: 8000                          # Development environment port

# Management endpoints configuration
management:
  endpoints:
    web:
      exposure:
        # Development environment exposes more endpoints for debugging
        include: health,info,refresh,configprops,unified-search
  endpoint:
    health:
      probes:
        enabled: true                 # Enable health probes
        
# Spring framework configuration
spring:
  application:
    name: ElasticProxyServer-Dev      # Development environment app name
  http:
    encoding:
      charset: UTF-8
      enabled: true
      force: true
  # Actuator health check ES configuration
  elasticsearch:
    uris: http://localhost:9200
    username: elastic
    password: your_password
    connection-timeout: 3s
    socket-timeout: 30s

# Main business Elasticsearch configuration
elasticsearch:
  url: http://localhost:9200          # Development environment ES address
  username: "elastic"                 # ES authentication username
  password: "your_password"           # ES authentication password (please replace with actual password, environment variables recommended)
  requestTimeoutSeconds: 30           # Request timeout
  
  # Development environment connection pool configuration (smaller)
  maxConnections: 50                  # Fewer connections for development
  pendingAcquireMaxCount: 1000        # Smaller waiting queue
  pendingAcquireTimeoutSeconds: 5
  connectTimeoutMillis: 3000
  maxIdleTimeSeconds: 30
  maxLifeTimeSeconds: 300
  
  # Cross-cluster search configuration (optional, development usually single cluster)
  clusters: {}                        # No cross-cluster in development

# Proxy configuration
proxy:
  logLevel: DEBUG                     # Detailed logging for development
  useElasticsearchMonitoring: false  # Disable monitoring in development to reduce interference
  allowList: []                       # No IP restrictions in development
  blockList: []
  
  # Relaxed access restrictions for development
  limits:
    rateLimit: 1000/minute            # Relaxed rate limiting for testing
    maxResultWindow: 50000            # Larger pagination window
    maxQueryDepth: 10                 # Allow deeper nesting
    maxAggregationSize: 1000          # Larger aggregation
    maxSearchTerms: 2000
    allowPrefixWildcard: true         # Allow wildcards in development
    strictQueryValidation: false      # Relaxed validation
    maxQueryComplexity: 200
    allowRegexpQuery: true            # Allow regex in development
    regexpMaxLength: 100
  
  # Unified search configuration
  unified-search:
    enabled: true                     # Enable unified search
    defaultMatchAnalyzer: "standard"
    maxConditions: 100                # Allow more conditions in development
    timeoutSeconds: 60                # Longer timeout for debugging
    
    # Field recognition configuration
    rangeFields: ["date", "timestamp", "created", "updated", "bizDate", "price", "amount"]
    exactMatchFields: ["id", "status", "type", "userId", "orderId"]
    
    # Development environment access control
    allowedIndices: ["*"]             # Allow access to all indices in development
    
    # Cache configuration
    enableQueryCache: false           # Disable cache in development for testing
    queryCacheTtlSeconds: 60
    
    # Monitoring configuration
    strictTypeDetection: false
    enableQueryLogging: true          # Enable logging in development for debugging
    enablePerformanceMetrics: true

Production Environment Complete Configuration

Complete production environment configuration based on actual config/application.yml file

application-prod.yml
# ===================================
# Production Environment Complete Configuration
# Based on actual config/application.yml file
# ===================================

# HTTP Server Configuration
server:
  port: 8000                          # Production service port

# Management endpoints configuration
management:
  endpoints:
    web:
      exposure:
        # Selected management endpoints for production
        include: health,info,prometheus,refresh,configprops,unified-search
  endpoint:
    health:
      probes:
        enabled: true                 # Kubernetes probe support

# Spring framework configuration
spring:
  application:
    name: ElasticProxyServer          # Production app name
  http:
    encoding:
      charset: UTF-8                  # UTF-8 encoding
      enabled: true
      force: true
  # Actuator health check dedicated ES configuration (separated from main business)
  elasticsearch:
    uris: http://172.168.0.100:19200
    username: elastic
    password: "******"
    connection-timeout: 3s            # Health check connection timeout
    socket-timeout: 30s               # Health check socket timeout

# Main business Elasticsearch configuration
elasticsearch:
  url: http://172.168.0.100:19200     # Main ES cluster address
  username: "elastic"                 # ES authentication username
  password: "******"                  # ES authentication password (please replace with actual password, environment variables recommended)
  requestTimeoutSeconds: 30           # Request timeout 30 seconds
  
  # Production environment connection pool configuration
  maxConnections: 200                 # Maximum concurrent connections
  pendingAcquireMaxCount: 10000       # Connection waiting queue size
  pendingAcquireTimeoutSeconds: 5     # Connection acquisition timeout
  connectTimeoutMillis: 3000          # TCP connection timeout 3 seconds
  maxIdleTimeSeconds: 30              # Maximum connection idle 30 seconds
  maxLifeTimeSeconds: 300             # Maximum connection lifetime 5 minutes
  
  # Cross-cluster search (CCS) configuration
  clusters:
    logs:                             # Log cluster alias
      url: http://172.168.0.100:19200
      username: "elastic"
      password: "******"
    metrics:                          # Metrics cluster alias
      url: http://172.168.0.100:19200
      username: "elastic"
      password: "******"

# Proxy configuration
proxy:
  logLevel: INFO                      # Production INFO level logging
  useElasticsearchMonitoring: true   # Enable ES monitoring (daily indexed)
  allowList: ["127.0.0.1", "10.0.0.0/24"]  # Production IP whitelist
  blockList: []                       # IP blacklist (currently empty)
  
  # Production environment access restriction configuration
  limits:
    rateLimit: 100/minute             # Production rate limit 100/minute
    maxResultWindow: 10000            # Maximum pagination window 10000
    maxQueryDepth: 5                  # Maximum nesting depth 5
    maxAggregationSize: 500           # Maximum aggregation size 500
    maxSearchTerms: 1000              # Maximum search terms 1000
    allowPrefixWildcard: false        # Disable prefix wildcards in production
    strictQueryValidation: false      # Query validation mode
    maxQueryComplexity: 100           # Maximum query complexity 100
    allowRegexpQuery: false           # Disable regex queries in production
    regexpMaxLength: 50               # Regex length limit 50 characters
  
  # Unified search complete configuration
  unified-search:
    # === Basic Configuration ===
    enabled: true                     # Enable unified search interface
    defaultMatchAnalyzer: "standard"  # Default analyzer
    maxConditions: 50                 # Maximum query conditions 50
    timeoutSeconds: 30                # Query timeout 30 seconds
    
    # === Intelligent Field Recognition Configuration ===
    # Range query fields (date, numeric types)
    rangeFields: ["date", "time", "timestamp", "created", "updated", "bizDate", "biz_date", "price", "amount", "count", "quantity", "age", "score", "version"]
    # Exact match fields (ID, status types)
    exactMatchFields: ["id", "code", "status", "type", "category", "corpid", "corp_id", "userId", "user_id", "orderId", "order_id", "transactionId", "transaction_id"]
    
    # === v1.3.6 Index Access Control ===
    allowedIndices: ["test1", "elastic-proxy-logs-*", "logs-*", "metrics-*", "business-*"]
    
    # === v1.3.6 Query Result Caching ===
    enableQueryCache: true            # Enable query caching
    queryCacheTtlSeconds: 300         # Cache TTL 5 minutes
    
    # === v1.3.7 Advanced Features ===
    strictTypeDetection: false        # Strict type detection mode
    enableQueryLogging: true          # Enable query logging
    enablePerformanceMetrics: true    # Enable performance metrics collection

πŸ†• Multi-Tenant Production Configuration

v1.4.0 enterprise multi-tenant configuration example with completely isolated tenant management

application-multi-tenant.yml
# ===================================
# v1.4.0 Multi-Tenant Production Configuration
# Enterprise multi-tenant isolation functionality
# ===================================

# HTTP Server Configuration
server:
  port: 8000                          # Multi-tenant proxy service port

# Management endpoints configuration
management:
  endpoints:
    web:
      exposure:
        # Include tenant management endpoints
        include: health,info,prometheus,refresh,tenants
  endpoint:
    health:
      probes:
        enabled: true                 # Kubernetes probe support

# Spring framework configuration
spring:
  application:
    name: ElasticProxyServer-MultiTenant
  http:
    encoding:
      charset: UTF-8
      enabled: true
      force: true
  # Actuator health check ES configuration
  elasticsearch:
    uris: http://172.168.0.100:19200
    username: elastic
    password: "******"
    connection-timeout: 3s
    socket-timeout: 30s

# Main business Elasticsearch configuration
elasticsearch:
  url: http://172.168.0.100:19200     # Default ES cluster address
  username: "elastic"                 # Default ES authentication username
  password: "******"                  # Default ES authentication password
  requestTimeoutSeconds: 30           # Request timeout 30 seconds
  
  # Production environment connection pool configuration
  maxConnections: 200                 # Maximum concurrent connections
  pendingAcquireMaxCount: 10000       # Connection waiting queue size
  pendingAcquireTimeoutSeconds: 5     # Connection acquisition timeout
  connectTimeoutMillis: 3000          # TCP connection timeout 3 seconds
  maxIdleTimeSeconds: 30              # Maximum connection idle 30 seconds
  maxLifeTimeSeconds: 300             # Maximum connection lifetime 5 minutes
  
  # Multi-cluster configuration (for tenant use)
  clusters:
    logs:                             # Log cluster alias
      url: http://172.168.0.100:19200
      username: "elastic"
      password: "******"
    metrics:                          # Metrics cluster alias
      url: http://172.168.0.101:9200
      username: "elastic"
      password: "******"
    business:                         # Business cluster alias
      url: http://172.168.0.102:9200
      username: "elastic"
      password: "******"

# Proxy configuration
proxy:
  logLevel: INFO                      # Production INFO level logging
  useElasticsearchMonitoring: true   # Enable ES monitoring
  allowList: ["127.0.0.1", "10.0.0.0/24"]  # Global IP whitelist
  blockList: []                       # Global IP blacklist
  
  # Global access restriction configuration
  limits:
    rateLimit: 100/minute             # Global rate limit
    maxResultWindow: 10000            # Global pagination limit
    maxQueryDepth: 5                  # Global query depth
    maxAggregationSize: 500
    maxSearchTerms: 1000
    allowPrefixWildcard: false        # Global disable prefix wildcards
    strictQueryValidation: false
    maxQueryComplexity: 100
    allowRegexpQuery: false
    regexpMaxLength: 50
  
  # === πŸ†• v1.4.0 Multi-Tenant Configuration ===
  multi-tenant:
    # Basic multi-tenant configuration
    enabled: true                     # Enable multi-tenant functionality
    defaultTenant: "default"          # Default tenant (backward compatibility)
    strictTenantIsolation: true       # Strict tenant isolation
    allowAnonymousAccess: false       # Disable anonymous access in production
    
    # Tenant cache configuration
    cache:
      tenantContextTtlMinutes: 15     # Tenant context cache TTL
      maxCachedTenants: 1000          # Maximum cached tenants
      basicAuthCacheTtlMinutes: 30    # Basic auth cache TTL
    
    # Tenant configuration
    tenants:
      # Default tenant (maintain backward compatibility)
      default:
        name: "Default Tenant"
        basicAuth:
          username: "default"
          password: "change-me-default-secret"
        elasticsearch:
          inherit: true                # Inherit global ES configuration
        limits:
          inherit: true                # Inherit global limits
        unified-search:
          inherit: true                # Inherit unified search configuration
        monitoring:
          enabled: true
          
      # Log analysis tenant
      tenant-logs:
        name: "Log Analysis Tenant"
        description: "Tenant dedicated to log data analysis"
        basicAuth:
          username: "logs-user"
          password: "change-me-logs-secret"
        allowedIps: ["10.1.0.0/24", "192.168.1.0/24"]
        elasticsearch:
          cluster: "logs"              # Use log cluster
          username: "es_logs_user"
          password: "change-me-es-logs-password"
        limits:
          rateLimit: "200/minute"      # Tenant-specific rate limit
          maxResultWindow: 15000
          allowedIndices: ["logs-*", "app-logs-*"]
          forbiddenIndices: ["system-*", ".security*"]
        unified-search:
          enabled: true
          allowedIndices: ["logs-*"]
        monitoring:
          enabled: true
          indexPrefix: "tenant-logs-proxy-logs"
          
      # Metrics analysis tenant
      tenant-metrics:
        name: "Metrics Analysis Tenant"
        description: "Tenant dedicated to performance metrics analysis"
        basicAuth:
          username: "metrics-user"
          password: "change-me-metrics-secret"
        allowedIps: ["10.2.0.0/24"]
        elasticsearch:
          cluster: "metrics"           # Use metrics cluster
          username: "es_metrics_user"
          password: "change-me-es-metrics-password"
        limits:
          rateLimit: "150/minute"
          maxResultWindow: 10000
          allowedIndices: ["metrics-*", "performance-*"]
          forbiddenIndices: ["logs-*", "business-*"]
        unified-search:
          enabled: true
          allowedIndices: ["metrics-*"]
          timeoutSeconds: 45
        monitoring:
          enabled: true
          indexPrefix: "tenant-metrics-proxy-logs"
          
      # Business data tenant
      tenant-business:
        name: "Business Data Tenant"
        description: "Business core data access tenant"
        basicAuth:
          username: "business-user"
          password: "change-me-business-secret"
        allowedIps: ["10.3.0.0/24", "192.168.10.0/24"]
        elasticsearch:
          cluster: "business"          # Use business cluster
          username: "es_business_user"
          password: "change-me-es-business-password"
        limits:
          rateLimit: "300/minute"      # Higher rate limit for business tenant
          maxResultWindow: 20000
          allowedIndices: ["business-*", "orders-*", "users-*"]
          forbiddenIndices: ["logs-*", "metrics-*", "system-*"]
          allowedOperations: ["search", "count", "explain"]
        unified-search:
          enabled: true
          allowedIndices: ["business-*", "orders-*"]
          maxConditions: 80
          timeoutSeconds: 30
        monitoring:
          enabled: true
          trackQueries: true
          trackPerformance: true
          indexPrefix: "tenant-business-proxy-logs"
  
  # Unified search global configuration
  unified-search:
    enabled: true                     # Global enable unified search
    defaultMatchAnalyzer: "standard"
    maxConditions: 50                 # Global default conditions
    timeoutSeconds: 30                # Global default timeout
    rangeFields: ["date", "time", "timestamp", "created", "updated", "bizDate", "price", "amount"]
    exactMatchFields: ["id", "code", "status", "type", "userId", "orderId"]
    allowedIndices: ["*"]             # Global default (will be overridden by tenant configuration)
    enableQueryCache: true            # Global query cache
    queryCacheTtlSeconds: 300
    strictTypeDetection: false
    enableQueryLogging: true
    enablePerformanceMetrics: true

Configuration Best Practices

Best practice recommendations for production environment configuration

Security Best Practices

  • πŸ” Configure index access control to limit accessible index patterns
  • πŸ”’ Use environment variables to store passwords, avoid plain text in configuration files
  • πŸ›‘οΈ Set production configuration file permissions to 600 (owner read/write only)
  • Use IP whitelist to restrict access sources
  • Enable strict query validation to prevent malicious queries
  • Disable prefix wildcards and regex queries
  • Set reasonable access rate limits
  • Regularly update passwords and access keys
  • Separate Actuator health check ES configuration

Performance Best Practices

  • ⚑ Enable query result caching for significant performance improvement
  • Adjust connection pool size based on load (maxConnections)
  • Optimize connection timeout and lifetime configuration
  • Configure reasonable query cache TTL time
  • Monitor cache hit rate and performance metrics
  • Use SSD storage for log files
  • Increase pendingAcquireMaxCount for high-load scenarios

v1.3.6 New Feature Configuration

  • πŸ“Š Configure intelligent field recognition to improve query accuracy
  • Set reasonable allowedIndices index access control
  • Enable query caching and monitor cache statistics
  • Configure rangeFields and exactMatchFields
  • Enable query logging and performance statistics
  • Regularly clean query cache
  • Use cross-cluster search alias routing

Operations Best Practices

  • Enable ES monitoring log recording (daily indexed)
  • Configure Actuator health check endpoints
  • Enable liveness and readiness probes
  • Regularly backup configuration files
  • Use configuration hot reload functionality (POST /actuator/refresh)
  • Set up Prometheus monitoring and alerting
  • Monitor cache usage and hit rate

πŸ†• v1.4.0 Multi-Tenant Best Practices

  • 🏒 Create independent tenants for different business lines
  • πŸ”‘ Configure independent ES credentials for each tenant
  • πŸ›‘οΈ Use strict tenant isolation mode (strictTenantIsolation: true)
  • πŸ“Š Configure independent monitoring index prefix for each tenant
  • πŸ”’ Configure tenant-level IP access control
  • ⚑ Adjust rate limiting policies based on tenant business characteristics
  • πŸ“‹ Regularly review tenant index access permissions
  • πŸ”„ Use tenant management API for connection testing
  • πŸ’Ύ Enable tenant context caching for performance optimization
  • πŸ“ˆ Monitor tenant cache hit rate and performance metrics

Configuration Hot Reload

Configuration changes take effect without restarting the service

Secure Configuration and Hot Reload
# Use environment variables to start service (recommended)
export ES_PASSWORD="your_actual_password"
export ES_URL="http://your-es-cluster:9200"
java -jar elasticproxy-server.jar

# After modifying configuration file, trigger hot reload
curl -X POST http://localhost:8000/actuator/refresh

# Verify configuration update
curl http://localhost:8000/actuator/health