Advanced Configs

Configure model parameters and advanced settings

Wondering what the advanced settings are and how to use them? We've outlined the most common advanced settings in our The AI Settings Dictionary blog post. Please check out this article for the most common settings - in which Msty offers quick configs for these common settings.

This document will cover the even more advanced settings that are not included in the dictionary. These settings you'd typically add as json to the Advanced Parameter or Provider Options sections of the Model Parameters section.

Please note that these settings are generally available across models; however, a model may not support all (or any) of these. If you try to add them to a model that doesn't support them, they will be ignored or an error may be thrown. Please check the documentation for the model provider you are using for their available settings.

Advanced Runtime Configurations

{
  "runtimeConfig": {
    "priority": "standard",           // Options: "low", "standard", "high"
    "timeout": 60000,                 // Request timeout in milliseconds
    "maxRetries": 3,                  // Maximum retry attempts
    "stream": true,                   // Enable streaming responses
    "streamChunkSize": 16,            // Number of tokens per streaming chunk
    "streamLatency": "low",           // Options: "low", "balanced", "consistent"
    "caching": {
      "enabled": true,                // Enable response caching
      "ttl": 3600                     // Cache time-to-live in seconds
    },
    "quotaManagement": {
      "tokensPerMinute": 10000,       // Rate limit for tokens
      "requestsPerDay": 1000          // Daily request limit
    },
    "logging": {
      "level": "info",                // Options: "debug", "info", "warn", "error"
      "includePrompts": false,        // Include prompts in logs
      "includeResponses": false       // Include responses in logs
    }
  }
}

Memory and Context Configurations

{
  "contextConfig": {
    "maxTokens": 128000,              // Maximum context window size
    "retentionStrategy": "recency",   // Options: "recency", "importance", "hybrid"
    "compressionRatio": 0.8,          // Context compression ratio (0.0-1.0)
    "memoryOptimization": "auto",     // Options: "auto", "aggressive", "conservative"
    "contextPrioritization": [        // Priority order for context management
      "user_instructions",
      "recent_exchanges",
      "relevant_facts",
      "system_instructions"
    ]
  }
}

Multimodal Configurations

{
  "multimodalConfig": {
    "visionEnabled": true,            // Enable vision capabilities
    "imageAnalysisDetail": "high",    // Options: "low", "medium", "high"
    "imageGenerationEnabled": false,  // Enable image generation
    "audioProcessingEnabled": false,  // Enable audio processing
    "videoAnalysisEnabled": false,    // Enable video analysis
    "modalityPreference": "balanced"  // Options: "text", "visual", "balanced"
  }
}

Output Formatting Configurations

{
  "outputConfig": {
    "responseFormat": "markdown",     // Options: "markdown", "plain", "json", "xml", "html"
    "verbosityLevel": "concise",      // Options: "minimal", "concise", "detailed", "comprehensive"
    "structurePreference": "narrative", // Options: "narrative", "analytical", "bullet", "dialog"
    "citationStyle": "academic",      // Options: "none", "informal", "academic", "legal"
    "languageModel": "formal",        // Options: "casual", "formal", "technical", "creative"
    "maxOutputTokens": 4096           // Maximum response length
  }
}

Safety and Guardrail Configurations

{
  "safetyConfig": {
    "contentFilters": {
      "harmfulContent": "strict",     // Options: "off", "standard", "strict"
      "hateContent": "standard",      // Options: "off", "standard", "strict"
      "sexualContent": "standard",    // Options: "off", "standard", "strict"
      "violenceContent": "standard"   // Options: "off", "standard", "strict"
    },
    "topicRestrictions": ["weapons", "illegal_activities"],
    "refusalBehavior": "explain",     // Options: "explain", "redirect", "minimal"
    "userVerification": false,        // Enable user verification for sensitive requests
    "contentWarnings": true           // Show content warnings when appropriate
  }
}

Sampling Configurations

{
  "samplingConfig": {
    "temperature": 0.7,               // Controls randomness (0.0-1.0)
    "top_p": 0.9,                     // Nucleus sampling parameter (0.0-1.0)
    "top_k": 40,                      // Limits vocabulary to top K tokens
    "frequencyPenalty": 0.5,          // Penalizes repetition (0.0-2.0)
    "presencePenalty": 0.5,           // Penalizes token reuse (0.0-2.0)
    "deterministicSampling": false,   // Forces deterministic outputs
    "seed": 123456                    // Random seed for reproducibility
  }
}

System Behavior Configurations

{
  "behaviorConfig": {
    "personality": "helpful",         // Options: "helpful", "concise", "creative", "analytical" 
    "roleplayMode": false,            // Enable character roleplay capabilities
    "expertiseDomains": ["coding", "science", "math"],
    "conversationalStyle": "balanced", // Options: "formal", "casual", "balanced", "technical"
    "initiativeLevel": "responsive",  // Options: "passive", "responsive", "proactive"
    "opinionExpression": "neutral",   // Options: "neutral", "balanced", "opinionated"
    "uncertaintyHandling": "transparent" // Options: "confident", "transparent", "conservative"
  }
}

Thinking and Reasoning Configurations

{
  "thinkingConfig": {
    "thinkingBudget": 0,              // Disable thinking mode (0) or set custom budget (positive integer)
    "thinkingDepth": "deep",          // Options: "shallow", "medium", "deep"
    "enableAutoThinking": false,      // Auto-enable thinking for complex queries
    "thinkingStrategy": "structured", // Options: "structured", "exploratory", "critical"
    "reasoningEffort": "low",         // Controls effort spent on reasoning 
    "microstat": true                 // Enable fine-grained statistical reasoning
  }
}

Tool Use and Integration Configurations

{
  "toolConfig": {
    "enabledTools": ["calculator", "web_search", "code_execution"],
    "toolCallThreshold": 0.7,         // Confidence threshold for tool calling (0.0-1.0)
    "maxToolCalls": 5,                // Maximum number of tool calls per request
    "autoToolSelection": true,        // Automatically select appropriate tools
    "toolPreferences": {
      "preferred": ["calculator"],    // Tools to prioritize
      "fallback": ["web_search"]      // Tools to use as fallback
    }
  }
}

Environments

Create and manage environments for your needs

Insights

View insights into token usage for your models