Skip to main content

Deployment on Azure

Preparation Before Deployment​

Required Resources​

  • A new subscription dedicated to product deployment, and ensure that this subscription can create OpenAI-related resources.
  • An Azure AD account with Contributor or higher permissions on the subscription.
    • You can add an existing AAD account as a Guest to the tenant and grant permissions, or create a dedicated AAD account for deployment.
    • The account must be able to create and configure AAD application registrations (e.g., grant the Application Developer role).

Required Tools​

Product Deployment Architecture and Resource List​

We provide two deployment options in the Terraform scripts, which can be selected as needed.

Standard Deployment Architecture

Private Endpoint Deployment Architecture, which achieves internal resource network isolation through Private Endpoint/Vnet, further enhancing security.

tip

When using Private Endpoint, in addition to the basic per-quantity charges, there will be additional network traffic fees.

Component Usage Description

TypeCustom NameDescription
Azure Front DoorCDN(Optional) Content Delivery Network, used to accelerate access to static resources
Azure Front DoorAPI Acceleration(Optional) API acceleration, used to speed up API access
API ManagementAPI EndpointAPI management endpoint, used to manage APIs
App ServiceAPI ServiceAPI service, used to handle API requests
App ServiceJob ServiceJob service, used to handle high-load tasks and scheduled jobs
Azure OpenAI ServiceLLM-PrimaryPrimary large language model, used as the main model for Agent Q&A (e.g., gpt-4o)
Azure OpenAI ServiceLLM-SecondarySecondary large language model, used as an auxiliary model for document processing (e.g., gpt-4o-mini)
Azure OpenAI ServiceEmbeddingVector embedding, used for processing vector data
Azure OpenAI ServiceText to ImageText-to-image generation, used for text-to-image tasks
Azure OpenAI ServiceWhisperSpeech recognition, used for speech recognition tasks
Azure AI Document IntelligenceDocument OCRDocument OCR recognition, used for document OCR tasks
Azure AI servicesDocument TranslateDocument translation, used for document translation tasks
Azure SQL DatabaseDataData storage, main database, most system data is stored here
Azure Database for PostgreSQLIndexIndex database, mainly stores document-related index data
Azure Cache for RedisCacheCache service, used for data caching
Storage AccountsFilesFile storage, used to store user-uploaded files
Key Vault-Key management, used to store keys
Azure Monitor-(Optional) Resource monitoring service, collects and analyzes resource metrics
Application Insights-(Optional) Application monitoring service, used for application performance monitoring and troubleshooting
Vnet/Subnet-(Optional) Defines logical isolation networks and subnets for Azure resources
Private Endpoint-(Optional) Securely connect to Azure services via private IP
Private DNS Zone-(Optional) Private DNS, resolves service FQDNs to private IPs
Managed Identity-(Optional) Managed identity, credentials for communication between resources
Public IP-(Optional) Public IP address, when using Private Endpoint deployment architecture, Standard tier API Management requires public IP configuration
Network Security Group-(Optional) Network security group, used to control network access to resources

Reference Configurations

tip

The above configurations are for reference only and should be adjusted according to actual load conditions.

Product Deployment​

Deployment Process

  • Execute deployment
    • Create AAD application registration
    • Prepare deployment package
    • Azure account login, subscription switch
    • Execute deployment
    • Update AAD application registration
  • Resource package upload, system startup
    • Install frontend deployment package, font package, icon resources
    • Install License
    • Start the system and check if it starts successfully
  • System check
    • Administrator logs into the system
    • Test if system functions are normal
  • Complete configuration
    • Modify system settings
    • Agent settings
    • Space settings
    • Test document upload

Terraform Deployment​

Create AAD Application Registration​

Create an AAD application registration in Azure Entra ID and obtain the client_id

Naming convention: appreg-[app-name]-[environment]

eg. appreg-serviceme-trial

Prepare Deployment Package​

tip

Please contact the product team to obtain the latest deployment package (including Terraform deployment package, font package, icon resources, web frontend deployment package, mobile deployment package, browser extension package)

Please refer to the README.md file in the Terraform deployment package and configure the parameters in the variables.tf file as instructed.

To enable the Private Endpoint feature, set the deploy_private_network variable to true in the variables.tf file. The IP address configuration for Vnet and subnet should be set in the network.tf file.

About the enable_public_network_access variable:

  • This variable controls public network access for all resources except APIM
  • It is recommended to set this option to false only after all deployments are complete
  • If turned off prematurely, it may affect the normal execution of Terraform scripts
  • Once turned off, resources including keyvault and databases cannot be accessed from outside the vnet; you will need to deploy a VPN Gateway or jumpbox

Azure Account Login, Subscription Switch​

Open PowerShell in the working directory of the deployment package and execute the following command to log in to Azure. TENANT_ID is the tenant ID provided by the customer.

Global Cloud login command

az login --tenant TENANT_ID

If it is not Global cloud, specify the --cloud parameter, for example:

China Cloud login command

az login --tenant TENANT_ID --cloud AzureChinaCloud

Set the subscription ID, [subscriptionId] is the subscription ID provided by the customer

az account set --subscription "[subscriptionId]"

Check the current subscription to ensure you are logged into the correct subscription

az account show

Execute Deployment​

First, check whether the changes to be executed meet expectations

terraform plan
+ means new resources
- means deleted resources
~ means updated resources

If there are no issues, formally apply these changes. Note that Terraform will check and output the changes again; after reviewing, you need to enter yes to confirm

terraform apply

Wait for the deployment to complete (it takes about 20~60 minutes, with APIM deployment taking longer)

If there are no errors, Terraform will indicate how many resources were created and that the deployment was successful.

If errors occur, troubleshoot according to the error message. Common issues include:

  • Network error or request timeout during execution
    • Re-running Terraform apply usually resolves the issue
  • Resource already exists error during execution
    • Check if the corresponding resource already exists via Azure Portal. If it does, delete it via Azure Portal. (Some resources have Purge Protection and may require special az cli commands to delete)
  • OpenAI resource creation failed due to insufficient quota
    • Check if the OpenAI resource quota in the region is sufficient. If not, switch to another region

Update AAD Application Registration​

Update the APIM URL in the AAD application registration (rule: $APIM_GATEWAY_URL + "/", e.g. https://xxx-apim.azure-api.net/ ), and make sure to check access token and ID token

Resource Package Upload, System Startup​

Install Frontend Deployment Package, Font Package, Icon Resources​

Upload the following resources to the corresponding containers in the Azure Storage Account. You can use Azure Storage Explorer or az cli commands to upload.

  • Upload the font package to the backend-core-fonts container
  • Upload the icon resources to the icons container
  • Upload the web frontend deployment package to the static-site container (do not overwrite the existing config.js file)
  • Upload the browser extension deployment package to the extension container (create manually if it does not exist)
  • For mobile deployment, create a mobile folder in the static-site container, configure the config.js in the mobile deployment package, and upload it to this folder

Install License​

Upload the License file to the static-site container of the Azure Storage Account

Start the System and Verify Deployment Status​

  1. Modify configuration to enable App Service

    • Open the variables.tf file
    • Update the value of the enable_app_service variable to true

  2. Apply configuration changes

    • Execute the terraform apply command
    • Wait for deployment to complete

  3. Check service running status

    • Log in to Azure Portal
    • Check the Monitor logs for the following services:
      • api_app_service
      • scheduler_app_service
      • pythoncore_app_service
    • Confirm that all services have started and are running normally
tip

If you find that a service fails to start, carefully check the error log information and troubleshoot and fix according to the specific error.

tip

For environments deployed with Private Endpoint, please note the following:

  1. After deployment, if you need to restrict public network access, you can set the enable_public_network_access variable to false
  2. This setting will disable public network access for all resources except APIM, improving system security
  3. Note that after disabling public network access:
    • Terraform will not be able to directly access keyvault and Storage resources
    • Subsequent terraform apply operations may fail
    • Solutions:
      • Option 1: Temporarily enable public network access, disable it after executing terraform apply
      • Option 2: Connect to the vnet via jumpbox or VPN Gateway to execute scripts

System Check​

Administrator Login to the System​

Log in to the system via the APIM URL. The default administrator account is servicemeadmin, and the default password is in the README.md file of the deployment package.

Test System Functionality​

Complete Configuration​

Agent Settings​

The accuracy and response format of Agent Q&A are greatly affected by the Prompt and related parameters.

The following links provide some examples of Agent settings, which can be modified according to the customer's scenario.

Before providing Agent to end users, sufficient testing is required to ensure that the relevant skills can be correctly invoked, and that the accuracy and response format of Q&A can meet user requirements.

Refer to System Configuration Best Practices

Space Settings​

Settings in Space and document properties will affect the document segmentation strategy during preprocessing. Please choose the most appropriate settings according to different situations to achieve better document accuracy.

tip

After adjusting the settings, please remember to regenerate the index for the relevant documents, otherwise the settings will only take effect for newly uploaded documents.

Refer to System Configuration Best Practices

Test Document Upload​

You can upload some sample documents or customer-provided documents to the system. After preprocessing is complete, test via Agent Q&A.

Modify System Settings​

Modify the system LOGO and system title as needed.