[azure] Application gateway와 private endpoint를 활용한 Open AI 연동

안녕하세요, 최근 Azure의 Application gateway와 Private Endpoint를 활용해 Open AI 서비스를 연동한 내용을 간단히 설명합니다.

 

[요구사항]

외부 SaaS 서비스에서 도메인을 통해 Azure의 Open AI 서비스를 안정적으로 사용

 

[테스트 구성]

 

 

 

1. Network 구성

- Network Subscription과 서비스(DEV/STG/PRD)용 Subscription을 분리하여 Peering으로 연결 구성

- Open AI의 특정 모델의 경우 East US 2 리전에 출시되어 East US 리전에 vNet을 구성하여 Global Peering으로 구성

- Network Subscription과 서비스(DEV/STG/PRD)용 Subscription 간에는 Private Endpoint를 통해 연결하여 보안적으로 안정적인 통신이 가능하도록 구성

 

2. Application Gateway 구성

- Routing Rule의 Path based Rule을 적용하여 리전 별로 Open AI 서비스 호출 시 구분되도록 구성

- WAF 정책을 적용하여 특정 IP만 접속가능하도록 Whitelist 적용

 

3. 추가 고려사항

- 도메인 적용 시 SSL 적용을 위해 Host 단위로 서비스를 구분하려면 Wildcard 인증서를 적용할 필요가 있음

(단일 인증서를 적용하여 Path로 구분할 경우 rewrite rule 적용 등에서 이슈 발생 시 트러블슈팅 등 운영관리 어려움 발생될 수 있음)

 

Azure 유의사항

- 인증서 내 Key vault 적용 시 포탈에서 적용되지 않는 경우가 있어 Azure CLI, Powershell 등의 명령어를 통해 적용해야함

 

 

 

[테스트 결과]

- REST API 호출 : 배포된 리소스의 특정 엔드포인트 URL에 배포 이름을 경로 변수나 쿼리 매개변수로 포함하여 HTTP 요청

 curl -vk "https://test.com/openai/deployments/gptTestkr/chat/completions?api-version=2025-01-01-preview" \
  -H "Content-Type: application/json" \
  -H "api-key: XXXXXXXXXXX" \
  -d '{"messages":[{"role":"user","content":"Hello KRC"}]}'
* Host cns-hub.com:443 was resolved.
* IPv6: (none)
* IPv4: 4.230.64.34
*   Trying 4.230.64.34:443...
* ALPN: curl offers h2,http/1.1
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384 / x25519 / RSASSA-PSS
* ALPN: server accepted h2
* Server certificate:
*  subject: CN=test.com
*  start date: Sep 25 09:47:07 2025 GMT
*  expire date: Sep 25 09:47:07 2026 GMT
*  issuer: C=US; ST=Arizona; L=Scottsdale; O=GoDaddy.com, Inc.; OU=http://certs.godaddy.com/repository/; CN=Go Daddy Secure Certificate Authority - G2
*  SSL certificate verify result: self-signed certificate in certificate chain (19), continuing anyway.
*   Certificate level 0: Public key type RSA (2048/112 Bits/secBits), signed using sha256WithRSAEncryption
*   Certificate level 1: Public key type RSA (2048/112 Bits/secBits), signed using sha256WithRSAEncryption
*   Certificate level 2: Public key type RSA (2048/112 Bits/secBits), signed using sha256WithRSAEncryption
* Connected to test.com (4.230.64.34) port 443
* using HTTP/2
* [HTTP/2] [1] OPENED stream for https://test.com/openai/deployments/gptTestkr/chat/completions?api-version=2025-01-01-preview
* [HTTP/2] [1] [:method: POST]
* [HTTP/2] [1] [:scheme: https]
* [HTTP/2] [1] [:authority: test.com]
* [HTTP/2] [1] [:path: /openai/deployments/gptTestkr/chat/completions?api-version=2025-01-01-preview]
* [HTTP/2] [1] [user-agent: curl/8.11.1]
* [HTTP/2] [1] [accept: */*]
* [HTTP/2] [1] [content-type: application/json]
* [HTTP/2] [1] [api-key: XXXXXXXXXXXXX]
* [HTTP/2] [1] [content-length: 52]
> POST /openai/deployments/gptTestkr/chat/completions?api-version=2025-01-01-preview HTTP/2
> Host: test.com
> User-Agent: curl/8.11.1
> Accept: */*
> Content-Type: application/json
> api-key: XXXXXXXXXXX
> Content-Length: 52
> 
* upload completely sent off: 52 bytes
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
< HTTP/2 200 
< date: Sun, 28 Sep 2025 07:46:00 GMT
< content-type: application/json
< content-length: 1246
< apim-request-id: aeb096fc-8d5f-4eb2-ab48-af49816b40d0
< strict-transport-security: max-age=31536000; includeSubDomains; preload
< x-content-type-options: nosniff
< x-ms-region: Korea Central
< x-ratelimit-remaining-requests: 249
< x-ratelimit-limit-requests: 250
< x-ratelimit-remaining-tokens: 249997
< x-ratelimit-limit-tokens: 250000
< azureml-model-session: d097-20250926132639
< x-ms-rai-invoked: true
< x-request-id: a2ccbd9a-75fb-41a3-b822-459987625eaa
< x-ms-deployment-name: gptTestkr
< 
{"choices":[{"content_filter_results":{"hate":{"filtered":false,"severity":"safe"},"protected_material_code":{"filtered":false,"detected":false},"protected_material_text":{"filtered":false,"detected":false},"self_harm":{"filtered":false,"severity":"safe"},"sexual":{"filtered":false,"severity":"safe"},"violence":{"filtered":false,"severity":"safe"}},"finish_reason":"stop","index":0,"logprobs":null,"message":{"annotations":[],"content":"Hello! How can I assist you today?","refusal":null,"role":"assistant"}}],"created":1759045560,"id":"chatcmpl-CKgOWsFWaHE6IHV2AUFduAFgDb31g","model":"gpt-4.1-mini-2025-04-14","object":"chat.completion","prompt_filter_results":[{"prompt_index":0,"content_filter_results":{"hate":{"filtered":false,"severity":"safe"},"jailbreak":{"filtered":false,"detected":false},"self_harm":{"filtered":false,"severity":"safe"},"sexual":{"filtered":false,"severity":"safe"},"violence":{"filtered":false,"severity":"safe"}}}],"system_fingerprint":"fp_c82f8028fa","usage":{"completion_tokens":10,"completion_tokens_details":{"accepted_prediction_tokens":0,"audio_tokens":0,"reasoning_tokens":0,"rejected_prediction_tokens":0},"prompt_tokens":10,"prompt_tokens_details":{"audio_tokens":0,"cached_tokens":0},"total_tokens":20}}
* Connection #0 to host test.com left intact

 

 curl -vk "https://test.com/eastus2/openai/deployments/gpt-4o-mini/chat/completions?api-version=2025-01-01-preview" \
  -H "Content-Type: application/json" \
  -H "api-key: XXXXXXXXXXXXXXXXXXXXX" \
  -d '{
    "messages":[{"role":"user","content":"Hello via AGW"}],
    "max_tokens":20
  }'
* Host test.com:443 was resolved.
* IPv6: (none)
* IPv4: 4.230.64.34
*   Trying 4.230.64.34:443...
* ALPN: curl offers h2,http/1.1
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384 / x25519 / RSASSA-PSS
* ALPN: server accepted h2
* Server certificate:
*  subject: CN=test.com
*  start date: Sep 25 09:47:07 2025 GMT
*  expire date: Sep 25 09:47:07 2026 GMT
*  issuer: C=US; ST=Arizona; L=Scottsdale; O=GoDaddy.com, Inc.; OU=http://certs.godaddy.com/repository/; CN=Go Daddy Secure Certificate Authority - G2
*  SSL certificate verify result: self-signed certificate in certificate chain (19), continuing anyway.
*   Certificate level 0: Public key type RSA (2048/112 Bits/secBits), signed using sha256WithRSAEncryption
*   Certificate level 1: Public key type RSA (2048/112 Bits/secBits), signed using sha256WithRSAEncryption
*   Certificate level 2: Public key type RSA (2048/112 Bits/secBits), signed using sha256WithRSAEncryption
* Connected to test.com (4.230.64.34) port 443
* using HTTP/2
* [HTTP/2] [1] OPENED stream for https://test.com/eastus2/openai/deployments/gpt-4o-mini/chat/completions?api-version=2025-01-01-preview
* [HTTP/2] [1] [:method: POST]
* [HTTP/2] [1] [:scheme: https]
* [HTTP/2] [1] [:authority: test.com]
* [HTTP/2] [1] [:path: /eastus2/openai/deployments/gpt-4o-mini/chat/completions?api-version=2025-01-01-preview]
* [HTTP/2] [1] [user-agent: curl/8.11.1]
* [HTTP/2] [1] [accept: */*]
* [HTTP/2] [1] [content-type: application/json]
* [HTTP/2] [1] [api-key: XXXXXXXXX]
* [HTTP/2] [1] [content-length: 85]
> POST /eastus2/openai/deployments/gpt-4o-mini/chat/completions?api-version=2025-01-01-preview HTTP/2
> Host: test.com
> User-Agent: curl/8.11.1
> Accept: */*
> Content-Type: application/json
> api-key: XXXXXXXXXXXXXXXX
> Content-Length: 85
> 
* upload completely sent off: 85 bytes
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
< HTTP/2 200 
< date: Sun, 28 Sep 2025 07:47:03 GMT
< content-type: application/json
< content-length: 1266
< apim-request-id: 42e1ee67-4f7c-44bc-9e9a-45d9c150b43c
< strict-transport-security: max-age=31536000; includeSubDomains; preload
< x-content-type-options: nosniff
< x-ms-region: East US 2
< x-ratelimit-remaining-requests: 2498
< x-ratelimit-limit-requests: 2500
< x-ratelimit-remaining-tokens: 249992
< x-ratelimit-limit-tokens: 250000
< azureml-model-session: d048-20250904053033
< x-ms-rai-invoked: true
< x-request-id: 9406edd7-6047-4549-b952-bfb487a49536
< x-ms-deployment-name: gpt-4o-mini
< 
{"choices":[{"content_filter_results":{"hate":{"filtered":false,"severity":"safe"},"protected_material_text":{"filtered":false,"detected":false},"self_harm":{"filtered":false,"severity":"safe"},"sexual":{"filtered":false,"severity":"safe"},"violence":{"filtered":false,"severity":"safe"}},"finish_reason":"length","index":0,"logprobs":null,"message":{"annotations":[],"content":"Hello! It seems like you're referencing a form of communication possibly related to Amateur Radio or packet radio (","refusal":null,"role":"assistant"}}],"created":1759045622,"id":"chatcmpl-CKgPWpRkmDaWGHMkzoataclS0DkTP","model":"gpt-4o-mini-2024-07-18","object":"chat.completion","prompt_filter_results":[{"prompt_index":0,"content_filter_results":{"hate":{"filtered":false,"severity":"safe"},"jailbreak":{"filtered":false,"detected":false},"self_harm":{"filtered":false,"severity":"safe"},"sexual":{"filtered":false,"severity":"safe"},"violence":{"filtered":false,"severity":"safe"}}}],"system_fingerprint":"fp_efad92c60b","usage":{"completion_tokens":20,"completion_tokens_details":{"accepted_prediction_tokens":0,"audio_tokens":0,"reasoning_tokens":0,"rejected_prediction_tokens":0},"prompt_tokens":11,"prompt_tokens_details":{"audio_tokens":0,"cached_tokens":0},"total_tokens":31}}
* Connection #0 to host cns-hub.com left intact

 

 

Response API 호출 : Response API는 Azure OpenAI Service에서 제공하는 새로운 개념으로, 최근에 도입되었거나 기능이 통합된 상태 저장(stateful) API

curl -k -X POST "https://test.com/eastus2/openai/responses?api-version=2025-04-01-preview" \
  -H "Content-Type: application/json" \
  -H "api-key: XXXXXXXXXXXX" \
  -d '{
    "model": "test",
    "input": [
      {"role": "system", "content": "You are a helpful assistant."},
      {"role": "user", "content": "Hello via Responses API"}
    ],
    "max_output_tokens": 50
  }'
{
  "id": "resp_68d8e77b140881978dead2cb68731aa60369fb87e0e58628",
  "object": "response",
  "created_at": 1759045499,
  "status": "completed",
  "background": false,
  "content_filters": null,
  "error": null,
  "incomplete_details": null,
  "instructions": null,
  "max_output_tokens": 50,
  "max_tool_calls": null,
  "model": "test",
  "output": [
    {
      "id": "msg_68d8e77b6c2481979bc692b8d3cbfe410369fb87e0e58628",
      "type": "message",
      "status": "completed",
      "content": [
        {
          "type": "output_text",
          "annotations": [],
          "text": "Hello! How can I assist you today? If you have any questions about using the Responses API or anything else, feel free to ask!"
        }
      ],
      "role": "assistant"
    }
  ],
  "parallel_tool_calls": true,
  "previous_response_id": null,
  "prompt_cache_key": null,
  "reasoning": {
    "effort": null,
    "summary": null
  },
  "safety_identifier": null,
  "service_tier": "default",
  "store": true,
  "temperature": 1.0,
  "text": {
    "format": {
      "type": "text"
    }
  },
  "tool_choice": "auto",
  "tools": [],
  "top_p": 1.0,
  "truncation": "disabled",
  "usage": {
    "input_tokens": 21,
    "input_tokens_details": {
      "cached_tokens": 0
    },
    "output_tokens": 29,
    "output_tokens_details": {
      "reasoning_tokens": 0
    },
    "total_tokens": 50
  },
  "user": null,
  "metadata": {}

 

 

[참고 문서]

 

https://learn.microsoft.com/ko-kr/azure/ai-foundry/openai/reference

https://learn.microsoft.com/ko-kr/azure/ai-foundry/openai/how-to/create-resource?pivots=web-portal

방법: Azure AI Foundry Models 리소스에서 Azure OpenAI 만들기 및 배포 - Azure OpenAI

https://learn.microsoft.com/ko-kr/azure/ai-foundry/openai/how-to/responses?tabs=python-key

Azure OpenAI 응답 API - Azure OpenAI

https://learn.microsoft.com/ko-kr/azure/app-service/troubleshoot-domain-ssl-certificates

도메인 및 TLS/SSL 인증서 문제 해결 - Azure App Service