Model Services¶

Model services are a solution for quickly deploying open-source or fine-tuned large language models as callable services. With one-click deployment, the complex management of models is simplified into a standardized service format, compatible with mainstream model service API call capabilities, meeting the demand for immediate use.

• Model services allow users to call selected models to perform tasks such as text generation, dialogue processing, image generation, etc.
• Supports online experience of models.

The details of the model service include basic information about the service, authorization methods, and call examples.

Basic Information¶

• Model Service Name: The name of the current service, used to identify this model service.
• Access Path Name: Each model service has a unique path name.
• Model Type: The model being used by the current service.
• Status: The current status of the service.
• Billing Method: The billing method for the current service.

Authorization Methods¶

• API-Key Authorization:

  • All API requests need to include an Authorization field in the HTTP Header for identity verification.
  • Format: Authorization: Bearer {API_KEY}
  • You can obtain the key through the "View My API-Key" link on the page.

• Security Advice: Store the API-Key on the backend server to avoid exposing the key in client-side code and prevent leakage.

API Call Example¶

• Request URL: The POST request URL is https://sh-02.d.run/v1/chat/completions

Request Example: Calling the API with curl¶

curl 'https://sh-02.d.run/v1/chat/completions' \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer <Your API Key here>" \
  -d '{
    "model": "u-8105f7322477/test",
    "messages": [{"role": "user", "content": "Say this is a test!"}],
    "temperature": 0.7
  }'

Parameter Explanation:

• model: The access path name of the model service (e.g., u-8105f7322477/test).

• messages: A list of dialogue history containing user input, for example:

  [{"role": "user", "content": "Say this is a test!"}]
  

• temperature: Controls the randomness of the generated results; a higher value results in more creative outputs, while a lower value yields more stable outputs.

Response Example¶

{
  "id": "cmp-1d033c426254417b7b0675303b1d300",
  "object": "chat.completion",
  "created": 1733724462,
  "model": "u-8105f7322477/test",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "I am a large language model. How can I assist you today?"
      },
      "tool_calls": []
    }
  ],
  "usage": {
    "prompt_tokens": 25,
    "completion_tokens": 15,
    "total_tokens": 40
  }
}

Response Field Explanation:

• id: A unique identifier for the generated result.
• model: The ID of the model service being called.
• choices: An array of results generated by the model.
  • message: The generated content.
  • content: The text content generated by the model.

• usage: Token usage for this call:

  • prompt_tokens: The number of tokens for user input.
  • completion_tokens: The number of tokens for the generated result.
  • total_tokens: The total usage.

• Integration Development Examples

Python Example Code¶

# Compatible with OpenAI Python library v1.0.0 and above

from openai import OpenAI

client = OpenAI(
    base_url="https://sh-02.d.run/v1/",
    api_key="<Your API Key here>"
)

messages = [
    {"role": "user", "content": "hello!"},
    {"role": "user", "content": "Say this is a test?"}
]

response = client.chat.completions.create(
    model="u-8105f7322477/test",
    messages=messages
)

content = response.choices[0].message.content

print(content)

Node.js Example Code¶

const OpenAI = require('openai');

const openai = new OpenAI({
  baseURL: 'https://sh-02.d.run/v1',
  apiKey: '<Your API Key here>',
});

async function getData() {
  try {
    const chatCompletion = await openai.chat.completions.create({
      model: 'u-8105f7322477/test',
      messages: [
        { role: 'user', content: 'hello!' },
        { role: 'user', content: 'how are you?' },
      ],
    });

    console.log(chatCompletion.choices[0].message.content);
  } catch (error) {
    if (error instanceof OpenAI.APIError) {
      console.error('API Error:', error.status, error.message);
      console.error('Error details:', error.error);
    } else {
      console.error('Unexpected error:', error);
    }
  }
}

getData();