Docs

Creating AI service assets

Last updated: Nov 21, 2024
Creating AI service assets

To deploy an AI service, you must create a repository asset in watsonx.ai Runtime that contains the AI service and upload the Python file to the asset.

Requirements for creating AI service assets

When you use an Integrated Development Environment (IDE) such as VSCode, eclipse, PyCharm, or more to build your generative AI application, you must create a Python file to store your AI service. Once you define the function, you must compress the AI service to create a gzip archive (.gz file format).

When you use the watsonx.ai Python client library to create your AI service asset, the library automatically stores the function in gzip archive for you. However, when you create an AI service asset by using the REST API, you must follow the process of manually compressing your Python file in a gzip archive.

You must use the runtime-24.1-py3.11 software specification to create and deploy an AI service asset that is coded in Python.

Creating AI service assets with Python client library

You can use the store_ai_service function of the watsonx.ai Python client library to create an AI service asset.

The following code sample shows how to create an AI service asset by using the Python client library:

documentation_request = {
    "application/json": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "type": "object",
        "properties": {
            "query": {"type": "string"},
            "parameters": {
                "properties": {
                    "max_new_tokens": {"type": "integer"},
                    "top_p": {"type": "number"},
                },
                "required": ["max_new_tokens", "top_p"],
            },
        },
        "required": ["query"],
    }
}

documentation_response = {
    "application/json": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "type": "object",
        "properties": {"query": {"type": "string"}, "result": {"type": "string"}},
        "required": ["query", "result"],
    }
}


meta_props = {
    client.repository.AIServiceMetaNames.NAME: "AI service example",
    client.repository.AIServiceMetaNames.DESCRIPTION: "This is AI service function",
    client.repository.AIServiceMetaNames.SOFTWARE_SPEC_ID: client.software_specifications.get_id_by_name(
        "runtime-24.1-py3.11"
    ),
    client.repository.AIServiceMetaNames.REQUEST_DOCUMENTATION: documentation_request,
    client.repository.AIServiceMetaNames.RESPONSE_DOCUMENTATION: documentation_response,
}

stored_ai_service_details = client.repository.store_ai_service(
    basic_generate_demo, meta_props
)

ai_service_id = client.repository.get_ai_service_id(stored_ai_service_details)
print("The AI service asset id:", ai_service_id)
Note:
  • The REQUEST_DOCUMENTATION and RESPONSE_DOCUMENTATION parameters are optional. You can use these parameters to store the schema of the request and response of generate and generate_stream functions.
  • The function call client.repository.store_ai_service saves the AI service function basic_generate_demo into a gzip file internally.

For more information, see watsonx.ai Python client library documentation for creating an AI service asset.

Creating an AI service asset with REST API

You can use the /ml/v4/ai_services REST API endpoint to create the AI services asset in the watsonx.ai Runtime repository. For more information, see watsonx.ai REST API documentation.

Learn more

Parent topic: Deploying AI services with direct coding