hbllmutils.model.load

This module provides functionality for loading Large Language Model (LLM) configurations and creating remote model instances.

The module handles loading LLM configurations from config files or directories, and creates remote model instances with appropriate parameters. It supports both pre-configured models and dynamically specified configurations.

Main Functions:

  • load_llm_model_from_config: Load LLM model with specified configuration from config file or parameters

  • load_llm_model: Convenience function to load LLM model from various input types

Type Aliases:

  • LLMModelTyping: Union type for model input (str or LLMModel instance)

LLMModelTyping

hbllmutils.model.load.LLMModelTyping

Type alias for model input, which can be either a string (model name) or an LLMModel instance.

alias of str | LLMModel

load_llm_model_from_config

hbllmutils.model.load.load_llm_model_from_config(config_file_or_dir: str | None = None, base_url: str | None = None, api_token: str | None = None, model_name: str | None = None, **params) → RemoteLLMModel[source]

Load a Large Language Model with specified configuration.

This function attempts to load LLM configuration from a config file or directory, and creates a remote model instance. It supports both pre-configured models from config files and dynamically specified configurations.

Parameters:

  • config_file_or_dir (Optional[str]) – Path to the configuration file or directory. If None, defaults to current directory.

  • base_url (Optional[str]) – Base URL for the LLM API endpoint. If provided, overrides config file settings.

  • api_token (Optional[str]) – API token for authentication. Required when base_url is provided without config file.

  • model_name (Optional[str]) – Name of the model to load. Required when base_url is provided without config file.

  • params – Additional parameters to pass to the model.

Returns:

An initialized LLM remote model instance.

Return type:

RemoteLLMModel

Raises:

  • FileNotFoundError – When config file is not found (handled internally).

  • KeyError – When specified model is not found in config (handled internally).

  • ValueError – When api_token is not specified but required, or when model_name is empty but required.

  • RuntimeError – When no model parameters are specified and no local configuration is available.

Example::
>>> # Load model from config file
>>> model = load_llm_model_from_config(config_file_or_dir='./config')

>>> # Load model with explicit parameters
>>> model = load_llm_model_from_config(
...     base_url='https://api.example.com',
...     api_token='your-token',
...     model_name='gpt-4'
... )

>>> # Load model from config with overrides
>>> model = load_llm_model_from_config(
...     config_file_or_dir='./config',
...     model_name='gpt-4',
...     base_url='https://custom-api.example.com'
... )

load_llm_model

hbllmutils.model.load.load_llm_model(model: str | LLMModel | None = None) → LLMModel[source]

Load a Large Language Model from various input types.

This is a convenience function that handles different types of model specifications. It can load a model by name from configuration, use an existing model instance, or load the default model from configuration.

Parameters:

model (Optional[LLMModelTyping]) – The model specification. Can be: - A string representing the model name to load from configuration - An LLMModel instance to use directly - None to load the default model from configuration

Returns:

An initialized LLM model instance.

Return type:

LLMModel

Raises:

  • TypeError – When model is not a string, LLMModel instance, or None.

  • ValueError – When model name is invalid or not found in configuration.

  • RuntimeError – When no model parameters are specified and no local configuration is available.

Example::
>>> # Load model by name from configuration
>>> model = load_llm_model('gpt-4')

>>> # Use an existing model instance
>>> existing_model = RemoteLLMModel(base_url='...', api_token='...', model_name='gpt-4')
>>> model = load_llm_model(existing_model)

>>> # Load default model from configuration
>>> model = load_llm_model()