o Ag[z@sddlZddlmZmZmZmZmZmZmZddl Z ddl m Z ddl m Z ddlmZddlmZmZddlmZmZddl mZGd d d e ZGd d d e ZdS) N)AnyDictIteratorListOptionalTupleUnion)CallbackManagerForLLMRun)LLM)GenerationChunk)convert_to_secret_strget_from_dict_or_env)Field SecretStr)Responsec s2eZdZUdZeddZeed< eeddZ eed< edddZ eed< edddZ eed < ed dZ e ed < d Zeeeefed < GdddZede fddZedeeeffddZedeeeffddZedefddZdedd ffdd ZdedeeeffddZd/deeedeeeffd d!Z d0d"eeeefdeeed ee defd#d$Z d%edefd&d'Z!d%ede"e#fd(d)Z$ d1d"eeeefdeeed*ee%dede"e#f d+d,Z& d1d"eeeefdeeed*ee%dedef d-d.Z'Z(S)2 SambaStudiou SambaStudio large language models. Setup: To use, you should have the environment variables ``SAMBASTUDIO_URL`` set with your SambaStudio environment URL. ``SAMBASTUDIO_API_KEY`` set with your SambaStudio endpoint API key. https://sambanova.ai/products/enterprise-ai-platform-sambanova-suite read extra documentation in https://docs.sambanova.ai/sambastudio/latest/index.html Example: .. code-block:: python from langchain_community.llms.sambanova import SambaStudio SambaStudio( sambastudio_url="your-SambaStudio-environment-URL", sambastudio_api_key="your-SambaStudio-API-key, model_kwargs={ "model" : model or expert name (set for Bundle endpoints), "max_tokens" : max number of tokens to generate, "temperature" : model temperature, "top_p" : model top p, "top_k" : model top k, "do_sample" : wether to do sample "process_prompt": wether to process prompt (set for Bundle generic v1 and v2 endpoints) }, ) Key init args — completion params: model: str The name of the model to use, e.g., Meta-Llama-3-70B-Instruct-4096 (set for Bundle endpoints). streaming: bool Whether to use streaming handler when using non streaming methods model_kwargs: dict Extra Key word arguments to pass to the model: max_tokens: int max tokens to generate temperature: float model temperature top_p: float model top p top_k: int model top k do_sample: bool wether to do sample process_prompt: wether to process prompt (set for Bundle generic v1 and v2 endpoints) Key init args — client params: sambastudio_url: str SambaStudio endpoint Url sambastudio_api_key: str SambaStudio endpoint api key Instantiate: .. code-block:: python from langchain_community.llms import SambaStudio llm = SambaStudio=( sambastudio_url = set with your SambaStudio deployed endpoint URL, sambastudio_api_key = set with your SambaStudio deployed endpoint Key, model_kwargs = { "model" : model or expert name (set for Bundle endpoints), "max_tokens" : max number of tokens to generate, "temperature" : model temperature, "top_p" : model top p, "top_k" : model top k, "do_sample" : wether to do sample "process_prompt" : wether to process prompt (set for Bundle generic v1 and v2 endpoints) } ) Invoke: .. code-block:: python prompt = "tell me a joke" response = llm.invoke(prompt) Stream: .. code-block:: python for chunk in llm.stream(prompt): print(chunk, end="", flush=True) Async: .. code-block:: python response = llm.ainvoke(prompt) await response defaultsambastudio_urlsambastudio_api_keyT)rexcludebase_url streaming_urlF streamingN model_kwargsc@eZdZdZdS)zSambaStudio.ConfigTN__name__ __module__ __qualname__populate_by_namer"r"i/var/www/html/development/chatbot/venv/lib/python3.10/site-packages/langchain_community/llms/sambanova.pyConfig|r$returncCdS)9Return whether this model can be serialized by Langchain.Tr"clsr"r"r#is_lc_serializablezSambaStudio.is_lc_serializablecCs dddS)Nrr)rrr"selfr"r"r# lc_secretsszSambaStudio.lc_secretscCsd|jid|jiS)Return a dictionary of identifying parameters. This information is used by the LangChain callback system, which is used for tracing purposes make it possible to monitor LLMs. rr)rrr-r"r"r#_identifying_paramsszSambaStudio._identifying_paramscCr')zReturn type of llm.zsambastudio-llmr"r-r"r"r# _llm_typer,zSambaStudio._llm_typekwargsc sTt|dd|d<tt|dd|d<||d\|d<|d<tjdi|dS) 'init and validate environment variablesrSAMBASTUDIO_URLrSAMBASTUDIO_API_KEYrrNr")r r _get_sambastudio_urlssuper__init__r.r3 __class__r"r#r9s zSambaStudio.__init__urlcCsdd|vr |}|}||fSd|vr|dd}|}||fS|}d|vr.d|d}||fStd)a3 Get streaming and non streaming URLs from the given URL Args: url: string with sambastudio base or streaming endpoint url Returns: base_url: string with url to do non streaming calls streaming_url: string with url to do streaming calls chat/completionsstreamzstream/rgenericzgeneric/streamUnsupported URL)replacejoinsplit ValueError)r.r=r stream_urlr"r"r#r7s   z!SambaStudio._get_sambastudio_urlsstopcCsB|durg}|jp i}|dg|}t|dkr||d<d|jvrKd|vr/|d|d<d|vr<|d|d<d |vrG|d |}|Sd |jvrnd|vr]|d|d<d|vrj|d|d<|}|Sd |jvrd|vr|d|d<d|vr|d|d<d d |D}|Std|jd)aD Get the tuning parameters to use when calling the LLM. Args: stop: Stop words to use when generating. Model output is cut off at the first occurrence of any of the stop substrings. Returns: The tuning parameters in the format required by api to use Nstop_sequencesrr> select_expertmodelmax_tokens_to_generate max_tokensprocess_promptapi/v2/predict/genericapi/predict/genericcSs&i|]\}}|t|jt|dqS))typevalue)rPrstr).0kvr"r"r# sz2SambaStudio._get_tuning_params..rA9only openai, generic v1 and generic v2 APIs are supported)rgetlenrkeyspopitemsrE)r.rG _model_kwargs_stop_sequences tuning_paramsr"r"r#_get_tuning_paramssP              zSambaStudio._get_tuning_paramspromptc Cst|tr|g}||}d|jvr6d|ddg}||d|}dd|D}d|jd d }nd |jvrq|d d rQt ddd|ddgd}n|d}d|dg}dd|D}||d}d|ji}nLd|jvr|d d r|d ddkrt ddd|ddgd}n |d}n|d}|r||d}n|g|d}d|ji}n t d|jdt } |r| j |j||dd} n | j |j||d d} | jd krtd!| jd"| jd"| S)#a Performs a post request to the LLM API. Args: prompt: The prompt to pass into the model stop: list of stop tokens streaming: wether to do a streaming call Returns: A request Response object r>userrrolecontent)messagesr?cSi|] \}}|dur||qSNr"rSkeyrQr"r"r#rVz/SambaStudio._handle_request..Bearer application/json Authorizationz Content-TyperNrMFzsambaverse-conversation-idN) message_idrdre)conversation_idrfitem0)idrQcSrgrhr"rir"r"r#rV2rk)r\paramsrjrOrQTrue)instancert) instancesrtrArWTheadersjsonr?2Sambanova / complete call failed with status code .) isinstancerRr`rr\rget_secret_valuerXrzdumpsrErequestsSessionpostrr status_code RuntimeErrortext) r.rarGrrt messages_dictdataryr\ http_sessionresponser"r"r#_handle_requests~                zSambaStudio._handle_requestrc Csz|}Wnty}z td|d|jd}~wwd|jvr/|dddd}|Sd |jvr@|d dd d }|Sd |jvrO|ddd }|Std|jd) Process a non streaming response from the api Args: response: A request Response object Returns completion: a string with model generation ;Sambanova /complete call failed couldn't get JSON response response: Nr>choicesrmessagererNr\rQ completionrO predictionsrArW)rz ExceptionrrrrEr.r response_dicterr"r"r#_process_responsehs,     zSambaStudio._process_responsec csLzddl}Wn tytdwd|jvr||}|D]{}|jdkr5td|jd|jdzP|jdkrt |jt rHt |j}n td|jd|jd| d rgtd|jd|jdt|d dkrz|d dd d }nd }t|d}|VWq!ty}z td|d|jd}~wwdSd|jvr|D]3} zt | }|ddddd}t|d}|VWqty}z td|d| d}~wwdSd|jvr|D]2} zt | }|dddd}t|d}|VWqty}z td|d| d}~wwdStd|jd) Process a streaming response from the api Args: response: An iterable request Response object Yields: GenerationChunk: a GenerationChunk with model partial generation rNTcould not import sseclient libraryPlease install it with `pip install sseclient-py`.r> error_event1Sambanova /complete call failed with status code r}[DONE]errorrdeltarerr3Error getting content chunk raw streamed response: data: rNresultr\rQ stream_tokenzline: rO responsesrArW) sseclient ImportErrorr SSEClienteventseventrrrr~rRrzloadsrXrYr r iter_linesrE) r.rrclientrrregenerated_chunkrliner"r"r#_process_stream_responses           &          z$SambaStudio._process_stream_response run_managerck<|j||dd}||D] }|r||j|VqdS)a\Call out to Sambanova's complete endpoint. Args: prompt: The prompt to pass into the model. stop: a list of strings on which the model should stop generating. run_manager: A run manager with callbacks for the LLM. Yields: chunk: GenerationChunk with model partial generation TrNrron_llm_new_tokenrr.rarGrr3rchunkr"r"r#_stream zSambaStudio._streamcKT|jrd}|jd|||d|D]}||j7}q|S|j||dd}||}|S)a Call out to Sambanova's complete endpoint. Args: prompt: The prompt to pass into the model. stop: a list of strings on which the model should stop generating. Returns: result: string with model generation rrarGrFrNr"rrrrrr.rarGrr3rrrr"r"r#_call  zSambaStudio._callrhNFNN))rrr __doc__rrrR__annotations__rrrrrboolrrrrr$ classmethodr+propertyr/r1r2r9rr7rr`rrrrrr rr rr __classcell__r"r"r;r#r s \$D  f$b   rc s2eZdZUdZeddZeed< eeddZ eed< eddZ eed< eddZ e ed < ed dZ eed < ed dZeed < eddZeeed< eddZeeed< eddidZeed< GdddZede fddZedeeeffddZedeeeffddZedefddZdeddffd d! Z  d0d"e e!eefd#ee!ed ee de"fd$d%Z#d&e"defd'd(Z$d&e"de%e&fd)d*Z'  d1d"e e!eefd#ee!ed+ee(dedef d,d-Z)  d1d"e e!eefd#ee!ed+ee(dede%e&f d.d/Z*Z+S)2SambaNovaCloudu SambaNova Cloud large language models. Setup: To use, you should have the environment variables: ``SAMBANOVA_URL`` set with SambaNova Cloud URL. defaults to http://cloud.sambanova.ai/ ``SAMBANOVA_API_KEY`` set with your SambaNova Cloud API Key. Example: .. code-block:: python from langchain_community.llms.sambanova import SambaNovaCloud SambaNovaCloud( sambanova_api_key="your-SambaNovaCloud-API-key, model = model name, max_tokens = max number of tokens to generate, temperature = model temperature, top_p = model top p, top_k = model top k ) Key init args — completion params: model: str The name of the model to use, e.g., Meta-Llama-3-70B-Instruct-4096 (set for CoE endpoints). streaming: bool Whether to use streaming handler when using non streaming methods max_tokens: int max tokens to generate temperature: float model temperature top_p: float model top p top_k: int model top k Key init args — client params: sambanova_url: str SambaNovaCloud Url defaults to http://cloud.sambanova.ai/ sambanova_api_key: str SambaNovaCloud api key Instantiate: .. code-block:: python from langchain_community.llms.sambanova import SambaNovaCloud SambaNovaCloud( sambanova_api_key="your-SambaNovaCloud-API-key, model = model name, max_tokens = max number of tokens to generate, temperature = model temperature, top_p = model top p, top_k = model top k ) Invoke: .. code-block:: python prompt = "tell me a joke" response = llm.invoke(prompt) Stream: .. code-block:: python for chunk in llm.stream(prompt): print(chunk, end="", flush=True) Async: .. code-block:: python response = llm.ainvoke(prompt) await response rr sambanova_urlsambanova_api_keyzMeta-Llama-3.1-8B-InstructrJFrirLgffffff? temperatureNtop_ptop_k include_usageTstream_optionsc@r)zSambaNovaCloud.ConfigTNrr"r"r"r#r${r%r$r&cCr')r(Fr"r)r"r"r#r+~r,z!SambaNovaCloud.is_lc_serializablecCsddiS)Nrr"r-r"r"r#r/szSambaNovaCloud.lc_secretscCs"|j|j|j|j|j|j|jdS)r0rJrrLrrrrrr-r"r"r#r1sz"SambaNovaCloud._identifying_paramscCr')z7Get the type of language model used by this chat model.zsambanovacloud-llmr"r-r"r"r#r2r,zSambaNovaCloud._llm_typer3c s>t|dddd|d<tt|dd|d<tjdi|dS) r4r SAMBANOVA_URLz,https://api.sambanova.ai/v1/chat/completionsrrSAMBANOVA_API_KEYNr")r r r8r9r:r;r"r#r9s  zSambaNovaCloud.__init__rarGc Cst|tr|g}d|ddg}|||j||j|j|j|jd}dd|D}d|j dd }t }|rE|j |j ||d d }n |j |j ||d d }|jd kratd|jd|jd|S)z Performs a post request to the LLM API. Args: prompt: The prompt to pass into the model. stop: list of stop tokens Returns: A request Response object rbrrc)rfr?rLrGrJrrrcSrgrhr"rir"r"r#rVrkz2SambaNovaCloud._handle_request..rlrmrnTrxFr{r|r})r~rRrLrJrrrr\rrrrrrrrr) r.rarGrrrryrrr"r"r#rsB     zSambaNovaCloud._handle_requestrc CsTz|}Wnty}z td|d|jd}~ww|dddd}|S)rrrNrrrre)rzrrrrr"r"r#rs z SambaNovaCloud._process_responsec cs4zddl}Wn tytdw||}|D]{}|jdkr0td|jd|jdzP|jdkrt|jt rCt |j}n td|jd|jd| drbtd|jd|jdt |d dkru|d dd d }nd }t|d }|VWqty}z td|d|jd}~wwdS)rrNrrrr}rrrrrerrrr)rrrrrrrrr~rRrzrrXrYr r) r.rrrrrrerrr"r"r#rsh         z'SambaNovaCloud._process_stream_responsercKr)Call out to SambaNovaCloud complete endpoint. Args: prompt: The prompt to pass into the model. stop: Optional list of stop words to use when generating. Returns: The string generated by the model. rrFrNr"rrr"r"r#r1rzSambaNovaCloud._callckr)rTrNrrr"r"r#rNrzSambaNovaCloud._streamrr),rrr rrrrRrrrrJrrrLintrfloatrrrrdictr$rr+rrr/rr1r2r9rrrrrrr rr rrrr"r"r;r#rs @  6<    r)rztypingrrrrrrrr langchain_core.callbacks.managerr #langchain_core.language_models.llmsr langchain_core.outputsr langchain_core.utilsr r pydanticrrrrrr"r"r"r#s$