o Ag(@sddlmZddlZddlmZddlmZmZmZm Z m Z ddl Z ddl m Z ddlmZddlmZddlmZdd lmZGd d d eZdS) ) annotationsN)StringIO)AnyDictIteratorListOptional)CallbackManagerForLLMRun)LLM)GenerationChunk)get_pydantic_field_names) ConfigDictc@seZdZUdZdZded< dZded< dZd ed < d Zd ed < dZ ded< dZ d ed< dZ ded< dZ ded< d Z d ed< dZd ed< dZded< dZded< dZded< dZd ed < d!Zd ed"< d#Zded$< d#Zded%< dZd ed&< d'Zded(< d)Zded*< ed+d,ZedGd.d/ZedHd1d2ZedId4d5Z dJdKd:d;Z  dLdMd?d@Z  dLdNdBdCZ!dOdEdFZ"dS)P LlamafileaOLlamafile lets you distribute and run large language models with a single file. To get started, see: https://github.com/Mozilla-Ocho/llamafile To use this class, you will need to first: 1. Download a llamafile. 2. Make the downloaded file executable: `chmod +x path/to/model.llamafile` 3. Start the llamafile in server mode: `./path/to/model.llamafile --server --nobrowser` Example: .. code-block:: python from langchain_community.llms import Llamafile llm = Llamafile() llm.invoke("Tell me a joke.") zhttp://localhost:8080strbase_urlNz Optional[int]request_timeoutFbool streamingintseedg?float temperature(top_kgffffff?top_pg?min_p n_predictrn_keepg?tfs_z typical_pg?repeat_penalty@ repeat_last_nT penalize_nlgpresence_penaltyfrequency_penaltymirostatg@ mirostat_taug? mirostat_etaforbid)extrareturncCsdS)N llamafile)selfr.r.i/var/www/html/development/chatbot/venv/lib/python3.10/site-packages/langchain_community/llms/llamafile.py _llm_typexszLlamafile._llm_type List[str]cs$gdfddt|jD}|S)N) rcachecallback_manager callbacksmetadatanamerrtagsverbosecustom_get_token_idscsg|]}|vr|qSr.r.).0k ignore_keysr.r0 sz/Llamafile._param_fieldnames..)r __class__)r/attrsr.r=r0_param_fieldnames|s  zLlamafile._param_fieldnamesDict[str, Any]cCs"i}|jD] }t||||<q|SN)rBgetattr)r/params fieldnamer.r.r0_default_paramss zLlamafile._default_paramsstopOptional[List[str]]kwargsrcKsV|j}|D] \}}||vr|||<q|dur"t|dkr"||d<|jr)d|d<|S)NrrITstream)rHitemslenr)r/rIrKrFr<vr.r.r0_get_parametersszLlamafile._get_parametersprompt run_manager"Optional[CallbackManagerForLLMRun]c Ks|jr2t"}|j|f||d|D]}||jq|}Wd|S1s+wY|S|jdd|i|}d|i|} ztj|j dddi| d|j d } Wntj j yitj d |j d w| d | _| d }|S)aRequest prompt completion from the llamafile server and return the output. Args: prompt: The prompt to use for generation. stop: A list of strings to stop generation when encountered. run_manager: **kwargs: Any additional options to pass as part of the generation request. Returns: The string generated by the model. )rIrRNrIrQ /completion Content-Typeapplication/jsonFurlheadersjsonrLtimeoutTCould not connect to Llamafile server. Please make sure that a server is running at .zutf-8contentr.)rr_streamwritetextgetvaluerPrequestspostrr exceptionsConnectionErrorraise_for_statusencodingrZ) r/rQrIrRrKbuffchunkrarFpayloadresponser.r.r0_callsJ        zLlamafile._callIterator[GenerationChunk]c ks|jdd|i|}d|vrd|d<d|i|}ztj|jdddi|d|jd}Wntjjy@tjd |jd wd |_|jdd D]}| |} t | d } |r_|j | j d| VqJdS)a"Yields results objects as they are generated in real time. It also calls the callback manager's on_llm_new_token event with similar parameters to the OpenAI LLM class method of the same name. Args: prompt: The prompts to pass into the model. stop: Optional list of stop words to use when generating. run_manager: **kwargs: Any additional options to pass as part of the generation request. Returns: A generator representing the stream of tokens being generated. Yields: Dictionary-like objects each containing a token Example: .. code-block:: python from langchain_community.llms import Llamafile llm = Llamafile( temperature = 0.0 ) for chunk in llm.stream("Ask 'Hi, how are you?' like a pirate:'", stop=["'"," "]): result = chunk["choices"][0] print(result["text"], end='', flush=True) rIrLTrQrTrUrVrWr\r]utf8)decode_unicode)ra)tokenNr.) rPrcrdrrrerfrh iter_lines_get_chunk_contentr on_llm_new_tokenra) r/rQrIrRrKrFrkrl raw_chunkr^rjr.r.r0r_s:&      zLlamafile._streamrjcCs*|dr|d}t|}|dS|S)zWhen streaming is turned on, llamafile server returns lines like: 'data: {"content":" They","multimodal":true,"slot_id":0,"stop":false}' Here, we convert this to a dict and return the value of the 'content' field zdata:zdata: r^) startswithlstriprZloads)r/rjcleaneddatar.r.r0rs1s  zLlamafile._get_chunk_content)r,r)r,r2)r,rCrD)rIrJrKrr,rC)NN) rQrrIrJrRrSrKrr,r) rQrrIrJrRrSrKrr,rn)rjrr,r)#__name__ __module__ __qualname____doc__r__annotations__rrrrrrrrrrr r!r#r$r%r&r'r(r)r model_configpropertyr1rBrHrPrmr_rsr.r.r.r0rsx                          ? Fr) __future__rrZiortypingrrrrrrc langchain_core.callbacks.managerr #langchain_core.language_models.llmsr langchain_core.outputsr langchain_core.utilsr pydanticr rr.r.r.r0s