o
Ag @ sh d dl Z d dlZd dlmZmZmZmZmZ d dlm Z m
Z
d dl m Z d dl
mZ G dd deZdS ) N)AnyDictListLiteralOptional)#AsyncCallbackManagerForRetrieverRun CallbackManagerForRetrieverRun)Document)
BaseRetrieverc @ s
e Zd ZU dZdZeed< dZeed< dZe e ed< dZ
e e ed< d Z e d
ed < d gZ
ee d
ed< dZeed< dZeed< i Ze eeef ed< dZe e ed< dZe e ed< dededee fddZdede dee fdd Z d edee fd d Z dS ) AskNewsRetrieverzAskNews retriever.
kr offsetNstart_timestamp
end_timestampnl)r kwmethodAll)r BusinessCrimePoliticsScienceSports
TechnologyMilitaryHealth
EntertainmentFinanceCultureClimate EnvironmentWorld
categoriesF
historicalg ?similarity_score_thresholdkwargs client_id
client_secretquery run_managerreturnc C s zddl m} W n ty tdw || jp tjd | jp#tjd dgd}|jjd
|| j | j
| j | j | j
| j| j| jdd d
d | j}| |S )zGet documents relevant to a query.
Args:
query: String to find relevant documents for
run_manager: The callbacks handler to use
Returns:
List of relevant documents
r )
AskNewsSDKOAskNews python package not found. Please install it with `pip install asknews`.ASKNEWS_CLIENT_IDASKNEWS_CLIENT_SECRETnewsr' r( scopesboth) r)
n_articlesr r r r# r$ r% r doc_start_delimiterdoc_end_delimiter return_typeN ) asknews_sdkr, ImportErrorr' osenvironr( r0 search_newsr
r r r r# r$ r% r r& _extract_documents)selfr) r* r, an_clientresponser: r: m/var/www/html/development/chatbot/venv/lib/python3.10/site-packages/langchain_community/retrievers/asknews.py_get_relevant_documents. s:
z(AskNewsRetriever._get_relevant_documentsc s zddl m} W n ty tdw || jp tjd | jp$tjd dgd}|jjd
|| j | j
| j | j | j
| j| j| jdd d
d | jI d H }| |S )zAsynchronously get documents relevant to a query.
Args:
query: String to find relevant documents for
run_manager: The callbacks handler to use
Returns:
List of relevant documents
r )AsyncAskNewsSDKr- r. r/ r0 r1 r5 r3 r4 ) r) r6 r r r r# r$ r% r r9 r7 r8 Nr: )r; rF r< r' r= r> r( r0 r? r
r r r r# r$ r% r r& r@ )rA r) r* rF rB rC r: r: rD _aget_relevant_documentsV s<
z)AskNewsRetriever._aget_relevant_documentsrC c sB ddl m} |tdjtj fddtt D }|S )z'Extract documents from an api response.r )SearchResponsez(.*?)c sT g | ]&}t | j| jj| jr tj| jnd j| jddqS )N)titlesourceimages) page_contentmetadata)r stripas_dictsrI article_urlstr image_url).0imatchessrr: rD
s
z7AskNewsRetriever._extract_documents..)asknews_sdk.dto.newsrH refindall as_stringDOTALLrangelen)rA rC rH docsr: rU rD r@ ~ s
z#AskNewsRetriever._extract_documents) __name__
__module__ __qualname____doc__r
int__annotations__r r r r r r r# r r$ boolr% floatr& r rQ r r' r( r r rE r rG r@ r: r: r: rD r
sB
(
(r )r= rZ typingr r r r r langchain_core.callbacksr r langchain_core.documentsr langchain_core.retrieversr
r r: r: r: rD s