HFg1ddlZddlmZmZddlmZmZddlmZm Z ddl m Z ddl m Z mZddlmZdd lmZGd d eZGd d eZy)N)ListOptional) SystemPromptAgentMessagePrompt) ActionResult ActionModel) BrowserState) HumanMessage SystemMessage)datetime)CustomAgentStepInfoc$eZdZdefdZdefdZy)CustomSystemPromptreturnc0d}|d|jdz }|S)z< Returns the important rules for the agent. aK CRITICAL: Your response MUST be a valid JSON object (not an array) with the exact structure shown below. Failure to follow this format will cause errors. 1. RESPONSE FORMAT: You must ALWAYS respond with valid JSON in this exact format: { "current_state": { "prev_action_evaluation": "Success|Failed|Unknown - Analyze the current elements and the image to check if the previous goals/actions are successful like intended by the task. Ignore the action result. The website is the ground truth. Also mention if something unexpected happened like new suggestions in an input field. Shortly state why/why not. Note that the result you output must be consistent with the reasoning you output afterwards. If you consider it to be 'Failed,' you should reflect on this during your thought.", "important_contents": "Output important contents closely related to user\'s instruction on the current page. If there is, please output the contents. If not, please output empty string ''.", "task_progress": "Task Progress is a general summary of the current contents that have been completed. Just summarize the contents that have been actually completed based on the content at current step and the history operations. Please list each completed item individually, such as: 1. Input username. 2. Input Password. 3. Click confirm button. Please return string type not a list.", "future_plans": "Based on the user's request and the current state, outline the remaining steps needed to complete the task. This should be a concise list of actions yet to be performed, such as: 1. Select a date. 2. Choose a specific time slot. 3. Confirm booking. Please return string type not a list.", "thought": "Think about the requirements that have been completed in previous operations and the requirements that need to be completed in the next one operation. If your output of prev_action_evaluation is 'Failed', please reflect and output your reflection here.", "summary": "Please generate a brief natural language description for the operation in next actions based on your Thought." }, "action": [ * actions in sequences, please refer to **Common action sequences**. Each output action MUST be formated as: \{action_name\: action_params\}* ] } EXTREMELY IMPORTANT JSON RULES: - Your ENTIRE response must be a single valid JSON object - DO NOT include any text, explanations, or comments outside the JSON structure - "current_state" MUST be an object, not an array or string - "action" MUST be an array of objects, even if there's only one action - Make sure all JSON keys and values are properly quoted with double quotes - Ensure all brackets, braces, and quotes are properly balanced - DO NOT use single quotes for JSON keys or values, use double quotes - NEVER return a plain string or a list as your top-level response - NEVER omit the "current_state" or "action" fields - ALWAYS include all required fields in "current_state" - ALWAYS wrap your response in a JSON object with { } at the top level - DO NOT include markdown formatting like ```json or ``` around your response EXAMPLE OF CORRECT JSON FORMAT: { "current_state": { "prev_action_evaluation": "Success - The page loaded correctly", "important_contents": "Search results for OpenAI", "task_progress": "1. Navigated to Google. 2. Entered search term.", "future_plans": "1. Click on the first search result.", "thought": "I need to click on the first search result to complete the task.", "summary": "Will click on the first search result for OpenAI." }, "action": [ {"click_element": {"index": 5}} ] } EXAMPLES OF INCORRECT JSON FORMAT (DO NOT DO THESE): 1. DO NOT return a list: [ { "current_state": {...}, "action": [...] } ] 2. DO NOT return a plain string: "I will navigate to Google and search for OpenAI" 3. DO NOT include markdown formatting: ```json { "current_state": {...}, "action": [...] } ``` 4. DO NOT include explanations outside the JSON: Here's my response: { "current_state": {...}, "action": [...] } 5. DO NOT use single quotes: { 'current_state': {...}, 'action': [...] } 2. ACTIONS: You can specify multiple actions to be executed in sequence. Common action sequences: - Form filling: [ {"input_text": {"index": 1, "text": "username"}}, {"input_text": {"index": 2, "text": "password"}}, {"click_element": {"index": 3}} ] - Navigation and extraction: [ {"go_to_url": {"url": "https://example.com"}}, {"extract_page_content": {}} ] 3. ELEMENT INTERACTION: - Only use indexes that exist in the provided element list - Each element has a unique index number (e.g., "33[:] [] Non-interactive text Notes: - Only elements with numeric indexes inside [] are interactive - [] elements provide context but cannot be interacted with )rs r input_formatzCustomSystemPrompt.input_formatsrN)__name__ __module__ __qualname__strrrrrrrr sNN`crrczeZdZddgddfdedeeedeeedee de dee f fd Z d d e d efd ZxZS)CustomAgentMessagePromptNistateactionsresultinclude_attributesmax_error_length step_infocDtt| |||||||_y)N)r"r$r%r&r')superr!__init__r#)rr"r#r$r%r&r' __class__s rr*z!CustomAgentMessagePrompt.__init__s1 &6U>DJ\HXAJ 7 9  r use_visionrc|jr2d|jjd|jjd}nd}tjj d}|d|z }|j jj|j}|j jxsddkD}|j jxsddkD}|dk7rI|rd |j jd |}nd |}|r|d |j jd }n|d}nd}d|d|jjd|jjd|jjd|j jd|j j d|d}|j"r>|j$r1|dz }|d|jjdz d|jjdz }t'|j$D]\}} |j"|} |d|dzdt)|j$d| j+ddz }| j,sW| j.r/|d|dzdt)|j$d| j.dz }| j0s| j0|j2 d} |d |dzdt)|j$d!| dz }|j j4r2|dk(r-t7d"|d#d$d%d&|j j4id'g(St7|(S))NzCurrent step: / z%Y-%m-%d %H:%MzCurrent date and time: )r%rz... z: pixels above - scroll or extract content to see more ... z[Start of page] z ... z9 pixels below - scroll or extract content to see more ...z [End of page]z empty pagez 1. Task: z. 2. Hints(Optional): z 3. Memory: z 4. Current url: z 5. Available tabs: z 6. Interactive elements: z z **Previous Actions** zPrevious step: r z zPrevious action z: T) exclude_unsetzResult of previous action zError of previous action z: ...r)typer image_urlurlzdata:image/png;base64,)r2r3)content)r' step_number max_stepsr nowstrftimer" element_treeclickable_elements_to_stringr% pixels_above pixels_belowtask add_infosmemoryr4tabsr#r$ enumeratelenmodel_dump_jsoninclude_in_memoryextracted_contenterrorr& screenshotr ) rr,step_info_descriptiontime_str elements_texthas_content_abovehas_content_belowstate_descriptionir$actionrGs rget_user_messagez)CustomAgentMessagePrompt.get_user_messages >>&4T^^5O5O4PPQRVR`R`RjRjQkkm$n !$& !<<>**+;<#:8*!EE //LL`d`w`wLx !ZZ449Q>!ZZ449Q> B  4::2233no|n}~#4M? C  $oVDJJ,C,C+DD}~$1/ A (M! ..    !    <>3M3Ma3O2PPQRVR`R`RjRjQkkn!o o &t{{3  6a!'7AwaDKK@P?QQSTZTjTjy}TjT~S@B&CC!++//)/I!a%PQRUVZVaVaRbQcceflf~f~e@B.CC)|| & d.C.C-C-D E)7AwaDKK@P?QQVW\V]]_`)  :: Z4%7#->? +&+/EdjjF[F[E\-]%^ $566r)T)rrrr rrrrlistrintrr*boolr rQ __classcell__)r+s@rr!r!s4837,.$'7;d;/0T,/0  !%S  "  34"H74H7<H7rr!)pdbtypingrrbrowser_use.agent.promptsrrbrowser_use.agent.viewsrrbrowser_use.browser.viewsr langchain_core.messagesr r r custom_viewsrrr!rrrr]s; !F=2?-ggTZ71Z7r