Web

Web Automation Nodes

Browser

The Browser node in the Integrail flow builder allows users to automate tasks that involve browsing web pages. This node can perform a variety of actions within a web browser, making it a versatile tool for extracting information or automating web-based tasks.

The browser node can be used multiple time over the agent execution, and preserves the same browser session. So you can utilize findVisibleElements, pass them to LLM mode such as ChatGTP, and then use another Browser node down the execution path to continue working with the page.

• Browser Program: This field defines the actions the node will execute in the browser. Users can input scripts or commands to control browsing behavior. The browser program is read by a simple JavaScript parser, so you can use variables and basic loops and conditional functions (for, while, if). During he process of writing of the program, you can use // to comment out specific lines if needed for testing.

• Outputs:

  • content: The last executed command/variable output.

  • firstScreenshot: A screenshot of the page at the start of the browsing session.

  • screenshots: Captures multiple screenshots during browsing.

  • next: Used for paginated browsing or continuous actions.

This node is ideal for automating data retrieval, capturing visual information from websites, and performing automated interactions with web pages.

Browser node API

This list outlines the functionality of various functions used in Browser node in order to access and manipulate web content. Whenever you see "cssSelector", it means that you will want to use the standard cssSelector syntax to pass the selector to the function.

You can use "inspect element" on any of webpage elements to find its property.

click

• Description: Performs a click on a web element identified by a CSS selector.

• Input:

  • cssSelector: A CSS selector string for identifying the target element.

• Output: None.

Example usage: click("#username");

Explanation: clicks on element that has "id="username"" in the source.

click("[name=bigbutton]");

Explanation: clicks on element that has "name="bigbutton"" in the source.

findVisibleElements

• Description: Finds visible HTML elements and adds a random string value to the specified attribute. This funciton is included in all existing llm functions, but can be useful if you want to continue working with browser node output, and then re-enter another browser node to work with the page.

• Input:

  • elements: A comma-separated string of HTML element names (div, p, textarea. input etc).

  • attribute: Any text string, it is used to mark returned elements for future usage in the program. Leave "" if you do not plan to use the output in any of LLM Browser functions.

  • selector (optional): return contents of only specific selector

• Output: Returns a string containing the HTML of the found elements.

Example usage:

findVisibleElements('div','data-llm-id');

Explanation: Returns all page elements of type div and adds a random string value as a new attribute called "data-llm-id". The new attribute is only useful if you plan to further use LLM browser functions such as llmClick. findVisibleElements('div','llm-data-id', "selector:#resultsTable");

Explanation: Only returns elements of type div that has id "resultsTable". Uses cssSelector syntax after the colon (":").

getInnerText

• Description: Retrieves the inner text of an HTML element identified by a CSS selector.

• Input:

  • cssSelector: A CSS selector string for identifying the target element.

• Output: Returns the inner text of the found HTML element.

getSecret

• Description: Retrieves a secret value by its name.

• Input:

  • secretName: The name of the secret.

• Output: Returns the value of the secret.

getURL

• Description: Retrieves the URL of the currently active browser page.

• Input: None.

• Output: Returns the current URL as a string.

getValue

• Description: Retrieves a stored value by its name.

• Input:

  • valueName: The name of the value.

• Output: Returns the value.

innerHtml

• Description: Captures and stores the inner HTML of an element for later use.

• Input:

  • cssSelector: A CSS selector string for identifying the target element.

• Output: None.

isElementPresent

• Description: Checks whether an element is present on the page.

• Input:

  • cssSelector: A CSS selector string for identifying the target element.

• Output: Returns a boolean indicating whether the element is present.

llmClick

• Description: Clicks on a web element based on a provided description.

• Input:

  • description: A description of the element to click.

• Output: None.

llmClickElement

• Description: Clicks on a web element identified by a description, selecting from a list of elements.

• Input:

  • elements: A comma-separated list of elements.

  • description: A description of the target element.

• Output: None.

llmLogin

• Description: Logs in by entering the username and password into the appropriate form fields.

• Input:

  • username: The username or email.

  • password: The password.

• Output: None.

llmSendKeys

• Description: Sends specified keys to an input element on the page.

• Input:

  • description: A description of the input element.

  • keys: The string of keys to send.

• Output: None.

llmSetValue

• Description: Sets a value in a specified element on the page.

• Input:

  • description: A description of the element.

  • value: The value to set.

• Output: None.

llmSetValueSkipVerify

• Description: Sets a value in a specified element on the page without verifying the action.

• Input:

  • description: A description of the element.

  • value: The value to set.

• Output: None.

llmText

• Description: Finds and retrieves a specified text element on the page.

• Input:

  • description: A description of the text to find.

• Output: Returns the text if found.

log

• Description: Logs a message to the server console.

• Input:

  • message: The message to log.

• Output: None.

logURL

• Description: Logs the current URL to the server console.

• Input: None.

• Output: None.

navigate

• Description: Navigates the browser to a specified URL.

• Input:

  • url: The target URL.

• Output: None.

navigateStatus

• Description: Navigates to a URL and returns the HTTP status of the page.

• Input:

  • url: The target URL.

• Output: Returns the HTTP status code of the page.

outerHtml

• Description: Captures and stores the outer HTML of an element for later use.

• Input:

  • cssSelector: A CSS selector string for identifying the target element.

• Output: None.

replaceInnerHtml

• Description: Replaces the content of a web page's element with new HTML content.

• Input:

  • cssSelector: A CSS selector string for identifying the target element.

  • htmlContent: The new HTML content to replace the current content.

• Output: None.

sendKeys

• Description: Sends keys to the browser page.

• Input:

  • text: The text or keys to send. Use newline ("\n") if certain elements require Enter press.

• Output: None.

sleep

• Description: Pauses execution for a specified duration.

• Input:

  • duration: The duration to sleep, e.g., "10s".

• Output: None.

submit

• Description: Submits a form identified by a CSS selector.

• Input:

  • cssSelector: A CSS selector string for identifying the form.

• Output: None.

takeScreenshot

• Description: Takes a screenshot of the current page.

• Input:

  • name: The name to save the screenshot under.

• Output: None.

text

• Description: A placeholder function with no defined behavior.

• Input: None.

• Output: None.

waitFileDownload

• Description: Waits for a file to be downloaded within a specified duration.

• Input:

  • duration: The duration to wait, e.g., "10s".

• Output: Returns true if the file was downloaded successfully, false otherwise.

waitFileDownloadStarted

• Description: Waits for a file download to start within a specified duration.

• Input:

  • duration: The duration to wait, e.g., "10s".

• Output: Returns true if the download started, false otherwise.

waitReady

• Description: Waits for a specified element to be ready on the page.

• Input:

  • cssSelector: A CSS selector string for identifying the target element.

• Output: None.

waitVisible

• Description: Waits for a specified element to become visible on the page.

• Input:

  • cssSelector: A CSS selector string for identifying the target element.

• Output: None.

---

Google Search

The Google Search node is designed to perform web searches using a search query and retrieve results from Google. This node is helpful for automating search tasks and extracting information from the web. Here's a breakdown of its key settings:

• Label: The name of the node, by default labeled Google Search. You can change this label to describe your task better.

• Search Query: This required field defines what you want to search for. You can specify a query term or phrase that will be used for the search, such as a keyword or question.

• Site: This optional field allows you to restrict the search to a specific website. By specifying a site, you can limit the search results to that particular domain, making the search more focused.

• Limit: This required field defines the maximum number of search results to return. The minimum value is 1, and in this case, it is set to 10.

• Fallback Outputs: This field is for specifying an alternative output or action in case the search doesn't return results or fails to execute.

• Results: The search results are displayed here as objects, which can be processed in subsequent steps or workflows.

Usage

This node is useful when you need to automate web searches or gather data from specific websites using Google search queries. It can be combined with other nodes for more complex workflows such as data extraction, web scraping, or analysis of search results.

---

Link Extractor

The Link Extractor node is designed to extract URLs from input text or content. This is particularly useful for workflows that involve web scraping, content analysis, or automated URL extraction tasks. Here's a breakdown of its key settings:

• Label: The name of the node, by default labeled Link Extractor, but you can customize it based on your specific task.

• Input: This required field specifies the content from which URLs will be extracted. It can be text, HTML, or any structured content containing URLs.

• Regex: You can define a regular expression (regex) pattern here to extract specific types of links or URLs from the input content. This is optional and allows you to filter or narrow down the extraction process.

• Content Type: This field specifies the type of content being processed, such as HTML, text, etc., to ensure the node handles the input appropriately.

• Limit: This required field defines the maximum number of URLs to extract from the input. The minimum value is 1, and you can set this according to how many links you want to process.

• Fallback Outputs: This section allows specifying an alternative action or response if the node fails to extract URLs or doesn't return valid results.

• URLs (Output): The extracted URLs are provided as an output, which can then be used in subsequent nodes or workflows.

Usage

This node is ideal when you need to automatically extract URLs from web pages, emails, or any other text content. It can be used in conjunction with web scraping workflows, content mining, or other automation processes.

---

Readability

The Readability node is used to transform or reformat the content attached to it into a more readable format. Here's an overview of how this node works:

• Label: You can customize the label for the node to make it easier to identify within the workflow.

• Attachment: This is the input field where you specify the content that you want to convert or format for readability. The attachment can be either:

  • String: A URL string pointing to the web page whose content you want to extract.

  • File: A file containing the content to be processed. Supported file types: pdf, htm/html, doc/docx, txt.

• Format: This dropdown allows you to choose the format in which the content will be processed. The available formats include:

  • HTML

  • Markdown

  • Readability

  • Plain

• Fallback Outputs: This section allows you to define an alternative action or result in case the main process fails.

---

We’d love to hear from you! Reach out to [e-mail]