Getting Started with Browserbear: How to Automate Browsers

In today's digital age, the ability to automate repetitive tasks has become increasingly important. One area where automation has been particularly useful is in web browsing. By automating the web browser, you can complete repetitive tasks without having to do it manually, which enables you to shift time and effort to other critical aspects of your work.

If you are a developer, you must have heard of automated browser tools and frameworks like Selenium, Puppeteer, and Playwright. These tools are great for browser automation but you will need to write code to give the browser instructions.

In this article, we will introduce a browser automation tool called Browserbear, a scalable, cloud-based service that requires minimum coding to automate the web browser, making web automation easier than ever for developers.

What You Can Do with Browser Automation

Testing

Testing is a crucial part of web development. Testing websites and web applications ensure that they are working as expected on different browsers and devices. By automating the browser to do testing, you can test more efficiently compared to manual testing and catch issues that might be difficult to find otherwise.

Web Scraping

Browser automation can be used to scrape and extract data from websites. This is useful for tasks like market research, competitive analysis, data mining, or any other purposes that require large amounts of data. With browser automation, you can gather data much faster than you could manually.

Task Automation

Many user actions can be automated using a browser automation tool. This includes actions like filling out forms, clicking buttons, and navigating to specific pages. By automating these actions, you can automate tasks such as creating user accounts, filling out surveys, or entering data into a web-based application.

Monitoring

Automated browsers can also be used to monitor websites and web applications for changes or errors. This can be useful for detecting issues such as broken links, page layout issues, or other problems that could affect user experience. Besides that, businesses can also use browser automation to monitor competitors’ prices or other latest updates.

What is Browserbear

Browserbear is a scalable, cloud-based browser automation tool that helps you to automate any browser task. From automated website testing, scraping data, taking scheduled screenshots for archiving, to automating other repetitive browser tasks, you can do it with Browserbear.

Similar to Bannerbear, the featured automated image and video generation tool from the same company, you can easily create no-code automated workflows by integrating it with other apps like Google Sheets,  Airtable, and more on Zapier. Besides that, you can also use the REST API to trigger tasks to run in the cloud and receive data from the completed tasks. As the tasks run in the cloud, you can create as many tasks as you need without affecting your machine's performance.

Creating an Browser Automation with Browserbear

Step 1. Create a Task

You will need a Browserbear account to create an automation task (create a free trial account ). After logging in to your account, go to Tasks and create a new task.

Enter a name for your task and click the “Save” button.

Step 2. Add Steps

Now, you can start adding the steps required for your task. Click on “Add Step” to add a new step.

On the next page, you can select an action from the Action dropdown. Different actions will require you to provide different details for the step.

For example, for the “Go” action, you will need to provide the URL that you want to visit and the Wait Until option so that Browserbear can know when to execute the next step after visiting the URL.

For some actions like “Assert”, “Click”, “Save Structured Data”, etc., you will need to locate some specific HTML elements to perform the action. For these actions, you can install the Browserbear Helper Chrome extension to get a JSON object that contains the XPath and the HTML snippet of the element, and add it to the Helper textbox.

This will help Browserbear to locate the HTML elements on the website. Click the “Save” button and you should see the step added to your task.

Add as many steps as you need to the task.

🐻 Bear Tips: Refer to How to Scrape Data from a Website Using Browserbear to learn how to use the Browserbear Helper Chrome extension.

Step 3. Run The Task

After you’re done adding all the steps needed, click “Run Task” to execute it.

You should see the status of the run when you scroll down the page.

Click on “Log” to view the result and details of the run.

If the action yields a result, you can see the result in the outputs (grouped by actions):

Whenever you create a task in Browserbear, you can access and run it via Zapier integration and REST API (making HTTP requests). This enables you to integrate the task with other applications such as Google Sheets, Notion, Airtable, etc., or make a scheduled trigger to create an automated workflow.

The Browserbear API is asynchronous. If you’re triggering the task using the REST API and want to receive the result automatically, you can use a webhook. The result will be sent to the webhook URL provided when the task has finished running.

Here’s an example in Python:

post_url = f"https://api.browserbear.com/v1/tasks/{task_uid}/runs"
data = {"webhook_url" : "your_webhook_url"}

response = requests.post(url, headers=headers, json=data)

Overwriting the Default Configurations

Although the automation tasks are created using Browserbear's UI interface, you can still modify the steps in your code by stating a new value for them, like the new URL for the “Go” action below:

post_url = f"https://api.browserbear.com/v1/tasks/{task_uid}/runs"
data = {
	"webhook_url" : "your_webhook_url",
	"steps": [
		# overwrite the original URL
	    {
	      "uid": "GNqV2ngBmly7O9dPRe", 
	      "action": "go",
	      "config": {
	        "url": "https://twitter.com/apple"
	      }
	    }
	  ]
}

response = requests.post(url, headers=headers, json=data)

When you make a POST request to the API endpoint, it will go to the new URL instead of the default one that is added from the Browserbear dashboard.

Sharing Your Browserbear Automation Task

You can share your Browserbear task by making it public. Then, you will get a shareable link to that task with the steps listed.

Any visitor with a Browserbear account will be able to duplicate the task into their account and use it immediately.

Browser Automation Examples/Templates

For common tasks like filling in a form, testing an HTML element, scraping a web page, and taking a screenshot, you can find ready-made Browserbear tasks in the Task Library.

Simply select a task and add it to your account to start automating immediately.

Let’s see what you can do with these tasks:

1. Welcome to Browserbear - Simple Form Submission

This task is an introduction to using Browserbear. It is a simple automation that shows you how to enter text into a form, hit submit, and take a screenshot after submitting the form.

In the outputs, you will see the details of the HTTP request for the “Go” action and the URL of the screenshot:

...
"outputs": {
    "ngJAvaqb8noB6jE531_go": {
     	...
      }
    },
    "gpXvMxmBjqOy35kd8e_take_screenshot": "https://media.browserbear.com/screenshots/74a1lBngYN4m3MLxR5/gpXvMxmBjqOy35kd8e/376010cf2affe8324c8b45ac1eace6a20db8bf52.jpg"
  }
...

Screenshot

2. Simple Check - Assertions on HTML Elements

This task uses the “Assert” action to check whether an HTML exists and its visibility on the web page. Besides that, you can also check if its text content matches or contains a particular text.

Result:

...
"outputs": {
    "1qXQ0djzOJKBxNKWJ3_go": {
	...
      }
    },
    "YaD8emgzJR6zE79Qno_assert": true
  }
...

3. Extract Job Data - Single Page Data Scraping

This task goes to an URL (Browserbear’s job board playground), extracts data from selected HTML elements, and saves them as structured data in JSON format. You can use it to extract data from a single web page that has no pagination.

Result:

...
"outputs": {
    "YaQ4LmEyGlpBj7xZe6_go": {
	...
      }
    },
    "Xgna6ZmbQQrbjGl5KM_save_structured_data": [
      {
        "title": "Community-Services Assistant",
        "company": "Cricket LLC",
        "country": "Virgin Islands, U.S.",
        "salary": "$14,000 / year",
        "link": "/jobs/55dJFeg_H0s-community-services-assistant/"
      },
      {
        "title": "Farming Associate",
        "company": "Greenlam LLC",
        "country": "Comoros",
        "salary": "$14,000 / year",
        "link": "/jobs/9F5EUh5gAKM-farming-associate/"
      },
...

4. Hacker News Articles - Multi-page Data Scraping

This task works similarly to the previous job data extraction task, but it caters to scenarios where there are multiple pages. Two extra steps ( “Click ” and “Go To Step” ) are added to navigate the browser to the next pages and then repeat the data scraping process.

The outputs result is similar to the previous task. For data that are scraped from the second page onwards, it will be added to the end of the same data array.

5. Wikipedia Screenshots - Website Screenshots

This task takes screenshots of the URL entered in three different sizes:

• 1024x768 pixels (default)
• 800x800 pixels
• 1440x900 pixels

After taking a screenshot, it will resize the window before taking another one.

Result:

...
"outputs": {
    "jWp2wREBNO8z9Gal4q_go": {
	...
      }
    },
    "AOxY25kzr3XbrDgELn_take_screenshot": "https://media.browserbear.com/screenshots/lgYOK3n34wlQjDE4zb/AOxY25kzr3XbrDgELn/32a6add67365e142c01ed94ffc0ec0db3940457a.jpg",
    "nvpxMrmbqWxBj7Y5V4_take_screenshot": "https://media.browserbear.com/screenshots/lgYOK3n34wlQjDE4zb/nvpxMrmbqWxBj7Y5V4/2224da170cf160665600ff3c996f175e235b369f.jpg",
    "qd26njEyDggbW3gol4_take_screenshot": "https://media.browserbear.com/screenshots/lgYOK3n34wlQjDE4zb/qd26njEyDggbW3gol4/2b4b263961e6eb1e9dd6fd3983ed5079dd545960.jpg"
  }
...

Conclusion

These are some of the ways you can use Browserbear to automate the web browser. While there are several automated browser tools and frameworks available, Browserbear helps you automate the web browser with less code while still allows you to modify/control it using code.

The importance of automation in today's digital age cannot be overstated. If you want to start automating the web browser, sign up for a Browserbear trial account! It only takes a few minutes to run your first web browser automation.