Browser Automation for Portals Without APIs

Many businesses depend on web portals that offer no API: a supplier's ordering site, a government filing system, a carrier's tracking page, a bank's statement download screen or an old internal application. Staff log in, click through the same screens and copy data by hand, often every day. Browser automation can take over that repetitive work by driving a real browser with a script, exactly as a person would. It is useful when used carefully, but brittle and risky when rushed. This guide explains when browser automation for portals is justified, what to check before you start, how to build scripts that survive page changes and how to keep logins, data and the portal owner's rules respected.

What browser automation is

Browser automation uses a tool to control a web browser programmatically: opening pages, filling in forms, clicking buttons, waiting for results and reading or downloading information. Modern frameworks such as Playwright can run browsers with or without a visible window and are widely used for testing web applications as well as automating tasks.

For portals without APIs, browser automation can:

• Download reports or statements on a schedule
• Submit routine forms with data from another system
• Check statuses, such as order or shipment progress
• Copy information from a portal into a CRM or database

Check whether you should automate at all

Before building anything, check:

• Is there an API, export feature or file feed you have missed? Ask the provider
• Do the portal's terms of use allow automated access? Some prohibit it
• Will automation put noticeable load on the portal? Keep frequency modest
• Does the data include personal or sensitive information? Plan protections
• Is the task stable enough to automate, or does it change constantly?

If an official integration exists or can be requested, it is usually more reliable. Our guide to RPA vs API integration compares screen-based automation with API approaches in more depth.

Map the task carefully

Write down the exact steps a person follows, including:

• Login and any additional verification steps
• Each page, field and button involved
• What success looks like at each step
• Common errors, pop-ups and messages
• Variations, such as different screens for different account types

Watching someone do the task several times often reveals steps they no longer notice, such as dismissing a notice or choosing a default option.

Use stable ways to find elements

Scripts break most often when they cannot find elements after a page changes. Prefer locators that reflect what users see and are less likely to change:

• Visible labels and button text
• Accessible roles and names, such as "button named Download"
• Dedicated test or data attributes, where the portal provides them

Avoid long, position-based selectors tied to the page's layout, which break with minor design changes.

Wait for the page, not the clock

Fixed delays, such as waiting five seconds after every click, make scripts slow and still fail when the portal is slower than usual. Use waiting that checks the page state instead. Playwright's documentation describes automatic actionability checks: before a click, for example, it ensures the locator resolves to exactly one element and that the element is visible, stable, able to receive events and enabled (Playwright: auto-waiting). Combine these with explicit checks for the result you expect, such as a confirmation message or a downloaded file.

Handle logins and credentials safely

Portal credentials deserve the same care as any other secret:

• Store them in a secrets manager, not in scripts or configuration files
• Use a dedicated account for automation where the portal allows it, so actions are traceable
• Limit that account's permissions to what the task needs
• Plan for multi-factor authentication with the portal owner rather than trying to bypass it
• Rotate credentials and review access regularly

Never try to work around security controls such as CAPTCHAs or verification steps. If the portal blocks automation, talk to the provider about an approved route.

Worked example: a portal automation plan

The table below is a hypothetical plan for automating a daily statement download from a supplier portal. It shows the decisions worth recording, not a specific implementation.

StepApproachCheck for successIf it failsLog inDedicated account, credentials from secrets managerDashboard heading visibleAlert owner; stop runOpen statements pageNavigate by menu labelStatements table visibleRetry once, then alertChoose date rangeFill date fields by labelSelected dates shownStop and alertDownload statementClick button by accessible nameFile saved with expected name and sizeRetry once, then alertHand off fileMove to processing folderFile present and readableAlert ownerLog outClick sign-outLogin page visibleLog warning only

Test against a safe environment

Where the portal offers a test or demo environment, build and test there first. If not, start with read-only steps, such as viewing or downloading, before automating anything that submits data. For submission tasks, run the script in a mode that stops before the final confirmation until you are confident it fills every field correctly.

Build in error handling and evidence

When something goes wrong, the script should stop cleanly, record what happened and alert someone. Capture a screenshot and the page address at the point of failure, while avoiding screenshots of sensitive data where possible. Keep a log of each run with timings and results.

Monitor for breakage

Portals change without notice. Expect scripts to break occasionally and design for quick detection:

• Alert when a run fails or produces unexpected output
• Check that downloaded files or extracted values look reasonable
• Track run times, since sudden slowdowns can signal changes
• Keep a manual fallback procedure so staff can do the task if automation is down

Respect the portal

Run automation at sensible times and frequencies, avoid unnecessary page loads and identify your organisation if the provider asks. Treat the portal owner as someone whose systems you are using, not an obstacle. Our guide to headless browser scraping discusses responsible automation of public websites, which raises related considerations.

Plan for maintenance

Assign an owner for each automated task, document the steps and locators and keep the scripts in version control. Budget time for updates when portals change. Over time, review whether an API or official export has become available.

Frequently Asked Questions

What is browser automation?

It is the use of a tool to control a web browser with a script, opening pages, filling forms, clicking buttons and reading or downloading data as a person would.

When is browser automation appropriate for a portal?

When no API or export is available, the portal's terms allow automated access, the task is stable and repetitive and data can be protected properly.

How do you make browser automation less brittle?

Use stable locators based on labels and roles, wait for page states rather than fixed delays and check for the expected result after each step.

Should automation bypass CAPTCHAs or multi-factor authentication?

No. Respect security controls and work with the portal provider on an approved approach, such as a dedicated account or official integration.

Automate portal work responsibly

We assess portals, build maintainable browser automation and set up monitoring and fallbacks. See our B2B lead mining and prospect verification service, or talk to us about browser automation.

About the Author

Manav Sharma is the founder of Insyrge and a CRM solutions architect. He works across Zoho, Salesforce, HubSpot, Odoo and more, designing the CRM setups, integrations and automations that keep sales and operations teams working from the same data.