Product Capabilities

Web Agents extract, collect, and filter data better than basic web scrapers

Connotate’s patented technology uses machine learning to automatically generate highly efficient code and supporting context, collectively referred to as a Web Agent or just Agent.

Agents navigate websites to find the precise information that you’re looking for – ignoring ads and other extraneous website content – turning unstructured Web pages into machine-readable data to support business processes.

Connotate’s Web data extraction and collection technology is superior to Web scrapers and Web scraping scripts because we recognized that change is a fundamental aspect of the Web – and we’ve built the Connotate solution for maximum resiliency, because websites change in format all the time.

Websites also post new content constantly. Our data extraction and collection solution is optimized to not only accurately detect Web content changes but also deliver just the precise changed content to achieve tremendous efficiency gains. We filter out the noise and remove duplicate data to make workflows more efficient and effective.

Connotate4 Comparison to Other Approaches
 Connotate4 HostedConnotate4 On-premOther Enterprise SolutionsPersonal EditionsHomegrown Scripts
YOUR DEVELOPMENT PLAN:     
Who can you use?We do it for youBusiness analystProgrammerBusiness AnalystProgrammer
Required skill levelNoneLowHighLowHigh
Average extraction routine development time per site (hours)Not Applicable1-2 hours3-6 hours1-2 hours1-2 days
SCALABILITY:     
Handles multi-million item extraction easily and quickly?
Splits a job into parallel efforts to maximize throughput and minimize time?
Handles extractions accessing thousands of sites easily?
MAINTENANCE:     
Adjusts automatically to changes in a target website design?
Is easy to repair?
OPERATIONS/MANAGEMENT:     
Has scheduling and Agent management tools?
Has reporting tools for website failures?
Has validation tools for data?
FEATURES/CAPABILITIES:     
Can extract from dynamic websites (JavaScript, Ajax, HTML) without writing code?
Automatically fills in forms and search boxes?
Allows user to automatically insert compiled lists of new parameters (e.g., product numbers, date ranges, URLs, etc.) into forms and search boxes?
Has automatic, 1-click change detection set-up?
Has in-line change detection (no post-processing required)?
Delivers highlighted changes via any format, including email?
Supports point-and click set-up of automated data transformations and normalization?
Easily integrates with BI and analytics platforms?

For more details, read What Connotate Does Better Than Web Scraping Scripts and Toolkits.

Deployment Options

At Connotate, we believe that technology should accommodate your business processes – and adapt to future growth and change.

To that end, we offer flexible, practical options to meet your specific business needs.

Hosted On-Premise Advanced Web Scraping Solutions

Connotate4 On-Premise

Using our point-and-click interface, non-technical staff can begin monitoring and collecting data from websites quickly and easily:

  • Our training sessions allow your staff to become proficient in creating Agents in one or two sessions
  • One non-programmer can manage access to thousands of websites
  • Job scheduling features are very user-friendly
  • Proactive alerts notify when websites go down or otherwise fail to return data
  • Data delivery options are also easy to set up

Connotate4 On-Premise patented visual abstraction technology, allowing non-developers to quickly set up monitoring for new sites, supporting rapid deployment at scale.

The solution also supports SOAP and REST Web Services APIs for integration into workflows, as well as rapid creation of database connections for any ODBC-compliant database including SQL Server, MySQL, Oracle and others.

IDEAL FOR…

Connotate4 On-Premise is ideal if you want to build your own applications, merge them into your workflow at will and create a data collection capability that can empower your entire organization as well as your specific department.

Connotate4 Hosted

Connotate4 builds and maintains hosted solutions at our own data centers, delivering the data to you in the format you request per your defined schedule or on a transaction basis.

  • Deploys and scales quickly without tapping into your computing resources
  • You leverage our professional services team’s years of experience working with a wide variety of deployments to continually manage your project and optimize a solution
  • Service level agreements (SLAs) and 24/7 maintenance options are available

IDEAL FOR…

Connotate4 Hosted is ideal for businesses or departments that don’t have the time or cycles to manage a project start to finish and/or interact with IT to use computing resources.

Bottom Line:

Connotate lowers your average cost and time-to-market by allowing you to pick the right solution for the right website and the right job.

For more information, please browse our Web Data Collection & Monitoring Resources.