Scrapy - Web Services

Advertisements

Previous Page
Next Page  

Description

A running Scrapy web crawler can be controlled via JSON-RPC. It is enabled by JSONRPC_ENABLED setting. This service provides access to the main crawler object via JSON-RPC 2.0 protocol. The endpoint for accessing the crawler object is −

http://localhost:6080/crawler

The following table contains some of the settings which show the behavior of web service −

Sr.No Setting & Description Default Value
1

JSONRPC_ENABLED

This refers to the boolean, which decides the web service along with its extension will be enabled or not.

 True
2

JSONRPC_LOGFILE

This refers to the file used for logging HTTP requests made to the web service. If it is not set the standard Scrapy log will be used.

 None
3

JSONRPC_PORT

This refers to the port range for the web service. If it is set to none, then the port will be dynamically assigned.

 [6080, 7030]
4

JSONRPC_HOST

This refers to the interface the web service should listen on.

 '127.0.0.1'
Advertisements