Scrapy proxy setup
Configure a NobleProxy dedicated static residential proxy in Scrapy with the exact fields the app asks for, then run its connection check before real tasks.
Quick answer
Add your NobleProxy host, port, username, and password to Scrapy's proxy settings, choose HTTP or SOCKS5, save, and run the built-in proxy check before you start tasks.
Configure a NobleProxy dedicated static residential proxy in Scrapy with the exact fields the app asks for, then run its connection check before real tasks.
Add your NobleProxy host, port, username, and password to Scrapy's proxy settings, choose HTTP or SOCKS5, save, and run the built-in proxy check before you start tasks.
| Proxy type / protocol | HTTP or SOCKS5 - both supported on every NobleProxy endpoint |
|---|---|
| Host / endpoint | The host exactly as issued in your NobleProxy dashboard |
| Port | The port paired with your endpoint in the dashboard |
| Username | Your NobleProxy credential (case sensitive) |
| Password | Your NobleProxy credential (case sensitive) |
| Static or rotating | Static - the same exit IP every session by design |
Field labels vary slightly between app versions; the NobleProxy values never change.
Set HTTP_PROXY and HTTPS_PROXY to http://username:password@host:port in the shell that launches Scrapy, replacing all four values with your dashboard credentials.
Keep the same endpoint for the whole run so target sites see one consistent residential IP for the session.
Start with one URL and confirm the response arrives through the proxy before scaling concurrency.
A 407 means credentials are wrong or missing from the URL; a timeout usually means the port is wrong or a firewall blocks the endpoint.
The tool reports the endpoint IP or country, and target pages load without credential prompts.
A refused connection, timeout, or repeated credential prompt means the host, port, username, or password does not match the dashboard values exactly.
Facts last reviewed 2026-09-01.
Scrapy reads proxy settings at middleware level and rotates nothing on its own. HTTP_PROXY and HTTPS_PROXY environment variables or a custom downloader middleware supply the endpoint; HttpProxyMiddleware is enabled by default and consumes those variables.
Record which profile uses which endpoint before you start. When every task or profile has one stable residential endpoint, its session history stays consistent across restarts and team handoffs.
NobleProxy endpoints support HTTP and SOCKS5. Pick the protocol Scrapy asks for in its proxy dialog; both use the same host, port, username, and password.
Use the host, port, username, and password exactly as issued in your NobleProxy dashboard. Enter them in the same fields Scrapy shows in its proxy configuration screen.
Yes. Map one dedicated static residential proxy to one profile and keep that mapping stable. Reusing the same endpoint for many profiles links their sessions together.
Resolve credential prompts and 407 responses during crawls.
View pageUnderstand the endpoint type you just configured.
View pageDiagnose tunnel failures between client and proxy.
View pageFix the matching proxy error.
View pageBuy self-serve now, or talk with us about a larger agency inventory.