A Beginner's Web Scraping Stack: VPS, Proxies and CAPTCHA Solvers on a Budget

You want to track competitor prices or collect public listings, and you've got a working script on your laptop. Moving it to something that runs every day takes three pieces: a server that's allowed to scrape, proxies that fit the target, and (sometimes) a CAPTCHA service. This guide covers each layer, adds up a realistic monthly bill, and ends with a short Python example.

All prices come from official pricing pages and were checked on 2026-10-05.

Ground rules first

Scraping public data is common and often legitimate, but how you do it matters.

• Read robots.txt and the site's terms. If a path or a whole site says no, respect it.
• Rate-limit yourself. Keep concurrency low per domain, sleep between requests, and back off on 429 or 503 responses. Your scraper shouldn't be noticeable in the site's load graphs.
• Stick to public pages. Don't bypass logins or paywalls, and be careful with personal data. Privacy laws such as GDPR and CCPA can apply even to information that's publicly visible.
• Treat CAPTCHAs as a signal. A CAPTCHA often means the site doesn't want automated traffic on that page. Only use solving services where you're authorized to access the content, and prefer an official API whenever one exists.

This is general guidance, not legal advice. For commercial projects, talk to a lawyer.

Layer 1: the VPS

Before comparing specs, check whether the host's acceptable use policy allows scraping. Getting your server terminated mid-project costs more than any price difference.

Host	Starting price	What the terms say
AlphaVPS	2 vCPU / 2 GB cloud VPS, €2.99/mo	Personal VPN/proxy use allowed; bans artificial traffic and port scanning; no explicit scraping clause
DMIT	Tier 1 TINY, $6.90/mo	No explicit scraping ban (public proxy services not allowed); CN2 GIA plans for China access
BandwagonHost	From $49.99/yr	Explicitly bans data mining and crawling. Don't use it for scraping.

For targets in Europe or the US on a small budget, AlphaVPS is enough. DMIT makes sense if you or your team need a stable connection to the server from mainland China. Our best VPS for scraping comparison goes deeper on terms and locations.

Layer 2: proxies

Pick the type based on how well the target defends itself.

Lightly protected sites: datacenter proxies. Cheapest and fastest.

• Webshare: 10 free proxies, or 100 for $2.99/month
• Proxy6: dedicated IPv4 at $1.5 per IP for one month (1–99 IPs); IPv6 at $0.258 per IP for one month (1–9 IPs); HTTPS and SOCKS5 on the same proxy
• PrivateProxy: static datacenter from $3/month with unmetered bandwidth

Note: PrivateProxy's terms prohibit bypassing CAPTCHAs and running fake accounts. If your workflow needs a CAPTCHA solver, use one of the other providers above.

Sites with stronger bot protection: rotating residential.

• DataImpulse: $1/GB, pay as you go, traffic never expires
• Novada: residential from $1.3/GB, a $10-for-10-GB new-user offer, and rotating datacenter at $0.50/GB

Targets that support IPv6: IPv6 is the cheapest per IP, but confirm the site is reachable over IPv6 first.

More numbers are in our cheapest proxies price comparison, and the trade-offs between types are in Residential vs ISP vs Mobile vs Datacenter Proxies.

Layer 3: CAPTCHA solving (only if you need it)

Service	reCAPTCHA v2	Turnstile	Image CAPTCHA	Notes
CapSolver	$0.80/1K	$1.20/1K	$0.40/1K	AI-based; no refunds
2Captcha	$1–2.99/1K	$1.45/1K	$0.50–1/1K	Human-powered; unused balance refundable
SolveCaptcha	$0.55/1K	$0.80/1K	$0.35/1K	AI plus human solvers; accepts PayPal and Alipay

For a detailed look at the two established services, see CapSolver vs 2Captcha.

What it costs: a small price-monitoring project

Say you use about 10 GB of residential traffic a month and hit roughly 2,000 reCAPTCHA v2 challenges:

Item	Choice	Monthly cost
VPS	AlphaVPS 2 vCPU / 2 GB	€2.99
Proxies	DataImpulse residential, 10 GB × $1	$10.00
CAPTCHA	CapSolver, 2,000 × $0.80/1K	$1.60
Total		€2.99 + $11.60

If the target is lightly protected and shows no CAPTCHAs, swap in Webshare's 100 datacenter proxies and drop the solver: €2.99 + $2.99 a month. Swapping CapSolver for SolveCaptcha in the first scenario brings the CAPTCHA line to $1.10.

These are list prices for one example workload. Your real bill depends on page size, retries and how often the target changes its defenses.

A minimal Python example

This uses requests, checks robots.txt before fetching, and slows down when the server pushes back. Replace the placeholders with the endpoint and credentials from your proxy dashboard.

import time
import requests
from urllib import robotparser

PROXY = "http://USERNAME:PASSWORD@PROXY_HOST:PORT"
proxies = {"http": PROXY, "https": PROXY}
HEADERS = {"User-Agent": "MyPriceBot/1.0 (contact: [email protected])"}

rp = robotparser.RobotFileParser("https://example.com/robots.txt")
rp.read()

for url in ["https://example.com/item/1", "https://example.com/item/2"]:
    if not rp.can_fetch(HEADERS["User-Agent"], url):
        print("Disallowed by robots.txt:", url)
        continue
    resp = requests.get(url, headers=HEADERS, proxies=proxies, timeout=20)
    print(url, resp.status_code)
    if resp.status_code in (429, 503):
        time.sleep(60)  # back off when rate-limited
    time.sleep(2)       # keep a polite pace

For a SOCKS5 proxy, install requests[socks] and use socks5h://USERNAME:PASSWORD@PROXY_HOST:PORT so DNS lookups also go through the proxy.

Running it every day

Once the script works, put it on the VPS and keep the operational side boring:

• Schedule it with cron or a systemd timer rather than leaving a terminal open.
• Log status codes per run. A sudden jump in 403s or CAPTCHA pages tells you the target changed something before your data quality does.
• Cap retries. Two or three retries with growing delays is plenty. Hammering a page that keeps failing burns proxy traffic and looks abusive.
• Cache what doesn't change. Re-downloading static pages every day costs bandwidth on both sides.
• Keep credentials out of the code. Read the proxy username and password from environment variables, especially if the script lives in a shared repository.

When to upgrade a layer

Change one layer at a time, and only when your logs say so:

• Success rate drops on a datacenter IP: test a few GB of rotating residential before switching everything.
• CAPTCHAs start appearing: first slow down. If they persist and you're authorized to access the content, add a solver.
• The VPS is maxed out: move up a plan with the same host before adding servers. On AlphaVPS, the next cloud tier is 2 vCPU / 4 GB at €3.99/month.

Where to start

Get the script working end to end with Webshare's free proxies or DataImpulse's $5 intro plan, measure your success rate, and only then pay for residential traffic or a CAPTCHA balance. Mobile proxies rarely make sense for scraping; our mobile proxy guide explains why.