How to Use a Proxy With Httpx

In this guide, I’ll walk you through how to use proxies with Httpx. I’ll show you how to set them up, use authentication, rotate between different proxies, and even work with premium residential proxies. Don’t worry — it’s easier than it sounds. Let’s dive in and get started!

What is Httpx?

Httpx is an advanced HTTP client for Python, built by the same team behind the popular requests library. However, Httpx comes with extra features:

• Support for synchronous and asynchronous requests
• HTTP/2 and connection pooling
• Streaming and timeout control

You can install it via pip:

pip install httpx

Let’s now see how to use proxies with Httpx.

Why Use a Proxy?

A proxy server acts as a go-between for your scraper and the target website. Instead of sending the request directly, your scraper sends it through a proxy, which forwards it to the destination.

Benefits of Using a Proxy

• Bypass IP bans: Websites can block your real IP if you send too many requests. A proxy prevents that.
• Geo-location control: You can simulate users from different countries.
• Better anonymity: Your actual IP is never exposed.
• Scalability: Send thousands of requests without raising red flags.

Basic Request Without a Proxy

Here’s a basic Httpx request that does not use a proxy:

# scraper.py
import httpx
response = httpx.get("https://httpbin.io/ip")
print(response.text)

This returns your actual IP address:

{
"origin": "123.45.67.89"
}

How to Set Your Proxy With Httpx in Python

Now, let’s send the same request through a proxy.

Step 1: Add a Proxy With Httpx

To use a proxy, define it with the proxies parameter:

# scraper.py
import httpx
proxies = {
"http://": "http://216.137.184.253:80",
"https://": "http://216.137.184.253:80"
}
response = httpx.get("https://httpbin.io/ip", proxies=proxies)
print(response.text)

Tip: Free proxies may be unstable or already blocked. Use them for testing, not production.

Step 2: Use Httpx Client With Proxy

You can also pass the proxy when creating a Client instance:

# scraper.py
import httpx
with httpx.Client(proxy="http://216.137.184.253:80") as client:
response = client.get("https://httpbin.io/ip")
print(response.text)

Using Client is helpful if you’re making multiple requests with the same proxy.

Step 3: Proxy Authentication (Username & Password)

Premium proxies (like those from Bright Data) often require authentication. You’ll need to include your username and password in the proxy URL.

Format:

http://<USERNAME>:<PASSWORD>@<PROXY_IP>:<PORT>

Example:

# scraper.py
import httpx
proxy_url = "http://myuser:[email protected]:80"
with httpx.Client(proxy=proxy_url) as client:
response = client.get("https://httpbin.io/ip")
print(response.text)

Replace myuser and mypass with your actual Bright Data proxy credentials.

Step 4: Rotate Proxies With Httpx

Rotating proxies helps you avoid rate limits or bans. You can maintain a list and choose a random one for each request.

Example:

# scraper.py
import httpx
import random
proxy_list = [
"http://20.210.113.32:8123",
"http://47.56.110.204:8989",
"http://50.174.214.216:80"
]
random_proxy = random.choice(proxy_list)
with httpx.Client(proxy=random_proxy) as client:
response = client.get("https://httpbin.io/ip")
print(response.text)

Each request will go through a randomly selected proxy.

Step 5: Handle Failures With Retry Logic

Some proxies may fail. Add retry logic to keep your scraper running smoothly:

# scraper.py
import httpx
import random
import time
proxies = [
"http://20.210.113.32:8123",
"http://47.56.110.204:8989",
"http://50.174.214.216:80"
]
url = "https://httpbin.io/ip"
for attempt in range(5):
proxy = random.choice(proxies)
try:
with httpx.Client(proxy=proxy, timeout=5) as client:
response = client.get(url)
print(response.text)
break
except httpx.RequestError:
print(f"Attempt {attempt 1} failed with proxy {proxy}")
time.sleep(2)

Step 6: Async Requests With Proxy

Httpx supports async requests with AsyncClient. This is useful for high-speed scraping.

Example:

# scraper_async.py
import httpx
import asyncio
async def fetch():
proxy_url = "http://216.137.184.253:80"
async with httpx.AsyncClient(proxy=proxy_url) as client:
response = await client.get("https://httpbin.io/ip")
print(response.text)
asyncio.run(fetch())

You can scale this by launching many coroutines in parallel.

Step 7: Use Bright Data Proxies With Httpx

Free proxies are useful for testing, but they’re slow, unsecure, unreliable, and often blocked.

For production scraping, use a premium proxy provider like Bright Data or IPRoyal.

Why I Chose Bright Data?

• Over 150M residential IPs worldwide
• Geo-targeting by country, city, or ASN
• Automatic IP rotation
• High-speed, low-latency performance
• Dashboard and API for real-time proxy management

How to Use Bright Data With Httpx

After signing up, you’ll get your proxy credentials.

Here’s how to plug them into your code:

# scraper.py
import httpx
proxy_url = "http://username:[email protected]:33335"
with httpx.Client(proxy=proxy_url) as client:
response = client.get("https://httpbin.io/ip")
print(response.text)

Replace username and password with your actual Bright Data proxy credentials.

Example Output

Each time you run the above script, you’ll get a different residential IP:

{
"origin": "178.62.45.183"
}
{
"origin": "191.101.54.225"
}

Bright Data handles IP rotation automatically if you configure it in your dashboard.

Bonus: Rotate Headers and User Agents

To further avoid detection, rotate headers like User-Agent to simulate different browsers.

Example:

import httpx
import random
user_agents = [
"Mozilla/5.0 (Windows NT 10.0; Win64; x64)",
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7)",
"Mozilla/5.0 (X11; Linux x86_64)"
]
headers = {
"User-Agent": random.choice(user_agents)
}
proxy = "http://username:[email protected]:22225"
with httpx.Client(proxy=proxy, headers=headers) as client:
response = client.get("https://httpbin.io/headers")
print(response.json())

This mimics real browser requests, helping you stay undetected.

Conclusion

Using proxies with Httpx makes your web scraping smarter and more reliable. You now know how to set up basic proxies, use login details, switch between different proxies, and work with high-quality residential ones. You also learned how to catch and handle errors smoothly. These skills help you avoid blocks and keep your scrapers running longer.

With this knowledge, you’re ready to explore the web more freely and gather data more safely. Start testing proxies today and see what you can build. With Httpx and Python, powerful scraping tools are just a few lines of code away. Keep experimenting and learning!