Making HTTP Requests with Python

Python provides a built-in module called http.client that allows you to make HTTP requests and interact with web servers. In this article, we will explore how to use the http.client module to send GET, POST, and PUT requests, handle responses, and work with headers.

Establishing HTTP Connections

To start making HTTP requests with Python, you need to establish a connection to the web server. The http.client module provides the HTTPConnection and HTTPSConnection classes for creating connections over HTTP and HTTPS protocols, respectively.

Here's an example of establishing an HTTP connection:

import http.client

connection = http.client.HTTPConnection('www.example.com', 80, timeout=10)

print(connection)

In this example, we create an HTTPConnection object by specifying the host (www.example.com), port (80), and an optional timeout value in seconds.

Sending HTTP GET Requests

Once you have established a connection, you can send HTTP GET requests to retrieve data from a server. Here's an example:

import http.client

connection = http.client.HTTPSConnection("www.example.com")

connection.request("GET", "/")

response = connection.getresponse()

print("Status: {} and reason: {}".format(response.status, response.reason))

connection.close()

In this example, we create an HTTPSConnection to establish a secure connection to www.example.com. We then send a GET request to the root path ("/") using the request() method. The getresponse() method is used to retrieve the response from the server. We print the response status and reason, and finally close the connection.

Note: If you encounter an SSL: CERTIFICATE_VERIFY_FAILED error, it may be due to missing or outdated SSL certificates on your system. On macOS, you can resolve this by running the Install Certificates.command file located in your Python installation directory.

Handling Response Headers

HTTP responses often include headers that provide additional information about the response, such as the content type and status code. You can retrieve the headers from the response object using the getheaders() method. Here's an example:

import http.client

import pprint

connection = http.client.HTTPSConnection("www.example.com")

connection.request("GET", "/")

response = connection.getresponse()

headers = response.getheaders()

pp = pprint.PrettyPrinter(indent=4)

pp.pprint("Headers: {}".format(headers))

In this example, we retrieve the headers from the response using getheaders() and print them using the pprint module for better readability.

Sending HTTP POST Requests

To send data to a server, you can use HTTP POST requests. Here's an example of sending a POST request with JSON data:

import http.client

import json

conn = http.client.HTTPSConnection('www.httpbin.org')

headers = {'Content-type': 'application/json'}

data = {'text': 'Hello HTTP #1 **cool**, and #1!'}

json_data = json.dumps(data)

conn.request('POST', '/post', json_data, headers)

response = conn.getresponse()

print(response.read().decode())

In this example, we establish an HTTPS connection to www.httpbin.org. We set the Content-type header to application/json to indicate that we're sending JSON data. We create a dictionary data and convert it to a JSON string using json.dumps(). We then send the POST request using the request() method, specifying the path ("/post"), JSON data, and headers. Finally, we read and decode the response.

Sending HTTP PUT Requests

Similar to POST requests, you can also send PUT requests using the http.client module. Here's an example:

import http.client

import json

conn = http.client.HTTPSConnection('www.httpbin.org')

headers = {'Content-type': 'application/json'}

data = {'text': 'Hello HTTP #1 **cool**, and #1!'}

json_data = json.dumps(data)

conn.request("PUT", "/put", json_data)

response = conn.getresponse()

print(response.status, response.reason)

In this example, we send a PUT request to www.httpbin.org with JSON data. We set the Content-type header to application/json and send the request using the request() method, specifying the path ("/put") and JSON data. We then print the response status and reason.

Summary

In this article, we explored how to make HTTP requests using Python's http.client module. We learned how to establish connections, send GET, POST, and PUT requests, handle response headers, and work with JSON data. The http.client module provides a simple and efficient way to interact with web servers and perform various HTTP operations in Python.

Remember to close the connection when you're done using it to free up resources. You can also refer to the official Python documentation for more details on the http.client module and its API.