Download a webpage using CURL in C
CURL is used for the data transfer to and from a data server. Curl can be used to download a webpage. Given below is the code that can be used to download any webpage of your choice to the location of your choice
For this purpose, you must fill the following two constants
- WEBPAGE_URL : This is the URL of the webpage which you want to download. Enter the link of any website or webpage
- DESTINATION_FILE: This is the destination where the downloaded webpage is stored. Give the absolute path of the file.
#include <curl/curl.h>
#include <stdio.h>
#include <stdlib.h>
#define WEBPAGE_URL "http://google.com"
#define DESTINATION_FILE "/home/user/data.txt"
size_t write_data( void *ptr, size_t size, size_t nmeb, void *stream)
{
return fwrite(ptr,size,nmeb,stream);
}
int main()
{
FILE * file = (FILE *)fopen(DESTINATION_FILE,"w+");
if(!file){
perror("File Open:");
exit(0);
}
CURL *handle = curl_easy_init();
curl_easy_setopt(handle,CURLOPT_URL,WEBPAGE_URL); /*Using the http protocol*/
curl_easy_setopt(handle,CURLOPT_WRITEFUNCTION, write_data);
curl_easy_setopt(handle,CURLOPT_WRITEDATA, file);
curl_easy_perform(handle);
curl_easy_cleanup(handle);
}
To compile this program, you must specify the library
$gcc webpage.c -lcurl
Note the -lcurl. This is to perform the dynamic linking of the libcurl library. Let’s look at the function curl_easy_setopt. This function is used to set the URL of the webpage to be downloaded. Note the flag CURLOPT_URL used to specify the location of the webpage. Similarly the function to write the downloaded webpage to the destination file is also set using curl_easy_setopt. In short, curl_easy_setopt is used to set the parameters before taking any action of downloading or uploading a web page
Comments:
4 Comments
Leave a Reply
Translate
Search
Custom Search
Thank You VERY much for that page. It helped me a lot
You can also refer the complete tutorial here
the corrected write_data function should be; one needs to explicitly convert void* back to FILE*.
size_t write_data(void* ptr, size_t size, size_t nmeb, void* stream)
{
return fwrite(ptr,size,nmeb, (FILE*)stream);
}//write_data
thanks for the tutorial. it helps breaking the down the complexity. I learn a lot from this site ….at least about libcurl.