aboutsummaryrefslogtreecommitdiff
path: root/http-fetch.c
Commit message (Collapse)AuthorAge
* http-fetch: nicer warning for a server with unreliable 404 statusJunio C Hamano2006-03-20
| | | | | | | | | | | | | | When a repository otherwise properly prepared is served by a dumb HTTP server that sends "No such page" output with 200 status for human consumption to a request for a page that does not exist, the users will get an alarming "File X corrupt" error message. Hint that they might be dealing with such a server at the end and suggest running fsck-objects to check if the result is OK (the pack-fallback code does the right thing in this case so unless a loose object file was actually corrupt the result should check OK). Signed-off-by: Junio C Hamano <junkio@cox.net>
* Fix broken slot reuse when fetching alternatesNick Hengeveld2006-03-15
| | | | | | | | | | When fetching alternates, http-fetch may reuse the slot to fetch non-http alternates if http-alternates does not exist. When doing so, it now needs to update the slot's finished status so run_active_slot waits for the non-http alternates request to finish. Signed-off-by: Nick Hengeveld <nickh@reactrix.com> Signed-off-by: Junio C Hamano <junkio@cox.net>
* Give no terminating LF to error() function.Junio C Hamano2006-02-22
| | | | Signed-off-by: Junio C Hamano <junkio@cox.net>
* http-fetch: Abort requests for objects which arrived in packsMark Wooding2006-02-07
| | | | | | | | | | In fetch_object, there's a call to release an object request if the object mysteriously arrived, say in a pack. Unfortunately, the fetch attempt for this object might already be in progress, and we'll leak the descriptor. Instead, try to tidy away the request. Signed-off-by: Mark Wooding <mdw@distorted.org.uk> Signed-off-by: Junio C Hamano <junkio@cox.net>
* http-fetch: Tidy control flow in process_alternate_responseMark Wooding2006-02-05
| | | | | | | It's a bit convoluted. Tidy it up. Signed-off-by: Mark Wooding <mdw@distorted.org.uk> Signed-off-by: Junio C Hamano <junkio@cox.net>
* http-fetch: Fix message reporting rename of object file.Mark Wooding2006-02-05
| | | | | | | | | move_temp_to_file returns 0 or -1. This is not a good thing to pass to strerror(3). Fortunately, someone already reported the error, so don't worry too much. Signed-off-by: Mark Wooding <mdw@distorted.org.uk> Signed-off-by: Junio C Hamano <junkio@cox.net>
* http-fetch: Fix object list corruption in fill_active_slots().Mark Wooding2006-02-05
| | | | | | | | | | | In fill_active_slots() -- if we find an object which has already arrived, say as part of a pack, /don't/ remove it from the list. It's already been prefetched and someone will ask for it later. Just label it as done and carry blithely on. (As it was, the code would dereference a freed object to continue through the list anyway.) Signed-off-by: Mark Wooding <mdw@distorted.org.uk> Signed-off-by: Junio C Hamano <junkio@cox.net>
* whitespace cleanup.Junio C Hamano2006-02-02
| | | | Signed-off-by: Junio C Hamano <junkio@cox.net>
* Use local structs for HTTP slot callback dataNick Hengeveld2006-01-31
| | | | | | | | There's no need for these structures to be static, and it could potentially cause problems down the road. Signed-off-by: Nick Hengeveld <nickh@reactrix.com> Signed-off-by: Junio C Hamano <junkio@cox.net>
* Fix HTTP request result processing after slot reuseNick Hengeveld2006-01-31
| | | | | | | | Add a way to store the results of an HTTP request when a slot finishes so the results can be processed after the slot has been reused. Signed-off-by: Nick Hengeveld <nickh@reactrix.com> Signed-off-by: Junio C Hamano <junkio@cox.net>
* Fix for http-fetch from file:// URLsNick Hengeveld2005-12-22
| | | | | | | Recognize missing files when using http-fetch with file:// URLs Signed-off-by: Nick Hengeveld <nickh@reactrix.com> Signed-off-by: Junio C Hamano <junkio@cox.net>
* http-fetch.c: fix objects/info/pack parsing.Junio C Hamano2005-12-21
| | | | | | | It failed to register the last pack in the objects/info/packs file. Also it had an independent overrun error. Signed-off-by: Junio C Hamano <junkio@cox.net>
* Make networking commands to work from a subdirectory.Junio C Hamano2005-11-28
| | | | | | | | | These are whole-tree operations and there is not much point making them operable from within a subdirectory, but it is easy to do so, and using setup_git_directory() upfront helps git:// proxy specification picked up from the correct place. Signed-off-by: Junio C Hamano <junkio@cox.net>
* Improve pack list response handlingNick Hengeveld2005-11-19
| | | | | | | | Better response handling for pack list requests - a 404 means we do have the list but it happens to be empty. Signed-off-by: Nick Hengeveld <nickh@reactrix.com> Signed-off-by: Junio C Hamano <junkio@cox.net>
* Make http-fetch request types more clearNick Hengeveld2005-11-19
| | | | | | | | | | Rename object request functions and data to make it more clear which type of request is being processed - this is a response to the introduction of slot callbacks and the definition of different types of requests such as alternates_request. Signed-off-by: Nick Hengeveld <nickh@reactrix.com> Signed-off-by: Junio C Hamano <junkio@cox.net>
* Isolate shared HTTP request functionalityNick Hengeveld2005-11-19
| | | | | | | | | | | | Move shared HTTP request functionality out of http-fetch and http-push, and replace the two fwrite_buffer/fwrite_buffer_dynamic functions with one fwrite_buffer function that does dynamic buffering. Use slot callbacks to process responses to fetch object transfer requests and push transfer requests, and put all of http-push into an #ifdef check for curl multi support. Signed-off-by: Nick Hengeveld <nickh@reactrix.com> Signed-off-by: Junio C Hamano <junkio@cox.net>
* Fix bunch of fd leaks in http-fetchPetr Baudis2005-11-15
| | | | | | | | | | | | | The current http-fetch is rather careless about fd leakage, causing problems while fetching large repositories. This patch does not reserve exhaustiveness, but I covered everything I spotted. I also left some safeguards in place in case I missed something, so that we get to know, sooner or later. Reported by Becky Bruce <becky.bruce@freescale.com>. Signed-off-by: Petr Baudis <pasky@suse.cz> Signed-off-by: Junio C Hamano <junkio@cox.net>
* Fix fd leak in http-fetchNick Hengeveld2005-11-12
| | | | | | | | Added a call to finish_request to clean up resources if the server returned a 404 and there are no alternates left to try. Signed-off-by: Nick Hengeveld <nickh@reactrix.com> Signed-off-by: Junio C Hamano <junkio@cox.net>
* Fix for multiple alternates requests in http-fetchNick Hengeveld2005-11-12
| | | | | | | | | Stop additional alternates requests from starting if one is already in progress. This adds an optional callback which is processed after a slot has finished running. Signed-off-by: Nick Hengeveld <nickh@reactrix.com> Signed-off-by: Junio C Hamano <junkio@cox.net>
* Show URL in the "Getting <foo> list" http-fetch messagesPetr Baudis2005-11-11
| | | | | Signed-off-by: Petr Baudis <pasky@suse.cz> Signed-off-by: Junio C Hamano <junkio@cox.net>
* Let git-clone/git-fetch follow HTTP redirectionsJosef Weidendorfer2005-11-10
| | | | | | | | | | | Otherwise, git-clone silently failed to clone a remote repository where redirections (ie. a response with a "Location" header line) are used. This includes the fixes from Nick Hengeveld. Signed-off-by: Josef Weidendorfer <Josef.Weidendorfer@gmx.de> Signed-off-by: Junio C Hamano <junkio@cox.net>
* sparse fixes for http-{fetch,push}.cPeter Hagervall2005-11-08
| | | | | | | | Make a bunch of needlessly global functions static, and replace two K&R-style declarations. Signed-off-by: Peter Hagervall <hager@cs.umu.se> Signed-off-by: Junio C Hamano <junkio@cox.net>
* http-fetch: do not use curl_message after releasing itJohannes Schindelin2005-11-06
| | | | | | | | When curl_message is released using curl_multi_remove_handle(), it's contents are undefined. Therefore, get the information before releasing it. Signed-off-by: Johannes Schindelin <Johannes.Schindelin@gmx.de> Signed-off-by: Junio C Hamano <junkio@cox.net>
* Remove the temp file if it is empty after the request has failedNick Hengeveld2005-11-03
| | | | | | | | | | | | | After using cg-update to pull, empty files named *.temp are left in the various subdirectories of .git/objects/. These are created by git-http-fetch to hold data as it's being fetched from the remote repository. They are left behind after a transfer error so that the next time git-http-fetch runs it can pick up where it left off. If they're empty though, it would make more sense to delete them rather than leaving them behind for the next attempt. Signed-off-by: Nick Hengeveld <nickh@reactrix.com> Signed-off-by: Junio C Hamano <junkio@cox.net>
* Silence confusing and false-positive curl error messagePetr Baudis2005-10-23
| | | | | | | | | | | git-http-fetch spits out curl 404 error message when unable to fetch an object, but that's confusing since no error really happened and the object is usually found in a pack it tries right after that. And if the object still cannot be retrieved, it will say another error message anyway. OTOH other HTTP errors (403 etc) are likely fatal and the user should be still informed about them. Signed-off-by: Petr Baudis <pasky@suse.cz> Signed-off-by: Junio C Hamano <junkio@cox.net>
* [PATCH 3/3] Allow running requests to finish after a pull errorNick Hengeveld2005-10-21
| | | | | | | Allow running requests to finish after a pull error Signed-off-by: Nick Hengeveld <nickh@reactrix.com> Signed-off-by: Junio C Hamano <junkio@cox.net>
* [PATCH 2/3] Switched back to loading alternates as neededNick Hengeveld2005-10-21
| | | | | | | Switched back to loading alternates as needed Signed-off-by: Nick Hengeveld <nickh@reactrix.com> Signed-off-by: Junio C Hamano <junkio@cox.net>
* [PATCH 1/3] Clean up CURL handles in unused request slotsNick Hengeveld2005-10-21
| | | | | | | Clean up CURL handles in unused request slots Signed-off-by: Nick Hengeveld <nickh@reactrix.com> Signed-off-by: Junio C Hamano <junkio@cox.net>
* Support for HTTP transfer timeouts based on transfer speedNick Hengeveld2005-10-19
| | | | | | | | | Add configuration settings to abort HTTP requests if the transfer rate drops below a threshold for a specified length of time. Environment variables override config file settings. Signed-off-by: Nick Hengeveld <nickh@reactrix.com> Signed-off-by: Junio C Hamano <junkio@cox.net>
* Sparse fixes for http-fetchPeter Hagervall2005-10-16
| | | | | | | | | | | | | | This patch cleans out all sparse warnings from http-fetch.c I'm a bit uncomfortable with adding extra #ifdefs to avoid either 'mixing declaration with code' or 'unused variable' warnings, but I figured that since those functions are already littered with #ifdefs I might just get away with it. Comments? [jc: I adjusted Peter's patch to address uncomfortableness issues.] Signed-off-by: Peter Hagervall <hager@cs.umu.se> Signed-off-by: Junio C Hamano <junkio@cox.net>
* Show curl error a bit better.Junio C Hamano2005-10-15
| | | | Signed-off-by: Junio C Hamano <junkio@cox.net>
* Some curl versions lack curl_easy_duphandle()Johannes Schindelin2005-10-15
| | | | | | | | | | | | | | | | | | | | | Hi, On Fri, 14 Oct 2005, Junio C Hamano wrote: > Johannes Schindelin <Johannes.Schindelin@gmx.de> writes: > > > This patch looks bigger than it really is: The code to get the > > default handle was refactored into a function, and is called > > instead of curl_easy_duphandle() if that does not exist. > > I'd like to take Nick's config file patch first, which > unfortunately interferes with your patch. I'd hate to ask you > this, but could you rebase it on top of Nick's patch, [...] No need to hate it. Here comes the rebased patch, and this time, I actually tested it a bit. Signed-off-by: Junio C Hamano <junkio@cox.net>
* Use config file settings for httpNick Hengeveld2005-10-14
| | | | | | | | Use "http." config file settings if they exist. Environment variables still work, and they will override config file settings. Signed-off-by: Nick Hengeveld <nickh@reactrix.com> Signed-off-by: Junio C Hamano <junkio@cox.net>
* git-http-fetch: Remove size limit for objects/info/{packs,alternates}Sergey Vlasov2005-10-14
| | | | | | | | | | | git-http-fetch received objects/info/packs into a fixed-size buffer and started to fail when this file became larger than the buffer. Change it to grow the buffer dynamically, and do the same thing for objects/info/alternates. Also add missing free() calls for these buffers. Signed-off-by: Sergey Vlasov <vsu@altlinux.ru> Signed-off-by: Junio C Hamano <junkio@cox.net>
* Quote reference names while fetching with curl.Junio C Hamano2005-10-11
| | | | | | | | | | | | | | curl_escape ought to do this, but we should not let it quote slashes (nobody said refs/tags cannot have subdirectories), so we roll our own safer version. With this, the last part of git-clone from Martin's moodle repository that used to fail now works, which reads: $ git-http-fetch -v -a -w 'tags/MOODLE_15_MERGED **INVALID**' \ 'tags/MOODLE_15_MERGED **INVALID**' \ http://locke.catalyst.net.nz/git/moodle.git/ Signed-off-by: Junio C Hamano <junkio@cox.net>
* Remove unused 'got_alternates' variable.Junio C Hamano2005-10-10
| | | | | | | The function fetch_alternates() is called only once from the main now. Signed-off-by: Junio C Hamano <junkio@cox.net>
* Use the same move_temp_to_file in git-http-fetch.Junio C Hamano2005-10-10
| | | | | | | | | | | | | | | | | The http commit walker cannot use the same temporary file creation code because it needs to use predictable temporary filename for partial fetch continuation purposes, but the code to move the temporary file to the final location should be usable from the ordinary object creation codepath. Export move_temp_to_file from sha1_file.c and use it, while losing the custom relink_or_rename function from http-fetch.c. Also the temporary object file creation part needs to make sure the leading path exists, in preparation of the really lazy fan-out directory creation. Signed-off-by: Junio C Hamano <junkio@cox.net>
* Restore functionality to allow proxies to cache objectsNick Hengeveld2005-10-10
| | | | | | | | | | | The parallel request changes didn't properly implement the previous patch to allow caching of retrieved objects by proxy servers. Restore the previous functionality such that by default requests include the "Pragma: no-cache" header, and this header is removed on requests for pack indexes, packs, and objects. Signed-off-by: Nick Hengeveld <nickh@reactrix.com> Signed-off-by: Junio C Hamano <junkio@cox.net>
* [PATCH] Don't fetch objects that exist in the local repositoryNick Hengeveld2005-10-10
| | | | | | | | | | | Be sure not to fetch objects that already exist in the local repository. The main process loop no longer performs this check, http-fetch now checks prior to starting a new request queue entry and when fetch_object() is called, and local-fetch now checks when fetch_object() is called. As discussed in this thread: http://marc.theaimsgroup.com/?t=112854890500001 Signed-off-by: Nick Hengeveld <nickh@reactrix.com>
* Set the parallel HTTP request limit via an environment variableNick Hengeveld2005-10-10
| | | | | | | | | Use an environment variable rather than a command-line argument to set the parallel HTTP request limit. This allows the setting to work whether git-http-fetch is run directly or via git-fetch. Signed-off-by: Nick Hengeveld <nickh@reactrix.com> Signed-off-by: Junio C Hamano <junkio@cox.net>
* Only compile parallel HTTP support with CURL >= 7.9.8Nick Hengeveld2005-10-10
| | | | | | | Only compile parallel HTTP support with CURL >= 7.9.8 Signed-off-by: Nick Hengeveld <nickh@reactrix.com> Signed-off-by: Junio C Hamano <junkio@cox.net>
* Add support for parallel HTTP transfersNick Hengeveld2005-10-10
| | | | | | | | | | | | | Add support for parallel HTTP transfers. Prefetch populates a queue of objects to transfer and starts feeding requests to an active request queue for processing; fetch_object keeps the active queue moving while the specified object is being transferred. The size of the active queue can be restricted using -r and defaults to 5 concurrent transfers. Requests for objects that are not prefetched are also processed via the active queue. Signed-off-by: Nick Hengeveld <nickh@reactrix.com> Signed-off-by: Junio C Hamano <junkio@cox.net>
* [PATCH] HTTP partial transfer support fix.Nick Hengeveld2005-10-01
| | | | | | | | Don't unlink the temp file when an object transfer fails, so next attempt will pick up where the failed transfer left off Signed-off-by: Nick Hengeveld <nickh@reactrix.com> Signed-off-by: Junio C Hamano <junkio@cox.net>
* Update partial HTTP transfers.Junio C Hamano2005-10-01
| | | | | | | | | | | | | | | | | | Add the sanity checks discussed on the list with Nick Hengeveld in <20050927000931.GA15615@reactrix.com>. * unlink of previous and rename from temp to previous can fail for reasons other than benign ones (missing previous and missing temp). Report these failures when we encounter them, to make diagnosing problems easier. * when rewinding the partially written result, make sure to truncate the file. Also verify the pack after downloading by calling verify_packfile(). Signed-off-by: Junio C Hamano <junkio@cox.net>
* [PATCH] HTTP partial transfer support for object, pack, and index transfersNick Hengeveld2005-10-01
| | | | | | | | | | HTTP partial transfer support for object, pack, and index transfers [jc: this should not be placed in "master" -- it does not have any fixes requested on the list.] Signed-off-by: Nick Hengeveld <nickh@reactrix.com> Signed-off-by: Junio C Hamano <junkio@cox.net>
* [PATCH] Old curl does not know about CURLOPT_SSLKEYJohannes Schindelin2005-09-29
| | | | | | | ... so try to set it only in later versions. Signed-off-by: Johannes Schindelin <Johannes.Schindelin@gmx.de> Signed-off-by: Junio C Hamano <junkio@cox.net>
* [PATCH] Make some needlessly global stuff staticPeter Hagervall2005-09-28
| | | | | | | Insert 'static' where appropriate. Signed-off-by: Peter Hagervall <hager@cs.umu.se> Signed-off-by: Junio C Hamano <junkio@cox.net>
* [PATCH] Support for more CURL SSL settings via environment variablesNick Hengeveld2005-09-28
| | | | | | | | | Added support for additional CURL SSL settings via environment variables. Client certificate/key files can be specified as well as alternate CA information. Signed-off-by: Nick Hengeveld <nickh@reactrix.com> Signed-off-by: Junio C Hamano <junkio@cox.net>
* [PATCH] Return CURL error message when object transfer failsNick Hengeveld2005-09-27
| | | | | | | | | | Return CURL error message when object transfer fails [jc: added similar curl_errorstr errors to places where we use curl_easy_perform() to run fetch that _must_ succeed.] Signed-off-by: Nick Hengeveld <nickh@reactrix.com> Signed-off-by: Junio C Hamano <junkio@cox.net>
* [PATCH] Implement --recover for git-*-fetchDaniel Barkalow2005-09-27
| | | | | | | | | With the --recover option, we verify that we have absolutely everything reachable from the target, not assuming that things reachable from refs will be complete. Signed-off-by: Daniel Barkalow <barkalow@iabervon.org> Signed-off-by: Junio C Hamano <junkio@cox.net>
generated by cgit v1.2.1 (git 2.18.0) at 2024-11-12 11:13:05 +0000