aboutsummaryrefslogtreecommitdiff
path: root/http-walker.c
Commit message (Collapse)AuthorAge
* Merge branch 'sp/maint-dumb-http-pack-reidx'Junio C Hamano2010-05-21
|\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * sp/maint-dumb-http-pack-reidx: http.c::new_http_pack_request: do away with the temp variable filename http-fetch: Use temporary files for pack-*.idx until verified http-fetch: Use index-pack rather than verify-pack to check packs Allow parse_pack_index on temporary files Extract verify_pack_index for reuse from verify_pack Introduce close_pack_index to permit replacement http.c: Remove unnecessary strdup of sha1_to_hex result http.c: Don't store destination name in request structures http.c: Drop useless != NULL test in finish_http_pack_request http.c: Tiny refactoring of finish_http_pack_request t5550-http-fetch: Use subshell for repository operations http.c: Remove bad free of static block
| * http.c: Don't store destination name in request structuresShawn O. Pearce2010-04-17
| | | | | | | | | | | | | | | | | | | | | | | | | | | | The destination name within the object store is easily computed on demand, reusing a static buffer held by sha1_file.c. We don't need to copy the entire path into the request structure for safe keeping, when it can be easily reformatted after the download has been completed. This reduces the size of the per-request structure, and removes yet another PATH_MAX based limit. Signed-off-by: Shawn O. Pearce <spearce@spearce.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | http: init and cleanup separately from http-walkerTay Ray Chuan2010-03-02
| | | | | | | | | | | | | | | | | | | | | | | | Previously, all our http operations were done with http-walker. With the new remote-curl helper, we find ourselves using http methods outside of http-walker - for example, fetching info/refs. Accomodate this by separating http_init() and http_cleanup() invocations from http-walker. Signed-off-by: Tay Ray Chuan <rctay89@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | http-walker: cleanup more thoroughlyTay Ray Chuan2010-03-02
|/ | | | | Signed-off-by: Tay Ray Chuan <rctay89@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* http*: add helper methods for fetching objects (loose)Tay Ray Chuan2009-06-06
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The code handling the fetching of loose objects in http-push.c and http-walker.c have been refactored into new methods and a new struct (object_http_request) in http.c. They are not meant to be invoked elsewhere. The new methods in http.c are - new_http_object_request - process_http_object_request - finish_http_object_request - abort_http_object_request - release_http_object_request and the new struct is http_object_request. RANGER_HEADER_SIZE and no_pragma_header is no longer made available outside of http.c, since after the above changes, there are no other instances of usage outside of http.c. Remove members of the transfer_request struct in http-push.c and http-walker.c, including filename, real_sha1 and zret, as they are used no longer used. Move the methods append_remote_object_url() and get_remote_object_url() from http-push.c to http.c. Additionally, get_remote_object_url() is no longer defined only when USE_CURL_MULTI is defined, since non-USE_CURL_MULTI code in http.c uses it (namely, in new_http_object_request()). Refactor code from http-push.c::start_fetch_loose() and http-walker.c::start_object_fetch_request() that deals with the details of coming up with the filename to store the retrieved object, resuming a previously aborted request, and making a new curl request, into a new function, new_http_object_request(). Refactor code from http-walker.c::process_object_request() into the function, process_http_object_request(). Refactor code from http-push.c::finish_request() and http-walker.c::finish_object_request() into a new function, finish_http_object_request(). It returns the result of the move_temp_to_file() invocation. Add a function, release_http_object_request(), which cleans up object request data. http-push.c and http-walker.c invoke this function separately; http-push.c::release_request() and http-walker.c::release_object_request() do not invoke this function. Add a function, abort_http_object_request(), which unlink()s the object file and invokes release_http_object_request(). Update http-walker.c::abort_object_request() to use this. Signed-off-by: Tay Ray Chuan <rctay89@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* http*: add helper methods for fetching packsTay Ray Chuan2009-06-06
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The code handling the fetching of packs in http-push.c and http-walker.c have been refactored into new methods and a new struct (http_pack_request) in http.c. They are not meant to be invoked elsewhere. The new methods in http.c are - new_http_pack_request - finish_http_pack_request - release_http_pack_request and the new struct is http_pack_request. Add a function, new_http_pack_request(), that deals with the details of coming up with the filename to store the retrieved packfile, resuming a previously aborted request, and making a new curl request. Update http-push.c::start_fetch_packed() and http-walker.c::fetch_pack() to use this. Add a function, finish_http_pack_request(), that deals with renaming the pack, advancing the pack list, and installing the pack. Update http-push.c::finish_request() and http-walker.c::fetch_pack to use this. Update release_request() in http-push.c and http-walker.c to invoke release_http_pack_request() to clean up pack request helper data. The local_stream member of the transfer_request struct in http-push.c has been removed, as the packfile pointer will be managed in the struct http_pack_request. Signed-off-by: Tay Ray Chuan <rctay89@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* http*: add http_get_info_packsTay Ray Chuan2009-06-06
| | | | | | | | | | | | | | | | | | | | | http-push.c and http-walker.c no longer have to use fetch_index or setup_index; they simply need to use http_get_info_packs, a new http method, in their fetch_indices implementations. Move fetch_index() and rename to fetch_pack_index() in http.c; this method is not meant to be used outside of http.c. It invokes end_url_with_slash with base_url; apart from that change, the code is identical. Move setup_index() and rename to fetch_and_setup_pack_index() in http.c; this method is not meant to be used outside of http.c. Do not immediately set ret to 0 in http-walker.c::fetch_indices(); instead do it in the HTTP_MISSING_TARGET case, to make it clear that the HTTP_OK and HTTP_MISSING_TARGET cases both return 0. Signed-off-by: Tay Ray Chuan <rctay89@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* http*: move common variables and macros to http.[ch]Tay Ray Chuan2009-06-06
| | | | | | | | | | | | | | | | Move RANGE_HEADER_SIZE to http.h. Create no_pragma_header, the curl header list containing the header "Pragma:" in http.[ch]. It is allocated in http_init, and freed in http_cleanup. This replaces the no_pragma_header in http-push.c, and the no_pragma_header member in walker_data in http-walker.c. Create http_is_verbose. It is to be used by methods in http.c, and is modified at the entry points of http.c's users, namely http-push.c (when parsing options) and http-walker.c (in get_http_walker). Signed-off-by: Tay Ray Chuan <rctay89@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* http*: copy string returned by sha1_to_hexTay Ray Chuan2009-06-06
| | | | | | | | | | | | In the fetch_index implementations in http-push.c and http-walker.c, the string returned by sha1_to_hex is assumed to stay immutable. This patch ensures that hex stays immutable by copying the string returned by sha1_to_hex (via xstrdup) and frees it subsequently. It also refactors free()'s and fclose()'s with labels. Signed-off-by: Tay Ray Chuan <rctay89@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* http-walker: verify remote packsTay Ray Chuan2009-06-06
| | | | | | | | | | | | | In c17fb6e ("Verify remote packs, speed up pending request queue"), changes were made to index fetching in http-push.c, particularly the methods fetch_index and setup_index. Since http-walker.c has similar code for index fetching, these improvements should apply to http-walker.c's fetch_index and setup_index. Invocations of free() of string memory are reproduced as well. Signed-off-by: Tay Ray Chuan <rctay89@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* http-push, http-walker: style fixesTay Ray Chuan2009-06-06
| | | | | | | | | | | | | | | | | | | | | - Use tabs to indent, instead of spaces. - Do not use curly-braces around a single statement body in if/while statement; - Do not start multi-line comment with description on the first line after "/*", i.e. /* * We prefer this over... */ /* comments like * this (notice the first line) */ Signed-off-by: Tay Ray Chuan <rctay89@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* Merge branch 'rc/maint-http-local-slot-fix' into rc/http-pushJunio C Hamano2009-06-06
|\ | | | | | | | | * rc/maint-http-local-slot-fix: http*: cleanup slot->local after fclose
| * http*: cleanup slot->local after fcloseTay Ray Chuan2009-06-06
| | | | | | | | | | | | | | | | | | | | | | | | | | Set slot->local to NULL after doing a fclose() on the file it points to. This prevents the passing of a FILE* pointer to a fclose()'d file to ftell() in http.c::run_active_slot(). This issue was raised by Clemens Buchacher on 30th May 2009: http://www.spinics.net/lists/git/msg104623.html Signed-off-by: Tay Ray Chuan <rctay89@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | replace direct calls to unlink(2) with unlink_or_warnAlex Riesen2009-04-29
|/ | | | | | | | | | | | | | | | | This helps to notice when something's going wrong, especially on systems which lock open files. I used the following criteria when selecting the code for replacement: - it was already printing a warning for the unlink failures - it is in a function which already printing something or is called from such a function - it is in a static function, returning void and the function is only called from a builtin main function (cmd_) - it is in a function which handles emergency exit (signal handlers) - it is in a function which is obvously cleaning up the lockfiles Signed-off-by: Alex Riesen <raa.lkml@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* Move chmod(foo, 0444) into move_temp_to_file()Johan Herland2009-03-27
| | | | | | | | | | | | When writing out a loose object or a pack (index), move_temp_to_file() is called to finalize the resulting file. These files (loose files and packs) should all have permission mode 0444 (modulo adjust_shared_perm()). Therefore, instead of doing chmod(foo, 0444) explicitly from each callsite (or even forgetting to chmod() at all), do the chmod() call from within move_temp_to_file(). Signed-off-by: Johan Herland <johan@herland.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* Merge branch 'lt/maint-wrap-zlib'Junio C Hamano2009-01-21
|\ | | | | | | | | | | | | | | | | | | * lt/maint-wrap-zlib: Wrap inflate and other zlib routines for better error reporting Conflicts: http-push.c http-walker.c sha1_file.c
| * Wrap inflate and other zlib routines for better error reportingLinus Torvalds2009-01-11
| | | | | | | | | | | | | | | | | | | | | | | | | | R. Tyler Ballance reported a mysterious transient repository corruption; after much digging, it turns out that we were not catching and reporting memory allocation errors from some calls we make to zlib. This one _just_ wraps things; it doesn't do the "retry on low memory error" part, at least not yet. It is an independent issue from the reporting. Some of the errors are expected and passed back to the caller, but we die when zlib reports it failed to allocate memory for now. Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | fix openssl headers conflicting with custom SHA1 implementationsNicolas Pitre2008-10-02
|/ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | On ARM I have the following compilation errors: CC fast-import.o In file included from cache.h:8, from builtin.h:6, from fast-import.c:142: arm/sha1.h:14: error: conflicting types for 'SHA_CTX' /usr/include/openssl/sha.h:105: error: previous declaration of 'SHA_CTX' was here arm/sha1.h:16: error: conflicting types for 'SHA1_Init' /usr/include/openssl/sha.h:115: error: previous declaration of 'SHA1_Init' was here arm/sha1.h:17: error: conflicting types for 'SHA1_Update' /usr/include/openssl/sha.h:116: error: previous declaration of 'SHA1_Update' was here arm/sha1.h:18: error: conflicting types for 'SHA1_Final' /usr/include/openssl/sha.h:117: error: previous declaration of 'SHA1_Final' was here make: *** [fast-import.o] Error 1 This is because openssl header files are always included in git-compat-util.h since commit 684ec6c63c whenever NO_OPENSSL is not set, which somehow brings in <openssl/sha1.h> clashing with the custom ARM version. Compilation of git is probably broken on PPC too for the same reason. Turns out that the only file requiring openssl/ssl.h and openssl/err.h is imap-send.c. But only moving those problematic includes there doesn't solve the issue as it also includes cache.h which brings in the conflicting local SHA1 header file. As suggested by Jeff King, the best solution is to rename our references to SHA1 functions and structure to something git specific, and define those according to the implementation used. Signed-off-by: Nicolas Pitre <nico@cam.org> Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
* Merge branch 'maint'Junio C Hamano2008-07-19
|\ | | | | | | | | | | | | | | | | * maint: GIT 1.5.6.4 builtin-rm: fix index lock file path http-fetch: do not SEGV after fetching a bad pack idx file rev-list: honor --quiet option api-run-command.txt: typofix
| * http-fetch: do not SEGV after fetching a bad pack idx fileJunio C Hamano2008-07-18
| | | | | | | | Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | move show_pack_info() where it belongsNicolas Pitre2008-06-24
|/ | | | | | | | | | This is called when verify_pack() has its verbose argument set, and verbose in this context makes sense only for the actual 'git verify-pack' command. Therefore let's move show_pack_info() to builtin-verify-pack.c instead and remove useless verbose argument from verify_pack(). Signed-off-by: Nicolas Pitre <nico@cam.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* Make walker.fetch_ref() take a struct ref.Daniel Barkalow2008-04-26
| | | | | | | | | | | | | | | | This simplifies a few things, makes a few things slightly more complicated, but, more importantly, allows that, when struct ref can represent a symref, http_fetch_ref() can return one. Incidentally makes the string that http_fetch_ref() gets include "refs/" (if appropriate), because that's how the name field of struct ref works. As far as I can tell, the usage in walker:interpret_target() wouldn't have worked previously, if it ever would have been used, which it wouldn't (since the fetch process uses the hash instead of the name of the ref there). Signed-off-by: Daniel Barkalow <barkalow@iabervon.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* Set proxy override with http_init()Mike Hommey2008-02-27
| | | | | | | | | | | | | | | | | | | | In transport.c, proxy setting (the one from the remote conf) was set through curl_easy_setopt() call, while http.c already does the same with the http.proxy setting. We now just use this infrastructure instead, and make http_init() now take the struct remote as argument so that it can take the http_proxy setting from there, and any other property that would be added later. At the same time, we make get_http_walker() take a struct remote argument too, and pass it to http_init(), which makes remote defined proxy be used for more than get_refs_via_curl(). We leave out http-fetch and http-push, which don't use remotes for the moment, purposefully. Signed-off-by: Mike Hommey <mh@glandium.org> Acked-by: Daniel Barkalow <barkalow@iabervon.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* Move fetch_ref from http-push.c and http-walker.c to http.cMike Hommey2007-12-14
| | | | | | | | Make the necessary changes to be ok with their difference, and rename the function http_fetch_ref. Signed-off-by: Mike Hommey <mh@glandium.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* Fix various memory leaks in http-push.c and http-walker.cMike Hommey2007-12-14
| | | | | Signed-off-by: Mike Hommey <mh@glandium.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* Use strbuf in http codeMike Hommey2007-12-14
| | | | | | | Also, replace whitespaces with tabs in some places Signed-off-by: Mike Hommey <mh@glandium.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* Avoid redundant declaration of missing_target()Mike Hommey2007-12-14
| | | | | Signed-off-by: Mike Hommey <mh@glandium.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* Fix random sha1 in error message in http-fetch and http-pushMike Hommey2007-12-14
| | | | | | | | | | When a downloaded ref doesn't contain a sha1, the error message displays a random sha1 because of uninitialized memory. This happens when cloning a repository that is already a clone of another one, in which case refs/remotes/origin/HEAD is a symref. Signed-off-by: Mike Hommey <mh@glandium.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* Print the real filename that we failed to open.André Goddard Rosa2007-11-25
| | | | | | | | | When we fail to open a temporary file to be renamed to something else, we reported the final filename, not the temporary file we failed to open. Signed-off-by: André Goddard Rosa <andre.goddard@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* Modularize commit-walkerDaniel Barkalow2007-09-19
This turns the extern functions to be provided by the backend into a struct of pointers, renames the functions to be more namespace-friendly, and updates http-fetch to this interface. It removes the unused include from http-push.c. It makes git-http-fetch a builtin (with the implementation a separate file, accessible directly). Signed-off-by: Daniel Barkalow <barkalow@iabervon.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>