aboutsummaryrefslogtreecommitdiff
path: root/sha1_file.c
Commit message (Collapse)AuthorAge
* Merge branch 'po/read-graft-line'Junio C Hamano2017-09-06
|\ | | | | | | | | | | | | | | | | | | | | | | Conversion from uchar[20] to struct object_id continues; this is to ensure that we do not assume sizeof(struct object_id) is the same as the length of SHA-1 hash (or length of longest hash we support). * po/read-graft-line: commit: rewrite read_graft_line commit: allocate array using object_id size commit: replace the raw buffer with strbuf in read_graft_line sha1_file: fix definition of null_sha1
| * sha1_file: fix definition of null_sha1Patryk Obara2017-08-17
| | | | | | | | | | | | | | | | | | | | | | The array is declared in cache.h as: extern const unsigned char null_sha1[GIT_MAX_RAWSZ]; Definition in sha1_file.c must match. Signed-off-by: Patryk Obara <patryk.obara@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | Merge branch 'jt/packmigrate'Junio C Hamano2017-08-26
|\ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Code movement to make it easier to hack later. * jt/packmigrate: (23 commits) pack: move for_each_packed_object() pack: move has_pack_index() pack: move has_sha1_pack() pack: move find_pack_entry() and make it global pack: move find_sha1_pack() pack: move find_pack_entry_one(), is_pack_valid() pack: move check_pack_index_ptr(), nth_packed_object_offset() pack: move nth_packed_object_{sha1,oid} pack: move clear_delta_base_cache(), packed_object_info(), unpack_entry() pack: move unpack_object_header() pack: move get_size_from_delta() pack: move unpack_object_header_buffer() pack: move {,re}prepare_packed_git and approximate_object_count pack: move install_packed_git() pack: move add_packed_git() pack: move unuse_pack() pack: move use_pack() pack: move pack-closing functions pack: move release_pack_memory() pack: move open_pack_index(), parse_pack_index() ...
| * | pack: move for_each_packed_object()Jonathan Tan2017-08-23
| | | | | | | | | | | | | | | Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | pack: move has_pack_index()Jonathan Tan2017-08-23
| | | | | | | | | | | | | | | Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | pack: move has_sha1_pack()Jonathan Tan2017-08-23
| | | | | | | | | | | | | | | Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | pack: move find_pack_entry() and make it globalJonathan Tan2017-08-23
| | | | | | | | | | | | | | | | | | | | | | | | This function needs to be global as it is used by sha1_file.c and will be used by packfile.c. Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | pack: move find_sha1_pack()Jonathan Tan2017-08-23
| | | | | | | | | | | | | | | Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | pack: move find_pack_entry_one(), is_pack_valid()Jonathan Tan2017-08-23
| | | | | | | | | | | | | | | Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | pack: move check_pack_index_ptr(), nth_packed_object_offset()Jonathan Tan2017-08-23
| | | | | | | | | | | | | | | Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | pack: move nth_packed_object_{sha1,oid}Jonathan Tan2017-08-23
| | | | | | | | | | | | | | | Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | pack: move clear_delta_base_cache(), packed_object_info(), unpack_entry()Jonathan Tan2017-08-23
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Both sha1_file.c and packfile.c now need read_object(), so a copy of read_object() was created in packfile.c. This patch makes both mark_bad_packed_object() and has_packed_and_bad() global. Unlike most of the other patches in this series, these 2 functions need to remain global. Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | pack: move unpack_object_header()Jonathan Tan2017-08-23
| | | | | | | | | | | | | | | Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | pack: move get_size_from_delta()Jonathan Tan2017-08-23
| | | | | | | | | | | | | | | Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | pack: move unpack_object_header_buffer()Jonathan Tan2017-08-23
| | | | | | | | | | | | | | | Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | pack: move {,re}prepare_packed_git and approximate_object_countJonathan Tan2017-08-23
| | | | | | | | | | | | | | | Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | pack: move install_packed_git()Jonathan Tan2017-08-23
| | | | | | | | | | | | | | | Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | pack: move add_packed_git()Jonathan Tan2017-08-23
| | | | | | | | | | | | | | | Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | pack: move unuse_pack()Jonathan Tan2017-08-23
| | | | | | | | | | | | | | | Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | pack: move use_pack()Jonathan Tan2017-08-23
| | | | | | | | | | | | | | | | | | | | | | | | The function open_packed_git() needs to be temporarily made global. Its scope will be restored to static in a subsequent commit. Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | pack: move pack-closing functionsJonathan Tan2017-08-23
| | | | | | | | | | | | | | | | | | | | | | | | The function close_pack_fd() needs to be temporarily made global. Its scope will be restored to static in a subsequent commit. Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | pack: move release_pack_memory()Jonathan Tan2017-08-23
| | | | | | | | | | | | | | | | | | | | | | | | The function unuse_one_window() needs to be temporarily made global. Its scope will be restored to static in a subsequent commit. Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | pack: move open_pack_index(), parse_pack_index()Jonathan Tan2017-08-23
| | | | | | | | | | | | | | | | | | | | | | | | alloc_packed_git() in packfile.c is duplicated from sha1_file.c. In a subsequent commit, alloc_packed_git() will be removed from sha1_file.c. Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | pack: move pack_report()Jonathan Tan2017-08-23
| | | | | | | | | | | | | | | Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | pack: move static state variablesJonathan Tan2017-08-23
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | sha1_file.c declares some static variables that store packfile-related state. Move them to packfile.c. They are temporarily made global, but subsequent commits will restore their scope back to static. Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | pack: move pack name-related functionsJonathan Tan2017-08-23
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently, sha1_file.c and cache.h contain many functions, both related to and unrelated to packfiles. This makes both files very large and causes an unclear separation of concerns. Create a new file, packfile.c, to hold all packfile-related functions currently in sha1_file.c. It has a corresponding header packfile.h. In this commit, the pack name-related functions are moved. Subsequent commits will move the other functions. Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | | Merge branch 'po/object-id'Junio C Hamano2017-08-26
|\ \ \ | |/ / |/| | | | | | | | | | | | | | | | | | | | | | | * po/object-id: sha1_file: convert index_stream to struct object_id sha1_file: convert hash_sha1_file_literally to struct object_id sha1_file: convert index_fd to struct object_id sha1_file: convert index_path to struct object_id read-cache: convert to struct object_id builtin/hash-object: convert to struct object_id
| * | sha1_file: convert index_stream to struct object_idPatryk Obara2017-08-20
| | | | | | | | | | | | | | | Signed-off-by: Patryk Obara <patryk.obara@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | sha1_file: convert hash_sha1_file_literally to struct object_idPatryk Obara2017-08-20
| | | | | | | | | | | | | | | | | | | | | Convert all remaining callers as well. Signed-off-by: Patryk Obara <patryk.obara@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | sha1_file: convert index_fd to struct object_idPatryk Obara2017-08-20
| | | | | | | | | | | | | | | | | | | | | Convert all remaining callers as well. Signed-off-by: Patryk Obara <patryk.obara@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | sha1_file: convert index_path to struct object_idPatryk Obara2017-08-20
| |/ | | | | | | | | | | | | Convert all remaining callers as well. Signed-off-by: Patryk Obara <patryk.obara@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | Merge branch 'sb/sha1-file-cleanup'Junio C Hamano2017-08-23
|\ \ | | | | | | | | | | | | | | | | | | Code clean-up. * sb/sha1-file-cleanup: sha1_file: make read_info_alternates static
| * | sha1_file: make read_info_alternates staticStefan Beller2017-08-15
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | read_info_alternates is not used from outside, so let's make it static. We have to declare the function before link_alt_odb_entry instead of moving the code around, link_alt_odb_entry calls read_info_alternates, which in turn calls link_alt_odb_entry. Signed-off-by: Stefan Beller <sbeller@google.com> Reviewed-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | | Merge branch 'jt/sha1-file-cleanup'Junio C Hamano2017-08-23
|\ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | Preparatory code clean-up. * jt/sha1-file-cleanup: sha1_file: remove read_packed_sha1() sha1_file: set whence in storage-specific info fn
| * | | sha1_file: remove read_packed_sha1()Jonathan Tan2017-08-11
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Use read_object() in its place instead. This avoids duplication of code. This makes force_object_loose() slightly slower (because of a redundant check of loose object storage), but only in the error case. Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | | sha1_file: set whence in storage-specific info fnJonathan Tan2017-08-11
| | |/ | |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Move the setting of oi->whence to sha1_loose_object_info() and packed_object_info(). This allows sha1_object_info_extended() to not need to know about the delta base cache. This will be useful during a future refactoring in which packfile-related functions, including the handling of the delta base cache, will be moved to a separate file. Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | | Merge branch 'rs/unpack-entry-leakfix'Junio C Hamano2017-08-22
|\ \ \ | | | | | | | | | | | | | | | | | | | | | | | | Memory leak in an error codepath has been plugged. * rs/unpack-entry-leakfix: sha1_file: release delta_stack on error in unpack_entry()
| * | | sha1_file: release delta_stack on error in unpack_entry()René Scharfe2017-08-10
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When unpack_entry() encounters a broken packed object, it returns early. It adjusts the reference count of the pack window, but leaks the buffer for a big delta stack in case the small automatic one was not enough. Jump to the cleanup code at end instead, which takes care of that. Signed-off-by: Rene Scharfe <l.s.r@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | | | Merge branch 'rs/find-pack-entry-bisection'Junio C Hamano2017-08-22
|\ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Code clean-up. * rs/find-pack-entry-bisection: sha1_file: avoid comparison if no packed hash matches the first byte
| * | | | sha1_file: avoid comparison if no packed hash matches the first byteRené Scharfe2017-08-09
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | find_pack_entry_one() uses the fan-out table of pack indexes to find out which entries match the first byte of the searched hash and does a binary search on this subset of the main index table. If there are no matching entries then lo and hi will have the same value. The binary search still starts and compares the hash of the following entry (which has a non-matching first byte, so won't cause any trouble), or whatever comes after the sorted list of entries. The probability of that stray comparison matching by mistake is low, but let's not take any chances and check when entering the binary search loop if we're actually done already. Signed-off-by: Rene Scharfe <l.s.r@web.de> Reviewed-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | | | | Merge branch 'jk/drop-sha1-entry-pos'Junio C Hamano2017-08-22
|\ \ \ \ \ | |_|_|/ / |/| | | | | | | | | | | | | | | | | | | | | | | | Code clean-up. * jk/drop-sha1-entry-pos: sha1_file: drop experimental GIT_USE_LOOKUP search
| * | | | sha1_file: drop experimental GIT_USE_LOOKUP searchJeff King2017-08-09
| | |/ / | |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Long ago in 628522ec14 (sha1-lookup: more memory efficient search in sorted list of SHA-1, 2007-12-29) we added sha1_entry_pos(), a binary search that uses the uniform distribution of sha1s to scale the selection of mid-points. As this was a performance experiment, we tied it to the GIT_USE_LOOKUP environment variable and never enabled it by default. This code was successful in reducing the number of steps in each search. But the overhead of the scaling ends up making it slower when the cache is warm. Here are best-of-five timings for running rev-list on linux.git, which will have to look up every object: $ time git rev-list --objects --all >/dev/null real 0m35.357s user 0m35.016s sys 0m0.340s $ time GIT_USE_LOOKUP=1 git rev-list --objects --all >/dev/null real 0m37.364s user 0m37.045s sys 0m0.316s The USE_LOOKUP version might have more benefit on a cold cache, as the time to fault in each page would dominate. But that would be for a single lookup. In practice, most operations tend to look up many objects, and the whole pack .idx will end up warm. It's possible that the code could be better optimized to compete with a naive binary search for the warm-cache case, and we could have the best of both worlds. But over the years nobody has done so, and this is largely dead code that is rarely run outside of the test suite. Let's drop it in the name of simplicity. This lets us remove sha1_entry_pos() entirely, as the .idx lookup code was the only caller. Note that sha1-lookup.c still contains sha1_pos(), which differs from sha1_entry_pos() in two ways: - it has a different interface; it uses a function pointer to access sha1 entries rather than a size/offset pair describing the table's memory layout - it only scales the initial selection of "mi", rather than each iteration of the search We can't get rid of this function, as it's called from several places. It may be that we could replace it with a simple binary search, but that's out of scope for this patch (and would need benchmarking). Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | | | Merge branch 'ew/fd-cloexec-fix'Junio C Hamano2017-07-20
|\ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Portability/fallback fix. * ew/fd-cloexec-fix: set FD_CLOEXEC properly when O_CLOEXEC is not supported
| * | | | set FD_CLOEXEC properly when O_CLOEXEC is not supportedEric Wong2017-07-17
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | FD_CLOEXEC only applies to the file descriptor, so it needs to be manipuluated via F_GETFD/F_SETFD. F_GETFL/F_SETFL are for file description flags. Verified via strace with o_cloexec set to zero. Signed-off-by: Eric Wong <e@80x24.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | | | | Merge branch 'sb/hashmap-customize-comparison'Junio C Hamano2017-07-13
|\ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Update the hashmap API so that data to customize the behaviour of the comparison function can be specified at the time a hashmap is initialized. * sb/hashmap-customize-comparison: hashmap: migrate documentation from Documentation/technical into header patch-ids.c: use hashmap correctly hashmap.h: compare function has access to a data field
| * | | | | hashmap.h: compare function has access to a data fieldStefan Beller2017-06-30
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When using the hashmap a common need is to have access to caller provided data in the compare function. A couple of times we abuse the keydata field to pass in the data needed. This happens for example in patch-ids.c. This patch changes the function signature of the compare function to have one more void pointer available. The pointer given for each invocation of the compare function must be defined in the init function of the hashmap and is just passed through. Documentation of this new feature is deferred to a later patch. This is a rather mechanical conversion, just adding the new pass-through parameter. However while at it improve the naming of the fields of all compare functions used by hashmaps by ensuring unused parameters are prefixed with 'unused_' and naming the parameters what they are (instead of 'unused' make it 'unused_keydata'). Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | | | | | Merge branch 'jt/unify-object-info'Junio C Hamano2017-07-05
|\ \ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Code clean-ups. * jt/unify-object-info: sha1_file: refactor has_sha1_file_with_flags sha1_file: do not access pack if unneeded sha1_file: teach sha1_object_info_extended more flags sha1_file: refactor read_object sha1_file: move delta base cache code up sha1_file: rename LOOKUP_REPLACE_OBJECT sha1_file: rename LOOKUP_UNKNOWN_OBJECT sha1_file: teach packed_object_info about typename
| * | | | | | sha1_file: refactor has_sha1_file_with_flagsJonathan Tan2017-06-26
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | has_sha1_file_with_flags() implements many mechanisms in common with sha1_object_info_extended(). Make has_sha1_file_with_flags() a convenience function for sha1_object_info_extended() instead. Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | | | | | sha1_file: do not access pack if unneededJonathan Tan2017-06-26
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently, regardless of the contents of the "struct object_info" passed to sha1_object_info_extended(), that function always accesses the packfile whenever it returns information about a packed object, since it needs to populate "u.packed". Add the ability to pass NULL, and use NULL-ness of the argument to activate an optimization in which sha1_object_info_extended() does not needlessly access the packfile. A subsequent patch will make use of this optimization. A similar optimization is not made for the cached and loose cases as it would not cause a significant performance improvement. Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | | | | | sha1_file: teach sha1_object_info_extended more flagsJonathan Tan2017-06-26
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Improve sha1_object_info_extended() by supporting additional flags. This allows has_sha1_file_with_flags() to be modified to use sha1_object_info_extended() in a subsequent patch. Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
generated by cgit v1.2.1 (git 2.18.0) at 2024-06-29 02:32:13 +0000