aboutsummaryrefslogtreecommitdiff
path: root/strbuf.c
Commit message (Collapse)AuthorAge
* strbuf: add strbuf_add_real_path()René Scharfe2017-02-27
| | | | | | | | | | | | | | | Add a function for appending the canonized absolute pathname of a given path to a strbuf. It keeps the existing contents intact, as expected of a function of the strbuf_add() family, while avoiding copying the result if the given strbuf is empty. It's more consistent with the rest of the strbuf API than strbuf_realpath(), which it's wrapping. Also add a semantic patch demonstrating its intended usage and apply it to the current tree. Using strbuf_add_real_path() instead of calling strbuf_addstr() and real_path() avoids an extra copy to a static buffer. Signed-off-by: Rene Scharfe <l.s.r@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* link_alt_odb_entry: handle normalize_path errorsJeff King2016-10-10
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When we add a new alternate to the list, we try to normalize out any redundant "..", etc. However, we do not look at the return value of normalize_path_copy(), and will happily continue with a path that could not be normalized. Worse, the normalizing process is done in-place, so we are left with whatever half-finished working state the normalizing function was in. Fortunately, this cannot cause us to read past the end of our buffer, as that working state will always leave the NUL from the original path in place. And we do tend to notice problems when we check is_directory() on the path. But you can see the nonsense that we feed to is_directory with an entry like: this/../../is/../../way/../../too/../../deep/../../to/../../resolve in your objects/info/alternates, which yields: error: object directory /to/e/deep/too/way//ects/this/../../is/../../way/../../too/../../deep/../../to/../../resolve does not exist; check .git/objects/info/alternates. We can easily fix this just by checking the return value. But that makes it hard to generate a good error message, since we're normalizing in-place and our input value has been overwritten by cruft. Instead, let's provide a strbuf helper that does an in-place normalize, but restores the original contents on error. This uses a second buffer under the hood, which is slightly less efficient, but this is not a performance-critical code path. The strbuf helper can also properly set the "len" parameter of the strbuf before returning. Just doing: normalize_path_copy(buf.buf, buf.buf); will shorten the string, but leave buf.len at the original length. That may be confusing to later code which uses the strbuf. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* Merge branch 'rs/strbuf-remove-fix'Junio C Hamano2016-09-21
|\ | | | | | | | | | | | | Code cleanup. * rs/strbuf-remove-fix: strbuf: use valid pointer in strbuf_remove()
| * strbuf: use valid pointer in strbuf_remove()René Scharfe2016-09-13
| | | | | | | | | | | | | | | | | | The fourth argument of strbuf_splice() is passed to memcpy(3), which is not supposed to handle NULL pointers. Let's be extra careful and use a valid empty string instead. It even shortens the source code. :) Signed-off-by: Rene Scharfe <l.s.r@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * Merge branch 'jk/getwholeline-getdelim-empty' into maintJunio C Hamano2016-04-14
| |\ | | | | | | | | | | | | | | | | | | | | | strbuf_getwholeline() did not NUL-terminate the buffer on certain corner cases in its error codepath. * jk/getwholeline-getdelim-empty: strbuf_getwholeline: NUL-terminate getdelim buffer on error
* | \ Merge branch 'rs/use-strbuf-addbuf'Junio C Hamano2016-07-25
|\ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | Code cleanup. * rs/use-strbuf-addbuf: strbuf: avoid calling strbuf_grow() twice in strbuf_addbuf() use strbuf_addbuf() for appending a strbuf to another
| * | | strbuf: avoid calling strbuf_grow() twice in strbuf_addbuf()René Scharfe2016-07-22
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Implement strbuf_addbuf() as a normal function in order to avoid calling strbuf_grow() twice, with the second callinside strbud_add() being a no-op. This is slightly faster and also reduces the text size a bit. Signed-off-by: Rene Scharfe <l.s.r@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | | | Merge branch 'sb/submodule-parallel-update'Junio C Hamano2016-04-06
|\ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | A major part of "git submodule update" has been ported to C to take advantage of the recently added framework to run download tasks in parallel. * sb/submodule-parallel-update: clone: allow an explicit argument for parallel submodule clones submodule update: expose parallelism to the user submodule helper: remove double 'fatal: ' prefix git submodule update: have a dedicated helper for cloning run_processes_parallel: rename parameters for the callbacks run_processes_parallel: treat output of children as byte array submodule update: direct error message to stderr fetching submodules: respect `submodule.fetchJobs` config option submodule-config: drop check against NULL submodule-config: keep update strategy around
| * | | | run_processes_parallel: treat output of children as byte arrayStefan Beller2016-03-01
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | We do not want the output to be interrupted by a NUL byte, so we cannot use raw fputs. Introduce strbuf_write to avoid having long arguments in run-command.c. Reviewed-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | | | | Merge branch 'jk/getwholeline-getdelim-empty'Junio C Hamano2016-04-03
|\ \ \ \ \ | |_|_|/ / |/| | | / | | |_|/ | |/| | | | | | | | | | | | | | strbuf_getwholeline() did not NUL-terminate the buffer on certain corner cases in its error codepath. * jk/getwholeline-getdelim-empty: strbuf_getwholeline: NUL-terminate getdelim buffer on error
| * | | strbuf_getwholeline: NUL-terminate getdelim buffer on errorJeff King2016-03-05
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Commit 0cc30e0 (strbuf_getwholeline: use getdelim if it is available, 2015-04-16) tries to clean up after getdelim() returns EOF, but gets one case wrong, which can lead in some obscure cases to us reading uninitialized memory. After getdelim() returns -1, we re-initialize the strbuf only if sb->buf is NULL. The thinking was that either: 1. We fed an existing allocated buffer to getdelim(), and at most it would have realloc'd, leaving our NUL in place. 2. We didn't have a buffer to feed, so we gave getdelim() NULL; sb->buf will remain NULL, and we just want to restore the empty slopbuf. But that second case isn't quite right. getdelim() may allocate a buffer, write nothing into it, and then return EOF. The resulting strbuf rightfully has sb->len set to "0", but is missing the NUL terminator in the first byte. Most call-sites are fine with this. They see the EOF and don't bother looking at the strbuf. Or they notice that sb->len is empty, and don't look at the contents. But there's at least one case that does neither, and relies on parsing the resulting (possibly zero-length) string: fast-import. You can see this in action with the new test (though we probably only notice failure there when run with --valgrind or ASAN). We can fix this by unconditionally resetting the strbuf when we have a buffer after getdelim(). That fixes case 2 above. Case 1 is probably already fine in practice, but it does not hurt for us to re-assert our invariants (especially because we are relying on whatever getdelim() happens to do, which may vary from platform to platform). Our fix covers that case, too. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | | | Merge branch 'jk/tighten-alloc'Junio C Hamano2016-02-26
|\ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Update various codepaths to avoid manually-counted malloc(). * jk/tighten-alloc: (22 commits) ewah: convert to REALLOC_ARRAY, etc convert ewah/bitmap code to use xmalloc diff_populate_gitlink: use a strbuf transport_anonymize_url: use xstrfmt git-compat-util: drop mempcpy compat code sequencer: simplify memory allocation of get_message test-path-utils: fix normalize_path_copy output buffer size fetch-pack: simplify add_sought_entry fast-import: simplify allocation in start_packfile write_untracked_extension: use FLEX_ALLOC helper prepare_{git,shell}_cmd: use argv_array use st_add and st_mult for allocation size computation convert trivial cases to FLEX_ARRAY macros use xmallocz to avoid size arithmetic convert trivial cases to ALLOC_ARRAY convert manual allocations to argv_array argv-array: add detach function add helpers for allocating flex-array structs harden REALLOC_ARRAY and xcalloc against size_t overflow tree-diff: catch integer overflow in combine_diff_path allocation ...
| * | | | use xmallocz to avoid size arithmeticJeff King2016-02-22
| |/ / / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | We frequently allocate strings as xmalloc(len + 1), where the extra 1 is for the NUL terminator. This can be done more simply with xmallocz, which also checks for integer overflow. There's no case where switching xmalloc(n+1) to xmallocz(n) is wrong; the result is the same length, and malloc made no guarantees about what was in the buffer anyway. But in some cases, we can stop manually placing NUL at the end of the allocated buffer. But that's only safe if it's clear that the contents will always fill the buffer. In each case where this patch does so, I manually examined the control flow, and I tried to err on the side of caution. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | | | Merge branch 'jc/strbuf-getline'Junio C Hamano2016-01-28
|\ \ \ \ | |_|/ / |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The preliminary clean-up for jc/peace-with-crlf topic. * jc/strbuf-getline: strbuf: give strbuf_getline() to the "most text friendly" variant checkout-index: there are only two possible line terminations update-index: there are only two possible line terminations check-ignore: there are only two possible line terminations check-attr: there are only two possible line terminations mktree: there are only two possible line terminations strbuf: introduce strbuf_getline_{lf,nul}() strbuf: make strbuf_getline_crlf() global strbuf: miniscule style fix
| * | | strbuf: give strbuf_getline() to the "most text friendly" variantJunio C Hamano2016-01-15
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Now there is no direct caller to strbuf_getline(), we can demote it to file-scope static that is private to strbuf.c and rename it to strbuf_getdelim(). Rename strbuf_getline_crlf(), which is designed to be the most "text friendly" variant, and allow it to take over this simplest name, strbuf_getline(), so we can add more uses of it without having to type _crlf over and over again in the coming steps. Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | | strbuf: introduce strbuf_getline_{lf,nul}()Junio C Hamano2016-01-15
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The strbuf_getline() interface allows a byte other than LF or NUL as the line terminator, but this is only because I wrote these codepaths anticipating that there might be a value other than NUL and LF that could be useful when I introduced line_termination long time ago. No useful caller that uses other value has emerged. By now, it is clear that the interface is overly broad without a good reason. Many codepaths have hardcoded preference to read either LF terminated or NUL terminated records from their input, and then call strbuf_getline() with LF or NUL as the third parameter. This step introduces two thin wrappers around strbuf_getline(), namely, strbuf_getline_lf() and strbuf_getline_nul(), and mechanically rewrites these call sites to call either one of them. The changes contained in this patch are: * introduction of these two functions in strbuf.[ch] * mechanical conversion of all callers to strbuf_getline() with either '\n' or '\0' as the third parameter to instead call the respective thin wrapper. After this step, output from "git grep 'strbuf_getline('" would become a lot smaller. An interim goal of this series is to make this an empty set, so that we can have strbuf_getline_crlf() take over the shorter name strbuf_getline(). Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | | strbuf: make strbuf_getline_crlf() globalJunio C Hamano2016-01-14
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Often we read "text" files that are supplied by the end user (e.g. commit log message that was edited with $GIT_EDITOR upon 'git commit -e'), and in some environments lines in a text file are terminated with CRLF. Existing strbuf_getline() knows to read a single line and then strip the terminating byte from the result, but it is handy to have a version that is more tailored for a "text" input that takes both '\n' and '\r\n' as line terminator (aka <newline> in POSIX lingo) and returns the body of the line after stripping <newline>. Recently reimplemented "git am" uses such a function implemented privately; move it to strbuf.[ch] and make it available for others. Note that we do not blindly replace calls to strbuf_getline() that uses LF as the line terminator with calls to strbuf_getline_crlf() and this is very much deliberate. Some callers may want to treat an incoming line that ends with CR (and terminated with LF) to have a payload that includes the final CR, and such a blind replacement will result in misconversion when done without code audit. Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | | strbuf: miniscule style fixJunio C Hamano2016-01-14
| |/ / | | | | | | | | | | | | | | | | | | We write one SP on each side of an operator, even inside an [] pair that computes the array index. Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | | strbuf: add strbuf_read_once to read without blockingStefan Beller2015-12-16
|/ / | | | | | | | | | | | | | | | | | | The new call will read from a file descriptor into a strbuf once. The underlying call xread is just run once. xread only reattempts reading in case of EINTR, which makes it suitable to use for a nonblocking read. Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | Merge branch 'tk/stripspace'Junio C Hamano2015-10-26
|\ \ | |/ | | | | | | | | | | | | | | | | The internal stripspace() function has been moved to where it logically belongs to, i.e. strbuf API, and the command line parser of "git stripspace" has been updated to use the parse_options API. * tk/stripspace: stripspace: use parse-options for command-line parsing strbuf: make stripspace() part of strbuf
| * strbuf: make stripspace() part of strbufTobias Klauser2015-10-16
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This function is also used in other builtins than stripspace, so it makes sense to have it in a more generic place. Since it operates on an strbuf and the function is declared in strbuf.h, move it to strbuf.c and add the corresponding prefix to its name, just like other API functions in the strbuf_* family. Also switch all current users of stripspace() to the new function name and keep a temporary wrapper inline function for any topic branches still using stripspace(). Reviewed-by: Matthieu Moy <Matthieu.Moy@imag.fr> Signed-off-by: Tobias Klauser <tklauser@distanz.ch> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | convert trivial sprintf / strcpy calls to xsnprintfJeff King2015-09-25
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | We sometimes sprintf into fixed-size buffers when we know that the buffer is large enough to fit the input (either because it's a constant, or because it's numeric input that is bounded in size). Likewise with strcpy of constant strings. However, these sites make it hard to audit sprintf and strcpy calls for buffer overflows, as a reader has to cross-reference the size of the array with the input. Let's use xsnprintf instead, which communicates to a reader that we don't expect this to overflow (and catches the mistake in case we do). Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | add reentrant variants of sha1_to_hex and find_unique_abbrevJeff King2015-09-25
|/ | | | | | | | | | | | | | | | | | | | | | | | | | | | | The sha1_to_hex and find_unique_abbrev functions always write into reusable static buffers. There are a few problems with this: - future calls overwrite our result. This is especially annoying with find_unique_abbrev, which does not have a ring of buffers, so you cannot even printf() a result that has two abbreviated sha1s. - if you want to put the result into another buffer, we often strcpy, which looks suspicious when auditing for overflows. This patch introduces sha1_to_hex_r and find_unique_abbrev_r, which write into a user-provided buffer. Of course this is just punting on the overflow-auditing, as the buffer obviously needs to be GIT_SHA1_HEXSZ + 1 bytes. But it is much easier to audit, since that is a well-known size. We retain the non-reentrant forms, which just become thin wrappers around the reentrant ones. This patch also adds a strbuf variant of find_unique_abbrev, which will be handy in later patches. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* Merge branch 'jh/strbuf-read-use-read-in-full'Junio C Hamano2015-08-25
|\ | | | | | | | | | | | | | | strbuf_read() used to have one extra iteration (and an unnecessary strbuf_grow() of 8kB), which was eliminated. * jh/strbuf-read-use-read-in-full: strbuf_read(): skip unnecessary strbuf_grow() at eof
| * strbuf_read(): skip unnecessary strbuf_grow() at eofJim Hill2015-08-10
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The loop in strbuf_read() uses xread() repeatedly while extending the strbuf until the call returns zero. If the buffer is sufficiently large to begin with, this results in xread() returning the remainder of the file to the end (returning non-zero), the loop extending the strbuf, and then making another call to xread() to have it return zero. By using read_in_full(), we can tell when the read reached the end of file: when it returns less than was requested, it's eof. This way we can avoid an extra iteration that allocates an extra 8kB that is never used. Signed-off-by: Jim Hill <gjthill@gmail.com> Reviewed-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | Merge branch 'jk/date-mode-format'Junio C Hamano2015-08-03
|\ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Teach "git log" and friends a new "--date=format:..." option to format timestamps using system's strftime(3). * jk/date-mode-format: strbuf: make strbuf_addftime more robust introduce "format" date-mode convert "enum date_mode" into a struct show-branch: use DATE_RELATIVE instead of magic number
| * | strbuf: make strbuf_addftime more robustJeff King2015-07-20
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The return value of strftime is poorly designed; when it returns 0, the caller cannot tell if the buffer was not large enough, or if the output was actually 0 bytes. In the original implementation of strbuf_addftime, we simply punted and guessed that our 128-byte hint would be large enough. We can do better, though, if we're willing to treat strftime like less of a black box. We can munge the incoming format to make sure that it never produces 0-length output, and then "fix" the resulting output. That lets us reliably grow the buffer based on strftime's return value. Clever-idea-by: Eric Sunshine <sunshine@sunshineco.com> Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | introduce "format" date-modeJeff King2015-06-29
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This feeds the format directly to strftime. Besides being a little more flexible, the main advantage is that your system strftime may know more about your locale's preferred format (e.g., how to spell the days of the week). Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | | Merge branch 'mh/strbuf-read-file-returns-ssize-t'Junio C Hamano2015-07-13
|\ \ \ | |/ / |/| | | | | | | | | | | | | | Avoid possible ssize_t to int truncation. * mh/strbuf-read-file-returns-ssize-t: strbuf: strbuf_read_file() should return ssize_t
| * | strbuf: strbuf_read_file() should return ssize_tMichael Haggerty2015-07-03
| | | | | | | | | | | | | | | | | | | | | | | | It is currently declared to return int, which could overflow for large files. Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | | strbuf_getwholeline: use getdelim if it is availableJeff King2015-04-16
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | We spend a lot of time in strbuf_getwholeline in a tight loop reading characters from a stdio handle into a buffer. The libc getdelim() function can do this for us with less overhead. It's in POSIX.1-2008, and was a GNU extension before that. Therefore we can't rely on it, but can fall back to the existing getc loop when it is not available. The HAVE_GETDELIM knob is turned on automatically for Linux, where we have glibc. We don't need to set any new feature-test macros, because we already define _GNU_SOURCE. Other systems that implement getdelim may need to other macros (probably _POSIX_C_SOURCE >= 200809L), but we can address that along with setting the Makefile knob after testing the feature on those systems. Running "git rev-parse refs/heads/does-not-exist" on a repo with an extremely large (1.6GB) packed-refs file went from (best-of-5): real 0m8.601s user 0m8.084s sys 0m0.524s to: real 0m6.768s user 0m6.340s sys 0m0.432s for a wall-clock speedup of 21%. Based on a patch from Rasmus Villemoes <rv@rasmusvillemoes.dk>. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | | strbuf_getwholeline: avoid calling strbuf_growJeff King2015-04-16
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | As with the recent speedup to strbuf_addch, we can avoid calling strbuf_grow() in a tight loop of single-character adds by instead checking strbuf_avail. Note that we would instead call strbuf_addch directly here, but it does more work than necessary: it will NUL-terminate the result for each character read. Instead, in this loop we read the characters one by one and then add the terminator manually at the end. Running "git rev-parse refs/heads/does-not-exist" on a repo with an extremely large (1.6GB) packed-refs file went from (best-of-5): real 0m10.948s user 0m10.548s sys 0m0.412s to: real 0m8.601s user 0m8.084s sys 0m0.524s for a wall-clock speedup of 21%. Helped-by: Eric Sunshine <sunshine@sunshineco.com> Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | | strbuf_getwholeline: use getc_unlockedJeff King2015-04-16
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | strbuf_getwholeline calls getc in a tight loop. On modern libc implementations, the stdio code locks the handle for every operation, which means we are paying a significant overhead. We can get around this by locking the handle for the whole loop and using the unlocked variant. Running "git rev-parse refs/heads/does-not-exist" on a repo with an extremely large (1.6GB) packed-refs file went from: real 0m18.900s user 0m18.472s sys 0m0.448s to: real 0m10.953s user 0m10.384s sys 0m0.580s for a wall-clock speedup of 42%. All times are best-of-3, and done on a glibc 2.19 system. Note that we call into strbuf_grow while holding the lock. It's possible for that function to call other stdio functions (e.g., printing to stderr when dying due to malloc error); however, the POSIX.1-2001 definition of flockfile makes it clear that the locks are per-handle, so we are fine unless somebody else tries to read from our same handle. This doesn't ever happen in the current code, and is unlikely to be added in the future (we would have to do something exotic like add a die_routine that tried to read from stdin). Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | | strbuf_getwholeline: use getc macroJeff King2015-04-16
|/ / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | strbuf_getwholeline calls fgetc in a tight loop. Using the getc form, which can be implemented as a macro, should be faster (and we do not care about it evaluating our argument twice, as we just have a plain variable). On my glibc system, running "git rev-parse refs/heads/does-not-exist" on a file with an extremely large (1.6GB) packed-refs file went from (best of 3 runs): real 0m19.383s user 0m18.876s sys 0m0.528s to: real 0m18.900s user 0m18.472s sys 0m0.448s for a wall-clock speedup of 2.5%. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | Merge branch 'jc/strbuf-add-lines-avoid-sp-ht-sequence'Junio C Hamano2015-01-07
|\ \ | | | | | | | | | | | | | | | | | | | | | | | | The commented output used to blindly add a SP before the payload line, resulting in "# \t<indented text>\n" when the payload began with a HT. Instead, produce "#\t<indented text>\n". * jc/strbuf-add-lines-avoid-sp-ht-sequence: strbuf_add_commented_lines(): avoid SP-HT sequence in commented lines
| * | strbuf_add_commented_lines(): avoid SP-HT sequence in commented linesJunio C Hamano2014-10-27
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The strbuf_add_commented_lines() function passes a pair of prefixes, one to be used for a non-empty line, and the other for an empty line, to underlying add_lines(). The former is set to a comment char followed by a SP, while the latter is set to just the comment char. This is designed to give a SP after the comment character, e.g. "# <user text>\n", on a line with some text, and to avoid emitting an unsightly "# \n" for an empty line. Teach this machinery to also use the latter space-less prefix when the payload line begins with a tab, to show e.g. "#\t<user text>\n"; otherwise we will end up showing "# \t<user text>\n" which is similarly unsightly. Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | | Merge branch 'rs/export-strbuf-addchars'Junio C Hamano2014-09-19
|\ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | Code clean-up. * rs/export-strbuf-addchars: strbuf: use strbuf_addchars() for adding a char multiple times strbuf: export strbuf_addchars()
| * | | strbuf: export strbuf_addchars()René Scharfe2014-09-08
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Move strbuf_addchars() to strbuf.c, where it belongs, and make it available for other callers. Signed-off-by: Rene Scharfe <l.s.r@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | | | Merge branch 'rs/strbuf-getcwd'Junio C Hamano2014-09-02
|\ \ \ \ | |_|/ / |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Reduce the use of fixed sized buffer passed to getcwd() calls by introducing xgetcwd() helper. * rs/strbuf-getcwd: use strbuf_add_absolute_path() to add absolute paths abspath: convert absolute_path() to strbuf use xgetcwd() to set $GIT_DIR use xgetcwd() to get the current directory or die wrapper: add xgetcwd() abspath: convert real_path_internal() to strbuf abspath: use strbuf_getcwd() to remember original working directory setup: convert setup_git_directory_gently_1 et al. to strbuf unix-sockets: use strbuf_getcwd() strbuf: add strbuf_getcwd()
| * | | abspath: convert absolute_path() to strbufRené Scharfe2014-08-26
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Move most of the code of absolute_path() into the new function strbuf_add_absolute_path() and in the process transform it to use struct strbuf and xgetcwd() instead of a PATH_MAX-sized buffer, which can be too small on some file systems. Signed-off-by: Rene Scharfe <l.s.r@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | | strbuf: add strbuf_getcwd()René Scharfe2014-07-28
| | |/ | |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Add strbuf_getcwd(), which puts the current working directory into a strbuf. Because it doesn't use a fixed-size buffer it supports arbitrarily long paths, provided the platform's getcwd() does as well. At least on Linux and FreeBSD it handles paths longer than PATH_MAX just fine. Suggested-by: Karsten Blees <karsten.blees@gmail.com> Helped-by: Duy Nguyen <pclouds@gmail.com> Signed-off-by: Rene Scharfe <l.s.r@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | | Merge branch 'jk/strip-suffix'Junio C Hamano2014-07-16
|\ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * jk/strip-suffix: prepare_packed_git_one: refactor duplicate-pack check verify-pack: use strbuf_strip_suffix strbuf: implement strbuf_strip_suffix index-pack: use strip_suffix to avoid magic numbers use strip_suffix instead of ends_with in simple cases replace has_extension with ends_with implement ends_with via strip_suffix add strip_suffix function sha1_file: replace PATH_MAX buffer with strbuf in prepare_packed_git_one()
| * | | implement ends_with via strip_suffixJeff King2014-06-30
| |/ / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The ends_with function is essentially a simplified version of strip_suffix, in which we throw away the stripped length. Implementing it as an inline on top of strip_suffix has two advantages: 1. We save a bit of duplicated code. 2. The suffix is typically a string literal, and we call strlen on it. By making the function inline, many compilers can replace the strlen call with a constant. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | | strbuf: add xstrfmt helperJeff King2014-06-19
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | You can use a strbuf to build up a string from parts, and then detach it. In the general case, you might use multiple strbuf_add* functions to do the building. However, in many cases, a single strbuf_addf is sufficient, and we end up with: struct strbuf buf = STRBUF_INIT; ... strbuf_addf(&buf, fmt, some, args); str = strbuf_detach(&buf, NULL); We can make this much more readable (and avoid introducing an extra variable, which can clutter the code) by introducing a convenience function: str = xstrfmt(fmt, some, args); Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | | Merge branch 'jk/http-errors'Junio C Hamano2014-06-16
|\ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Propagate the error messages from the webserver better to the client coming over the HTTP transport. * jk/http-errors: http: default text charset to iso-8859-1 remote-curl: reencode http error messages strbuf: add strbuf_reencode helper http: optionally extract charset parameter from content-type http: extract type/subtype portion of content-type t5550: test display of remote http error messages t/lib-httpd: use write_script to copy CGI scripts test-lib: preserve GIT_CURL_VERBOSE from the environment
| * | | strbuf: add strbuf_reencode helperJeff King2014-05-27
| |/ / | | | | | | | | | | | | | | | | | | | | | This is a convenience wrapper around `reencode_string_len` and `strbuf_attach`. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | | Merge branch 'jk/strbuf-tolower'Junio C Hamano2014-06-16
|\ \ \ | | | | | | | | | | | | | | | | * jk/strbuf-tolower: strbuf: add strbuf_tolower function
| * | | strbuf: add strbuf_tolower functionJeff King2014-05-23
| |/ / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is a convenience wrapper to call tolower on each character of the string. This makes config's lowercase() function obsolete, though note that because we have a strbuf, we are careful to operate over the whole strbuf, rather than assuming that a NUL is the end-of-string. We could continue to offer a pure-string lowercase, but there would be no callers (in most pure-string cases, we actually duplicate and lowercase the duplicate, for which we have the xstrdup_tolower wrapper). Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | | Merge branch 'jk/daemon-tolower'Junio C Hamano2014-06-16
|\ \ \ | | | | | | | | | | | | | | | | * jk/daemon-tolower: daemon/config: factor out duplicate xstrdup_tolower
| * | | daemon/config: factor out duplicate xstrdup_tolowerJeff King2014-05-23
| | |/ | |/| | | | | | | | | | | | | | | | | | | | | | | | | We have two implementations of the same function; let's drop that to one. We take the name from daemon.c, but the implementation (which is just slightly more efficient) from the config code. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>