aboutsummaryrefslogtreecommitdiff
path: root/convert.c
Commit message (Collapse)AuthorAge
* Merge branch 'ls/filter-process'Junio C Hamano2016-10-31
|\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The smudge/clean filter API expect an external process is spawned to filter the contents for each path that has a filter defined. A new type of "process" filter API has been added to allow the first request to run the filter for a path to spawn a single process, and all filtering need is served by this single process for multiple paths, reducing the process creation overhead. * ls/filter-process: contrib/long-running-filter: add long running filter example convert: add filter.<driver>.process option convert: prepare filter.<driver>.process option convert: make apply_filter() adhere to standard Git error handling pkt-line: add functions to read/write flush terminated packet streams pkt-line: add packet_write_gently() pkt-line: add packet_flush_gently() pkt-line: add packet_write_fmt_gently() pkt-line: extract set_packet_header() pkt-line: rename packet_write() to packet_write_fmt() run-command: add clean_on_exit_handler run-command: move check_pipe() from write_or_die to run_command convert: modernize tests convert: quote filter names in error messages
| * convert: add filter.<driver>.process optionLars Schneider2016-10-17
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Git's clean/smudge mechanism invokes an external filter process for every single blob that is affected by a filter. If Git filters a lot of blobs then the startup time of the external filter processes can become a significant part of the overall Git execution time. In a preliminary performance test this developer used a clean/smudge filter written in golang to filter 12,000 files. This process took 364s with the existing filter mechanism and 5s with the new mechanism. See details here: https://github.com/github/git-lfs/pull/1382 This patch adds the `filter.<driver>.process` string option which, if used, keeps the external filter process running and processes all blobs with the packet format (pkt-line) based protocol over standard input and standard output. The full protocol is explained in detail in `Documentation/gitattributes.txt`. A few key decisions: * The long running filter process is referred to as filter protocol version 2 because the existing single shot filter invocation is considered version 1. * Git sends a welcome message and expects a response right after the external filter process has started. This ensures that Git will not hang if a version 1 filter is incorrectly used with the filter.<driver>.process option for version 2 filters. In addition, Git can detect this kind of error and warn the user. * The status of a filter operation (e.g. "success" or "error) is set before the actual response and (if necessary!) re-set after the response. The advantage of this two step status response is that if the filter detects an error early, then the filter can communicate this and Git does not even need to create structures to read the response. * All status responses are pkt-line lists terminated with a flush packet. This allows us to send other status fields with the same protocol in the future. Helped-by: Martin-Louis Bright <mlbright@gmail.com> Reviewed-by: Jakub Narebski <jnareb@gmail.com> Signed-off-by: Lars Schneider <larsxschneider@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * convert: prepare filter.<driver>.process optionLars Schneider2016-10-17
| | | | | | | | | | | | | | | | Refactor the existing 'single shot filter mechanism' and prepare the new 'long running filter mechanism'. Signed-off-by: Lars Schneider <larsxschneider@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * convert: make apply_filter() adhere to standard Git error handlingLars Schneider2016-10-17
| | | | | | | | | | | | | | | | | | | | | | | | | | apply_filter() returns a boolean that tells the caller if it "did convert or did not convert". The variable `ret` was used throughout the function to track errors whereas `1` denoted success and `0` failure. This is unusual for the Git source where `0` denotes success. Rename the variable and flip its value to make the function easier readable for Git developers. Signed-off-by: Lars Schneider <larsxschneider@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * convert: quote filter names in error messagesLars Schneider2016-10-17
| | | | | | | | | | | | | | | | Git filter driver commands with spaces (e.g. `filter.sh foo`) are hard to read in error messages. Quote them to improve the readability. Signed-off-by: Lars Schneider <larsxschneider@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | i18n: convert mark error messages for translationVasco Almeida2016-10-17
|/ | | | | | | | | Mark error messages about CRLF for translation. Update test to reflect changes. Signed-off-by: Vasco Almeida <vascomalmeida@sapo.pt> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* convert: Correct NNO tests and missing `LF will be replaced by CRLF`Torsten Bögershausen2016-08-14
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When a non-reversible CRLF conversion is done in "git add", a warning is printed on stderr (or Git dies, depending on checksafe) The function commit_chk_wrnNNO() in t0027 was written to test this, but did the wrong thing: Instead of looking at the warning from "git add", it looked at the warning from "git commit". This is racy because "git commit" may not have to do CRLF conversion at all if it can use the sha1 value from the index (which depends on whether "add" and "commit" run in a single second). Correct t0027 and replace the commit for each and every file with a commit of all files in one go. The function commit_chk_wrnNNO() should be renamed in a separate commit. Now that t0027 does the right thing, it detects a bug in covert.c: This sequence should generate the warning `LF will be replaced by CRLF`, but does not: $ git init $ git config core.autocrlf false $ printf "Line\r\n" >file $ git add file $ git commit -m "commit with CRLF" $ git config core.autocrlf true $ printf "Line\n" >file $ git add file "git add" calls crlf_to_git() in convert.c, which calls check_safe_crlf(). When has_cr_in_index(path) is true, crlf_to_git() returns too early and check_safe_crlf() is not called at all. Factor out the code which determines if "git checkout" converts LF->CRLF into will_convert_lf_to_crlf(). Update the logic around check_safe_crlf() and "simulate" the possible LF->CRLF conversion at "git checkout" with help of will_convert_lf_to_crlf(). Thanks to Jeff King <peff@peff.net> for analyzing t0027. Reported-By: Johannes Schindelin <Johannes.Schindelin@gmx.de> Signed-off-by: Torsten Bögershausen <tboegi@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* convert: unify the "auto" handling of CRLFTorsten Bögershausen2016-07-06
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Before this change, $ echo "* text=auto" >.gitattributes $ echo "* eol=crlf" >>.gitattributes would have the same effect as $ echo "* text" >.gitattributes $ git config core.eol crlf Since the 'eol' attribute had higher priority than 'text=auto', this may corrupt binary files and is not what most users expect to happen. Make the 'eol' attribute to obey 'text=auto' and now $ echo "* text=auto" >.gitattributes $ echo "* eol=crlf" >>.gitattributes behaves the same as $ echo "* text=auto" >.gitattributes $ git config core.eol crlf In other words, $ echo "* text=auto eol=crlf" >.gitattributes has the same effect as $ git config core.autocrlf true and $ echo "* text=auto eol=lf" >.gitattributes has the same effect as $ git config core.autocrlf input Signed-off-by: Torsten Bögershausen <tboegi@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* convert.c: ident + core.autocrlf didn't workTorsten Bögershausen2016-04-25
| | | | | | | | | | | | | | | When the ident attributes is set, get_stream_filter() did not obey core.autocrlf=true, and the file was checked out with LF. Change the rule when a streaming filter can be used: - if an external filter is specified, don't use a stream filter. - if the worktree eol is CRLF and "auto" is active, don't use a stream filter. - Otherwise the stream filter can be used. Add test cases in t0027. Signed-off-by: Torsten Bögershausen <tboegi@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* Merge branch 'tb/conversion'Junio C Hamano2016-02-26
|\ | | | | | | | | | | | | | | | | | | | | | | | | Code simplification. * tb/conversion: convert.c: correct attr_action() convert.c: simplify text_stat convert.c: refactor crlf_action convert.c: use text_eol_is_crlf() convert.c: remove input_crlf_action() convert.c: remove unused parameter 'path' t0027: add tests for get_stream_filter()
| * convert.c: correct attr_action()Torsten Bögershausen2016-02-23
| | | | | | | | | | | | | | | | | | | | | | | | | | df747b81 (convert.c: refactor crlf_action, 2016-02-10) introduced a bug to "git ls-files --eol". The "text" attribute was shown as "text eol=lf" or "text eol=crlf", depending on core.autocrlf or core.eol. Correct this and add test cases in t0027. Signed-off-by: Torsten Bögershausen <tboegi@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * convert.c: simplify text_statTorsten Bögershausen2016-02-10
| | | | | | | | | | | | | | | | | | | | | | Simplify the statistics: lonecr counts the CR which is not followed by a LF, lonelf counts the LF which is not preceded by a CR, crlf counts CRLF combinations. This simplifies the evaluation of the statistics. Signed-off-by: Torsten Bögershausen <tboegi@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * convert.c: refactor crlf_actionTorsten Bögershausen2016-02-10
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Refactor the determination and usage of crlf_action. Today, when no "crlf" attribute are set on a file, crlf_action is set to CRLF_GUESS. Use CRLF_UNDEFINED instead, and search for "text" or "eol" as before. After searching for line ending attributes, save the value in struct conv_attrs.crlf_action attr_action, so that get_convert_attr_ascii() is able report the attributes. Replace the old CRLF_GUESS usage: CRLF_GUESS && core.autocrlf=true -> CRLF_AUTO_CRLF CRLF_GUESS && core.autocrlf=false -> CRLF_BINARY CRLF_GUESS && core.autocrlf=input -> CRLF_AUTO_INPUT Save the action in conv_attrs.crlf_action (as before) and change all callers. Make more clear, what is what, by defining: - CRLF_UNDEFINED : No attributes set. Temparally used, until core.autocrlf and core.eol is evaluated and one of CRLF_BINARY, CRLF_AUTO_INPUT or CRLF_AUTO_CRLF is selected - CRLF_BINARY : No processing of line endings. - CRLF_TEXT : attribute "text" is set, line endings are processed. - CRLF_TEXT_INPUT: attribute "input" or "eol=lf" is set. This implies text. - CRLF_TEXT_CRLF : attribute "eol=crlf" is set. This implies text. - CRLF_AUTO : attribute "auto" is set. - CRLF_AUTO_INPUT: core.autocrlf=input (no attributes) - CRLF_AUTO_CRLF : core.autocrlf=true (no attributes) Signed-off-by: Torsten Bögershausen <tboegi@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * convert.c: use text_eol_is_crlf()Torsten Bögershausen2016-02-08
| | | | | | | | | | | | | | | | | | Add a helper function to find out, which line endings text files should get at checkout, depending on core.autocrlf and core.eol configuration variables. Signed-off-by: Torsten Bögershausen <tboegi@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * convert.c: remove input_crlf_action()Torsten Bögershausen2016-02-08
| | | | | | | | | | | | | | | | | | | | | | | | Integrate the code of input_crlf_action() into convert_attrs(), so that ca.crlf_action is always valid after calling convert_attrs(). Keep a copy of crlf_action in attr_action, this is needed for get_convert_attr_ascii(). Remove eol_attr from struct conv_attrs, as it is now used temporally. Signed-off-by: Torsten Bögershausen <tboegi@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * convert.c: remove unused parameter 'path'Torsten Bögershausen2016-02-08
| | | | | | | | | | | | | | | | Some functions get a parameter path, but don't use it. Remove the unused parameter. Signed-off-by: Torsten Bögershausen <tboegi@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | Merge branch 'ls/clean-smudge-override-in-config'Junio C Hamano2016-02-10
|\ \ | |/ |/| | | | | | | | | | | | | Clean/smudge filters defined in a configuration file of lower precedence can now be overridden to be a pass-through no-op by setting the variable to an empty string. * ls/clean-smudge-override-in-config: convert: treat an empty string for clean/smudge filters as "cat"
| * convert: treat an empty string for clean/smudge filters as "cat"Lars Schneider2016-01-29
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Once a lower-priority configuration file defines a clean or smudge filter, there is no convenient way to override it to produce as-is output. Even though the configuration mechanism implements "the last one wins" semantics, you cannot set them to an empty string and expect them to work, as apply_filter() would try to run the empty string as an external command and fail. The conversion is not done, but the function would still report a failure to convert. Even though resetting the variable to "cat" (i.e. pass the data back as-is and report success) is an obvious and a viable way to solve this, it is wasteful to spawn an external process just as a workaround. Instead, teach apply_filter() to treat an empty string as a no-op filter that always returns successfully its input as-is without conversion. Signed-off-by: Lars Schneider <larsxschneider@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | ls-files: add eol diagnosticsTorsten Bögershausen2016-01-18
|/ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When working in a cross-platform environment, a user may want to check if text files are stored normalized in the repository and if .gitattributes are set appropriately. Make it possible to let Git show the line endings in the index and in the working tree and the effective text/eol attributes. The end of line ("eolinfo") are shown like this: "-text" binary (or with bare CR) file "none" text file without any EOL "lf" text file with LF "crlf" text file with CRLF "mixed" text file with mixed line endings. The effective text/eol attribute is one of these: "", "-text", "text", "text=auto", "text eol=lf", "text eol=crlf" git ls-files --eol gives an output like this: i/none w/none attr/text=auto t/t5100/empty i/-text w/-text attr/-text t/test-binary-2.png i/lf w/lf attr/text eol=lf t/t5100/rfc2047-info-0007 i/lf w/crlf attr/text eol=crlf doit.bat i/mixed w/mixed attr/ locale/XX.po to show what eol convention is used in the data in the index ('i'), and in the working tree ('w'), and what attribute is in effect, for each path that is shown. Add test cases in t0027. Helped-By: Eric Sunshine <sunshine@sunshineco.com> Signed-off-by: Torsten Bögershausen <tboegi@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* convert trivial sprintf / strcpy calls to xsnprintfJeff King2015-09-25
| | | | | | | | | | | | | | | | | | We sometimes sprintf into fixed-size buffers when we know that the buffer is large enough to fit the input (either because it's a constant, or because it's numeric input that is bounded in size). Likewise with strcpy of constant strings. However, these sites make it hard to audit sprintf and strcpy calls for buffer overflows, as a reader has to cross-reference the size of the array with the input. Let's use xsnprintf instead, which communicates to a reader that we don't expect this to overflow (and catches the mistake in case we do). Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* filter_buffer_or_fd(): ignore EPIPEJunio C Hamano2015-05-20
| | | | | | | | | | | We are explicitly ignoring SIGPIPE, as we fully expect that the filter program may not read our output fully. Ignore EPIPE that may come from writing to it as well. A new test was stolen from Jeff's suggestion. Helped-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* Merge branch 'sp/stream-clean-filter'Junio C Hamano2014-10-08
|\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | When running a required clean filter, we do not have to mmap the original before feeding the filter. Instead, stream the file contents directly to the filter and process its output. * sp/stream-clean-filter: sha1_file: don't convert off_t to size_t too early to avoid potential die() convert: stream from fd to required clean filter to reduce used address space copy_fd(): do not close the input file descriptor mmap_limit: introduce GIT_MMAP_LIMIT to allow testing expected mmap size memory_limit: use git_env_ulong() to parse GIT_ALLOC_LIMIT config.c: add git_env_ulong() to parse environment variable convert: drop arguments other than 'path' from would_convert_to_git()
| * convert: stream from fd to required clean filter to reduce used address spaceSteffen Prohaska2014-08-28
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The data is streamed to the filter process anyway. Better avoid mapping the file if possible. This is especially useful if a clean filter reduces the size, for example if it computes a sha1 for binary data, like git media. The file size that the previous implementation could handle was limited by the available address space; large files for example could not be handled with (32-bit) msysgit. The new implementation can filter files of any size as long as the filter output is small enough. The new code path is only taken if the filter is required. The filter consumes data directly from the fd. If it fails, the original data is not immediately available. The condition can easily be handled as a fatal error, which is expected for a required filter anyway. If the filter was not required, the condition would need to be handled in a different way, like seeking to 0 and reading the data. But this would require more restructuring of the code and is probably not worth it. The obvious approach of falling back to reading all data would not help achieving the main purpose of this patch, which is to handle large files with limited address space. If reading all data is an option, we can simply take the old code path right away and mmap the entire file. The environment variable GIT_MMAP_LIMIT, which has been introduced in a previous commit is used to test that the expected code path is taken. A related test that exercises required filters is modified to verify that the data actually has been modified on its way from the file system to the object store. Signed-off-by: Steffen Prohaska <prohaska@zib.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | run-command: introduce CHILD_PROCESS_INITRené Scharfe2014-08-20
|/ | | | | | | | | | | | Most struct child_process variables are cleared using memset first after declaration. Provide a macro, CHILD_PROCESS_INIT, that can be used to initialize them statically instead. That's shorter, doesn't require a function call and is slightly more readable (especially given that we already have STRBUF_INIT, ARGV_ARRAY_INIT etc.). Helped-by: Johannes Sixt <j6t@kdbg.org> Signed-off-by: Rene Scharfe <l.s.r@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* use skip_prefix to avoid magic numbersJeff King2014-06-20
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | It's a common idiom to match a prefix and then skip past it with a magic number, like: if (starts_with(foo, "bar")) foo += 3; This is easy to get wrong, since you have to count the prefix string yourself, and there's no compiler check if the string changes. We can use skip_prefix to avoid the magic numbers here. Note that some of these conversions could be much shorter. For example: if (starts_with(arg, "--foo=")) { bar = arg + 6; continue; } could become: if (skip_prefix(arg, "--foo=", &bar)) continue; However, I have left it as: if (skip_prefix(arg, "--foo=", &v)) { bar = v; continue; } to visually match nearby cases which need to actually process the string. Like: if (skip_prefix(arg, "--foo=", &v)) { bar = atoi(v); continue; } Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* replace {pre,suf}fixcmp() with {starts,ends}_with()Christian Couder2013-12-05
| | | | | | | | | | | | | | | | | | | | | | | Leaving only the function definitions and declarations so that any new topic in flight can still make use of the old functions, replace existing uses of the prefixcmp() and suffixcmp() with new API functions. The change can be recreated by mechanically applying this: $ git grep -l -e prefixcmp -e suffixcmp -- \*.c | grep -v strbuf\\.c | xargs perl -pi -e ' s|!prefixcmp\(|starts_with\(|g; s|prefixcmp\(|!starts_with\(|g; s|!suffixcmp\(|ends_with\(|g; s|suffixcmp\(|!ends_with\(|g; ' on the result of preparatory changes in this series. Signed-off-by: Christian Couder <chriscool@tuxfamily.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* typofix: in-code commentsOndřej Bílka2013-07-22
| | | | | Signed-off-by: Ondřej Bílka <neleai@seznam.cz> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* Merge branch 'lf/read-blob-data-from-index'Junio C Hamano2013-04-21
|\ | | | | | | | | | | | | | | | | Reduce duplicated code between convert.c and attr.c. * lf/read-blob-data-from-index: convert.c: remove duplicate code read_blob_data_from_index(): optionally return the size of blob data attr.c: extract read_index_data() as read_blob_data_from_index()
| * convert.c: remove duplicate codeLukas Fleischer2013-04-17
| | | | | | | | | | | | | | | | | | The has_cr_in_index() function is an almost 1:1 copy of read_blob_data_from_index() with some additions. Use the latter instead of using copy-pasted code. Signed-off-by: Lukas Fleischer <git@cryptocrack.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | convert some config callbacks to parse_config_keyJeff King2013-01-23
|/ | | | | | | | | | | | | | | | | These callers can drop some inline pointer arithmetic and magic offset constants, making them more readable and less error-prone (those constants had to match the lengths of strings, but there is no automatic verification of that fact). The "ep" pointer (presumably for "end pointer"), which points to the final key segment of the config variable, is given the more standard name "key" to describe its function rather than its derivation. Signed-off-by: Jeff King <peff@peff.net> Reviewed-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* Merge branch 'jb/required-filter'Junio C Hamano2012-02-28
|\ | | | | | | | | | | | | | | * jb/required-filter: Add a setting to require a filter to be successful Conflicts: convert.c
| * Add a setting to require a filter to be successfulJehan Bing2012-02-17
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | By default, a missing filter driver or a failure from the filter driver is not an error, but merely makes the filter operation a no-op pass through. This is useful to massage the content into a shape that is more convenient for the platform, filesystem, and the user to use, and the content filter mechanism is not used to turn something unusable into usable. However, we could also use of the content filtering mechanism and store the content that cannot be directly used in the repository (e.g. a UUID that refers to the true content stored outside git, or an encrypted content) and turn it into a usable form upon checkout (e.g. download the external content, or decrypt the encrypted content). For such a use case, the content cannot be used when filter driver fails, and we need a way to tell Git to abort the whole operation for such a failing or missing filter driver. Add a new "filter.<driver>.required" configuration variable to mark the second use case. When it is set, git will abort the operation when the filter driver does not exist or exits with a non-zero status code. Signed-off-by: Jehan Bing <jehan@orb.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * Merge branch 'jc/maint-lf-to-crlf-keep-crlf'Junio C Hamano2011-12-22
| |\ | | | | | | | | | | | | * jc/maint-lf-to-crlf-keep-crlf: lf_to_crlf_filter(): resurrect CRLF->CRLF hack
* | \ Merge branch 'jk/maint-avoid-streaming-filtered-contents'Junio C Hamano2012-02-26
|\ \ \ | | | | | | | | | | | | | | | | | | | | | | | | * jk/maint-avoid-streaming-filtered-contents: do not stream large files to pack when filters are in use teach dry-run convert_to_git not to require a src buffer teach convert_to_git a "dry run" mode
| * | | teach dry-run convert_to_git not to require a src bufferJeff King2012-02-24
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When we call convert_to_git in dry-run mode, it may still want to look at the source buffer, because some CRLF conversion modes depend on analyzing the source to determine whether it is in fact convertible CRLF text. However, the main motivation for convert_to_git's dry-run mode is that we would decide which method to use to acquire the blob's data (streaming versus in-core). Requiring this source analysis creates a chicken-and-egg problem. We are better off simply guessing that anything we can't analyze will end up needing conversion. This patch lets a caller specify a NULL src buffer when using dry-run mode (and only dry-run mode). A non-zero return value goes from "we would convert" to "we might convert"; a zero return value remains "we would definitely not convert". Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | | teach convert_to_git a "dry run" modeJeff King2012-02-24
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Some callers may want to know whether convert_to_git will actually do anything before performing the conversion itself (e.g., to decide whether to stream or handle blobs in-core). This patch lets callers specify the dry run mode by passing a NULL destination buffer. The return value, instead of indicating whether conversion happened, will indicate whether conversion would occur. For readability, we also include a wrapper function which makes it more obvious we are not actually performing the conversion. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | | | Ignore SIGPIPE when running a filter driverJehan Bing2012-02-21
|/ / / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | If a filter is not defined or if it fails, git should behave as if the filter is a no-op passthru. However, if the filter exits before reading all the content, depending on the timing, git could be killed with SIGPIPE when it tries to write to the pipe connected to the filter. Ignore SIGPIPE while processing the filter to give us a chance to check the return value from a failed write, in order to detect and act on this mode of failure in a more controlled way. Signed-off-by: Jehan Bing <jehan@orb.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | | Merge branch 'jc/maint-lf-to-crlf-keep-crlf' into maintJunio C Hamano2011-12-28
|\ \ \ | |/ / |/| / | |/ | | * jc/maint-lf-to-crlf-keep-crlf: lf_to_crlf_filter(): resurrect CRLF->CRLF hack
| * lf_to_crlf_filter(): resurrect CRLF->CRLF hackJunio C Hamano2011-12-18
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The non-streaming version of the filter counts CRLF and LF in the whole buffer, and returns without doing anything when they match (i.e. what is recorded in the object store already uses CRLF). This was done to help people who added files from the DOS world before realizing they want to go cross platform and adding .gitattributes to tell Git that they only want CRLF in their working tree. The streaming version of the filter does not want to read the whole thing before starting to work, as that defeats the whole point of streaming. So we instead check what byte follows CR whenever we see one, and add CR before LF only when the LF does not immediately follow CR already to keep CRLF as is. Reported-and-tested-by: Ralf Thielow Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | Merge branch 'cn/maint-lf-to-crlf-filter' into maintJunio C Hamano2011-12-21
|\ \ | |/ | | | | | | | | * cn/maint-lf-to-crlf-filter: lf_to_crlf_filter(): tell the caller we added "\n" when draining convert: track state in LF-to-CRLF filter
| * lf_to_crlf_filter(): tell the caller we added "\n" when drainingJunio C Hamano2011-12-16
| | | | | | | | | | | | | | | | | | This can only happen when the input size is multiple of the buffer size of the cascade filter (16k) and ends with an LF, but in such a case, the code forgot to tell the caller that it added the "\n" it could not add during the last round. Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * convert: track state in LF-to-CRLF filterCarlos Martín Nieto2011-11-28
| | | | | | | | | | | | | | | | | | | | | | | | | | | | There may not be enough space to store CRLF in the output. If we don't fill the buffer, then the filter will keep getting called with the same short buffer and will loop forever. Instead, always store the CR and record whether there's a missing LF if so we store it in the output buffer the next time the function gets called. Reported-by: Henrik Grubbström <grubba@roxen.com> Signed-off-by: Carlos Martín Nieto <cmn@elego.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | convert.c: Fix return type of git_path_check_eol()Ramsay Jones2011-11-21
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The git_path_check_eol() function converts a string value to the corresponding 'enum eol' value. However, the function is currently declared to return an 'enum crlf_action', which causes sparse to complain thus: SP convert.c convert.c:736:50: warning: mixing different enum types convert.c:736:50: int enum crlf_action versus convert.c:736:50: int enum eol In order to suppress the warning, we simply correct the return type in the function declaration. Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | convert: don't mix enum with intRamkumar Ramachandra2011-11-15
|/ | | | | Signed-off-by: Ramkumar Ramachandra <artagnon@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* Merge branch 'tr/maint-ident-to-git-memmove'Junio C Hamano2011-09-02
|\ | | | | | | | | * tr/maint-ident-to-git-memmove: Use memmove in ident_to_git
| * Use memmove in ident_to_gitThomas Rast2011-08-29
| | | | | | | | | | | | | | | | | | convert_to_git sets src=dst->buf if any of the preceding conversions actually did any work. Thus in ident_to_git we have to use memmove instead of memcpy as far as src->dst copying is concerned. Signed-off-by: Thomas Rast <trast@student.ethz.ch> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | Rename git_checkattr() to git_check_attr()Michael Haggerty2011-08-04
| | | | | | | | | | | | | | Suggested by: Junio Hamano <gitster@pobox.com> Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | streaming: filter cascadingJunio C Hamano2011-05-26
| | | | | | | | | | | | | | This implements an internal "cascade" filter mechanism that plugs two filters in series. Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | streaming filter: ident filterJunio C Hamano2011-05-26
| | | | | | | | | | | | | | Add support for "ident" filter on the output codepath. This does not work with lf-to-crlf filter together (yet). Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | Add LF-to-CRLF streaming conversionJunio C Hamano2011-05-26
| | | | | | | | | | | | | | If we do not have to guess or validate by scanning the input, we can just stream this through. Signed-off-by: Junio C Hamano <gitster@pobox.com>
generated by cgit v1.2.1 (git 2.18.0) at 2024-12-26 05:54:19 +0000