aboutsummaryrefslogtreecommitdiff
path: root/vcs-svn
Commit message (Collapse)AuthorAge
* Merge branch 'jn/vcs-svn-cleanup'Junio C Hamano2017-08-26
|\ | | | | | | | | | | | | | | | | | | Code clean-up. * jn/vcs-svn-cleanup: vcs-svn: move remaining repo_tree functions to fast_export.h vcs-svn: remove repo_delete wrapper function vcs-svn: remove custom mode constants vcs-svn: remove more unused prototypes and declarations
| * vcs-svn: move remaining repo_tree functions to fast_export.hJonathan Nieder2017-08-23
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | These used to be for manipulating the in-memory repo_tree structure, but nowadays they are convenience wrappers to handle a few git-vs-svn mismatches: 1. Git does not track empty directories but Subversion does. When looking up a path in git that Subversion thinks exists and finding nothing, we can safely assume that the path represents a directory. This is needed when a later Subversion revision modifies that directory. 2. Subversion allows deleting a file by copying. In Git fast-import we have to handle that more explicitly as a deletion. These are details of the tool's interaction with git fast-import. Move them to fast_export.c, where other such details are handled. This way the function names do not start with a repo_ prefix that would clash with the repository object introduced in v2.14.0-rc0~38^2~16 (repository: introduce the repository object, 2017-06-22) or an svn_ prefix that would clash with libsvn (in case someone wants to link this code with libsvn some day). Signed-off-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * vcs-svn: remove repo_delete wrapper functionJonathan Nieder2017-08-23
| | | | | | | | | | | | | | | | | | | | | | Since v1.7.10-rc0~118^2~4^2~4^2~3 (vcs-svn: pass paths through to fast-import, 2010-12-13) this is an alias for fast_export_delete. Remove the unnecessary layer of indirection. No functional change intended. Signed-off-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * vcs-svn: remove custom mode constantsJonathan Nieder2017-08-23
| | | | | | | | | | | | | | | | | | | | | | In the rest of Git, these modes are spelled as S_IFDIR, S_IFREG | 0644, S_IFREG | 0755, and S_IFLNK. Use the same constants in svn-fe for simplicity and consistency. No functional change intended. Signed-off-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * vcs-svn: remove more unused prototypes and declarationsJonathan Nieder2017-08-23
| | | | | | | | | | | | | | | | | | | | | | I forgot to remove these in v1.7.10-rc0~118^2~4^2~5^2~4 (vcs-svn: eliminate repo_tree structure, 2010-12-10). This finishes what was started in commit 36f63b50 (vcs-svn: remove unused prototypes, 2017-08-21). Signed-off-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | Merge branch 'bc/vcs-svn-cleanup'Junio C Hamano2017-08-26
|\ \ | |/ | | | | | | | | | | | | Code clean-up. * bc/vcs-svn-cleanup: vcs-svn: rename repo functions to "svn_repo" vcs-svn: remove unused prototypes
| * vcs-svn: rename repo functions to "svn_repo"brian m. carlson2017-08-20
| | | | | | | | | | | | | | | | | | | | There were several functions in the Subversion code that started with "repo_". This namespace is also used by the Git struct repository code. Rename these functions to start with "svn_repo" to avoid any future conflicts. Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * vcs-svn: remove unused prototypesbrian m. carlson2017-08-20
| | | | | | | | | | | | | | | | | | | | The Subversion code had prototypes for several functions which were not ever defined or used. These functions all had names starting with "repo_", some of which conflict with those in repository.h. To avoid the conflict, remove those unused prototypes. Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * Merge branch 'mr/vcs-svn-printf-ulong' into maintJunio C Hamano2016-09-29
| |\ | | | | | | | | | | | | | | | | | | Code cleanup. * mr/vcs-svn-printf-ulong: vcs-svn/fast_export: fix timestamp fmt specifiers
* | | timestamp_t: a new data type for timestampsJohannes Schindelin2017-04-27
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Git's source code assumes that unsigned long is at least as precise as time_t. Which is incorrect, and causes a lot of problems, in particular where unsigned long is only 32-bit (notably on Windows, even in 64-bit versions). So let's just use a more appropriate data type instead. In preparation for this, we introduce the new `timestamp_t` data type. By necessity, this is a very, very large patch, as it has to replace all timestamps' data type in one go. As we will use a data type that is not necessarily identical to `time_t`, we need to be very careful to use `time_t` whenever we interact with the system functions, and `timestamp_t` everywhere else. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | | PRItime: introduce a new "printf format" for timestampsJohannes Schindelin2017-04-23
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently, Git's source code treats all timestamps as if they were unsigned longs. Therefore, it is okay to write "%lu" when printing them. There is a substantial problem with that, though: at least on Windows, time_t is *larger* than unsigned long, and hence we will want to switch away from the ill-specified `unsigned long` data type. So let's introduce the pseudo format "PRItime" (currently simply being defined to "lu") to make it easier to change the data type used for timestamps. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | | Merge branch 'mr/vcs-svn-printf-ulong'Junio C Hamano2016-09-21
|\ \ \ | |/ / |/| / | |/ | | | | | | Code cleanup. * mr/vcs-svn-printf-ulong: vcs-svn/fast_export: fix timestamp fmt specifiers
| * vcs-svn/fast_export: fix timestamp fmt specifiersMike Ralphson2016-09-14
| | | | | | | | | | | | | | Two instances of %ld being used for unsigned longs Signed-off-by: Mike Ralphson <mike.ralphson@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | vcs-svn: use error_errno()Nguyễn Thái Ngọc Duy2016-05-09
|/ | | | | Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* strbuf: introduce starts_with() and ends_with()Christian Couder2013-12-05
| | | | | | | | | | | | | | | | | | | | | | | prefixcmp() and suffixcmp() share the common "cmp" suffix that typically are used to name functions that can be used for ordering, but they can't, because they are not antisymmetric: prefixcmp("foo", "foobar") < 0 prefixcmp("foobar", "foo") == 0 We in fact do not use these functions for ordering. Replace them with functions that just check for equality. Add starts_with() and end_with() that will be used to replace prefixcmp() and suffixcmp(), respectively, as the first step. These are named after corresponding functions/methods in programming languages, like Java, Python and Ruby. In vcs-svn/fast_export.c, there was already an ends_with() function that did the same thing. Let's use the new one instead while at it. Signed-off-by: Christian Couder <chriscool@tuxfamily.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* remote-svn: add incremental importFlorian Achleitner2012-10-07
| | | | | | | | | | | | | | | | | | | Search for a note attached to the ref to update and read it's 'Revision-number:'-line. Start import from the next svn revision. If there is no next revision in the svn repo, svnrdump terminates with a message on stderr an non-zero return value. This looks a little weird, but there is no other way to know whether there is a new revision in the svn repo. On the start of an incremental import, the parent of the first commit in the fast-import stream is set to the branch name to update. All following commits specify their parent by a mark number. Previous mark files are currently not reused. Signed-off-by: Florian Achleitner <florian.achleitner.2.6.31@gmail.com> Acked-by: David Michael Barr <b@rr-dav.id.au> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* Create a note for every imported commit containing svn metadataFlorian Achleitner2012-10-07
| | | | | | | | | | | | | | | | | | | To provide metadata from svn dumps for further processing, e.g. branch detection, attach a note to each imported commit that stores additional information. The notes are currently hard-coded in refs/notes/svn/revs. Currently the following lines from the svn dump are directly accumulated in the note. This can be refined as needed. - "Revision-number" - "Node-path" - "Node-kind" - "Node-action" - "Node-copyfrom-path" - "Node-copyfrom-rev" Signed-off-by: Florian Achleitner <florian.achleitner.2.6.31@gmail.com> Acked-by: David Michael Barr <b@rr-dav.id.au> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* vcs-svn: add fast_export_note to create notesDmitry Ivankov2012-10-07
| | | | | | | | | | | | | | | | | | | | | | | fast_export lacked a method to writes notes to fast-import stream. Add two new functions fast_export_note which is similar to fast_export_modify. And also add fast_export_buf_to_data to be able to write inline blobs that don't come from a line_buffer or from delta application. To be used like this: fast_export_begin_commit("refs/notes/somenotes", ...) fast_export_note("refs/heads/master", "inline") fast_export_buf_to_data(&data) or maybe fast_export_note("refs/heads/master", sha1) Signed-off-by: Dmitry Ivankov <divanorama@gmail.com> Signed-off-by: Florian Achleitner <florian.achleitner.2.6.31@gmail.com> Acked-by: David Michael Barr <b@rr-dav.id.au> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* remote-svn, vcs-svn: Enable fetching to private refsFlorian Achleitner2012-10-07
| | | | | | | | | | | | | | | The reference to update by the fast-import stream is hard-coded. When fetching from a remote the remote-helper shall update refs in a private namespace, i.e. a private subdir of refs/. This namespace is defined by the 'refspec' capability, that the remote-helper advertises as a reply to the 'capabilities' command. Extend svndump and fast-export to allow passing the target ref. Update svn-fe to be compatible. Signed-off-by: Florian Achleitner <florian.achleitner.2.6.31@gmail.com> Acked-by: David Michael Barr <b@rr-dav.id.au> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* Add svndump_init_fd to allow reading dumps from arbitrary FDsFlorian Achleitner2012-10-07
| | | | | | | | | | The existing function only allows reading from a filename or from stdin. Allow passing of a FD and an additional FD for the back report pipe. This allows us to retrieve the name of the pipe in the caller. Signed-off-by: Florian Achleitner <florian.achleitner.2.6.31@gmail.com> Acked-by: David Michael Barr <b@rr-dav.id.au> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* Merge branch 'jn/vcs-svn'Junio C Hamano2012-07-13
|\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | vcs-svn updates to clean-up compilation, lift 32-bit limitations, etc. * jn/vcs-svn: vcs-svn: allow 64-bit Prop-Content-Length vcs-svn: suppress a signed/unsigned comparison warning vcs-svn: suppress a signed/unsigned comparison warning vcs-svn: suppress signed/unsigned comparison warnings vcs-svn: use strstr instead of memmem vcs-svn: use constcmp instead of prefixcmp vcs-svn: simplify cleanup in apply_one_window vcs-svn: avoid self-assignment in dummy initialization of pre_off vcs-svn: drop no-op reset methods vcs-svn: suppress -Wtype-limits warning vcs-svn: allow import of > 4GiB files vcs-svn: rename check_overflow and its arguments for clarity
| * vcs-svn: allow 64-bit Prop-Content-LengthJonathan Nieder2012-07-05
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently the vcs-svn/ library only pays attention to the presence of the Prop-Content-Length field and doesn't care about its value, but some day we might care about the value. Parse it as an off_t instead of arbitrarily limiting to 32 bits for intuitiveness. So now you can import from a dump with more than 2 GiB of properties for a node. In practice that isn't likely to happen often, and this is mostly meant as a cleanup. Based-on-patch-by: David Barr <davidbarr@google.com> Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
| * vcs-svn: suppress a signed/unsigned comparison warningJonathan Nieder2012-07-05
| | | | | | | | | | | | | | | | | | All callers pass a nonnegative delta_len, so the code is already safe. Add an assertion to ensure that remains so and add a cast to keep clang and gcc -Wsign-compare from worrying. Reported-by: David Barr <davidbarr@google.com> Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
| * vcs-svn: suppress a signed/unsigned comparison warningDavid Barr2012-07-05
| | | | | | | | | | | | | | | | The preceding code checks that view->max_off is nonnegative and (off + width) fits in an off_t, so this code is already safe. Signed-off-by: David Barr <davidbarr@google.com> Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
| * vcs-svn: suppress signed/unsigned comparison warningsDavid Barr2012-07-05
| | | | | | | | | | | | | | | | | | | | | | | | These are already safe because both sides of the comparison are nonnegative. This would normally not be important because Git is not -Wsign-compare clean anyway, but we like to keep the vcs-svn/ lib to a higher standard for convenience using it in other projects. Signed-off-by: David Barr <davidbarr@google.com> Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
| * vcs-svn: use strstr instead of memmemDavid Barr2012-07-05
| | | | | | | | | | | | | | | | | | | | | | memmem is a GNU extension. Avoiding it makes the code clearer and makes it easier for projects that don't share git's compat/ code, such as the standalone svn-dump-fast-export project, to reuse the vcs-svn/ library. Signed-off-by: David Barr <davidbarr@google.com> Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
| * vcs-svn: use constcmp instead of prefixcmpDavid Barr2012-07-05
| | | | | | | | | | | | | | | | | | | | | | | | Since the length of t is already known, we can simplify a little by using memcmp() instead of strncmp() to carry out a prefix comparison. All nearby code already does this. Noticed in the standalone svn-dump-fast-export project which has not needed to implement prefixcmp() yet. Signed-off-by: David Barr <davidbarr@google.com> Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
| * vcs-svn: simplify cleanup in apply_one_windowDavid Barr2012-07-05
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently the cleanup code looks like this: free resources return 0; error_out: free resources return -1; Avoid duplicating the "free resources" part by keeping the return value in a variable and sharing code between the success and exceptional case: ret = 0; out: free resources return ret; Noticed in the svn-dump-fast-export project, where using the error() macro in void context produces a warning. Signed-off-by: David Barr <davidbarr@google.com> Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
| * vcs-svn: avoid self-assignment in dummy initialization of pre_offDavid Barr2012-07-05
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Without this change, clang complains: vcs-svn/svndiff.c:298:3: warning: Assigned value is garbage or undefined off_t pre_off = pre_off; /* stupid GCC... */ ^ ~~~~~~~ This code uses an old and common idiom for suppressing an "uninitialized variable" warning, and clang is wrong to warn about it. The idiom tells the compiler to leave the variable uninitialized, which saves a few bytes of code size, and, more importantly, allows valgrind to check at runtime that the variable is properly initialized by the time it is used. But MSVC and clang do not know that idiom, so let's avoid it in vcs-svn/ code. Initialize pre_off to -1, a recognizably meaningless value, to allow future code changes that cause pre_off to be used before it is initialized to be caught early. Signed-off-by: David Barr <davidbarr@google.com> Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
| * vcs-svn: drop no-op reset methodsDavid Barr2012-07-05
| | | | | | | | | | | | | | | | Since v1.7.5~42^2~6 (vcs-svn: remove buffer_read_string) buffer_reset() does nothing thus fast_export_reset() also. Signed-off-by: David Barr <davidbarr@google.com> Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
| * vcs-svn: suppress -Wtype-limits warningJonathan Nieder2012-02-02
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | On 32-bit architectures with 64-bit file offsets, gcc 4.3 and earlier produce the following warning: CC vcs-svn/sliding_window.o vcs-svn/sliding_window.c: In function `check_overflow': vcs-svn/sliding_window.c:36: warning: comparison is always false \ due to limited range of data type The warning appears even when gcc is run without any warning flags (PR12963). In later versions it can be reproduced with -Wtype-limits, which is implied by -Wextra. On 64-bit architectures it really is possible for a size_t not to be representable as an off_t so the check being warned about is not actually redundant. But even false positives are distracting. Avoid the warning by making the "len" argument to check_overflow a uintmax_t; no functional change intended. Reported-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk> Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
| * vcs-svn: allow import of > 4GiB filesJonathan Nieder2012-02-02
| | | | | | | | | | | | | | | | | | | | | | | | | | | | There is no reason in principle that an svn-format dump would not be able to represent a file whose length does not fit in a 32-bit integer. Use off_t consistently (instead of uint32_t) to represent file lengths so we can handle that. Most of our code is already ready to do that without this patch and already passes values of type off_t around. The type mismatch due to stragglers was noticed with gcc -Wtype-limits. Inspired-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk> Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
| * vcs-svn: rename check_overflow and its arguments for clarityRamsay Jones2012-02-02
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The canonical interpretation of a range a,b is as an interval [a,b), not [a,a+b), so this function taking argument names a and b feels unnatural. Use more explicit names "offset" and "len" to make the arguments' type and function clearer. While at it, rename the function to convey that we are making sure the sum of this offset and length do not overflow an off_t, not a size_t. [jn: split out from a patch from Ramsay Jones, then improved with advice from Thomas Rast, Dmitry Ivankov, and David Barr] Signed-off-by: Jonathan Nieder <jrnieder@gmail.com> Improved-by: Dmitry Ivankov <divanorama@gmail.com>
* | remove superfluous newlines in error messagesPete Wyckoff2012-04-30
| | | | | | | | | | | | | | | | The error handling routines add a newline. Remove the duplicate ones in error messages. Signed-off-by: Pete Wyckoff <pw@padd.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | vcs-svn: suppress a -Wtype-limits warningJonathan Nieder2012-02-02
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | On 32-bit architectures with 64-bit file offsets, gcc 4.3 and earlier produce the following warning: CC vcs-svn/sliding_window.o vcs-svn/sliding_window.c: In function `check_overflow': vcs-svn/sliding_window.c:36: warning: comparison is always false \ due to limited range of data type The warning appears even when gcc is run without any warning flags (this is gcc bug 12963). In later versions the same warning can be reproduced with -Wtype-limits, which is implied by -Wextra. On 64-bit architectures it really is possible for a size_t not to be representable as an off_t so the check this is warning about is not actually redundant. But even false positives are distracting. Avoid the warning by making the "len" argument to check_overflow a uintmax_t; no functional change intended. Reported-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk> Signed-off-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | vcs-svn: allow import of > 4GiB filesJonathan Nieder2012-02-02
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | There is no reason in principle that an svn-format dump would not be able to represent a file whose length does not fit in a 32-bit integer. Use off_t consistently to represent file lengths (in place of using uint32_t in some contexts) so we can handle that. Most svn-fe code is already ready to do that without this patch and passes values of type off_t around. The type mismatch from stragglers was noticed with gcc -Wtype-limits. While at it, tighten the parsing of the Text-content-length field to make sure it is a number and does not overflow, and tighten other overflow checks as that value is passed around and manipulated. Inspired-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk> Signed-off-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | vcs-svn: rename check_overflow arguments for clarityRamsay Jones2012-02-02
|/ | | | | | | | | | | | | Code using the argument names a and b just doesn't look right (not sure why!). Use more explicit names "offset" and "len" to make their type and meaning clearer. Also rename check_overflow() to check_offset_overflow() to clarify that we are making sure that "len" bytes beyond "offset" still fits the type to represent an offset. Signed-off-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* vcs-svn/svndiff.c: squelch false "unused" warning from gccJunio C Hamano2012-01-27
| | | | | | | | | | Curiously, pre_len given to read_length() does not trigger the same warning even though the code structure is the same. Most likely this is because read_offset() is used only once and inlining it will make gcc realize that it has a chance to do more flow analysis. Alas, the analysis is flawed, so it does not help X-<. Signed-off-by: Junio C Hamano <gitster@pobox.com>
* Merge branch 'svn-fe' of git://repo.or.cz/git/jrn into jn/svn-feJunio C Hamano2012-01-27
|\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This simplifies svn-fe a great deal and fulfills a longstanding wish: support for dumps with deltas in them, and incremental imports. The cost is that commandline usage of the svn-fe tool becomes a little more complicated since it no longer keeps state itself but instead reads blobs back from fast-import in order to copy them between revisions and apply deltas to them. Also removes a couple of custom data structures and replaces them with strbufs like other parts of Git. * 'svn-fe' of git://repo.or.cz/git/jrn: (32 commits) vcs-svn: reset first_commit_done in fast_export_init vcs-svn: do not initialize report_buffer twice vcs-svn: avoid hangs from corrupt deltas vcs-svn: guard against overflow when computing preimage length vcs-svn: cap number of bytes read from sliding view test-svn-fe: split off "test-svn-fe -d" into a separate function vcs-svn: implement text-delta handling vcs-svn: let deltas use data from preimage vcs-svn: let deltas use data from postimage vcs-svn: verify that deltas consume all inline data vcs-svn: implement copyfrom_data delta instruction vcs-svn: read instructions from deltas vcs-svn: read inline data from deltas vcs-svn: read the preimage when applying deltas vcs-svn: parse svndiff0 window header vcs-svn: skeleton of an svn delta parser vcs-svn: make buffer_read_binary API more convenient vcs-svn: learn to maintain a sliding view of a file Makefile: list one vcs-svn/xdiff object or header per line vcs-svn: avoid using ls command twice ... Conflicts: Makefile contrib/svn-fe/svn-fe.txt
| * vcs-svn: reset first_commit_done in fast_export_initDmitry Ivankov2011-06-23
| | | | | | | | | | | | | | | | | | | | | | first_commit_done has zero as a default value, but it is not reset back to zero in fast_export_init. Reset it back to zero so that each export will have proper initial state. Signed-off-by: Dmitry Ivankov <divanorama@gmail.com> Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
| * vcs-svn: do not initialize report_buffer twiceDmitry Ivankov2011-06-21
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When importing from a dump with deltas, first fast_export_init calls buffer_fdinit, and then init_report_buffer calls fdopen once again when processing the first delta. The second initialization is redundant and leaks a FILE *. Remove the redundant on-demand initialization to fix this. Initializing directly in fast_export_init is simpler and lets the caller pass an int specifying which fd to use instead of hard-coding REPORT_FILENO. Signed-off-by: Dmitry Ivankov <divanorama@gmail.com> Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
| * vcs-svn: avoid hangs from corrupt deltasJonathan Nieder2011-06-15
| | | | | | | | | | | | | | | | | | | | A corrupt Subversion-format delta can request reads past the end of the preimage. Set sliding_view::max_off so such corruption is caught when it appears rather than blocking in an impossible-to-fulfill read() when input is coming from a socket or pipe. Inspired-by: Ramkumar Ramachandra <artagnon@gmail.com> Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
| * vcs-svn: guard against overflow when computing preimage lengthJonathan Nieder2011-06-15
| | | | | | | | | | | | | | | | | | | | | | | | | | Signed integer overflow produces undefined behavior in C and off_t is a signed type. For predictable behavior, add some checks to protect in advance against overflow. On 32-bit systems ftell as called by buffer_tmpfile_prepare_to_read is likely to fail with EOVERFLOW when reading the corresponding postimage, and this patch does not fix that. So it's more of a futureproofing measure than a complete fix. Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
| * Merge branch 'db/delta-applier' into db/text-deltaJonathan Nieder2011-06-15
| |\ | | | | | | | | | | | | | | | * db/delta-applier: vcs-svn: cap number of bytes read from sliding view test-svn-fe: split off "test-svn-fe -d" into a separate function
| | * vcs-svn: cap number of bytes read from sliding viewJonathan Nieder2011-06-15
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Introduce a "max_off" field in struct sliding_view, roughly representing a maximum number of bytes that can be read from "file". If it is set to a nonnegative integer, a call to move_window() attempting to put the right endpoint beyond that offset will return an error instead. The idea is to use this when applying Subversion-format deltas to prevent reads past the end of the preimage (which has known length). Without such a check, corrupt deltas would cause svn-fe to block indefinitely when data in the input pipe is exhausted. Inspired-by: Ramkumar Ramachandra <artagnon@gmail.com> Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
| * | vcs-svn: implement text-delta handlingDavid Barr2011-05-26
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Handle input in Subversion's dumpfile format, version 3. This is the format produced by "svnrdump dump" and "svnadmin dump --deltas", and the main difference between v3 dumpfiles and the dumpfiles already handled is that these can include nodes whose properties and text are expressed relative to some other node. To handle such nodes, we find which node the text and properties are based on, handle its property changes, use the cat-blob command to request the basis blob from the fast-import backend, use the svndiff0_apply() helper to apply the text delta on the fly, writing output to a temporary file, and then measure that postimage file's length and write its content to the fast-import stream. The temporary postimage file is shared between delta-using nodes to avoid some file system overhead. The svn-fe interface needs to be more complicated to accomodate the backward flow of information from the fast-import backend to svn-fe. The backflow fd is not needed when parsing streams without deltas, though, so existing scripts using svn-fe on v2 dumps should continue to work. NEEDSWORK: generalize interface so caller sets the backflow fd, close temporary file before exiting Signed-off-by: David Barr <david.barr@cordelta.com> Signed-off-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: David Barr <david.barr@cordelta.com> Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
| * | Merge branch 'db/delta-applier' into db/text-deltaJonathan Nieder2011-05-26
| |\ \ | | |/ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * db/delta-applier: vcs-svn: let deltas use data from preimage vcs-svn: let deltas use data from postimage vcs-svn: verify that deltas consume all inline data vcs-svn: implement copyfrom_data delta instruction vcs-svn: read instructions from deltas vcs-svn: read inline data from deltas vcs-svn: read the preimage when applying deltas vcs-svn: parse svndiff0 window header vcs-svn: skeleton of an svn delta parser vcs-svn: make buffer_read_binary API more convenient vcs-svn: learn to maintain a sliding view of a file Makefile: list one vcs-svn/xdiff object or header per line Conflicts: Makefile vcs-svn/LICENSE
| | * vcs-svn: let deltas use data from preimageJonathan Nieder2011-03-28
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The copyfrom_source instruction appends data from the preimage buffer to the end of output. Its arguments are a length and an offset relative to the beginning of the source view. With this change, the delta applier is able to reproduce all 5,636,613 blobs in the early history of the ASF repository. Tested with mkfifo backflow svn-fe <svn-asf-public-r0:940166 3<backflow | git fast-import --cat-blob-fd=3 3>backflow with svn-asf-public-r0:940166 produced by whatever version of Subversion the dumps in /dump/ on svn.apache.org use (presumably 1.6.something). Improved-by: Ramkumar Ramachandra <artagnon@gmail.com> Improved-by: David Barr <david.barr@cordelta.com> Signed-off-by: Jonathan Nieder <jrnieder@gmail.com> Acked-by: Ramkumar Ramachandra <artagnon@gmail.com>
| | * vcs-svn: let deltas use data from postimageJonathan Nieder2011-03-27
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The copyfrom_target instruction copies appends data that is already present in the current output view to the end of output. (The offset argument is relative to the beginning of output produced in the current window.) The region copied is allowed to run past the end of the existing output. To support that case, copy one character at a time rather than calling memcpy or memmove. This allows copyfrom_target to be used once to repeat a string many times. For example: COPYFROM_DATA 2 COPYFROM_OUTPUT 10, 0 DATA "ab" would produce the output "ababababababababababab". Signed-off-by: Jonathan Nieder <jrnieder@gmail.com> Acked-by: Ramkumar Ramachandra <artagnon@gmail.com>
| | * vcs-svn: verify that deltas consume all inline dataJonathan Nieder2011-03-27
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | By constraining the format of deltas, we can more easily detect corruption and other breakage. Requiring deltas not to provide unconsumed data also opens the possibility of ignoring the declared amount of novel data and simply streaming the data as needed to fulfill copyfrom_data requests. Signed-off-by: Jonathan Nieder <jrnieder@gmail.com> Acked-by: Ramkumar Ramachandra <artagnon@gmail.com>
generated by cgit v1.2.1 (git 2.18.0) at 2024-12-29 09:11:24 +0000