SY> |--==> "APA" == Adrian Aichner <Adrian.Aichner(a)t-online.de&gt; writes: APA> Hello All, Hello Steve, APA> many of you must be experiencing the same problems. SY> Not if you're careful. Rephrase that to: Not if your're not using a native Windows XEmacs. Perhaps that is what you mean by being careful :-) APA> I had long wanted to write about the breakage I see. APA> Some Facts: APA> The package-index file contains APA> (package-get-update-base-entry (quote ...)) APA> forms. SY> Yes. APA> The file is not meant to be hand-edited: APA> ;; Package Index file -- Do not edit manually. APA> ;;;＠＠＠ APA> (package-get-update-base-entry (quote APA> ... SY> True. APA> Building a package adds a APA> (package-get-update-base-entry (quote ...)) APA> form at the beginning of the package-index file in the staging area. SY> Not quite correct. If the entry doesn't already exist it will get SY> written to the top of the file. However, if it does exist, the old SY> entry is overwritten. Not for me. APA> The Problem: APA> When a package-index file is read, the last APA> (package-get-update-base-entry (quote ...)) APA> form for a certain package in that file supercedes earlier ones. SY> In the normal course of events there is only one entry per package. SY> So this doesn't happen. Aha, this must be a problem with my platform then. APA> This means that the Package Release Engineer has to fiddle with the APA> package-index file or remove the package-index file and rebuild all APA> packages from scratch. SY> The only "fiddling" that I do with the package-index file is make sure SY> I have the right version of it in $STAGING. SY> Building all the packages from scratch for me is a *BIG PAIN* because SY> I would have to re-release each and every package each time I wanted SY> to release a single package. Because of the md5sums. Each time you SY> build a package you get a *different* md5sum, regardless of whether SY> anything has changed in the package. Yes, I know. APA> Two Questions: APA> 1. Is it a valuable feature for the package-index file to contain APA> multiple instances of the same package to provide a building history APA> of packages? We don't seem to have the interfaces to make use of this APA> feature today. SY> No. At least not until the package system evolves into something a SY> lot more intelligent than what we have now. OK. I guess I'll go bug-hunting. APA> 2. Shouldn't the latest addition to the package-index file supercede APA> older definitions? SY> It does. APA> Has anybody been working on these problems already? SY> Not me. I don't see a problem. APA> I think item 2 really needs fixing. SY> It ain't broke. Yes it is :-) Later, Adrian SY> -- SY> |---<Steve Youngs>---------------<GnuPG KeyID: 9E7E2820>---| SY> | XEmacs - It's not just an editor. | SY> | It's a way of life. | SY> |------------------------------------&lt;youngs(a)xemacs.org&gt;---| -- Adrian Aichner mailto:adrian＠xemacs.org http://www.xemacs.org/

SY> |--==> "APA" == Adrian Aichner <Adrian.Aichner(a)t-online.de&gt; writes: >> SY> |--==> "APA" == Adrian Aichner <Adrian.Aichner(a)t-online.de&gt; writes: APA> Hello All, Hello Steve, APA> many of you must be experiencing the same problems. SY> Not if you're careful. APA> Rephrase that to: Not if your're not using a native Windows APA> XEmacs. Perhaps that is what you mean by being careful :-) SY> Well it wasn't, but if the shoe fits... :-) SY> Seriously, Adrian, it didn't even occur to me that this could SY> be a platform specific problem. And as I don't have access to SY> any Windows machines, I probably won't be able to help, sorry. That's OK. I have stumbled around a bit and here is the story: package-info is created from package-info.in by batch-update-package-info in package-info.el. batch-hack-package-index in hack-package-index.el creates package-index.LATEST.pgp. Both defuns use insert-file-contents-literally to insert file contents. All this literal handling of file content is in vain because package-info.in is not treated as a binary file in CVS. My package-info.in files contain CRLF line-endings when checked out under Windows. The file is literally inserted into package-index.LATEST.pgp and the CR characters cause the replacement of old entries to fail. I think the resulting package-index.LATEST.pgp needs to be a binary file. Windows and MacOS, who make a distinction between text files and binary files, need to be able to create the same file as would be generated on UNIX. I can think of these reasons, why: o PGP signing o MD5 checksum o MULE characters may be embedded in package-info, at least for mule-packages. So, should we make the package-info.in files in the packages tree be of type binary in CVS? Best regards, Adrian SY> -- SY> |---<Steve Youngs>---------------<GnuPG KeyID: 9E7E2820>---| SY> | XEmacs - It's not just an editor. | SY> | It's a way of life. | SY> |------------------------------------&lt;youngs(a)xemacs.org&gt;---| -- Adrian Aichner mailto:adrian＠xemacs.org http://www.xemacs.org/

APA> So, should we make the package-info.in files in the packages APA> tree be of type binary in CVS? Stephen> No. CVS binary has different semantics from MS-DOS/Mac binary. I Stephen> think this is a dangerous idea. Please explain the danger to me. This excerpt from the CVS manual does not convince me: How to store binary files ========================= There are two issues with using CVS to store binary files. The first is that CVS by default converts line endings between the canonical form in which they are stored in the repository (linefeed only), and the form appropriate to the operating system in use on the client (for example, carriage return followed by line feed for Windows NT). The second is that a binary file might happen to contain data which looks like a keyword (*note Keyword substitution::.), so keyword expansion must be turned off. If we store the file with \n line endings in binary form, that's what we'll have inside and outside of CVS. A nuisance will be that CVS cannot merge these files for us. But XEmacs can. Stephen> Fix the files that call i-f-c-l to include text to do a Stephen> post-processing phase of undossing the buffer. Or maybe Stephen> you don't need to i-f-c-l at all, just i-f-c. Or maybe Perhaps. We would just have to make sure that the resulting package-index.LATEST.pgp gets stored with the same line-endings on all platforms packages can be built on. I presume this would be \n line endings only. Stephen> the manipulations on package-index should be done on Lisp Stephen> structures, not on buffer text. Stephen> I'm working on the package code at the moment, I'll add Stephen> that to the list. Great. I'll leave that in your trusting hands and offer to test your new code on Windows 2000. Best regards, Adrian Stephen> -- Stephen> University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN Stephen> Institute of Policy and Planning Sciences Tel/fax: +81 (298) 53-5091 Stephen> _________________ _________________ _________________ _________________ Stephen> What are those straight lines for? "XEmacs rules." -- Adrian Aichner mailto:adrian＠xemacs.org http://www.xemacs.org/

Stephen> No. CVS binary has different semantics from MS-DOS/Mac Stephen> binary. I think this is a dangerous idea. APA> Please explain the danger to me. Stephen> I prefer not to trust that the manual explains completely Stephen> what the differences are. Especially in the case of CVS; Stephen> I know that the manual is very incomplete about topics Stephen> like branches, and I know that the software itself is Stephen> buggy. And I thought you had facts :-) Stephen> I don't like relying on such software to Do The Right Stephen> Thing, when we can (and should; why are we manipulating Stephen> Lisp forms as text?) fix the problem internally. I agree that this is a better approach. Stephen> -- Stephen> University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN Stephen> Institute of Policy and Planning Sciences Tel/fax: +81 (298) 53-5091 Stephen> _________________ _________________ _________________ _________________ Stephen> What are those straight lines for? "XEmacs rules." -- Adrian Aichner mailto:adrian＠xemacs.org http://www.xemacs.org/

S> |--==> "APA" == Adrian Aichner <Adrian.Aichner(a)t-online.de&gt; writes: APA> So, should we make the package-info.in files in the packages tree be APA> of type binary in CVS? S> I have no problem with doing that. How do I do it? Short answer: csv admin -kb FILE Long answer: How to store binary files ========================= There are two issues with using CVS to store binary files. The first is that CVS by default converts line endings between the canonical form in which they are stored in the repository (linefeed only), and the form appropriate to the operating system in use on the client (for example, carriage return followed by line feed for Windows NT). The second is that a binary file might happen to contain data which looks like a keyword (*note Keyword substitution::), so keyword expansion must be turned off. The `-kb' option available with some CVS commands insures that neither line ending conversion nor keyword expansion will be done. Here is an example of how you can create a new file using the `-kb' flag: $ echo '$Id$' > kotest $ cvs add -kb -m"A test file" kotest $ cvs ci -m"First checkin; contains a keyword" kotest If a file accidentally gets added without `-kb', one can use the `cvs admin' command to recover. For example: $ echo '$Id$' > kotest $ cvs add -m"A test file" kotest $ cvs ci -m"First checkin; contains a keyword" kotest $ cvs admin -kb kotest $ cvs update -A kotest # For non-unix systems: # Copy in a good copy of the file from outside CVS $ cvs commit -m "make it binary" kotest When you check in the file `kotest' the file is not preserved as a binary file, because you did not check it in as a binary file. The `cvs admin -kb' command sets the default keyword substitution method for this file, but it does not alter the working copy of the file that you have. If you need to cope with line endings (that is, you are using CVS on a non-unix system), then you need to check in a new copy of the file, as shown by the `cvs commit' command above. On unix, the `cvs update -A' command suffices. However, in using `cvs admin -k' to change the keyword expansion, be aware that the keyword expansion mode is not version controlled. This means that, for example, that if you have a text file in old releases, and a binary file with the same name in new releases, CVS provides no way to check out the file in text or binary mode depending on what version you are checking out. There is no good workaround for this problem. You can also set a default for whether `cvs add' and `cvs import' treat a file as binary based on its name; for example you could say that files who names end in `.exe' are binary. *Note Wrappers::. There is currently no way to have CVS detect whether a file is binary based on its contents. The main difficulty with designing such a feature is that it is not clear how to distinguish between binary and non-binary files, and the rules to apply would vary considerably with the operating system.

|--==> "APA" == Adrian Aichner <Adrian.Aichner(a)t-online.de&gt; writes: SY> |--==> "APA" == Adrian Aichner <Adrian.Aichner(a)t-online.de&gt; writes: APA> So, should we make the package-info.in files in the packages tree be APA> of type binary in CVS? SY> I have no problem with doing that. How do I do it? APA> Hi Steve, Stephen is actively working to fix this problem. APA> I am convinced now, that storing package-info.in files in binary form APA> is a bad thing. Please let's not do it! Okay. I'll scratch it off my todo list. -- |---<Steve Youngs>---------------<GnuPG KeyID: 9E7E2820>---| | XEmacs - It's not just an editor. | | It's a way of life. | |------------------------------------&lt;youngs(a)xemacs.org&gt;---|

|--==> "Hrvoje" == Hrvoje Niksic <hniksic(a)arsdigita.com&gt; writes: Hrvoje> Steve Youngs <youngs(a)xemacs.org&gt; writes: Hrvoje> a. Why? Personally, I blame GNU/tar. Try doing this: tar cvzf test.tar.gz /some/directory/ md5sum test.tar.gz > test.md5 rm test.tar.gz tar cvzf test.tar.gz /some/directory/ md5sum test.tar.gz >> test.md5 cat test.md5 For some reason you'll have two different md5's. Hrvoje> b. Can that be changed? Don't know. -- |---<Steve Youngs>---------------<GnuPG KeyID: 9E7E2820>---| | XEmacs - It's not just an editor. | | It's a way of life. | |------------------------------------&lt;youngs(a)xemacs.org&gt;---|

Adrian.Aichner(a)t-online.de (Adrian Aichner) writes: Yes. See my response to Steven.

Steve Youngs <youngs(a)xemacs.org&gt; writes: The first fault in the reasoning is that you're actually using two programs: tar and gzip. If I try the test like this: tar cf x.tar test-dir tar cf y.tar test-dir ...the resulting files are the same. However, if I do this: tar czf x.tar.gz test-dir tar czf y.tar.gz test-dir ...they differ. The problem is that `gzip' "helpfully" adds a timestamp to the file, even it is completely unneeded for TAR files which keep their own timestamp information. Hopefully, there is a flag that removes this. So if I try: tar cf - test-dir | gzip -cn > x.tar.gz tar cf - test-dir | gzip -cn > y.tar.gz ...the files are the same. Does this work for you?