10:01 p.m.
Don't get too excited. This isn't *that* large file support. A
recent discussion on the Fedora devel list pointed out that btrfs is
now using 64 bit inode numbers even on 32-bit systems, and so some
stat() calls are failing spuriously (with EOVERFLOW) just because the
inode number cannot be stored in the 32-bit stat structure, even if
the caller is not interested in the inode number.
The attached patch is intended to avoid this problem by building with
_FILE_OFFSET_BITS == 64 when possible, and by recognizing that certain
integer values may overflow a Lisp fixnum. There is only so much that
can be done about the latter problem on a system without bignums, and
this patch doesn't fix all of those problems. However, on 32-bit
systems with bignums, this patch improves the situation.
--
Jerry James
http://www.jamezone.org/
_______________________________________________
XEmacs-Patches mailing list
XEmacs-Patches(a)xemacs.org
http://lists.xemacs.org/mailman/listinfo/xemacs-patches
7:56 a.m.
New subject: [A] Large file support
Looks good.
--
Regards,
Mike
_______________________________________________
XEmacs-Patches mailing list
XEmacs-Patches(a)xemacs.org
http://lists.xemacs.org/mailman/listinfo/xemacs-patches
8:20 a.m.
Jerry James writes:
The attached patch is intended to avoid this problem by building
with
_FILE_OFFSET_BITS == 64 when possible,
"When possible" doesn't seem like the right thing to do, at least if
that's defined by "compiler supports a 64-bit integer type." But
AFAICS what the patch actually does anyway is use off_t where
available, which seems to be TRT. Right?
and by recognizing that certain integer values may overflow a Lisp
fixnum. There is only so much that can be done about the latter
problem on a system without bignums, and this patch doesn't fix all
of those problems. However, on 32-bit systems with bignums, this
patch improves the situation.
Is there any harm in allowing bignums in *all* fields? It seems to me
that on most installations we'll never see bignums, so there's no
space penalty. And since we need to check numerical type anyway for
any numerical manipulation (including comparison -- they might be
floats), there should be no performance penalty period.
+dnl check for large file support
+AC_SYS_LARGEFILE
+AC_FUNC_FSEEKO
+AC_CHECK_SIZEOF(off_t)
What happens if off_t isn't supported by the system? (See definition
of OFF_T macro below.)
diff -r 1af0602ff9a2 lib-src/insert-data-in-exec.c
＠＠ -55,8 +56,8 ＠＠
{
FILE *te, *xe, *dump;
unsigned char *xed, *p;
- long size, size_dump, size1, i;
- long max_size, offset;
+ size_t size1, offset;
+ OFF_T size, size_dump, max_size, i;
Why would these be off_t and not size_t? Just because they're being
added to a ptr_t doesn't seem right. AFAICS they don't get passed to
FSEEK or compared to the value of FTELL.
+ printf("%" PRIdOFF "\n", size);
Doncha just *love* C?
+ size1 = fread(xed, 1, (size_t) size, te);
+ if(size1 != (size_t) size) {
Wink, wink, nudge, nudge, say no more, eh?
diff -r 1af0602ff9a2 src/config.h.in
--- a/src/config.h.in Mon Mar 04 10:24:55 2013 -0700
+++ b/src/config.h.in Mon Mar 04 14:53:03 2013 -0700
＠＠ -757,6 +757,31 ＠＠
#undef SIZEOF_LONG_LONG
#undef SIZEOF_VOID_P
#undef SIZEOF_DOUBLE
+#undef SIZEOF_OFF_T
+
+/* Large file support */
+#undef AC_FUNC_FSEEKO
+#ifdef AC_FUNC_FSEEKO
+# define OFF_T off_t
+# define FSEEK(stream, offset, whence) fseeko (stream, offset, whence)
+# define FTELL(stream) ftello (stream)
+#else
+# define OFF_T long
This is why I'm worried about off_t being supported or not.
+# define FSEEK(stream, offset, whence) fseek (stream, offset,
whence)
+# define FTELL(stream) ftell (stream)
+#endif
+#if SIZEOF_OFF_T == SIZEOF_INT
+# define PRIdOFF "d"
+# define PRIxOFF "x"
+#elif SIZEOF_OFF_T == SIZEOF_LONG
+# define PRIdOFF "ld"
+# define PRIxOFF "lx"
+#elif SIZEOF_OFF_T == SIZEOF_LONG_LONG
+# define PRIdOFF "lld"
+# define PRIxOFF "llx"
+#else
+# error cannot determine how to print an off_t
Doesn't this happen if off_t isn't defined by the system?
+#endif
_______________________________________________
XEmacs-Patches mailing list
XEmacs-Patches(a)xemacs.org
http://lists.xemacs.org/mailman/listinfo/xemacs-patches
11:46 p.m.
On Tue, Mar 5, 2013 at 1:20 AM, Stephen J. Turnbull <stephen(a)xemacs.org> wrote:
"When possible" doesn't seem like the right thing to
do, at least if
that's defined by "compiler supports a 64-bit integer type." But
AFAICS what the patch actually does anyway is use off_t where
available, which seems to be TRT. Right?
Sorry, what I meant by "when possible" is "when the C library knows
what _FILE_OFFSET_BITS is and handles it as defined by POSIX".
Is there any harm in allowing bignums in *all* fields? It seems to
me
that on most installations we'll never see bignums, so there's no
space penalty. And since we need to check numerical type anyway for
any numerical manipulation (including comparison -- they might be
floats), there should be no performance penalty period.
No harm. You're right, the uid and gid fields may also overflow a
Lisp integer. I'll make that change.
What happens if off_t isn't supported by the system? (See
definition
of OFF_T macro below.)
Then configure defines it to be "long int". This is part of the
expansion of AC_TYPE_OFF_T (line 2643 of configure.ac), which is
before the added AC_CHECK_SIZEOF(off_t) on line 2727.
> diff -r 1af0602ff9a2 lib-src/insert-data-in-exec.c
> ＠＠ -55,8 +56,8 ＠＠
> {
> FILE *te, *xe, *dump;
> unsigned char *xed, *p;
> - long size, size_dump, size1, i;
> - long max_size, offset;
> + size_t size1, offset;
> + OFF_T size, size_dump, max_size, i;
Why would these be off_t and not size_t? Just because they're being
added to a ptr_t doesn't seem right. AFAICS they don't get passed to
FSEEK or compared to the value of FTELL.
Let's see ... size and size_dump are assigned the return value of
FTELL(), which might really be ftello(), so an off_t. Of course, both
are passed to read(), where they need to be size_t, as well as
compared to size1, which is a size_t. So we could make them size_t,
if you prefer. We should really check that the return value of
FTELL() is not -1; if it isn't, then casting it to a size_t is fine.
max_size is OFF_T only because it is compared to size_dump, so if we
change size_dump to size_t, then max_size should change, too.
Finally, i is compared to an expression involving size. So I really
have no objection to changing them to size_t. Check the attached
patch, where I've done just that.
Doncha just *love* C?
http://www.netfunny.com/rhf/jokes/90q2/ode.html
Need I say more?
--
Jerry James
http://www.jamezone.org/
_______________________________________________
XEmacs-Patches mailing list
XEmacs-Patches(a)xemacs.org
http://lists.xemacs.org/mailman/listinfo/xemacs-patches
3:57 a.m.
Jerry James writes:
No harm. You're right, the uid and gid fields may also overflow
a
Lisp integer. I'll make that change.
Great! BTW, kudos to whoever designed the make_integer
vs. make_fixnum API. It's perfect for this kind of thing, looks nice
either way.
> What happens if off_t isn't supported by the system? (See
definition
> of OFF_T macro below.)
Then configure defines it to be "long int". This is part of the
expansion of AC_TYPE_OFF_T (line 2643 of configure.ac), which is
before the added AC_CHECK_SIZEOF(off_t) on line 2727.
I suspected it would be like that, but you never know with autoconf. :-(
Check the attached patch, where I've done just that.
It may be just me :-), but I think it reads a lot better.
Thanks for putting up with my crankiness!
Steve
_______________________________________________
XEmacs-Patches mailing list
XEmacs-Patches(a)xemacs.org
http://lists.xemacs.org/mailman/listinfo/xemacs-patches
4321
days inactive
4323
days old
4 comments
3 participants
participants (3)
-
Jerry James -
Michael Sperber -
Stephen J. Turnbull