[PATCH] Re: coding system oddity
14 years, 9 months
Aidan Kehoe
Aidan Kehoe
Ar an seachtú lá de mí Meitheamh, scríobh Stephen J. Turnbull:
> Aidan Kehoe writes:
> > Ar an seachtú lá de mí Meitheamh, scríobh Stephen J. Turnbull:
>
> > > Ben's original concept for the new detection mechanism was to assign
> > > likelihoods and pick the best of the matching coding systems.
> > > However, it's possible that there would be no match or a tie
> > > (depending on how the algorithm is tuned), in which case it might be
> > > natural for the result to be undecided.
> > >
> > > What do you think, Aidan?
>
> > It may be that coding_stream_detected_coding_system *should* never
> > give undecided, even if no data have been seen, in which case the
> > correct thing to do to fix this bug is to change
> > coding_stream_detected_coding_system to do that.
>
> I guess it's fair to say your answer is "I'm not ready to say yet", then?
‘Yet’ is perhaps a bit strong. Ben might be able to answer the question, if
he remembers. He may not, and he may not answer the question if we ask for
reasons independent of that. If either of these is the case, the thing is
unanswerable, and the decision as to the semantics of c_s_d_c_s is ours.
> My personal take is that it is a good idea to allow c_s_d_c_s to
> return undecided, because the final decision should be made at the
> Lisp level, not in C. I think your fix is in the right place, even if
> it's not the final version.
>
> > It may also be that undecided is a reasonable value there. It’s just
> > not clear, and my patch as written is minimally invasive and fixes the
> > issue.
>
> > Your warning below risks having people think that autodetection
> > succeeding means that the characters in the buffer reflect what
> > whoever wrote the octets to disk meant them to mean, which is not
> > true, our autodetection gets things wrong all the time.
>
> That's always true, though. Ie, AFAIK currently autodetection will
> always return a system that can save all the characters in the buffer
> at the time, if it returns a coding system.
No; this used to be especially not true. It’s less not true now, but it is
still the case that the autodetection code may well return a coding system
that will lose data if that coding system is used to decode and then encode
the entire file. Consider that the UTF-8 coding systems used to lose data in
this context all the time, despite that the autodetection code correctly
recognised UTF-8; and that the ISO-2022-1 coding systems do not preserve
some distinct error sequences on decoding followed by encoding.
> [...] That's why I want the warning.
OK; I disagree with you, but the downside from doing it your way is not that
significant, let’s go with that.
--
¿Dónde estará ahora mi sobrino Yoghurtu Nghe, que tuvo que huir
precipitadamente de la aldea por culpa de la escasez de rinocerontes?
_______________________________________________
XEmacs-Patches mailing list
XEmacs-Patches(a)xemacs.org
http://calypso.tux.org/cgi-bin/mailman/listinfo/xemacs-patches
- 1 participants
- 2 comments
[PATCH (pkgs)] Dired fixes for Fedora
14 years, 9 months
Jerry James
Jerry James
PATCH packages
I'm considering adding the following patch to Fedora's package build.
The first hunk fixes the problem that dired doesn't pass the --dired
switch to ls on Fedora. The second hunk deals with a new version of
GNU coreutils that prints a dot just after the mode bits to indicate
that the entity has an SELinux security context, but no other
alternate access method (i.e., no ACLs). I'll add an appropriate
ChangeLog entry later. Right now, cvs.xemacs.org won't talk to me, so
I can't do a new cvs diff.
I sent email about both problems with dired-report-bug on June 23 and
have yet to receive a response.
Index: xemacs-packages/dired/dired.el
===================================================================
RCS file: /pack/xemacscvs/XEmacs/packages/xemacs-packages/dired/dired.el,v
retrieving revision 1.11
diff -d -u -r1.11 dired.el
--- xemacs-packages/dired/dired.el 13 Jan 2008 14:51:20 -0000 1.11
+++ xemacs-packages/dired/dired.el 24 Jun 2009 16:33:11 -0000
＠＠ -219,7 +219,7 ＠＠
;;;###autoload
(defcustom dired-use-ls-dired
- (and (string-match "gnu" system-configuration)
+ (and (string-match "gnu\\|linux" system-configuration)
;; Only supported for XEmacs >= 21.5 and GNU Emacs >= 21.4 (I think)
(or (and (featurep 'xemacs)
(fboundp 'emacs-version>=)
＠＠ -2849,7 +2849,7 ＠＠
(if (save-excursion
(goto-char bol)
(re-search-forward
- "[^ ][-r][-w][^ ][-r][-w][^ ][-r][-w][^ ][-+ 0-9＠]"
+ "[^ ][-r][-w][^ ][-r][-w][^ ][-r][-w][^ ][-+ 0-9＠.]"
bof t))
(progn
(setq modes-start (match-beginning 0)
--
Jerry James
http://www.jamezone.org/
_______________________________________________
XEmacs-Patches mailing list
XEmacs-Patches(a)xemacs.org
http://calypso.tux.org/cgi-bin/mailman/listinfo/xemacs-patches
- 3 participants
- 4 comments
Re: [COMMIT] Check CCL available at run time, not compile time, in PGG.
14 years, 9 months
Aidan Kehoe
Aidan Kehoe
Ar an dara lá de mí Iúil, scríobh Jerry James:
> On Sun, Jan 25, 2009 at 11:46 PM, Aidan Kehoe<kehoea(a)parhasard.net> wrote:
> >
> > Ar an cúigiú lá is fiche de mí Eanair, scríobh Vin Shelton:
> >
> > > Where is register-ccl-program defined?
> >
> > mule-ccl.c. Whoops. I won’t get to fixing that this week, sorry.
>
> Did this patch ever get repaired, or are we still missing pgg-parse-crc24?
We’re still missing it.
--
¿Dónde estará ahora mi sobrino Yoghurtu Nghe, que tuvo que huir
precipitadamente de la aldea por culpa de la escasez de rinocerontes?
_______________________________________________
XEmacs-Patches mailing list
XEmacs-Patches(a)xemacs.org
http://calypso.tux.org/cgi-bin/mailman/listinfo/xemacs-patches
- 1 participants
- 0 comments
[PATCH 21.4] Large image crashes
14 years, 9 months
Jerry James
Jerry James
PATCH 21.4
Here is the 21.4 version of the previous patch. The lisp.h hunk is a
little ugly. Recommendations on how to deal with the absence of
UINT_64_T in 21.4 are welcome.
Index: src/ChangeLog
===================================================================
RCS file: /pack/xemacscvs/XEmacs/xemacs/src/ChangeLog,v
retrieving revision 1.290.2.127
diff -d -u -r1.290.2.127 ChangeLog
--- src/ChangeLog 2009/02/01 00:42:58 1.290.2.127
+++ src/ChangeLog 2009/07/01 22:36:04
＠＠ -1,3 +1,12 ＠＠
+2009-06-09 Jerry James <james(a)xemacs.org>
+
+ * lisp.h: Define UINT_64_BIT the way 21.5 does.
+ * glyphs-eimage.c (jpeg_instantiate):
+ (gif_instantiate):
+ (png_instantiate):
+ (tiff_instantiate): Check for integer overflow before allocating
+ memory for an image.
+
2009-01-31 Vin Shelton <acs(a)xemacs.org>
* regex.c (re_search_2): Really apply Julian Bradfield's patch for
Index: src/glyphs-eimage.c
===================================================================
RCS file: /pack/xemacscvs/XEmacs/xemacs/src/glyphs-eimage.c,v
retrieving revision 1.9.2.5
diff -d -u -r1.9.2.5 glyphs-eimage.c
--- src/glyphs-eimage.c 2007/08/20 19:52:28 1.9.2.5
+++ src/glyphs-eimage.c 2009/07/01 22:36:04
＠＠ -407,6 +407,7 ＠＠
*/
{
+ UINT_64_BIT pixels_sq;
int jpeg_gray = 0; /* if we're dealing with a grayscale */
/* Step 4: set parameters for decompression. */
＠＠ -429,7 +430,10 ＠＠
jpeg_start_decompress (&cinfo);
/* Step 6: Read in the data and put into EImage format (8bit RGB triples)*/
-
+ pixels_sq =
+ (UINT_64_BIT) cinfo.output_width * (UINT_64_BIT) cinfo.output_height;
+ if (pixels_sq > ((size_t) -1) / 3)
+ signal_image_error ("JPEG image too large to instantiate", instantiator);
unwind.eimage = (unsigned char*) xmalloc (cinfo.output_width *
cinfo.output_height * 3);
if (!unwind.eimage)
signal_image_error("Unable to allocate enough memory for
image", instantiator);
＠＠ -671,6 +675,7 ＠＠
{
ColorMapObject *cmo = unwind.giffile->SColorMap;
int i, j, row, pass, interlace, slice;
+ UINT_64_BIT pixels_sq;
unsigned char *eip;
/* interlaced gifs have rows in this order:
0, 8, 16, ..., 4, 12, 20, ..., 2, 6, 10, ..., 1, 3, 5, ... */
＠＠ -679,6 +684,9 ＠＠
height = unwind.giffile->SHeight;
width = unwind.giffile->SWidth;
+ pixels_sq = (UINT_64_BIT) width * (UINT_64_BIT) height;
+ if (pixels_sq > ((size_t) -1) / (3 * unwind.giffile->ImageCount))
+ signal_image_error ("GIF image too large to instantiate", instantiator);
unwind.eimage = (unsigned char*)
xmalloc (width * height * 3 * unwind.giffile->ImageCount);
if (!unwind.eimage)
＠＠ -937,11 +945,15 ＠＠
{
int y;
unsigned char **row_pointers;
+ UINT_64_BIT pixels_sq;
height = info_ptr->height;
width = info_ptr->width;
+ pixels_sq = (UINT_64_BIT) width * (UINT_64_BIT) height;
+ if (pixels_sq > ((size_t) -1) / 3)
+ signal_image_error ("PNG image too large to instantiate", instantiator);
/* Wow, allocate all the memory. Truly, exciting. */
- unwind.eimage = xnew_array_and_zero (unsigned char, width * height * 3);
+ unwind.eimage = xnew_array_and_zero (unsigned char, pixels_sq * 3);
/* libpng expects that the image buffer passed in contains a
picture to draw on top of if the png has any transparencies.
This could be a good place to pass that in... */
＠＠ -994,7 +1006,7 ＠＠
png_set_expand (png_ptr);
/* send grayscale images to RGB too */
if (info_ptr->color_type == PNG_COLOR_TYPE_GRAY ||
- info_ptr->color_type == PNG_COLOR_TYPE_GRAY_ALPHA)
+ info_ptr->color_type == PNG_COLOR_TYPE_GRAY_ALPHA)
png_set_gray_to_rgb (png_ptr);
/* we can't handle alpha values */
if (info_ptr->color_type & PNG_COLOR_MASK_ALPHA)
＠＠ -1268,6 +1280,7 ＠＠
uint32 *raster;
unsigned char *ep;
+ UINT_64_BIT pixels_sq;
assert (!NILP (data));
＠＠ -1290,12 +1303,15 ＠＠
TIFFGetField (unwind.tiff, TIFFTAG_IMAGEWIDTH, &width);
TIFFGetField (unwind.tiff, TIFFTAG_IMAGELENGTH, &height);
- unwind.eimage = (unsigned char *) xmalloc (width * height * 3);
+ pixels_sq = (UINT_64_BIT) width * (UINT_64_BIT) height;
+ if (pixels_sq >= 1 << 29)
+ signal_image_error ("TIFF image too large to instantiate", instantiator);
+ unwind.eimage = (unsigned char *) xmalloc (pixels_sq * 3);
/* #### This is little more than proof-of-concept/function testing.
It needs to be reimplemented via scanline reads for both memory
compactness. */
- raster = (uint32*) _TIFFmalloc (width * height * sizeof (uint32));
+ raster = (uint32*) _TIFFmalloc ((tsize_t) (pixels_sq * sizeof (uint32)));
if (raster != NULL)
{
int i,j;
Index: src/lisp.h
===================================================================
RCS file: /pack/xemacscvs/XEmacs/xemacs/src/lisp.h,v
retrieving revision 1.45.2.10
diff -d -u -r1.45.2.10 lisp.h
--- src/lisp.h 2005/02/01 03:55:02 1.45.2.10
+++ src/lisp.h 2009/07/01 22:36:05
＠＠ -265,6 +265,11 ＠＠
/*#define REGISTER register*/
/*#endif*/
+#if SIZEOF_LONG == 8
+#define UINT_64_BIT unsigned long
+#elif SIZEOF_LONG_LONG == 8
+#define UINT_64_BIT unsigned long long
+#endif
/* EMACS_INT is the underlying integral type into which a Lisp_Object must fit.
In particular, it must be large enough to contain a pointer.
--
Jerry James
http://www.jamezone.org/
_______________________________________________
XEmacs-Patches mailing list
XEmacs-Patches(a)xemacs.org
http://calypso.tux.org/cgi-bin/mailman/listinfo/xemacs-patches
- 1 participants
- 0 comments
[AC 21.5] Large image crashes
14 years, 9 months
Jerry James
Jerry James
APPROVE COMMIT 21.5
This patch has been committed. When trying to open very large image
files (or image files with header files that claim the image is very
large), we multiply length times width to get the number of pixels in
the image (and possibly multiply that number if a pixel occupies more
than 1 byte). The multiplication can overflow, resulting in passing
negative or insufficiently positive size values to malloc. This patch
checks whether the multiplication will overflow. If so, XEmacs
refuses to attempt to load the image.
diff -r f2a991ff6db0 src/ChangeLog
--- a/src/ChangeLog Mon Jun 29 08:20:47 2009 -0600
+++ b/src/ChangeLog Wed Jul 01 15:42:54 2009 -0600
＠＠ -1,3 +1,11 ＠＠
+2009-06-09 Jerry James <james(a)xemacs.org>
+
+ * glyphs-eimage.c (jpeg_instantiate):
+ (gif_instantiate):
+ (png_instantiate):
+ (tiff_instantiate): Check for integer overflow before allocating
+ memory for an image.
+
2009-06-20 Stephen Turnbull <stephen(a)xemacs.org>
* callint.c (Finteractive): Document that (interactive) must
diff -r f2a991ff6db0 src/glyphs-eimage.c
--- a/src/glyphs-eimage.c Mon Jun 29 08:20:47 2009 -0600
+++ b/src/glyphs-eimage.c Wed Jul 01 15:42:54 2009 -0600
＠＠ -409,6 +409,7 ＠＠
*/
{
+ UINT_64_BIT pixels_sq;
int jpeg_gray = 0; /* if we're dealing with a grayscale */
/* Step 4: set parameters for decompression. */
＠＠ -431,7 +432,10 ＠＠
jpeg_start_decompress (&cinfo);
/* Step 6: Read in the data and put into EImage format (8bit RGB triples)*/
-
+ pixels_sq =
+ (UINT_64_BIT) cinfo.output_width * (UINT_64_BIT) cinfo.output_height;
+ if (pixels_sq > ((size_t) -1) / 3)
+ signal_image_error ("JPEG image too large to instantiate", instantiator);
unwind.eimage =
xnew_binbytes (cinfo.output_width * cinfo.output_height * 3);
if (!unwind.eimage)
＠＠ -677,6 +681,7 ＠＠
{
ColorMapObject *cmo = unwind.giffile->SColorMap;
int i, j, row, pass, interlace, slice;
+ UINT_64_BIT pixels_sq;
Binbyte *eip;
/* interlaced gifs have rows in this order:
0, 8, 16, ..., 4, 12, 20, ..., 2, 6, 10, ..., 1, 3, 5, ... */
＠＠ -685,6 +690,9 ＠＠
height = unwind.giffile->SHeight;
width = unwind.giffile->SWidth;
+ pixels_sq = (UINT_64_BIT) width * (UINT_64_BIT) height;
+ if (pixels_sq > ((size_t) -1) / (3 * unwind.giffile->ImageCount))
+ signal_image_error ("GIF image too large to instantiate", instantiator);
unwind.eimage =
xnew_binbytes (width * height * 3 * unwind.giffile->ImageCount);
if (!unwind.eimage)
＠＠ -948,11 +956,15 ＠＠
{
int y;
Binbyte **row_pointers;
+ UINT_64_BIT pixels_sq;
height = info_ptr->height;
width = info_ptr->width;
+ pixels_sq = (UINT_64_BIT) width * (UINT_64_BIT) height;
+ if (pixels_sq > ((size_t) -1) / 3)
+ signal_image_error ("PNG image too large to instantiate", instantiator);
/* Wow, allocate all the memory. Truly, exciting. */
- unwind.eimage = xnew_array_and_zero (Binbyte, width * height * 3);
+ unwind.eimage = xnew_array_and_zero (Binbyte, (size_t) (pixels_sq * 3));
/* libpng expects that the image buffer passed in contains a
picture to draw on top of if the png has any transparencies.
This could be a good place to pass that in... */
＠＠ -1299,6 +1311,7 ＠＠
uint32 *raster;
Binbyte *ep;
+ UINT_64_BIT pixels_sq;
assert (!NILP (data));
＠＠ -1321,12 +1334,15 ＠＠
TIFFGetField (unwind.tiff, TIFFTAG_IMAGEWIDTH, &width);
TIFFGetField (unwind.tiff, TIFFTAG_IMAGELENGTH, &height);
- unwind.eimage = xnew_binbytes (width * height * 3);
+ pixels_sq = (UINT_64_BIT) width * (UINT_64_BIT) height;
+ if (pixels_sq >= 1 << 29)
+ signal_image_error ("TIFF image too large to instantiate", instantiator);
+ unwind.eimage = xnew_binbytes (pixels_sq * 3);
/* #### This is little more than proof-of-concept/function testing.
It needs to be reimplemented via scanline reads for both memory
compactness. */
- raster = (uint32*) _TIFFmalloc (width * height * sizeof (uint32));
+ raster = (uint32*) _TIFFmalloc ((tsize_t) (pixels_sq * sizeof (uint32)));
if (raster != NULL)
{
int i, j;
--
Jerry James
http://www.jamezone.org/
_______________________________________________
XEmacs-Patches mailing list
XEmacs-Patches(a)xemacs.org
http://calypso.tux.org/cgi-bin/mailman/listinfo/xemacs-patches
- 1 participants
- 0 comments