9:53 a.m.
I've been doing too much alignment hacking lately. But I still have
trouble understanding how pdump alignment works.
Olivier, it seems to me that:
- objects should be aligned when they are written. dumper.c still
occasionally (even after my recent proposed patch) adds alignment
_after_ writing an object. This requires knowledge of the next
object. Bad. If you know the alignment of an object, you can write it to
a stdio stream aligned by using ftell() to get the current offset in
the file. pdump_dump_data() aligns objects after they are written.
- every struct and Lisp_Object type should have an alignment, not just
a size, stored with its description. Currently pdump doesn't have
enough information to correctly align every object. We can do this
accurately now because we have a working ALIGNOF macro. This
matters spacewise because on some platforms everything has alignment
of 4, except for doubles, which have alignment 8. Almost all
Lisp_Objects require storage alignment of 4 except for things like
Lisp_Float. We shouldn't have to align to max_align_t.
- pdump_add_entry() uses pdump_align_table to get possible alignment
of an object. But it seems to use the _size_, not the _address_ of
the object to do this. I would like to simply use ALIGNOF, but the
actual type of the object seems not to be available here. See
above.
Please comment. I would like to further nuke some of the current
alignment code, like this cruft at the end of pdump_dump_data():
write (pdump_fd, desc ? pdump_buf : elmt->obj, size*count);
if (elmt->is_lrecord && ((size*count) & 3)
write (pdump_fd, "\0\0\0", 4-((size*count) & 3));
(This is partially fixed by my other patch with PDUMP_ALIGN_STREAM, but...)
Why check is_lrecord? Don't structs have similar alignment problems?
12:24 p.m.
[Martin, I bounced you original message to xemacs-beta, I hope you
don't mind]
On Mon, Jan 22, 2001 at 06:53:36PM +0900, Martin Buchholz wrote:
I've been doing too much alignment hacking lately. But I still
have
trouble understanding how pdump alignment works.
I'm using these facts of life:
1- Alignments are powers of two
2- sizeof() is required to return a properly-aligned size (that's in
ANSI)
By properly aligned size I mean if p is aligned for the type, p_size
is aligned too.
This means that you can get an upper-bound for the alignement of a
given structure from its size by finding the higher power of two that
is a divisor of the size. That's the point of the pdump_align_table
array. The table goes all the way to 2^8 which is overkill though.
We should stop at 2^3 or maybe 2^4 (and use a 8 or 16 elements table).
The other trick I use is to dump the data ordered by decreasing
alignment constraint. After all, if something is aligned for 8 bytes,
it is aligned for 4, 2 and 1 too. And by (2) we know that if we're
aligned for 8 bytes, and we write an 8-bytes aligned structure, we
will still be 8-bytes aligned afterwards.
So writing the data by decreasing alignment constraints ensures the
alignment at zero cost in padding.
This "scaning by decreasing alignment constraint" thingy is done by
pdump_scan_by_alignment btw.
That would be nice and cool, except that there is one exception. Some
lrecords are lying a little. Well, to be more precise, they give
sizes that are not the result of a sizeof, and don't necessarily
respect (2). In particular, the opaques do that. If that case, the
minimal alignment is sizeof(lrecord_header) (i.e. 4, but it should
have been written as a sizeof).
Please comment. I would like to further nuke some of the current
alignment code, like this cruft at the end of pdump_dump_data():
write (pdump_fd, desc ? pdump_buf : elmt->obj, size*count);
if (elmt->is_lrecord && ((size*count) & 3)
write (pdump_fd, "\0\0\0", 4-((size*count) & 3));
This is the management of the exception. The other half is in
pdump_allocate_offset.
(This is partially fixed by my other patch with PDUMP_ALIGN_STREAM,
but...)
Which aligns too hard though...
OG.
2:14 p.m.
>>>> "OG" == Olivier Galibert
<galibert(a)pobox.com> writes:
OG> [Martin, I bounced you original message to xemacs-beta, I hope you
OG> don't mind]
OG> On Mon, Jan 22, 2001 at 06:53:36PM +0900, Martin Buchholz wrote:
> I've been doing too much alignment hacking lately. But I
still have
> trouble understanding how pdump alignment works.
OG> I'm using these facts of life:
OG> 1- Alignments are powers of two
OG> 2- sizeof() is required to return a properly-aligned size (that's in
OG> ANSI)
OG> By properly aligned size I mean if p is aligned for the type, p_size
OG> is aligned too.
OG> This means that you can get an upper-bound for the alignement of a
OG> given structure from its size by finding the higher power of two that
OG> is a divisor of the size. That's the point of the pdump_align_table
OG> array. The table goes all the way to 2^8 which is overkill though.
OG> We should stop at 2^3 or maybe 2^4 (and use a 8 or 16 elements table).
We should stop at ALIGNOF (max_align_t).
OG> The other trick I use is to dump the data ordered by decreasing
OG> alignment constraint. After all, if something is aligned for 8 bytes,
OG> it is aligned for 4, 2 and 1 too. And by (2) we know that if we're
OG> aligned for 8 bytes, and we write an 8-bytes aligned structure, we
OG> will still be 8-bytes aligned afterwards.
OG> So writing the data by decreasing alignment constraints ensures the
OG> alignment at zero cost in padding.
OG> This "scaning by decreasing alignment constraint" thingy is done by
OG> pdump_scan_by_alignment btw.
OG> That would be nice and cool, except that there is one exception. Some
OG> lrecords are lying a little. Well, to be more precise, they give
OG> sizes that are not the result of a sizeof, and don't necessarily
OG> respect (2). In particular, the opaques do that. If that case, the
OG> minimal alignment is sizeof(lrecord_header) (i.e. 4, but it should
OG> have been written as a sizeof).
Excellent explanation. Thanks. Need to dream about this some.
Actually a Lisp_Opaque has a lcrecord_header at the front, not an
lrecord_header. And the alignment of an lrecord_header is much less
than that of an lcrecord_header. But almost all lisp objects which
are lrecords have the same alignment requirements as lcrecords, since
they typically contain pointers. So we're not losing much here.
> Please comment. I would like to further nuke some of the
current
> alignment code, like this cruft at the end of pdump_dump_data():
>
> write (pdump_fd, desc ? pdump_buf : elmt->obj, size*count);
> if (elmt->is_lrecord && ((size*count) & 3)
> write (pdump_fd, "\0\0\0", 4-((size*count) & 3));
OG> This is the management of the exception. The other half is in
OG> pdump_allocate_offset.
> (This is partially fixed by my other patch with
PDUMP_ALIGN_STREAM, but...)
OG> Which aligns too hard though...
Yes. I have just changed it to align to ALIGNOF (struct
lcrecord_header) from ALIGNOF (max_align_t) which is a win on those
many platforms where double is aligned to 8, while pointers are
aligned to 4.
6:15 a.m.
>>>> "OG" == Olivier Galibert
<galibert(a)pobox.com> writes:
OG> That would be nice and cool, except that there is one exception. Some
OG> lrecords are lying a little. Well, to be more precise, they give
OG> sizes that are not the result of a sizeof, and don't necessarily
OG> respect (2). In particular, the opaques do that. If that case, the
OG> minimal alignment is sizeof(lrecord_header) (i.e. 4, but it should
OG> have been written as a sizeof).
Design Proposal - I propose that:
We require the lcrecords _not_ to lie, but instead require the
size_in_bytes method to have the same alignment constraints as ANSI C
imposes.
So, for example, when a Lisp_Opaque is allocated, it (actually
make_opaque()) aligns its size to ALIGNOF (lcrecord_header) before
calling alloc_lcrecord.
And the size_in_bytes method will return the same size.
(Note that OPAQUE_SIZE still returns the old unaligned size, so
clients of opaque.c don't have to change.)
The ALIGN_SIZE and ALIGN_PTR methods should go into lisp.h with the
definition of ALIGNOF.
Then we can remove the special handling for lrecords in dumper.c, and
remove the is_lrecord member from struct pdump_entry_list_elmt, since
it is only used to kludge around the incorrect sizes.
6:30 a.m.
On Tue, Jan 23, 2001 at 03:15:56PM +0900, Martin Buchholz wrote:
We require the lcrecords _not_ to lie, but instead require the
size_in_bytes method to have the same alignment constraints as ANSI C
imposes.
Makes sense. The only lrecords not using sizeof directly are opaque,
lstream, specifier and window_config. Lstream and specifier
implementations are structs, their sizes are correctly aligned. Ditto
for saved_window. In practice we're not even dumping lstreams or
window configs.
So the only problem is the opaques. Quite reasonable to move the fix
there, I'd say.
Then we can remove the special handling for lrecords in dumper.c,
and
remove the is_lrecord member from struct pdump_entry_list_elmt, since
it is only used to kludge around the incorrect sizes.
Yup.
OG.
8704
days inactive
8705
days old
4 comments
2 participants
participants (2)
-
Martin Buchholz -
Olivier Galibert