12:20 a.m.
* Joachim Schrod <jschrod(a)acm.org> writes:
>>>>> "BW" == Ben Wing <ben(a)666.com>
writes:
BW> We really need to do things by HTTP.
BW> What will this involve?
There are ca. 5 places in package-get.el where remote files are
accessed. These places are not specially functions, stuff like
file-exists-p and insert-file-contents-literally, that rely on EFS to
do the actual work.
These places have to generalized. They should dispatch over a
transport protocol identifier that gets added to a download site
description.
Interesting. So you are thinking of something that sets the transport
method based on something in `package-get-download-sites' and
`package-get-pre-release-download-sites'? I never thought of doing
that. Could be cool. Could also be too restrictive and a
maintenance PITA. Interesting, nonetheless.
The bigger problem is the actual HTTP access. Currently, XEmacs
provides HTTP access methods only in W3. Clearly, it's a bad
decision to make the package package dependent on such a mass of
code. That means that we need to implement a new module. My
thoughts are on an http.el along the lines of smtp.el, built
upon #'open-network-stream.
Here is something I was working on a year or so ago. It gives a local
transport, FTP (via EFS), HTTP, and scp. All that needs to be done is
to hook it into package-get.el. This package-transport.el will
obviously live in core.
Of course, we also need to get Steve Youngs' agreement with this
approach; after all, package-get.el is him. As he just answered on
xemacs-beta; he doesn't want package-get.el to depend on something not
in core, and the http.el above would not be in core. Or would it? But
even if we put it in core, there remains the question how we handle
the upgrade path of those installations where it isn't there yet. (I
thought about adding it to the package package; but that's not a clean
way, IMO.)
Simple, you do it in such a way that there is zero impact on existing
package retrieval mechanisms.
If we agree on a design, I would submit an implementation. Next
Sunday, I'll go for a one-week holiday and will have some time for
hacking there.
Probably not necessary unless you feel package-transport.el is flawed
(very possible, I haven't looked at since 2003). As I said above, all
that needs to be done is to hook package-transport into package-get.
If you want to do that, go for it! :-)
--
|---<Steve Youngs>---------------<GnuPG KeyID: A94B3003>---|
| In space, |
| No one can hear you rip a stinky |
|------------------------------------<steve(a)sxemacs.org>---|
9:02 a.m.
New subject: HTTP fetching
Steve Youngs wrote:
* Joachim Schrod <jschrod(a)acm.org> writes:
> These places have to generalized. They should dispatch over a
> transport protocol identifier that gets added to a download site
> description.
Interesting. So you are thinking of something that sets the transport
method based on something in `package-get-download-sites' and
`package-get-pre-release-download-sites'? I never thought of doing
that. Could be cool. Could also be too restrictive and a
maintenance PITA. Interesting, nonetheless.
Your package-transport.el looks great; I'll hook it up to package-get.el.
Concerning my proposal above: It started with a design question. Is the
transport protocol a property of a download site, or an overall property? I.e.,
can we assume that _all_ XEmacs mirrors provide at least the protocols HTTP and
FTP? If not, explaining the package mechanism is a nightmare. One cannot say,
choose a mirror and then choose a transport method. Which one? Would you need to
look it up on the XEmacs Web Site? IMO too complex for a new XEmacs user, and
PUI should work for a newbie without much headache.
When we add a fourth element to a download site description list, this 4th
element could be your package-transport-method. When an XEmacs mirror is set up,
the mirror maintainer tells his access methods to the XEmacs guys, and they can
record these access methods in the site description. Not much pain in
maintenance; but easier for the end user.
This 4th element would be optional and default to ftp, if the site name exists
and to local if it's nil. Thus, existing personal package-get-remote
customizations still work.
What do you think of this? Or am I wrong with my approach that the available
transport protocol are a download site property?
Joachim
--
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Joachim Schrod Email: jschrod(a)acm.org
Roedermark, Germany
10:16 a.m.
New subject: HTTP fetching
* Joachim Schrod <jschrod(a)acm.org> writes:
Steve Youngs wrote:
> Interesting. So you are thinking of something that sets the
> transport method based on something in
> `package-get-download-sites' and
> `package-get-pre-release-download-sites'? I never thought of
> doing that. Could be cool. Could also be too restrictive and a
> maintenance PITA. Interesting, nonetheless.
Your package-transport.el looks great; I'll hook it up to
package-get.el.
Cool.
Concerning my proposal above: It started with a design question. Is
the transport protocol a property of a download site, or an overall
property? I.e., can we assume that _all_ XEmacs mirrors provide at
least the protocols HTTP and FTP?
No. The only thing you could assume is that they serve those
directories up via FTP. Some of them _might_ serve via HTTP as well,
but that is definitely _NOT_ the case for all of them.
Someone from the Review Board (preferably Norbert as he is the XPRM)
should get in contact with _all_ of the mirrors and ask them if they'd
be willing to serve up the package directories via HTTP as well as
FTP.
Unfortunately you can't just jump in a browser and see which ones will
respond. Not all of the sites are accessible world
wide. (mirror.aarnet.edu.au is a classic example, it can only be
accessed from within Australia)
If not, explaining the package mechanism is a nightmare. One cannot
say, choose a mirror and then choose a transport method. Which one?
Would you need to look it up on the XEmacs Web Site? IMO too complex
for a new XEmacs user, and PUI should work for a newbie without much
headache.
Exactly one of the reasons that package-transport.el has laid dormant
on my hard disk since 2003. :-)
When we add a fourth element to a download site description list,
this 4th element could be your package-transport-method. When an
XEmacs mirror is set up, the mirror maintainer tells his access
methods to the XEmacs guys, and they can record these access
methods in the site description. Not much pain in maintenance;
but easier for the end user.
That does sound pretty good.
This 4th element would be optional and default to ftp, if the site
name exists and to local if it's nil. Thus, existing personal
package-get-remote customizations still work.
Good.
What do you think of this? Or am I wrong with my approach that the
available transport protocol are a download site property?
I think this is a good idea. The idea certainly gets my approval.
--
|---<Steve Youngs>---------------<GnuPG KeyID: A94B3003>---|
| In space, |
| No one can hear you rip a stinky |
|------------------------------------<steve(a)sxemacs.org>---|
2:28 a.m.
New subject: HTTP fetching
>>>> "Joachim" == Joachim Schrod
<jschrod(a)acm.org> writes:
Joachim> Your package-transport.el looks great; I'll hook it up to
Joachim> package-get.el.
url.el (from the w3 package) plus some config junk elsewhere in that
package is sufficient to get http running, IIRC.
Joachim> Concerning my proposal above: It started with a design
Joachim> question. Is the transport protocol a property of a
Joachim> download site, or an overall property?
It's both. The fetch mechanism needs to know what transport to use,
obviously, and that varies by site. On the other hand, locally XEmacs
doesn't know that FTP is dropped at the firewall until the user
expresses a preference. This preference would be implemented as a
filter and ordering on URL schema (or whatever our equivalent is going
to be).
Joachim> I.e., can we assume that _all_ XEmacs mirrors provide at
Joachim> least the protocols HTTP and FTP?
No, you can't. The mirror list should provide full URLs or equivalent
information in an extension of the current format for backward compatibility.
Joachim> If not, explaining the package mechanism is a nightmare.
Not much we can do about that, as not all mirrors are going to expose
the same namespace by ftp and by http even if they do permit both.
The best we can do is to hide it by providing full URLs.
--
Institute of Policy and Planning Sciences http://turnbull.sk.tsukuba.ac.jp
University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
Ask not how you can "do" free software business;
ask what your business can "do for" free software.
4:46 a.m.
New subject: HTTP fetching
* Stephen J Turnbull <stephen(a)xemacs.org> writes:
>>>>> "Joachim" == Joachim Schrod
<jschrod(a)acm.org> writes:
Joachim> Your package-transport.el looks
great; I'll hook it up to
Joachim> package-get.el.
url.el (from the w3 package) plus some config junk elsewhere in
that package is sufficient to get http running, IIRC.
Overkill. Plus url.el is in packages... we are trying to _remove_
PUI's dependence on packages. And splitting url.el out of W3 package
and into core would not be the right way either.
But I won't lie to you and say I didn't use url.el as a reference for
writing package-transport.el. :-)
--
|---<Steve Youngs>---------------<GnuPG KeyID: A94B3003>---|
| In space, |
| No one can hear you rip a stinky |
|------------------------------------<steve(a)sxemacs.org>---|
3:29 p.m.
New subject: HTTP fetching
>>>> "SJT" == Stephen J Turnbull
<stephen(a)xemacs.org> writes:
>>>> "Joachim" == Joachim Schrod
<jschrod(a)acm.org> writes:
Joachim> Your package-transport.el looks great;
I'll hook it up to
Joachim> package-get.el.
SJT> url.el (from the w3 package) plus some config junk elsewhere in
SJT> that package is sufficient to get http running, IIRC.
As Steve wrote already, he implemented basic HTTP fetching in his
package-transport.el; I think I'll start with it. We'll need to see
what is really necessary to fetch packages from an XEmacs proxie, and
may add this later. Due to personal circumstances, I might be
interested to add proxy support, including authenticated proxies.
(Several of my clients allow Internet access only by means of
authenticated proxies.)
Joachim> Concerning my proposal above: It started with a design
Joachim> question. Is the transport protocol a property of a
Joachim> download site, or an overall property?
SJT> It's both. The fetch mechanism needs to know what transport to
SJT> use, obviously, and that varies by site. On the other hand,
SJT> locally XEmacs doesn't know that FTP is dropped at the firewall
SJT> until the user expresses a preference. This preference would be
SJT> implemented as a filter and ordering on URL schema (or whatever
SJT> our equivalent is going to be).
Interesting point, this is an important UI issue. As I've written
already, I think that the transport protocol is part of a download
site description. If a site offers two download possibilities, that's
two logical sites; e.g., ftp://ftp.xemacs.org/pub/xemacs/packages/ and
http://ftp.xemacs.org/pub/xemacs/packages/ are two different entries
in the download site menu list. But this way the menu list gets quite
long and unwieldy, further means to structure or filter it would be
good.
When the user customizes a preferred transport protocol, one can show
only sites with that protocol. Or, one can have the transport protocol
as submenus for the sites iff a site has more than one protocol.
This might need some experimentation to get good usability.
Joachim
--
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Joachim The most exciting phrase to hear in science, the
Rödermark, Germany one that heralds new discoveries, is not "Eureka!"
<jschrod(a)acm.org> (I found it!) but "That's funny..." [Isaac
Asimov]
7226
days inactive
7228
days old
5 comments
3 participants
participants (3)
-
Joachim Schrod -
Stephen J. Turnbull -
Steve Youngs