Automatic smoketests (was Re: Smoketest fails on jde.)
On Tue, 2002-11-19 at 11:57, Mats Lidell wrote:
Ville> Yep, but building non-mule packages with non-mule isn't
:) And
Ville> it's good to build with 21.1.14 too.
It's a matter of how you want to use the CPU-cycles. Now I build once
an hour so anyone doing a check-in can get feed back within one to two
hours. Building for more configurations I would have to reduce the
frequency or make the build scripts smarter.
Maybe fast turn around on a hourly basis and a full build for all
interesting configurations once a day or week or so.
Makes sense to me. 21.1.14 is important to include in the tests because
we still support packages on it (and will for some time), but the tester
base isn't that large.
Ville> Anyway, I have a setup of 21.1.14, 21.4.10 and 21.5.9 (all
Ville> mule) for running the smoketests, but I haven't set up an
Ville> automatic job to run the tests, mostly because I'm not sure how
Ville> I would like the logs handled.
I was thinking of a web page with red and green-lights. All of this
assumes of course that the people checking in things in the package
tree uses this web-page for feed back. (It's silly that I should cut
and past things from the log to the mail-list when developers simply
can find the info by them selves. Basically all is there now except
that there are no fancy web-page. Just visit the latest log in your
browser, wait for it to download and read the last few lines.)
Sounds neat. This, combined with a [HEADS UP] mail to some list if the
build fails would be even better. I'm afraid that simply setting up the
web pages doesn't draw enough attention...
It would also be an easy thing to just take the tail of the log to
se
whether the build succeeded or not.
That's already handled in my smoketest script, not by tailing but by
examining the return code of each build task. Or did I miss a point
here?
With a better log handling we could also do a "make -k" in
order to
spot more than one error on each build. But we would then need some
way to extract or highlight the errors since a normal log is about
3MB.
Ah, nice ideas, thanks a bunch. "make -k" would of course be only
needed for the build-the-whole-friggin-packages-tree task, not when
building individual packages separately. I'll start hacking something
along these ideas tomorrow, if not today.
--
\/ille Skyttä
scop at xemacs.org