[Pull request] Eunit controls

Nicolas Thauvin nicolas at corporama.com
Wed Feb 16 12:38:49 EST 2011


On may summarize like this:
  Should we add the patch in rebar to retrieve its module list, or should  
we let the user specify its own modules/functions in generators ?

Both options are ok (yet I think generators are a bit more complex to  
handle).

The thing is, as there is not a clear wish for this patch, it may be  
better to let it aside, rebar should remain simple to use and maintain.

Nicolas


On Wed, 16 Feb 2011 18:23:31 +0100, Joe Williams <joe at joetify.com> wrote:

>
>
> On Feb 16, 2011, at 9:09 AM, Joe Williams wrote:
>
>>
>> I mentioned it on the pull request  
>> (https://github.com/basho/rebar/pull/34) but I am not sure this is  
>> worth the added complexity when there other ways to do this in the test  
>> generator easily enough.
>
> Sorry if that sounded harsh. I suppose it comes down to how users would  
> prefer to do this, in the rebar.config or in a module.
>
> For example:
>
> foo_test_() ->
>     {inparallel, [fun foo/0, fun bar/0]}.
>
> vs
>
> [{inparallel, [foo, bar]}]
>
> I think I land on the minimalist side, since there is already an easy  
> way to do it I'm not sure rebar needs a second.
>
> If rebar users prefer it being in the rebar.config we can certainly add  
> it since making users happy is what rebar is about. :)
>
> -Joe
>
>
>> On Feb 13, 2011, at 2:58 PM, Nicolas Thauvin wrote:
>>
>>> Sorry for the late reply on this thread, I've been very busy lately...
>>>
>>> I wonder why your particular example did not work:
>>>
>>> Trying:
>>> {eunit_controls, [{timeout, 10, {mochiweb, multiple_https_GET_test}},  
>>> '_']}.
>>>
>>> I got:
>>> [long tests list...]
>>> mochiweb_multipart: multipart_body_test...ok
>>>  [done in 0.533 s]
>>> mochiweb: multiple_https_GET_test...[6.172 s] ok
>>> =======================================================
>>>  All 160 tests passed.
>>>
>>>
>>> (I confess I had to add a timer:sleep to make it last longer that 5s.)
>>>
>>> This patch was first designed to specify inparallel directive in a  
>>> single place to help speeding up test runs. At a first glance, I  
>>> thought it was easier to set it up as a configuration parameter to be  
>>> able to order suite of tests accesssing a shared ressource. As rebar  
>>> gives a list of modules in input, the eunit_controls thing adds the  
>>> '_' wildcard to process modules that are not part of the parameter. So  
>>> far so good. But in big projects, the parameter can quickly grow over  
>>> control.
>>>
>>> One drawback is that when you run tests in parallel, they are slower.  
>>> And they end in timeout. And then you see how messy eunit is with  
>>> inparallel to handle these. And then you wonder if the patch is a good  
>>> idea.
>>>
>>> One alternative is to use test generators, as Tuncer and someone in  
>>> the patch comments said.
>>> I pushed a version of the patch that do not try to handle timeout,  
>>> with a comment in rebar.config.sample that explains they should appear  
>>> in a test generator.
>>>
>>> I kept module expansion (allowing to specify {M, F} tests and remember  
>>> that other functions in the module should be executed)  as it is  
>>> useful if two tests must be executed in order in the same module.
>>> I didn't try test generators for in_parallel. How could we get rebar  
>>> module list from there ?
>>>
>>> After a short period of use, I think eunit_controls is not intuitive  
>>> to use when your tests crash sometimes (one time out of ten) when a  
>>> shared ressource is accessed by two tests at the same time. You then  
>>> have to sort your tests depending on the shared ressource access,  
>>> that's not really fun.
>>> As it's not just working out of the box, I wonder if it is compatible  
>>> with rebar. I mean, rebar works out of the box, eunit_controls don't.
>>>
>>> The point is that it can really help to speed up eunit runs.
>>>
>>> For the buggy eunit part, I think I will have a look at it someday if  
>>> someone is not pissed of before ;)
>>>
>>> Nicolas
>>>
>>> On Sun, 13 Feb 2011 10:04:42 +0100, Kenji Rikitake  
>>> <kenji.rikitake at acm.org> wrote:
>>>
>>>> My idea is that EUnit has to be patched to set the timeout parameter
>>>> throughout the eunit:test/2; it's not a rebar issue.
>>>>
>>>> R14B01's lib/eunit/src/eunit_internal.hrl has the hardcoded value of
>>>> timeout (5 seconds) as
>>>> -- quote --
>>>> -define(DEFAULT_TEST_TIMEOUT, 5000).
>>>> -- unquote --
>>>> only used in eunit_proc.erl.
>>>> And the automatic test timeout (60 seconds) is
>>>> -- quote --
>>>> -define(AUTO_TIMEOUT, 60000).   %% auto test time limit
>>>>
>>>> %% TODO: pass options to server, such as default timeout?
>>>> -- unquote --
>>>> in eunit_server.erl; I find no way to change the default value from
>>>> eunit:test/1 or anywhere else in a legitimate way.
>>>>
>>>> BTW I tested the eunit_control patch of eunit_controls for building
>>>> mochiweb.
>>>> Setting
>>>> -- quote --
>>>> {eunit_controls, [timeout, 10, {mochiweb, multiple_https_GET_test}}.
>>>> -- unquote --
>>>> in rebar.config made rebar to test mochiweb:multiple_https_GET_test/0
>>>> before testing other modules, but the timeout change did not cover
>>>> the whole test.
>>>>
>>>> For the timeout parameter, adding a functionality to EUnit might be  
>>>> the
>>>> first thing to do.
>>>>
>>>> Just my JPY 2 thought,
>>>> Kenji Rikitake
>>>>
>>>> ++> Tuncer Ayaz <tuncer.ayaz at gmail.com> [2011-02-11 17:33:32 +0100]:
>>>>> On Sun, Jan 23, 2011 at 11:00 PM, Nicolas Thauvin wrote:
>>>>> > There is a fixed version of eunit controls in my branch:
>>>>> >
>>>>> > https://github.com/nthauvin/rebar/commits/eunit_controls
>>>>>
>>>>> I may be missing the point, but what does the patch solve that we  
>>>>> cannot
>>>>> do in a test generator?
>>>
>>> _______________________________________________
>>> rebar mailing list
>>> rebar at lists.basho.com
>>> http://lists.basho.com/mailman/listinfo/rebar_lists.basho.com
>>
>> Name: Joseph A. Williams
>> Email: joe at joetify.com
>> Blog: http://www.joeandmotorboat.com/
>> Twitter: http://twitter.com/williamsjoe
>>
>> _______________________________________________
>> rebar mailing list
>> rebar at lists.basho.com
>> http://lists.basho.com/mailman/listinfo/rebar_lists.basho.com
>
> Name: Joseph A. Williams
> Email: joe at joetify.com
> Blog: http://www.joeandmotorboat.com/
> Twitter: http://twitter.com/williamsjoe



More information about the rebar mailing list