FEAT: Budget mechanism for the principal loop of DSpot

[danglotb]

Hi,

I propose to implement an interface that represents the budget allocated to the amplification of one test class, particularly for the main loop of DSpot.

For now, we can specify a given number of iteration. The problem is that the time execution can vary a lot between different run.

One first step is to provide an interface and mechanism (breaking the current process if the budget is consumed) for this.

We will two "natural" implementation:

1. the current one, which the number of iteration
2. a specific amount of time, which would be used by default IHMO (It is required to find a proper default value.)

[andrewbwogi]

Hi @danglotb, I'll look into this.

[danglotb]

Hello @andrewbwogi,

That would be great but I have to warn you: I'm currently preparing a HUGE pull request so we might have issue about merging here.

Thank you

[danglotb]

Sorry, I should have been more specific:

My pull request is about to remove the properties file and pass all the arguments on the command line.

Also, I'm removing the singleton InputConfiguration.

It should be open by tomorrow.

Anyway, we can still resolve the merging issue after.

Thank you again, I think it is a major feature for DSpot and it would be awesome that you implement it.

[andrewbwogi]

Hi @danglotb, if I understand you correctly, the task is to create a budget mechanism that breaks the amplification loop after a specified amount of time. Do we resume the iteration with whatever tests have been generated so far?

For instance, when input amplification takes too long, we would feed the finished tests we have to assertion generation. We pass along tests finished on time in a similar way for assertion generation and resume the iteration after test selection. I think test selection would be left as is for now since the cut off places are inside external programs.

[danglotb]

Hello @andrewbwogi

It depends at which steps/granularity you want to introduce this time budget.

I see two granularities:

1. at the main loop: instead of a given number of iteration, the user allocates a given amount of time. If the time expires, DSpot outputs want it could found and stop.
2. at different steps of the DSpot's algorithm: input-amplification, assertion-amplification, and test-selection.

In the first case, what we would like to have is an interface that tells if DSpot should continue or not. This interface would be used here: Amplification#amplification() instead of the iterations (we can imagine an implementation of the mentioned interface that is based on iterations, and I think it would be the first implementation that we have).

The second case, that I guess you are targetting is much more complicated. However, another issue is also opened about it: #624.

For instance, when input amplification takes too long, we would feed the finished tests we have to assertion generation. We pass along tests finished on time in a similar way for assertion generation and resume the iteration after test selection. I think test selection would be left as is for now since the cut off places are inside external programs.

This looks good to me to close #624. If you could do that it would be awesome! (difficult to configure well but still awesome!). This is true that we cannot cut off external tools (like PIT) and there is not so much to do in assertion generation (most of the time passed is to execute the test IMHO). Targetting a timed out on the input amplification looks the best to do.

To quickly answer the question, I would say that yes, we resume the iteration with whatever tests have been generated so far and it is related to #624.

Thank you @andrewbwogi! See you soon.