public abstract class SutController extends Object implements SutHandler, CustomizationHandler
| Constructor and Description |
|---|
SutController() |
| Modifier and Type | Method and Description |
|---|---|
void |
addTableToInserted(List<String> tables)
collect info about what table are manipulated by evo in order to generate data directly into it
|
CustomizedCallResultCode |
categorizeBasedOnResponse(Object response)
categorize result based on response
|
void |
cleanAccessedTables()
perform smart db clean by cleaning the data in accessed table
|
ExtraHeuristicsDto |
computeExtraHeuristics() |
void |
computeMongoHeuristics(ExtraHeuristicsDto dto) |
boolean |
customizeMockingDatabase(List<MockDatabaseDto> databaseDtos,
boolean enabled)
implement how to enable/disable customized mock objects for database
|
boolean |
customizeMockingRPCExternalService(List<MockRPCExternalServiceDto> externalServiceDtos,
boolean enabled)
implement how to enable/disable mocking of RPC based external services
|
boolean |
customizeRPCTestOutput(List<MockRPCExternalServiceDto> externalServiceDtos,
List<String> sqlInsertions,
List<EvaluatedRPCActionDto> actions)
here we provide additional handling on the generated RPC tests
|
boolean |
doEmploySmartDbClean() |
void |
enableComputeSqlHeuristicsOrExtractExecution(boolean enableSqlHeuristics,
boolean enableSqlExecution) |
InsertionResultsDto |
execInsertionsIntoDatabase(List<InsertionDto> insertions,
InsertionResultsDto... previous)
Execute the given data insertions into the database (if any)
|
MongoInsertionResultsDto |
execInsertionsIntoMongoDatabase(List<MongoInsertionDto> insertions) |
void |
executeAction(RPCActionDto dto,
ActionResponseDto responseDto)
execute a RPC request based on the specified dto
|
void |
executeHandleLocalAuthenticationSetup(RPCActionDto dto,
ActionResponseDto responseDto) |
Object |
executeRPCEndpoint(String json)
execute an RPC endpoint with evomaster driver
|
RPCProblemDto |
extractRPCProblemDto(boolean isSutRunning) |
void |
extractRPCSchema()
extract endpoints info of the RPC interface by reflection based on the specified service interface name
|
int |
getActionIndex() |
abstract List<AdditionalInfo> |
getAdditionalInfoList() |
abstract List<TargetInfo> |
getAllCoveredTargetInfos() |
abstract BootTimeInfoDto |
getBootTimeInfoDto() |
protected BootTimeInfoDto |
getBootTimeInfoDto(BootTimeObjectiveInfo info) |
Connection |
getConnection()
Deprecated.
this is now set in DbSpecification
|
Connection |
getConnectionIfExist()
TODO further handle multiple connections
|
String |
getControllerHost() |
int |
getControllerPort() |
int |
getControllerServerPort() |
List<CustomizedRequestValueDto> |
getCustomizedValueInRequests()
specify candidate values in requests
|
String |
getDatabaseDriverName()
Deprecated.
this method is no longer needed
|
Map<Class,Integer> |
getExceptionImportanceLevels()
specify importance levels for exceptions
lower value more important, 0 is the most important exception which needs to be fixed earliest
|
abstract String |
getExecutableFullPath() |
List<ExtraHeuristicsDto> |
getExtraHeuristics() |
abstract List<AuthenticationDto> |
getInfoForAuthentication()
If the application uses some sort of authentication, these details
need to be provided here.
|
abstract void |
getJvmDtoSchema(List<String> dtoNames) |
Map<Integer,LocalAuthSetupSchema> |
getLocalAuthSetupSchemaMap() |
abstract String |
getPackagePrefixesToCover()
A "," separated list of package prefixes or class names.
|
abstract SutInfoDto.OutputFormat |
getPreferredOutputFormat()
Test cases could be outputted in different language (e.g., Java and Kotlin),
using different testing libraries (e.g., JUnit 4 or 5).
|
abstract ProblemInfo |
getProblemInfo()
Depending of which kind of SUT we are dealing with (eg, REST, GraphQL or SPA frontend),
there is different info that must be provided.
|
Object |
getRPCClient(String interfaceName)
return an instance of a client of an RPC service.
|
Map<String,InterfaceSchema> |
getRPCSchema() |
DbSchemaDto |
getSqlDatabaseSchema()
Extra information about the SQL Database Schema, if any is present.
|
abstract List<TargetInfo> |
getTargetInfos(Collection<Integer> ids) |
abstract UnitsInfoDto |
getUnitsInfoDto() |
protected UnitsInfoDto |
getUnitsInfoDto(UnitsInfoRecorder recorder) |
Map<String,List<RPCActionDto>> |
handleSeededTests(boolean isSUTRunning)
parse seeded tests for RPC
|
void |
handleSql(String sql)
Deprecated.
|
void |
initMongoHandler() |
void |
initSqlHandler()
This is needed only during test generation (not execution),
and it is automatically called by the EM controller after
the SUT is started.
|
abstract boolean |
isInstrumentationActivated()
Check if bytecode instrumentation is on.
|
abstract boolean |
isSutRunning()
Check if the system under test (SUT) is running and fully initialized
|
boolean |
mockDatabasesWithCustomizedHandling(String mockDatabaseObjectDtos,
boolean enabled)
a method to employ customized mocking for database
|
boolean |
mockRPCExternalServicesWithCustomizedHandling(String externalServiceDtos,
boolean enabled)
a method to employ customized mocking of RPC based external services
|
void |
newAction(ActionDto dto)
As some heuristics are based on which action (eg HTTP call, or click of button)
in the test sequence is executed, and their order, we need to keep track of which
action does cover what.
|
abstract void |
newActionSpecificHandler(ActionDto dto) |
abstract void |
newSearch()
Re-initialize all internal data to enable a completely new search phase
which should be independent from previous ones
|
void |
newTest()
Re-initialize some internal data needed before running a new test
|
abstract void |
newTestSpecificHandler() |
String |
packagesToSkipInstrumentation()
Comma , separated list of package prefixes of classes to skip.
|
void |
postSearchAction(PostSearchActionDto dto)
handling post actions after the search
|
String |
readFileAsStringFromTestResource(String fileName) |
void |
registerOrExecuteInitSqlCommandsIfNeeded()
handle specified init sql script after SUT is started.
|
boolean |
resetCustomizedMethodForMockObject()
a method to reset mocked external services with customized method
|
void |
resetDatabase(List<String> tablesToClean)
reset database if the smart db cleaning is employed
|
void |
resetExtraHeuristics() |
List<SeededRPCTestDto> |
seedRPCTests()
specify seeded tests for RPC
|
void |
setControllerHost(String controllerHost) |
void |
setControllerPort(int controllerPort) |
abstract void |
setExecutingAction(boolean executingAction) |
abstract void |
setExecutingInitMongo(boolean executingInitMongo) |
abstract void |
setExecutingInitSql(boolean executingInitSql) |
abstract void |
setKillSwitch(boolean b) |
List<CustomizedNotNullAnnotationForRPCDto> |
specifyCustomizedNotNullAnnotation()
specify customized annotation indicating that field in DTO is not null if any
|
boolean |
startTheControllerServer()
Start the controller as a RESTful server.
|
boolean |
stopTheControllerServer() |
boolean |
verifySqlConnection()
Deprecated.
|
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, waitgetDbSpecifications, getMongoConnection, handleLocalAuthenticationSetup, resetStateOfSUT, setupForGeneratedTest, startSut, stopSutpublic final boolean startTheControllerServer()
public final boolean stopTheControllerServer()
public final int getControllerServerPort()
public final int getControllerPort()
public final void setControllerPort(int controllerPort)
public final String getControllerHost()
public final void setControllerHost(String controllerHost)
public InsertionResultsDto execInsertionsIntoDatabase(List<InsertionDto> insertions, InsertionResultsDto... previous)
SutHandlerexecInsertionsIntoDatabase in interface SutHandlerinsertions - DTOs for each insertion to executeprevious - an array of insertion results which were executed before this executionpublic MongoInsertionResultsDto execInsertionsIntoMongoDatabase(List<MongoInsertionDto> insertions)
execInsertionsIntoMongoDatabase in interface SutHandlerpublic int getActionIndex()
@Deprecated public final void handleSql(String sql)
sql - command as a stringpublic final void enableComputeSqlHeuristicsOrExtractExecution(boolean enableSqlHeuristics,
boolean enableSqlExecution)
public final void initSqlHandler()
public final void initMongoHandler()
public final Connection getConnectionIfExist()
public final boolean doEmploySmartDbClean()
public final void resetExtraHeuristics()
public final List<ExtraHeuristicsDto> getExtraHeuristics()
public final ExtraHeuristicsDto computeExtraHeuristics()
public final void computeMongoHeuristics(ExtraHeuristicsDto dto)
public final void registerOrExecuteInitSqlCommandsIfNeeded()
registerOrExecuteInitSqlCommandsIfNeeded in interface SutHandlerpublic final void cleanAccessedTables()
public void addTableToInserted(List<String> tables)
tables - a list of name of tablespublic final DbSchemaDto getSqlDatabaseSchema()
SutHandler.getDbSpecifications()public final Map<String,InterfaceSchema> getRPCSchema()
public Map<Integer,LocalAuthSetupSchema> getLocalAuthSetupSchemaMap()
public RPCProblemDto extractRPCProblemDto(boolean isSutRunning)
public final void extractRPCSchema()
extractRPCSchema in interface SutHandlerpublic Map<String,List<RPCActionDto>> handleSeededTests(boolean isSUTRunning)
@Deprecated public final boolean verifySqlConnection()
public abstract void newSearch()
public void postSearchAction(PostSearchActionDto dto)
dto - contains required info for the post handlingpublic final void newTest()
public final void newAction(ActionDto dto)
dto - the DTO with the information about the action (eg its index in the test)public final void executeHandleLocalAuthenticationSetup(RPCActionDto dto, ActionResponseDto responseDto)
public final void executeAction(RPCActionDto dto, ActionResponseDto responseDto)
dto - is the action DTO to be executedpublic Object executeRPCEndpoint(String json) throws Exception
SutHandlerexecute an RPC endpoint with evomaster driver
executeRPCEndpoint in interface SutHandlerjson - contains info of an RPC endpointExceptionpublic abstract void newTestSpecificHandler()
public abstract void newActionSpecificHandler(ActionDto dto)
public abstract boolean isInstrumentationActivated()
public abstract boolean isSutRunning()
Check if the system under test (SUT) is running and fully initialized
How to implement this method depends on the library/framework used
to build the application.
In Spring applications, this can be done with something like:
ctx != null && ctx.isRunning(), where ctx is a field where
ConfigurableApplicationContext should be stored when starting
the application.
public abstract String getPackagePrefixesToCover()
A "," separated list of package prefixes or class names. For example, "com.foo.,com.bar.Bar". This is used to specify for which classes we want to measure code coverage.
Note: be careful of using something as general as "com." or "org.", as most likely ALL your third-party libraries would be instrumented as well, which could have a severe impact on performance.
public abstract List<AuthenticationDto> getInfoForAuthentication()
If the application uses some sort of authentication, these details need to be provided here. Even if EvoMaster can have access to the database, it would not be able to recover hashed passwords.
To test the application, there is the need to provide auth for at least 1 user (and more if they have different authorization roles). When EvoMaster generates test cases, it can decide to use the credential of any user provided by this method.
What type of info to provide here depends on the auth mechanism, e.g.,
Basic or cookie-based (using CookieLoginDto).
To simplify the creation of these DTOs with auth info, you can look
at AuthUtils.
If the credential are stored in a database, be careful on how the
method resetStateOfSUT is implemented.
If you delete all data with DbCleaner, then you will need as well to
recreate the auth details.
This can be put in a script, executed then with SqlScriptRunner.
null if
* none is necessary@Deprecated public final Connection getConnection()
If the system under test (SUT) uses a SQL database, we need to have a configured connection to access it.
This method is related to SutHandler.resetStateOfSUT().
When accessing a Connection object to reset the state of
the application, we suggest to save it to field (eg when starting the
application), and return such field here, e.g., return connection;.
This connection object will be used by EvoMaster to analyze the state of
the database to create better test cases.
null if the SUT does not use any SQL database@Deprecated public final String getDatabaseDriverName()
org.h2.Driver.
This is needed for when we intercept SQL commands with P6Spynull if the SUT does not use any SQL databasepublic abstract List<TargetInfo> getTargetInfos(Collection<Integer> ids)
public abstract List<TargetInfo> getAllCoveredTargetInfos()
public abstract List<AdditionalInfo> getAdditionalInfoList()
public abstract ProblemInfo getProblemInfo()
Depending of which kind of SUT we are dealing with (eg, REST, GraphQL or SPA frontend), there is different info that must be provided. For example, in a RESTful API, you need to speficy where the OpenAPI/Swagger schema is located.
The interface ProblemInfo provides different implementations, like
RestProblem.
You will need to instantiate one of such classes, and return it here in this method.
public abstract SutInfoDto.OutputFormat getPreferredOutputFormat()
public abstract UnitsInfoDto getUnitsInfoDto()
public abstract void setKillSwitch(boolean b)
public abstract void setExecutingInitSql(boolean executingInitSql)
public abstract void setExecutingInitMongo(boolean executingInitMongo)
public abstract void setExecutingAction(boolean executingAction)
public abstract BootTimeInfoDto getBootTimeInfoDto()
protected BootTimeInfoDto getBootTimeInfoDto(BootTimeObjectiveInfo info)
public abstract String getExecutableFullPath()
protected UnitsInfoDto getUnitsInfoDto(UnitsInfoRecorder recorder)
public Object getRPCClient(String interfaceName)
SutHandlerreturn an instance of a client of an RPC service.
This method must be blocking until the SUT is initialized.
This method is only required when the problem is RPC for the moment, otherwise return null
might change string interfaceName to class interfacegetRPCClient in interface SutHandlerinterfaceName - a full name of an interfacepublic CustomizedCallResultCode categorizeBasedOnResponse(Object response)
CustomizationHandlercategorize result based on response
categorizeBasedOnResponse in interface CustomizationHandlerresponse - is a given responsepublic List<CustomizedRequestValueDto> getCustomizedValueInRequests()
CustomizationHandlerspecify candidate values in requests
getCustomizedValueInRequests in interface CustomizationHandlerpublic List<CustomizedNotNullAnnotationForRPCDto> specifyCustomizedNotNullAnnotation()
CustomizationHandlerspecify customized annotation indicating that field in DTO is not null if any
specifyCustomizedNotNullAnnotation in interface CustomizationHandlerpublic List<SeededRPCTestDto> seedRPCTests()
CustomizationHandlerspecify seeded tests for RPC
seedRPCTests in interface CustomizationHandlerpublic boolean customizeRPCTestOutput(List<MockRPCExternalServiceDto> externalServiceDtos, List<String> sqlInsertions, List<EvaluatedRPCActionDto> actions)
CustomizationHandlerhere we provide additional handling on the generated RPC tests
customizeRPCTestOutput in interface CustomizationHandlerexternalServiceDtos - represent info is to mock responses of external servicessqlInsertions - represent a sequence of SQL insertionsactions - represent a list of RPC actions to execute in this test with returned responsespublic boolean customizeMockingRPCExternalService(List<MockRPCExternalServiceDto> externalServiceDtos, boolean enabled)
CustomizationHandlerimplement how to enable/disable mocking of RPC based external services
customizeMockingRPCExternalService in interface CustomizationHandlerexternalServiceDtos - contains info about how to setup responsesenabled - reflect to enable (set it true) or disable (set it false) the specified external service dtos.
Note that null [externalServiceDtos] with false [enabled] means that all existing external service setup should be disabled.public boolean customizeMockingDatabase(List<MockDatabaseDto> databaseDtos, boolean enabled)
CustomizationHandlerimplement how to enable/disable customized mock objects for database
customizeMockingDatabase in interface CustomizationHandlerdatabaseDtos - contains info about how to mock databases based on commandNameenabled - reflect to enable (set it true) or disable (set it false) mock objects for sql command based on commandName
Note that null [databaseDtos] with false [enabled] means that all existing mock objects should be disabled.public void resetDatabase(List<String> tablesToClean)
SutHandlerreset database if the smart db cleaning is employed
resetDatabase in interface SutHandlerpublic String packagesToSkipInstrumentation()
public final boolean resetCustomizedMethodForMockObject()
a method to reset mocked external services with customized method
resetCustomizedMethodForMockObject in interface SutHandlerpublic final boolean mockRPCExternalServicesWithCustomizedHandling(String externalServiceDtos, boolean enabled)
a method to employ customized mocking of RPC based external services
mockRPCExternalServicesWithCustomizedHandling in interface SutHandlerexternalServiceDtos - contains info about how to setup responses with json format, note that the json should
be able to be converted to a list of MockRPCExternalServiceDtoenabled - reflect to enable (set it true) or disable (set it false) the specified external service dtos.
Note that null [externalServiceDtos] with false [enabled] means that all existing external service setup should be disabled.public boolean mockDatabasesWithCustomizedHandling(String mockDatabaseObjectDtos, boolean enabled)
SutHandlera method to employ customized mocking for database
mockDatabasesWithCustomizedHandling in interface SutHandlermockDatabaseObjectDtos - contains info about how to set up mock object for databases with json format, note that the json should
be able to be converted to a list of MockDatabaseDtoenabled - reflect to enable (set it true) or disable (set it false) the specified mock object
Note that null [mockDatabaseObjectDtos] with false [enabled] means that all existing mock objects for databases should be disabled.public final String readFileAsStringFromTestResource(String fileName)
fileName - the name of file which exist in the same directory of the classpublic Map<Class,Integer> getExceptionImportanceLevels()
CustomizationHandlerspecify importance levels for exceptions lower value more important, 0 is the most important exception which needs to be fixed earliest
getExceptionImportanceLevels in interface CustomizationHandlerCopyright © 2016–2023. All rights reserved.