profile
viewpoint
Krishnan Mahadevan krmahadevan @testng-team Bangalore http://rationaleemotions.com/ Yet another "lost soul", in pursuit of being accepted as a "developer"..!

cbeust/testng 1549

TestNG testing framework

krmahadevan/di 3

Examples of how Dependency Injection Containers can be done via different frameworks

krmahadevan/go-design-patterns 1

This repository is intended to just collate my learnings on working with design patterns in golang.

krmahadevan/selenium-1 1

Selenium/Webdriver client for Go

krmahadevan/action-maven-publish 0

📦 GitHub Action for automatically publishing Maven packages

krmahadevan/allure-bamboo 0

Allure plugin for Atlassian Bamboo

krmahadevan/benchmarking 0

A Hello world sort of project to learn JMH

krmahadevan/coroutines 0

A pure Java implementation of cooperative concurrency, aka coroutines

krmahadevan/fun-pro 0

Meant to consolidate my learnings about Functional Programming in Java

issue openedcbeust/testng

New feature TestNG - getFactoryMethodParamsInfo on ConfigurationMethod

TestNG Version

7.3.0

Expected behavior

I would like to access the factory parameters from ConfigurationMethod. This should work when method.getTestMethod() is ConfigurationMethod @Override public void beforeInvocation(final IInvokedMethod method, final ITestResult testResult, final ITestContext context) { method.getTestMethod().getFactoryMethodParamsInfo() }

Actual behavior

method.getTestMethod().getFactoryMethodParamsInfo() returns null and not ParameterInfo class

Is the issue reproductible on runner?

I use IntelliJ and maven project.

  • [ ] Shell
  • [x] Maven
  • [ ] Gradle
  • [ ] Ant
  • [ ] Eclipse
  • [x] IntelliJ
  • [ ] NetBeans

Test case sample

I attached the sample. testng_test.zip.zip

created time in an hour

issue commentcbeust/testng

Test with retry analyzer fails when running verify

I'm off work until next Monday. Did set up a reminder to check this ASAP after I'm back.

On Sun, Nov 22, 2020, 11:31 PM Krishnan Mahadevan notifications@github.com wrote:

@javydreamercsw https://github.com/javydreamercsw - Both mvn clean test and mvn clean verify (verify wont make a difference because in either case, the surefire plugin is going to be called, since verify internally calls test phase)

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/cbeust/testng/issues/2417#issuecomment-731935482, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAVP2P7Y7MHTV2W6TENPB73SRHXUDANCNFSM4TU227LQ .

javydreamercsw

comment created time in 12 hours

issue commentcbeust/testng

Using softAssert and "hard" Asserts together, softAsserts are swallowed by the "hard" Asserts

I simply extended the softassert class with hardasserts. Its the easiest and cleanest way possible. https://gist.github.com/hwulfmeyer/fba24d20a8b7b9b230c6147ace0df39e

That way I can e.g. call softAssert.assertEquals() for a soft assert and then softAssert.hardAssertEquals() if I need a hard assert. My Base class contains an IHookAble which I simply call for the testResult and see if it successfully went through in case there was no failed hardAssert.

Example Code:

public abstract class BaseTest implements IHookable {
    public MySoftAssert softAssert;

    @Override
    public void run(IHookCallBack iHookCallBack, ITestResult iTestResult) {
        iHookCallBack.runTestMethod(iTestResult);
        if(iTestResult.getThrowable() == null){
            softAssert.assertAll("Test successful, evaluating Soft Asserts:");
        }
    }
}
hwulfmeyer

comment created time in 21 hours

issue commentcbeust/testng

TestNG creates multiple Guice Module Instances

Hi,

That is a showstopper for me the upgrade to anything newer than 6.14.13 (I have reproduced the same problem with 7.0.0). Is there any timeline that it would get released?

Regards Jacek

krmahadevan

comment created time in a day

startedzsh-users/zsh-autosuggestions

started time in a day

startedromkatv/powerlevel10k

started time in a day

startedryanoasis/nerd-fonts

started time in a day

Pull request review commentcbeust/testng

SkipException for multiple testclasses causes error in the forked process

+package test.listeners.github2415;++import org.testng.IInvokedMethod;+import org.testng.IInvokedMethodListener;+import org.testng.ITestResult;+import org.testng.SkipException;++import java.util.HashMap;+import java.util.Map;++public class SkipAfterPreviousTestClassFailureListener implements IInvokedMethodListener {++    private boolean hasFailures = false;+    public static Map<String, Integer> status = new HashMap<>();++    public void beforeInvocation(IInvokedMethod invokedMethod, ITestResult testResult) {+        status.put(testResult.getTestClass().getName() + testResult.getName(), testResult.getStatus());+        if (hasFailures) {+            throw new SkipException("SkipException");

I'd prefer that @krmahadevan confirm my opinion before but I think we have to support both. And if we need to choose only one option, I prefer the second one because configuration methods are not supposed to throw an exception by definition (where the SkipException is a specific case).

bj-9527

comment created time in 2 days

Pull request review commentcbeust/testng

SkipException for multiple testclasses causes error in the forked process

+package test.listeners.github2415;++import org.testng.IInvokedMethod;+import org.testng.IInvokedMethodListener;+import org.testng.ITestResult;+import org.testng.SkipException;++import java.util.HashMap;+import java.util.Map;++public class SkipAfterPreviousTestClassFailureListener implements IInvokedMethodListener {++    private boolean hasFailures = false;+    public static Map<String, Integer> status = new HashMap<>();++    public void beforeInvocation(IInvokedMethod invokedMethod, ITestResult testResult) {+        status.put(testResult.getTestClass().getName() + testResult.getName(), testResult.getStatus());+        if (hasFailures) {+            throw new SkipException("SkipException");

@juherr I tried to set testResult as Skipped (testResult.setStatus(ITestResult.SKIP)) instead of throwing SkipException in beforeInvocation method. It doesn't stop test methods from second class even though there is testResult set as SKIP.

bj-9527

comment created time in 2 days

issue commentcbeust/testng

@AfterClass not run when using groups and include methods in suite xml

Hi All,

I am facing similar issue, when I am using group then testNG control is not going to @Afterclass or @AfterSuit method and also extent report is not generating. Please help

deeg42

comment created time in 2 days

Pull request review commentcbeust/testng

Fix 2405

 public Injector getInjector(List<Module> moduleInstances) {    @Override   public Injector getInjector(IClass iClass) {+    if (hasNoGuiceAnnotations(iClass)) {

The behavior is not what I expected but why not if it is working?

And what if the logic is inverted and getInjector never called if the class hasNoGuiceAnnotations? https://github.com/cbeust/testng/blob/518bd75212805eb5f6f6a9518616abb5fd454159/src/main/java/org/testng/internal/ClassImpl.java#L103 and apply a similar pattern there https://github.com/cbeust/testng/blob/518bd75212805eb5f6f6a9518616abb5fd454159/src/main/java/org/testng/internal/Parameters.java#L646-L651

In fact, I don't know what is the expected behavior with dataProviderClass because they never have any @Guice annotation.

krmahadevan

comment created time in 2 days

Pull request review commentcbeust/testng

Fix 2405

 public Injector getInjector(List<Module> moduleInstances) {    @Override   public Injector getInjector(IClass iClass) {+    if (hasNoGuiceAnnotations(iClass)) {

I agree with your explanation but how it is possible to load com.google.inject.Injectoror com.google.inject.Module if guice.jar is missing (both are importing in the current class)?

krmahadevan

comment created time in 2 days

Pull request review commentcbeust/testng

SkipException for multiple testclasses causes error in the forked process

+package test.listeners.github2415;++import org.testng.IInvokedMethod;+import org.testng.IInvokedMethodListener;+import org.testng.ITestResult;+import org.testng.SkipException;++import java.util.HashMap;+import java.util.Map;++public class SkipAfterPreviousTestClassFailureListener implements IInvokedMethodListener {++    private boolean hasFailures = false;+    public static Map<String, Integer> status = new HashMap<>();++    public void beforeInvocation(IInvokedMethod invokedMethod, ITestResult testResult) {+        status.put(testResult.getTestClass().getName() + testResult.getName(), testResult.getStatus());+        if (hasFailures) {+            throw new SkipException("SkipException");

I thought again about the issue and I think that the best way to do what you want is testResult.setStatus(ITestResult.SKIP) instead of throw new SkipException("SkipException"). Please, could you add this additional test case in order to confirm it?

But supporting SkipException is a good feature too.

Once said that the fix should be catching SkipException and setting status to ITestResult.SKIP) somewhere. I see the catch but I can't remember how the result could be SKIP (maybe a default value which can change a day).

@krmahadevan WDYT?

bj-9527

comment created time in 2 days

Pull request review commentcbeust/testng

Fix JQuery vulnerability

 public void generateReport(           throw new RuntimeException("Couldn't find resource header");         }         for (String fileName : RESOURCES) {-          try (InputStream is = getClass().getResourceAsStream(TESTNG_RESOURCE_PREFIX + fileName)) {+          try (InputStream is = load(fileName)) {             if (is == null) {               throw new AssertionError("Couldn't find resource: " + fileName);             }-            Files.copyFile(is, new File(m_outputDirectory, fileName));+            Files.copyFile(is, new File(outputDirectory, fileName));           }         }         all = Files.readFile(header);-        Utils.writeUtf8File(m_outputDirectory, "index.html", xsb, all);+        Utils.writeUtf8File(outputDirectory, "index.html", xsb, all);       }     } catch (IOException e) {       Logger.getLogger(Main.class).error(e.getMessage(), e);     }   }++  private InputStream load(String fileName) {+    InputStream is = getClass().getResourceAsStream(Main.TESTNG_RESOURCE_PREFIX + fileName);+    if (is == null) {

Proposition:

String path;
if (fileName.equals("jquery.min.js") {
  path = "/META-INF/resources/webjars/jquery/3.5.1/jquery.min.js";
} else {
  path = Main.TESTNG_RESOURCE_PREFIX + fileName;
}
return getClass().getResourceAsStream(path);
krmahadevan

comment created time in 2 days

Pull request review commentcbeust/testng

Fix JQuery vulnerability

 public void generateReport(           throw new RuntimeException("Couldn't find resource header");         }         for (String fileName : RESOURCES) {-          try (InputStream is = getClass().getResourceAsStream(TESTNG_RESOURCE_PREFIX + fileName)) {+          try (InputStream is = load(fileName)) {             if (is == null) {               throw new AssertionError("Couldn't find resource: " + fileName);             }-            Files.copyFile(is, new File(m_outputDirectory, fileName));+            Files.copyFile(is, new File(outputDirectory, fileName));           }         }         all = Files.readFile(header);-        Utils.writeUtf8File(m_outputDirectory, "index.html", xsb, all);+        Utils.writeUtf8File(outputDirectory, "index.html", xsb, all);       }     } catch (IOException e) {       Logger.getLogger(Main.class).error(e.getMessage(), e);     }   }++  private InputStream load(String fileName) {+    InputStream is = getClass().getResourceAsStream(Main.TESTNG_RESOURCE_PREFIX + fileName);+    if (is == null) {

It will allow us to manage the path strategy and keep the default fallback.

krmahadevan

comment created time in 2 days

issue commentcbeust/testng

Test with retry analyzer fails when running verify

Whic command you ran for that output?

On Sun, Nov 22, 2020, 11:29 AM Krishnan Mahadevan notifications@github.com wrote:

@javydreamercsw https://github.com/javydreamercsw - I cant reproduce the problem.

Here's the execution output of the attached sample project https://github.com/cbeust/testng/files/5579907/2417_sample.zip

[INFO] ------------------------------------------------------- [INFO] T E S T S [INFO] ------------------------------------------------------- [INFO] Running com.rationaleemotions.issue2417.RetryAnalyzerNGTest Nov 22, 2020 10:57:53 PM com.rationaleemotions.issue2417.RetryAnalyzerTransformer <init> WARNING: com.rationaleemotions.issue2417.RetryAnalyzerTransformer was instantiated. Retrying test testDataProviderRetry(0) Retrying test testDataProviderRetry(1) Retrying test testDataProviderRetry(2) Retrying test testDataProviderRetry(0) Retrying test testDataProviderRetry(1) Retrying test testDataProviderRetry(2) Retrying test testDataProviderRetry(0) Retrying test testDataProviderRetry(1) Retrying test testDataProviderRetry(2) Retrying test testRetry(0) Retrying test testRetry(1) Retrying test testRetry(2) [WARNING] Tests run: 16, Failures: 0, Errors: 0, Skipped: 12, Time elapsed: 0.536 s - in com.rationaleemotions.issue2417.RetryAnalyzerNGTest [INFO] [INFO] Results: [INFO] [INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0 [INFO] [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 3.189 s [INFO] Finished at: 2020-11-22T22:57:54+05:30 [INFO] ------------------------------------------------------------------------

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/cbeust/testng/issues/2417#issuecomment-731781355, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAVP2PZV3BUDUQEQAPB7PXTSRFDBFANCNFSM4TU227LQ .

javydreamercsw

comment created time in 2 days

Pull request review commentcbeust/testng

SkipException for multiple testclasses causes error in the forked process

 public ITestResultNotifier getNotifier() {               testMethod, System.currentTimeMillis(), new Throwable(okToProceed));       m_notifier.addSkippedTest(testMethod, result);       InvokedMethod invokedMethod = new InvokedMethod(System.currentTimeMillis(), result);-      invokeListenersForSkippedTestResult(result, invokedMethod);+      try {+        invokeListenersForSkippedTestResult(result, invokedMethod);+      } catch (Exception e) {

I think Exception is a bit too large. What if we only catch SkipException?

bj-9527

comment created time in 4 days

Pull request review commentcbeust/testng

SkipException for multiple testclasses causes error in the forked process

       <class name="test.listeners.github1296.GitHub1296Test"/>       <class name="test.listeners.github1465.IssueTest"/>       <class name="test.listeners.github2385.IssueTest"/>+      <class name="test.listeners.issue2415.IssueTest"/>

Could you replace "issue" by "github"?

bj-9527

comment created time in 4 days

Pull request review commentcbeust/testng

Fix 2405

 public Injector getInjector(List<Module> moduleInstances) {    @Override   public Injector getInjector(IClass iClass) {+    if (hasNoGuiceAnnotations(iClass)) {

I think the fix won't work because the Injector definition won't exist if guice is not in the classpath.

krmahadevan

comment created time in 4 days

issue commentcbeust/testng

Using softAssert and "hard" Asserts together, softAsserts are swallowed by the "hard" Asserts

@hwulfmeyer at my organisation we noticed/wanted something similar, we might have soft assertions for content checks (like missing periods) and have hard assertions that are primarily used to stop a test when it's obvious that it can't proceed. An example of this would be doing a hard assert for whether the page has loaded properly. Instead of getting a NoSuchElementException since the next element isn't there, we'd get the assertion error.

I can share our solution here if you're still looking to see what you can do, or even I could look to see if this could be a feature added to TestNG for this, but essentially it'd work like this:

-add an overloaded assertAll(Throwable) method to accept a throwable (which you'd pass in from the hard assert that failed) -add a method to SoftAssert to tell you if any soft assertion failures were caught (hasSoftAssertErrors()) -implement 'IHookable' in your base test, and within the run() method catch when a hard assert is thrown -if there are any soft assert failures, you then pass in the hard assert failure to the overloaded assertAll(Throwable) method, and then attach the hard assert along with the other soft asserts.

This ensures that any soft assertion failures are still caught and are not suppressed by hard asserts. We found it to be a relatively robust solution.

You may not want to stop your test for content failures, but you may way to stop if when some API setup call fails for example mid-test. This allows just that :D

hwulfmeyer

comment created time in 5 days

issue commentcbeust/testng

SkipException for multiple testclasses causes error in the forked process

I think maybe wait for the next release :)

dominikQAEngineer

comment created time in 6 days

issue commentcbeust/testng

SkipException for multiple testclasses causes error in the forked process

Hi @bj-9527 thank You for the solution. Do I have to do anything with that or just wait for another release?

dominikQAEngineer

comment created time in 6 days

issue commentcbeust/testng

SkipException for multiple testclasses causes error in the forked process

Hi @dominikQAEngineer ,

I have found the problem and raised a PR, you can check it.

BR

dominikQAEngineer

comment created time in 6 days

PR opened cbeust/testng

SkipException for multiple testclasses causes error in the forked process

SkipException for multiple testclasses causes error in the forked process

Fixes #2415 .

Did you remember to?

  • [ ] Add test case(s)
  • [ ] Update CHANGES.txt

We encourage pull requests that:

  • Add new features to TestNG (or)
  • Fix bugs in TestNG

If your pull request involves fixing SonarQube issues then we would suggest that you please discuss this with the TestNG-dev before you spend time working on it.

+93 -1

0 comment

7 changed files

pr created time in 6 days

issue openedcbeust/testng

I tried to run maven project from command prompt but got weird errors, please help

C:\Users\pc1\eclipse-workspace\Spicejet>mvn test [INFO] Scanning for projects... [INFO] [INFO] ----------------------< SpiceJet:SpicejetFlights >---------------------- [INFO] Building SpicejetFlights 0.0.1-SNAPSHOT [INFO] --------------------------------[ jar ]--------------------------------- [INFO] [INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ SpicejetFlights --- [WARNING] File encoding has not been set, using platform encoding Cp1252, i.e. build is platform dependent! [WARNING] Using platform encoding (Cp1252 actually) to copy filtered resources, i.e. build is platform dependent! [INFO] Copying 2 resources [INFO] [INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ SpicejetFlights --- [INFO] Nothing to compile - all classes are up to date [INFO] [INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ SpicejetFlights --- [WARNING] Using platform encoding (Cp1252 actually) to copy filtered resources, i.e. build is platform dependent! [INFO] Copying 0 resource [INFO] [INFO] --- maven-compiler-plugin:3.1:testCompile (default-testCompile) @ SpicejetFlights --- [INFO] Nothing to compile - all classes are up to date [INFO] [INFO] --- maven-surefire-plugin:3.0.0-M5:test (default-test) @ SpicejetFlights --- [INFO] [INFO] ------------------------------------------------------- [INFO] T E S T S [INFO] ------------------------------------------------------- [INFO] Running TestSuite [ERROR] org/testng/TestNG [INFO] [INFO] Results: [INFO] [INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0 [INFO] [INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] Total time: 4.035 s [INFO] Finished at: 2020-11-18T20:12:56+05:30 [INFO] ------------------------------------------------------------------------ [ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:3.0.0-M5:test (default-test) on project SpicejetFlights: There are test failures. [ERROR] [ERROR] Please refer to C:\Users\pc1\eclipse-workspace\Spicejet\target\surefire-reports for the individual test results. [ERROR] Please refer to dump files (if any exist) [date].dump, [date]-jvmRun[N].dump and [date].dumpstream. [ERROR] There was an error in the forked process [ERROR] org/testng/TestNG [ERROR] org.apache.maven.surefire.booter.SurefireBooterForkException: There was an error in the forked process [ERROR] org/testng/TestNG [ERROR] at org.apache.maven.plugin.surefire.booterclient.ForkStarter.fork(ForkStarter.java:733) [ERROR] at org.apache.maven.plugin.surefire.booterclient.ForkStarter.run(ForkStarter.java:305) [ERROR] at org.apache.maven.plugin.surefire.booterclient.ForkStarter.run(ForkStarter.java:265) [ERROR] at org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeProvider(AbstractSurefireMojo.java:1314) [ERROR] at org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeAfterPreconditionsChecked(AbstractSurefireMojo.java:1159) [ERROR] at org.apache.maven.plugin.surefire.AbstractSurefireMojo.execute(AbstractSurefireMojo.java:932) [ERROR] at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:137) [ERROR] at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:210) [ERROR] at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:156) [ERROR] at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:148) [ERROR] at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:117) [ERROR] at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:81) [ERROR] at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:56) [ERROR] at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128) [ERROR] at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:305) [ERROR] at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:192) [ERROR] at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:105) [ERROR] at org.apache.maven.cli.MavenCli.execute(MavenCli.java:957) [ERROR] at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:289) [ERROR] at org.apache.maven.cli.MavenCli.main(MavenCli.java:193) [ERROR] at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [ERROR] at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) [ERROR] at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [ERROR] at java.base/java.lang.reflect.Method.invoke(Method.java:566) [ERROR] at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:282) [ERROR] at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:225) [ERROR] at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:406) [ERROR] at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:347) [ERROR] [ERROR] -> [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException

created time in 7 days

push eventcbeust/testng

Lorenzo Orsatti

commit sha 6f10ed589be16e3a5f39daa55c7cbd530ee47d12

fix system-out testsuite child element system-out is a simple element. Since system out can include xml tags too, Jenkins junit (included now into resources folder) will fail to parse junit reports. With this change we wrap system-out content in a CDATA section, so that the parser will be happy. At the same time we format the system out input once per line (i.e. with line break).

view details

Julien Herr

commit sha 49070a7cf44c1cff0b5ab62ac4a3b9a36b3eefe6

Merge pull request #2420 from lorsatti/fix/junit-reporter-system-out JUnitReport: Fix system-out testsuite child element (Issue #2419)

view details

push time in 7 days

PR merged cbeust/testng

JUnitReport: Fix system-out testsuite child element (Issue #2419)

system-out is a simple element. Since system out can include xml tags too, Jenkins JUnit (included now into resources folder) will fail to parse JUnit reports.

With this change we wrap system-out content in a CDATA section, so that the parser will be happy.

At the same time we format the system out input once per line (i.e. with line break).

Fixes #2419

+173 -29

0 comment

5 changed files

lorsatti

pr closed time in 7 days

issue closedcbeust/testng

TestNG JUnit reports are not valid if system output contains XML tags

Jenkins JUnit plugin can't parse current JUnit reports if testsuite system-out child element contains XML tags. Excerpt from log:

WARNING: The file 'TEST-MyFeature.xml' is an invalid file.
WARNING: At line 29 of file:TEST-MyFeature:cvc-type.3.1.2: Element 'system-out' is a simple type, so it must have no element information item [children].
WARNING: The result file 'TEST-MyFeature.xml' for the metric 'JUnit' is not valid. The result file has been skipped.

For Jenkins JUnit plugin (see https://github.com/jenkinsci/junit-plugin/blob/d02d08f48e4729e66d6a07fa5c69f154657ff6dd/src/main/java/hudson/tasks/junit/SuiteResult.java#L273 and https://github.com/junit-team/junit5/blob/main/platform-tests/src/test/resources/jenkins-junit.xsd) is this an element of simple type string.

TestNG Version

7.1+

Expected behavior

TestNG generated JUnit report files are correctly parsed in Jenkins, when XML tags are present in system-out element.

Actual behavior

Jenkins can't parse TestNG generated JUnit report files, if the system-out element contains XML tags.

Is the issue reproducible on runner?

  • [ ] Shell
  • [x] Maven
  • [x] Gradle
  • [ ] Ant
  • [ ] Eclipse
  • [ ] IntelliJ
  • [ ] NetBeans

Test case sample

public class TestClassSample {

  public static final String MESSAGE_1 = "Teenage Mutant Ninja Turtles";
  public static final String MESSAGE_2 = "Teenage Mutant Ninja Turtles: <i>Out of the Shadows</i>";

  @Test
  public void testReporter() {
    Reporter.log(MESSAGE_1, true);
    Reporter.log(MESSAGE_2, true);
  }

}

closed time in 7 days

lorsatti

issue openedcbeust/testng

Request for formal release

TestNG Version

7.3.0

Expected behavior

We have uplifted to the latest version of TestNG, but there are some issues blocked us, like #2403 and #2357

Actual behavior

Do we have a plan to do a formal release, a small version uplift like 7.3.1 is OK to us.

Is the issue reproductible on runner?

  • [ ] Shell
  • [ ] Maven
  • [ ] Gradle
  • [ ] Ant
  • [ ] Eclipse
  • [ ] IntelliJ
  • [ ] NetBeans

Test case sample

Please, share the test case (as small as possible) which shows the issue

created time in 7 days

PR opened cbeust/testng

fix system-out testsuite child element

system-out is a simple element. Since system out can include xml tags too, Jenkins JUnit (included now into resources folder) will fail to parse JUnit reports.

With this change we wrap system-out content in a CDATA section, so that the parser will be happy.

At the same time we format the system out input once per line (i.e. with line break).

Fixes #2419

+172 -29

0 comment

4 changed files

pr created time in 8 days

more