Excluding lines?

Oct 2, 2014 at 8:21 PM
I've seen code coverage tools that would allow you to exclude code from coverage reports by lines - by adding structured comments, or #pragma statements, etc.

As far as I can see, OpenCover does not support this.

Is this something that it is possible for OpenCover to do?

If yes, are there any plans to support it?
Coordinator
Oct 2, 2014 at 9:51 PM
Currently OpenCover does not support this and has no plans to do so at the moment.

Why would this feature even be needed? I would need a good use case for this other than it is implemented by product X. I currently can't think of any scenario I would need to exclude lines in a method that I couldn't solve through refactoring; assuming I needed to cover those lines. I find most usages of an exclude is hiding some sort of code smell.

I could see how a coverage tool that instruments the source code could do this but OpenCover instruments the IL so to implement this feature would require source code analysis during execution (as I doubt the pragmas are in the PDB file) something OpenCover does not need to do at the moment. However this is for the RAW reporting.

Alternatively, you could modify a reporting tool, like ReportGenerator, to skip those lines during it's highlighting process when it generates the coverage report.

Thoughts?
Oct 6, 2014 at 2:23 PM
What we’re seeing are scattered catch blocks, that are not covered by our tests, because our tests can’t force the specific exceptions we’re catching to be thrown.

A purist would say that we should be mocking everything that our unit under test, to be a proper unit test. To that, I’ll say nonsense.

Unit tests are valuable, but integration tests are also valuable, and we’re not going to stop writing integration tests in our unit testing frameworks. But when we’re running a test against, say, a database, there are exceptions that can be raised that we need to handle, that we cannot force the database framework to throw. Even when we’re testing against an in-memory testing database like Effort.

We could, of course, write a test that mocked out the database entirely, but that’s an enormous amount of work, to little purpose. Entity Framework is not easily mockable.

We’d prefer to be able to mark specific sections of code to be ignored, in the code coverage report.

As I said, this is a functionality that other code coverage tools provide, which should suggest that there are some people who find it useful. Given the way OpenCover seems to be implemented, I’d not be surprised if it simply weren’t possible to flag individual lines to be ignored. The information you’d need to do this may simply not exist in the assemblies and pdbs.

But if it is possible, I really think it should be considered.

What I really think it comes down to is what do we expect to see, from a coverage report.

There seem to be some who think that the percentage covered is a meaningful number. I disagree. True, if I see 20% coverage, I know that things are a mess. But if I see two modules with 85% and 95% coverage, which should I be concerned about? Truth is, there isn't enough information to tell.

What I want to know, when I run a coverage report, is that every line of code under test has either been covered by a test, or has been determined to not need testing.

In other words, I want to see 100% coverage, after filtering out the sections of code I have examined and determined that tests aren't necessary.

I can't get that with OpenCover, as it exists right now.
Coordinator
Oct 7, 2014 at 5:50 AM
Edited Oct 7, 2014 at 5:51 AM
An alternative approach rather than mocking is to use actions e.g. if I had this method
public void SaveCustomers(ILogger logger)
{
  CustomersEntities ctx = CustomersEntities.Context;//)
  try
  {
    // awsome stuff with EntityFramework
    ctx.SaveChanges(); 
  }
  catch(Exception ex)
  {
    // do some awesome logging
    logger.Write(ex);
    throw;
  }
}
now as you say it is hard to get the EF to crash on demand so you can test that you exceptions are logged - however you could do this
public void SaveCustomers(ILogger logger)
{
  CustomersEntities ctx = CustomersEntities.Context;//)
  CallWrapper(() => {
    // awsome stuff with EntityFramework
    ctx.SaveChanges(); 
  }, logger);
}
internal void CallWrapper(Action doSomething, ILogger logger)
{
  try
  {
    doSomething();
  }
  catch(Exception ex)
  {
    // do some awesome logging
    logger.Write(ex);
    throw;
  }
}
now you can test your exception handling independently if you really wanted achieve that 100% coverage.