Log volume problem

When a problem occurs in our production system, we want to have in our logs all the information necessary to find the cause of the error. In rather complex systems, this leads to the collection of a large amount of data: which processing stages were completed, what were the arguments of some functions when called, what results were returned by calls to external services, etc. The problem here is that we have to collect all this information, even if there is no error. And this leads to an increase in the volume of our log storage, for which we have to pay.

Log levels (error, warning, information, ...) don't help much here. Usually the application has some target log level (for example, information). This means that all records with the level equal to or higher than this target level are logged, and all other records are discarded. But at the moment when an error occurs, it is these debug level entries that we are interested in, which are usually discarded. If the problem repeats frequently, we can temporarily lower the target level, collect all the necessary information, and then return the target level back. In this case, the rate of increase in the volume of the log storage increases only temporarily. But if the error is rare, this approach (although possible) is not very convenient because it leads to the collection of a large amount of data.

Can we improve the situation? I think we can. But here I have to say that in this article I will not offer a ready-made solution. This is just some idea that should be implemented in existing logging systems, as it requires changes to their source code.

Ok. Let's begin.

Comparison of HTTP libraries

In .NET applications, we often need to make HTTP calls. In these cases, we can use the standard HttpClient class or some other library. For example, I have already used Refit and RestSharp. But I have never decided which one to use. Always the library was already utilized in the project I was working with. Therefore, I decided to compare these libraries to form my own meaningful opinion, which one is better and why. This is what I will do in this article.

But how should I compare these libraries? I have no doubt that they all can send HTTP requests and receive responses. After all, these libraries wouldn't have become so popular if they couldn't do that. Therefore, I'm more interested in additional features that are in demand in large corporative applications.

Ok, let's start.

Testing of dependency tree

Today, the use of dependency containers is widespread. The constructor of your classes accepts instances of other classes, they depend on other classes, etc. And the dependency container manages the construction of the entire instance tree.

This system has its price. For example, during testing, you must create instances of all the dependencies of a class in order to test this class. You can use something like Moq for this task. But in this case, there is a problem with class changes. If you want to add or remove any constructor parameter, you will also have to change the tests, even if this parameter does not affect them.

There is another task that we want to solve during testing. Let's say we want to test the work of not one isolated class, but the joint work of several classes in some part of our system. Our dependency container creates a whole tree of instances of various classes. And you want to test the whole tree. Let's see how we can do this, what obstacles we will face and how we can overcome them.