I have this setup:
main.py /module /module/__init__.py (empty) /module.py
And here is the code for my two files,
import logging from module import module logger = logging.getLogger(__name__) def test(): logger.warning('in main.py/test') def main(): handler = logging.StreamHandler() handler.setLevel(logging.INFO) formatter = logging.Formatter('%(asctime)s %(name)s/%(module)s [%(levelname)s]: %(message)s', '%Y-%m-%d %H:%M:%S') handler.setFormatter(formatter) logger.addHandler(handler) logger.warning('in main.py/main') module.something() if __name__ == "__main__": main()
import logging logger = logging.getLogger(__name__) def something(): logger.warning('in module.py/something')
So, what I noticed is that this outputs the following (notice how the module logger has no formatting):
2019-10-01 09:03:40 __main__/main [WARNING]: in main.py/main in module.py/something
It only seems like only after I make an edit in
main.py to change
logger = logging.getLogger( __ name __ ) to
logger = logging.getLogger() or add
logger = logging.getLogger() after
def main(): that it logs like this (which is what I want):
2019-10-01 09:04:13 root/main [WARNING]: in main.py/main 2019-10-01 09:04:13 module.module/module [WARNING]: in module.py/something
Why is that? I thought that because
main.py is importing
module.py, it is naturally higher on the hierarchical scale so
module.py would inherit the logger settings as defined in
main.py. Do need to explicitly set the root logger (with
logger = logging.getLogger()) in main for the inheritance to work? Did I not configure my folder structure correctly to make
module.py‘s logger inherit
main.py‘s logger settings, or is folder structure irrelevant?
The reason I ask is because I thought one should use
logger = logging.getLogger( __ name __ ) throughout (even in
main.py) and then based on the import structure (or folder structure?), that would determine the hierarchy and loggers would inherit accordingly. And the reason I was making that assumption is because what if I was importing
main.py into another program? I guess my point is, I want to make logging as generic as possible such that I can import one module into another and it always inherits the parent’s logger settings. Is there a way to display the underlying hierarchy of all the modules for debugging/learning purposes?
The logging hierarchy has nothing to do with file structure in your program. The hierarchy is determined only by the names of the loggers. When you configure a logger, all loggers with its name in the prefix of their name are its children and inherit its configuration unless explicitly stated otherwise.
In your example, logging setup has more to do with execution sequence and the names you’ve chosen than anything else. When your program runs, it does the following:
logging.pyfrom the standard library because of
from module import module
- Sets the
- Create a
- Create a
- Run the main function
Some consequences of this sequence of events:
module.loggeris created before
main.logger. This doesn’t affect the behavior you’re seeing, but it’s worth noting under the circumstances.
__main__if you invoke
mainas a script. The behavior you see wouldn’t change if it was called
main, e.g. from
python -m main.
moduleis clearly not in the same hierarchy as
main. Both are descendants of the root logger along different branches.
The last item is really the answer to your question. If you want all the loggers in your program to share the same default logging method, you should configure the root logger, or ensure that they have the same name prefix, which you then configure as if it was the root logger.
You could make all the loggers inherit from
module/module.py, you would do
logger = logging.getLogger('__main__.' + __name__)
The issue here is that the name
__main__ is hard coded. You don’t have a guarantee that it will be
main. You could try
import main in
module so you could do
main.__name__ + '.' + __name__, but that wouldn’t work as expected. If
main was run as
__main__, importing it will actually create a second module object with an entirely separate logging hierarchy.
This is why the root logger has no name. It provides exactly the maintainability and consistency you want. You don’t have to jump through hoops trying to figure out the root name.
That being said, you should still have
main.py logging to the
main logger. The root logger should only be set up in the import guard. That way, if
main is imported as a regular module, it will respect the logging setup of the driver it is running under.
It is conventional to set up the anonymous root logger in the driver of your program. Don’t try to inherit loggers from
__main__ or the driver module name.