[Twisted-Python] running twisted with supervisord -- logging question

[Jonathan Vanasco]

Thanks Hynek and Christopher. 

Hynek- That's basically what I want -- but i am always quickly LOST when it comes to logging. It is my weakness.

Christopher-- The code is definitely contradicting me. I am admittedly lost.

I'll have to play around with this more in depth later -- but now I have an temporarily acceptable solution.

For the most part, I'm not logging anything myself.   I'm really concerned with all the debug messages that various PyPi packages send to the standard library's logging facility.  While running the twisted daemon outside of supervisord, this stuff ends up on the console( which i believe is both stdout/stderr ).

When I wrote "not having supervisord handle the logging" , I should have chosen better words.  It could have been stated as "I don't want /var/log/supervisord.log to handle the logging".  I don't care who handles the logging, I just wanted debug data from my twisted app in it's own file, and not in another.

I was trying to route the stderr into the /var/log/myapp-twisted/log ; but it was going into both that AND the /var/log/supervisord.log file

I tried using different permutations of the `-l` format, but without a bit of configuring, I could only get the "log.debug()" lines in twisted code to output; not the submodules.

After playing around, this seems to work the best:

[supervisord]
logfile=/var/log/supervisord.log
loglevel=warn

[myapp]
redirect_stderr = true
stdout_logfile = /var/log/myapp-twisted/twisted.log

It's not a perfect solution, but works for now --  I just raised the loglevel for supervisord to `warn`, so it doesn't doubly log any data.  The downside is that i'm not getting the debug messages from other apps that supervisord handles.  






On Aug 24, 2014, at 2:36 AM, Hynek Schlawack wrote:

> I would strongly urge you to do what Christopher told you, and not just for Twisted but in general.
> 
> Logging everything to stdout/stderr and then using some proper system tool to catch and process that instead of the finicky stdlib (or Twisted’s for that matter) logging will save you a lot of headaches and gain some love if you work with ops people.  Another alternative is using syslog which is directly supported by twistd.



On Aug 24, 2014, at 1:56 AM, Christopher Armstrong wrote:
> 
> You’re seemingly contradicting yourself: you say you don’t want supervisor to handle logging, but the supervisor configuration you've shown is handling the logging, by routing its stdout to /var/log/myapp-twisted/twisted.log (which I think is a fine thing to do, fwiw).
> 
> If this isn’t working, then the problem is probably in your python code, not the supervisor configuration: are you sure it’s writing logs to stdout? Again, we need to see your actual python code (or a minimal reproducible example) to diagnose this.
> 
> Since you’re not using twisted’s logging system, I really can’t make any guesses here. If you *were* using twisted’s logging system, I would point out that you should add “-l -“ to your twistd command line in that supervisor config, to make twisted’s logs go to stdout as well (maybe you want to do this even if your application isn’t using twisted’s log system, so that both the logs of twisted and of your application are going to the same place).



-------------- next part --------------
An HTML attachment was scrubbed...
URL: </pipermail/twisted-python/attachments/20140825/30d77db6/attachment-0002.html>