Duplicate logging when using multiple Pylot instances
We are using the pylot in an automatic setup and use it by importing the Pylot
class:
from proffastpylot.prfpylot.pylot import Pylot
def run(site: str, date: str):
Pylot(f"{PROJECT_DIR}/inputs/{site}_{date}.yml", logginglevel="info").run()
When running it like this, the logs seem to duplicate in a very weird way. The logs in between the pylot's logs are from our automation.
Initializing the environment
Determining the next timeseries to process
20211111/ma - Preparing all input files
20211111/ma - Running the proffast pylot
2022-04-08 16:25:17,996, INFO: ++++ Welcome to PROFFASTpylot ++++
2022-04-08 16:25:17,999, INFO: Running preprocess with 1 task(s) ...
2022-04-08 16:25:17,999, INFO: Running preprocess at 2021-11-11 ...
2022-04-08 16:39:16,259, INFO: Finished preprocessing.
2022-04-08 16:39:16,259, INFO: Running pcxs with 1 task(s) ...
2022-04-08 16:39:16,260, INFO: Running pcxs at 2021-11-11 ...
2022-04-08 16:41:50,799, INFO: Finished pcxs.
2022-04-08 16:41:50,800, INFO: Running invers with 1 task(s) ...
2022-04-08 16:46:28,021, INFO: Finished invers.
2022-04-08 16:46:28,471, INFO: The combined results of PROFFAST were written to /home/esm/automated-proffast-pylot/outputs/ma_SN061_20211111_20211111/combined_invparms_ma_20211111-20211111.csv.
2022-04-08 16:46:28,471, INFO: Removing temporary files ...
2022-04-08 16:46:28,524, INFO: Done.
20211111/mb - Preparing all input files
20211111/mb - Running the proffast pylot
2022-04-08 16:46:28,531, INFO: ++++ Welcome to PROFFASTpylot ++++
2022-04-08 16:46:28,531, INFO: ++++ Welcome to PROFFASTpylot ++++
2022-04-08 16:46:28,533, INFO: Running preprocess with 1 task(s) ...
2022-04-08 16:46:28,533, INFO: Running preprocess with 1 task(s) ...
2022-04-08 16:46:28,533, INFO: Running preprocess at 2021-11-11 ...
2022-04-08 16:46:28,533, INFO: Running preprocess at 2021-11-11 ...
2022-04-08 16:57:32,527, INFO: Finished preprocessing.
2022-04-08 16:57:32,527, INFO: Finished preprocessing.
2022-04-08 16:57:32,528, INFO: Running pcxs with 1 task(s) ...
2022-04-08 16:57:32,528, INFO: Running pcxs with 1 task(s) ...
2022-04-08 16:57:32,529, INFO: Running pcxs at 2021-11-11 ...
2022-04-08 16:57:32,529, INFO: Running pcxs at 2021-11-11 ...
2022-04-08 17:00:03,103, INFO: Finished pcxs.
2022-04-08 17:00:03,103, INFO: Finished pcxs.
2022-04-08 17:00:03,103, INFO: Running invers with 1 task(s) ...
2022-04-08 17:00:03,103, INFO: Running invers with 1 task(s) ...
2022-04-08 17:03:30,645, INFO: Finished invers.
2022-04-08 17:03:30,645, INFO: Finished invers.
2022-04-08 17:03:30,828, INFO: The combined results of PROFFAST were written to /home/esm/automated-proffast-pylot/outputs/mb_SN086_20211111_20211111/combined_invparms_mb_20211111-20211111.csv.
2022-04-08 17:03:30,828, INFO: The combined results of PROFFAST were written to /home/esm/automated-proffast-pylot/outputs/mb_SN086_20211111_20211111/combined_invparms_mb_20211111-20211111.csv.
2022-04-08 17:03:30,828, INFO: Removing temporary files ...
2022-04-08 17:03:30,828, INFO: Removing temporary files ...
2022-04-08 17:03:30,887, INFO: Done.
20211111/mc - Preparing all input files
20211111/mc - Running the proffast pylot
2022-04-08 17:03:30,899, INFO: ++++ Welcome to PROFFASTpylot ++++
2022-04-08 17:03:30,899, INFO: ++++ Welcome to PROFFASTpylot ++++
2022-04-08 17:03:30,900, INFO: Running preprocess with 1 task(s) ...
2022-04-08 17:03:30,900, INFO: Running preprocess with 1 task(s) ...
2022-04-08 17:03:30,900, INFO: Running preprocess at 2021-11-11 ...
2022-04-08 17:03:30,900, INFO: Running preprocess at 2021-11-11 ...
2022-04-08 17:09:24,574, INFO: Finished preprocessing.
2022-04-08 17:09:24,574, INFO: Finished preprocessing.
2022-04-08 17:09:24,584, INFO: Running pcxs with 1 task(s) ...
2022-04-08 17:09:24,584, INFO: Running pcxs with 1 task(s) ...
2022-04-08 17:09:24,584, INFO: Running pcxs at 2021-11-11 ...
2022-04-08 17:09:24,584, INFO: Running pcxs at 2021-11-11 ...
2022-04-08 17:11:53,634, INFO: Finished pcxs.
2022-04-08 17:11:53,634, INFO: Finished pcxs.
2022-04-08 17:11:53,635, INFO: Running invers with 1 task(s) ...
2022-04-08 17:11:53,635, INFO: Running invers with 1 task(s) ...
2022-04-08 17:13:34,505, INFO: Finished invers.
2022-04-08 17:13:34,505, INFO: Finished invers.
2022-04-08 17:13:34,619, INFO: The combined results of PROFFAST were written to /home/esm/automated-proffast-pylot/outputs/mc_SN115_20211111_20211111/combined_invparms_mc_20211111-20211111.csv.
2022-04-08 17:13:34,619, INFO: The combined results of PROFFAST were written to /home/esm/automated-proffast-pylot/outputs/mc_SN115_20211111_20211111/combined_invparms_mc_20211111-20211111.csv.
2022-04-08 17:13:34,620, INFO: Removing temporary files ...
2022-04-08 17:13:34,620, INFO: Removing temporary files ...
2022-04-08 17:13:34,673, INFO: Done.
2022-04-08 17:13:34,673, INFO: Done.
20211111/md - Preparing all input files
20211111/md - Running the proffast pylot
2022-04-08 17:13:34,681, INFO: ++++ Welcome to PROFFASTpylot ++++
2022-04-08 17:13:34,681, INFO: ++++ Welcome to PROFFASTpylot ++++
2022-04-08 17:13:34,681, INFO: ++++ Welcome to PROFFASTpylot ++++
2022-04-08 17:13:34,682, INFO: Running preprocess with 1 task(s) ...
2022-04-08 17:13:34,682, INFO: Running preprocess with 1 task(s) ...
2022-04-08 17:13:34,682, INFO: Running preprocess with 1 task(s) ...
2022-04-08 17:13:34,683, INFO: Running preprocess at 2021-11-11 ...
2022-04-08 17:13:34,683, INFO: Running preprocess at 2021-11-11 ...
2022-04-08 17:13:34,683, INFO: Running preprocess at 2021-11-11 ...
2022-04-08 17:15:00,810, INFO: Finished preprocessing.
2022-04-08 17:15:00,810, INFO: Finished preprocessing.
2022-04-08 17:15:00,810, INFO: Finished preprocessing.
2022-04-08 17:15:00,812, INFO: Running pcxs with 1 task(s) ...
2022-04-08 17:15:00,812, INFO: Running pcxs with 1 task(s) ...
2022-04-08 17:15:00,812, INFO: Running pcxs with 1 task(s) ...
2022-04-08 17:15:00,813, INFO: Running pcxs at 2021-11-11 ...
2022-04-08 17:15:00,813, INFO: Running pcxs at 2021-11-11 ...
2022-04-08 17:15:00,813, INFO: Running pcxs at 2021-11-11 ...
2022-04-08 17:17:29,329, INFO: Finished pcxs.
2022-04-08 17:17:29,329, INFO: Finished pcxs.
2022-04-08 17:17:29,329, INFO: Finished pcxs.
2022-04-08 17:17:29,330, INFO: Running invers with 1 task(s) ...
2022-04-08 17:17:29,330, INFO: Running invers with 1 task(s) ...
2022-04-08 17:17:29,330, INFO: Running invers with 1 task(s) ...
2022-04-08 17:17:51,277, INFO: Finished invers.
2022-04-08 17:17:51,277, INFO: Finished invers.
2022-04-08 17:17:51,277, INFO: Finished invers.
2022-04-08 17:17:51,324, INFO: The combined results of PROFFAST were written to /home/esm/automated-proffast-pylot/outputs/md_SN116_20211111_20211111/combined_invparms_md_20211111-20211111.csv.
2022-04-08 17:17:51,324, INFO: The combined results of PROFFAST were written to /home/esm/automated-proffast-pylot/outputs/md_SN116_20211111_20211111/combined_invparms_md_20211111-20211111.csv.
2022-04-08 17:17:51,324, INFO: The combined results of PROFFAST were written to /home/esm/automated-proffast-pylot/outputs/md_SN116_20211111_20211111/combined_invparms_md_20211111-20211111.csv.
2022-04-08 17:17:51,325, INFO: Removing temporary files ...
2022-04-08 17:17:51,325, INFO: Removing temporary files ...
2022-04-08 17:17:51,325, INFO: Removing temporary files ...
2022-04-08 17:17:51,371, INFO: Done.
2022-04-08 17:17:51,371, INFO: Done.
20211111/me - Preparing all input files
20211111/me - Running the proffast pylot
2022-04-08 17:17:51,379, INFO: ++++ Welcome to PROFFASTpylot ++++
2022-04-08 17:17:51,379, INFO: ++++ Welcome to PROFFASTpylot ++++
2022-04-08 17:17:51,379, INFO: ++++ Welcome to PROFFASTpylot ++++
2022-04-08 17:17:51,380, INFO: Running preprocess with 1 task(s) ...
2022-04-08 17:17:51,380, INFO: Running preprocess with 1 task(s) ...
2022-04-08 17:17:51,380, INFO: Running preprocess with 1 task(s) ...
2022-04-08 17:17:51,380, INFO: Running preprocess at 2021-11-11 ...
2022-04-08 17:17:51,380, INFO: Running preprocess at 2021-11-11 ...
2022-04-08 17:17:51,380, INFO: Running preprocess at 2021-11-11 ...
2022-04-08 17:30:24,782, INFO: Finished preprocessing.
2022-04-08 17:30:24,782, INFO: Finished preprocessing.
... and so on ...
Edited by Moritz Makowski