Prev
Index
Next
centroid_dashboard: Thu May 28 13:19:59 2026
File: /proj/sot/ska/data/ska_trend/centroid_dashboard/logs/daily.0/centroid_dashboard.log
| Task | Status | Age | MaxAge | Last Date |
|---|
| centroid_dashboard |
OK
|
0.89 | 1.0 |
Thu May 28 13:19:59 2026 |
No errors
File contents:
2026-05-28 13:16:21,824 main: Processing from 2026:141:17:16:21.823 to 2026:148:17:16:21.823
2026-05-28 13:16:29,966 get_observations: Found observation 32349 at 2026:142:01:53:11.983 with 1 manvrs
2026-05-28 13:16:29,966 get_observations: Found observation 30511 at 2026:142:07:06:25.790 with 1 manvrs
2026-05-28 13:16:29,967 get_observations: Found observation 41471 at 2026:142:16:20:23.473 with 1 manvrs
2026-05-28 13:16:29,967 get_observations: Found observation 41470 at 2026:142:17:52:48.661 with 1 manvrs
2026-05-28 13:16:29,968 get_observations: Found observation 41469 at 2026:142:19:23:21.822 with 1 manvrs
2026-05-28 13:16:29,968 get_observations: Found observation 41468 at 2026:142:23:35:40.252 with 1 manvrs
2026-05-28 13:16:29,968 get_observations: Found observation 41467 at 2026:143:01:13:22.082 with 1 manvrs
2026-05-28 13:16:29,969 get_observations: Found observation 32351 at 2026:143:03:08:58.995 with 1 manvrs
2026-05-28 13:16:29,969 get_observations: Found observation 31222 at 2026:143:06:27:34.081 with 1 manvrs
2026-05-28 13:16:29,970 get_observations: Found observation 41466 at 2026:143:20:54:58.569 with 1 manvrs
2026-05-28 13:16:29,970 get_observations: Found observation 32350 at 2026:143:21:46:02.021 with 1 manvrs
2026-05-28 13:16:29,970 get_observations: Found observation 31282 at 2026:144:03:59:56.946 with 1 manvrs
2026-05-28 13:16:29,971 get_observations: Found observation 30166 at 2026:144:09:57:43.377 with 1 manvrs
2026-05-28 13:16:29,971 get_observations: Found observation 32348 at 2026:144:18:52:41.123 with 1 manvrs
2026-05-28 13:16:29,972 get_observations: Found observation 29759 at 2026:144:22:22:47.310 with 1 manvrs
2026-05-28 13:16:29,972 get_observations: Found observation 30269 at 2026:144:23:23:46.387 with 1 manvrs
2026-05-28 13:16:29,972 get_observations: Found observation 41465 at 2026:145:07:41:03.238 with 1 manvrs
2026-05-28 13:16:29,973 get_observations: Found observation 41464 at 2026:145:10:55:40.252 with 1 manvrs
2026-05-28 13:16:29,973 get_observations: Found observation 41463 at 2026:145:13:55:40.252 with 1 manvrs
2026-05-28 13:16:29,974 get_observations: Found observation 41462 at 2026:145:15:35:40.252 with 1 manvrs
2026-05-28 13:16:29,974 get_observations: Found observation 41461 at 2026:145:17:25:40.252 with 1 manvrs
2026-05-28 13:16:29,975 get_observations: Found observation 30131 at 2026:145:18:33:49.065 with 1 manvrs
2026-05-28 13:16:29,975 get_observations: Found observation 31374 at 2026:145:23:17:01.302 with 1 manvrs
2026-05-28 13:16:29,976 get_observations: Found observation 32360 at 2026:146:05:56:52.000 with 1 manvrs
2026-05-28 13:16:29,976 get_observations: Found observation 31375 at 2026:146:10:06:45.175 with 1 manvrs
2026-05-28 13:16:29,976 get_observations: Found observation 30417 at 2026:146:16:11:30.123 with 1 manvrs
2026-05-28 13:16:29,977 get_observations: Found observation 32347 at 2026:146:19:30:50.711 with 1 manvrs
2026-05-28 13:16:29,977 get_observations: Found observation 32364 at 2026:146:22:28:04.859 with 1 manvrs
2026-05-28 13:16:29,978 get_observations: Found observation 30097 at 2026:147:01:42:15.834 with 1 manvrs
2026-05-28 13:16:29,978 get_observations: Found observation 30430 at 2026:147:10:32:55.895 with 1 manvrs
2026-05-28 13:16:29,978 get_observations: Found observation 30437 at 2026:147:13:47:22.592 with 1 manvrs
2026-05-28 13:16:29,979 get_observations: Found observation 41460 at 2026:147:22:36:09.951 with 1 manvrs
2026-05-28 13:16:29,979 get_observations: Found observation 41459 at 2026:148:00:55:40.252 with 1 manvrs
2026-05-28 13:16:29,980 get_observations: Found observation 41458 at 2026:148:02:55:40.252 with 1 manvrs
2026-05-28 13:16:29,980 get_observations: Found observation 41457 at 2026:148:04:45:40.252 with 1 manvrs
2026-05-28 13:16:29,980 get_observations: Found observation 41456 at 2026:148:05:55:40.252 with 1 manvrs
2026-05-28 13:16:29,981 get_observations: Found observation 41455 at 2026:148:07:55:40.252 with 1 manvrs
2026-05-28 13:16:29,981 get_observations: Found observation 30202 at 2026:148:11:05:24.459 with 1 manvrs
2026-05-28 13:16:29,982 get_observations: Found observation 30398 at 2026:148:14:28:11.045 with 1 manvrs
2026-05-28 13:16:29,982 main: Found 39 observations
2026-05-28 13:16:29,982 main: ********************************************************************************
2026-05-28 13:16:29,982 main: Processing observation 32349
2026-05-28 13:16:29,982 main: ********************************************************************************
2026-05-28 13:16:29,988 obs_link_from_info: Found prev obsid 31377
2026-05-28 13:16:30,005 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-05-28 13:16:30,007 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/MAY1826A/last.html
2026-05-28 13:16:30,013 process_obs: ObsID 32349 already processed, skipping
2026-05-28 13:16:30,013 main:
2026-05-28 13:16:30,013 main: ********************************************************************************
2026-05-28 13:16:30,013 main: Processing observation 30511
2026-05-28 13:16:30,013 main: ********************************************************************************
2026-05-28 13:16:30,021 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-05-28 13:16:30,033 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/MAY1826A/last.html
2026-05-28 13:16:30,054 process_obs: ObsID 30511 already processed, skipping
2026-05-28 13:16:30,054 main:
2026-05-28 13:16:30,054 main: ********************************************************************************
2026-05-28 13:16:30,055 main: Processing observation 41471
2026-05-28 13:16:30,055 main: ********************************************************************************
2026-05-28 13:16:30,057 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-05-28 13:16:30,061 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/MAY1826A/last.html
2026-05-28 13:16:30,083 process_obs: ObsID 41471 already processed, skipping
2026-05-28 13:16:30,083 main:
2026-05-28 13:16:30,083 main: ********************************************************************************
2026-05-28 13:16:30,083 main: Processing observation 41470
2026-05-28 13:16:30,083 main: ********************************************************************************
2026-05-28 13:16:30,085 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-05-28 13:16:30,089 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/MAY1826A/last.html
2026-05-28 13:16:30,103 process_obs: ObsID 41470 already processed, skipping
2026-05-28 13:16:30,103 main:
2026-05-28 13:16:30,103 main: ********************************************************************************
2026-05-28 13:16:30,103 main: Processing observation 41469
2026-05-28 13:16:30,103 main: ********************************************************************************
2026-05-28 13:16:30,105 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-05-28 13:16:30,109 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/MAY1826A/last.html
2026-05-28 13:16:30,120 process_obs: ObsID 41469 already processed, skipping
2026-05-28 13:16:30,120 main:
2026-05-28 13:16:30,120 main: ********************************************************************************
2026-05-28 13:16:30,120 main: Processing observation 41468
2026-05-28 13:16:30,120 main: ********************************************************************************
2026-05-28 13:16:30,125 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-05-28 13:16:30,131 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/MAY1826A/last.html
2026-05-28 13:16:30,142 process_obs: ObsID 41468 already processed, skipping
2026-05-28 13:16:30,142 main:
2026-05-28 13:16:30,142 main: ********************************************************************************
2026-05-28 13:16:30,142 main: Processing observation 41467
2026-05-28 13:16:30,142 main: ********************************************************************************
2026-05-28 13:16:30,145 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-05-28 13:16:30,148 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/MAY1826A/last.html
2026-05-28 13:16:30,160 process_obs: ObsID 41467 already processed, skipping
2026-05-28 13:16:30,160 main:
2026-05-28 13:16:30,160 main: ********************************************************************************
2026-05-28 13:16:30,160 main: Processing observation 32351
2026-05-28 13:16:30,160 main: ********************************************************************************
2026-05-28 13:16:30,162 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-05-28 13:16:30,166 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/MAY1826A/last.html
2026-05-28 13:16:30,181 process_obs: ObsID 32351 already processed, skipping
2026-05-28 13:16:30,181 main:
2026-05-28 13:16:30,181 main: ********************************************************************************
2026-05-28 13:16:30,181 main: Processing observation 31222
2026-05-28 13:16:30,181 main: ********************************************************************************
2026-05-28 13:16:30,186 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-05-28 13:16:30,190 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/MAY1826A/last.html
2026-05-28 13:16:30,216 process_obs: ObsID 31222 already processed, skipping
2026-05-28 13:16:30,216 main:
2026-05-28 13:16:30,216 main: ********************************************************************************
2026-05-28 13:16:30,216 main: Processing observation 41466
2026-05-28 13:16:30,216 main: ********************************************************************************
2026-05-28 13:16:30,220 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-05-28 13:16:30,230 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/MAY1826A/last.html
2026-05-28 13:16:30,256 process_obs: ObsID 41466 already processed, skipping
2026-05-28 13:16:30,256 main:
2026-05-28 13:16:30,256 main: ********************************************************************************
2026-05-28 13:16:30,256 main: Processing observation 32350
2026-05-28 13:16:30,256 main: ********************************************************************************
2026-05-28 13:16:30,258 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-05-28 13:16:30,261 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/MAY1826A/last.html
2026-05-28 13:16:30,278 process_obs: ObsID 32350 already processed, skipping
2026-05-28 13:16:30,278 main:
2026-05-28 13:16:30,278 main: ********************************************************************************
2026-05-28 13:16:30,278 main: Processing observation 31282
2026-05-28 13:16:30,278 main: ********************************************************************************
2026-05-28 13:16:30,281 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-05-28 13:16:30,285 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/MAY1826A/last.html
2026-05-28 13:16:30,300 process_obs: ObsID 31282 already processed, skipping
2026-05-28 13:16:30,300 main:
2026-05-28 13:16:30,300 main: ********************************************************************************
2026-05-28 13:16:30,300 main: Processing observation 30166
2026-05-28 13:16:30,300 main: ********************************************************************************
2026-05-28 13:16:30,305 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-05-28 13:16:30,308 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/MAY1826A/last.html
2026-05-28 13:16:30,351 process_obs: ObsID 30166 already processed, skipping
2026-05-28 13:16:30,351 main:
2026-05-28 13:16:30,351 main: ********************************************************************************
2026-05-28 13:16:30,351 main: Processing observation 32348
2026-05-28 13:16:30,351 main: ********************************************************************************
2026-05-28 13:16:30,355 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-05-28 13:16:30,367 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/MAY1826A/last.html
2026-05-28 12:16:36,666 get_centroid_resids_from_file: Reading centroid residuals from /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/MAY1826A/32348/centroid_resids.pkl
2026-05-28 12:16:36,682 write_centroid_resids: Centroid residuals file exists, skipping
2026-05-28 12:16:36,682 plot_crs_time: Plot file exists, skipping
2026-05-28 12:16:36,682 plot_crs_scatter: Plot file exists, skipping
2026-05-28 12:16:38,741 plot_n_kalman_delta_roll: Writing plot file /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/MAY1826A/32348/n_kalman_delta_roll.png
2026-05-28 12:16:43,052 write_index_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/MAY1826A/32348/index.html
2026-05-28 12:16:43,055 write_info_json: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/MAY1826A/32348/info.json
2026-05-28 12:16:43,059 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/32/32348/index.html
2026-05-28 12:16:43,061 main:
2026-05-28 12:16:43,061 main: ********************************************************************************
2026-05-28 12:16:43,061 main: Processing observation 29759
2026-05-28 12:16:43,061 main: ********************************************************************************
2026-05-28 12:16:43,063 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-05-28 12:16:43,064 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/MAY2526A/last.html
2026-05-28 12:16:43,073 process_obs: ObsID 29759 already processed, skipping
2026-05-28 12:16:43,073 main:
2026-05-28 12:16:43,073 main: ********************************************************************************
2026-05-28 12:16:43,073 main: Processing observation 30269
2026-05-28 12:16:43,073 main: ********************************************************************************
2026-05-28 12:16:43,074 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-05-28 12:16:43,076 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/MAY2526A/last.html
2026-05-28 12:16:43,087 process_obs: ObsID 30269 already processed, skipping
2026-05-28 12:16:43,087 main:
2026-05-28 12:16:43,087 main: ********************************************************************************
2026-05-28 12:16:43,087 main: Processing observation 41465
2026-05-28 12:16:43,087 main: ********************************************************************************
2026-05-28 12:16:43,088 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-05-28 12:16:43,090 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/MAY2526A/last.html
2026-05-28 12:16:43,095 process_obs: ObsID 41465 already processed, skipping
2026-05-28 12:16:43,095 main:
2026-05-28 12:16:43,095 main: ********************************************************************************
2026-05-28 12:16:43,095 main: Processing observation 41464
2026-05-28 12:16:43,095 main: ********************************************************************************
2026-05-28 12:16:43,096 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-05-28 12:16:43,097 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/MAY2526A/last.html
2026-05-28 12:16:43,113 process_obs: ObsID 41464 already processed, skipping
2026-05-28 12:16:43,113 main:
2026-05-28 12:16:43,113 main: ********************************************************************************
2026-05-28 12:16:43,113 main: Processing observation 41463
2026-05-28 12:16:43,113 main: ********************************************************************************
2026-05-28 12:16:43,114 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-05-28 12:16:43,115 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/MAY2526A/last.html
2026-05-28 12:16:43,126 process_obs: ObsID 41463 already processed, skipping
2026-05-28 12:16:43,126 main:
2026-05-28 12:16:43,126 main: ********************************************************************************
2026-05-28 12:16:43,126 main: Processing observation 41462
2026-05-28 12:16:43,126 main: ********************************************************************************
2026-05-28 12:16:43,127 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-05-28 12:16:43,128 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/MAY2526A/last.html
2026-05-28 12:16:43,141 process_obs: ObsID 41462 already processed, skipping
2026-05-28 12:16:43,141 main:
2026-05-28 12:16:43,141 main: ********************************************************************************
2026-05-28 12:16:43,141 main: Processing observation 41461
2026-05-28 12:16:43,141 main: ********************************************************************************
2026-05-28 12:16:43,143 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-05-28 12:16:43,145 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/MAY2526A/last.html
2026-05-28 12:16:43,166 process_obs: ObsID 41461 already processed, skipping
2026-05-28 12:16:43,166 main:
2026-05-28 12:16:43,166 main: ********************************************************************************
2026-05-28 12:16:43,166 main: Processing observation 30131
2026-05-28 12:16:43,166 main: ********************************************************************************
2026-05-28 12:16:43,167 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-05-28 12:16:43,169 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/MAY2526A/last.html
2026-05-28 12:16:43,196 process_obs: ObsID 30131 already processed, skipping
2026-05-28 12:16:43,196 main:
2026-05-28 12:16:43,196 main: ********************************************************************************
2026-05-28 12:16:43,196 main: Processing observation 31374
2026-05-28 12:16:43,196 main: ********************************************************************************
2026-05-28 12:16:43,200 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-05-28 12:16:43,227 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/MAY2526A/last.html
2026-05-28 12:16:43,252 process_obs: ObsID 31374 already processed, skipping
2026-05-28 12:16:43,252 main:
2026-05-28 12:16:43,252 main: ********************************************************************************
2026-05-28 12:16:43,252 main: Processing observation 32360
2026-05-28 12:16:43,252 main: ********************************************************************************
2026-05-28 12:16:43,254 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-05-28 12:16:43,255 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/MAY2526A/last.html
2026-05-28 12:16:43,267 process_obs: ObsID 32360 already processed, skipping
2026-05-28 12:16:43,267 main:
2026-05-28 12:16:43,267 main: ********************************************************************************
2026-05-28 12:16:43,267 main: Processing observation 31375
2026-05-28 12:16:43,267 main: ********************************************************************************
2026-05-28 12:16:43,268 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-05-28 12:16:43,270 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/MAY2526A/last.html
2026-05-28 12:16:43,283 process_obs: ObsID 31375 already processed, skipping
2026-05-28 12:16:43,283 main:
2026-05-28 12:16:43,283 main: ********************************************************************************
2026-05-28 12:16:43,283 main: Processing observation 30417
2026-05-28 12:16:43,283 main: ********************************************************************************
2026-05-28 12:16:43,284 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-05-28 12:16:43,285 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/MAY2526A/last.html
2026-05-28 12:16:43,306 process_obs: ObsID 30417 already processed, skipping
2026-05-28 12:16:43,306 main:
2026-05-28 12:16:43,306 main: ********************************************************************************
2026-05-28 12:16:43,306 main: Processing observation 32347
2026-05-28 12:16:43,306 main: ********************************************************************************
2026-05-28 12:16:43,307 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-05-28 12:16:43,308 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/MAY2526A/last.html
2026-05-28 12:16:45,739 get_centroid_resids: Computing centroid residuals from telemetry
2026-05-28 12:16:46,430 write_centroid_resids: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/MAY2526A/32347/centroid_resids.pkl with 4 slots
2026-05-28 12:16:46,468 plot_crs_time: Writing plot file /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/MAY2526A/32347/centroid_resids_time.png
2026-05-28 12:16:46,728 plot_crs_scatter: Writing plot file /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/MAY2526A/32347/centroid_resids_scatter.png
2026-05-28 12:16:46,856 plot_n_kalman_delta_roll: No OBC - GND deltas available, not plotting
2026-05-28 12:16:46,993 plot_n_kalman_delta_roll: Writing plot file /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/MAY2526A/32347/n_kalman_delta_roll.png
2026-05-28 12:16:49,710 write_index_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/MAY2526A/32347/index.html
2026-05-28 12:16:49,712 write_info_json: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/MAY2526A/32347/info.json
2026-05-28 12:16:49,715 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/32/32347/index.html
2026-05-28 12:16:49,716 main:
2026-05-28 12:16:49,716 main: ********************************************************************************
2026-05-28 12:16:49,716 main: Processing observation 32364
2026-05-28 12:16:49,716 main: ********************************************************************************
2026-05-28 12:16:49,717 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-05-28 12:16:49,718 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/MAY2526A/last.html
2026-05-28 12:16:51,921 get_centroid_resids: Computing centroid residuals from telemetry
2026-05-28 12:16:52,360 write_centroid_resids: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/MAY2526A/32364/centroid_resids.pkl with 5 slots
2026-05-28 12:16:52,408 plot_crs_time: Writing plot file /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/MAY2526A/32364/centroid_resids_time.png
2026-05-28 12:16:52,659 plot_crs_scatter: Writing plot file /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/MAY2526A/32364/centroid_resids_scatter.png
2026-05-28 12:16:53,774 plot_n_kalman_delta_roll: Writing plot file /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/MAY2526A/32364/n_kalman_delta_roll.png
2026-05-28 12:16:53,917 write_index_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/MAY2526A/32364/index.html
2026-05-28 12:16:53,919 write_info_json: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/MAY2526A/32364/info.json
2026-05-28 12:16:53,922 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/32/32364/index.html
2026-05-28 12:16:53,923 main:
2026-05-28 12:16:53,923 main: ********************************************************************************
2026-05-28 12:16:53,923 main: Processing observation 30097
2026-05-28 12:16:53,923 main: ********************************************************************************
2026-05-28 12:16:53,924 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-05-28 12:16:53,925 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/MAY2526A/last.html
2026-05-28 12:16:58,609 get_centroid_resids: Computing centroid residuals from telemetry
2026-05-28 12:16:59,457 write_centroid_resids: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/MAY2526A/30097/centroid_resids.pkl with 5 slots
2026-05-28 12:16:59,524 plot_crs_time: Writing plot file /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/MAY2526A/30097/centroid_resids_time.png
2026-05-28 12:16:59,903 plot_crs_scatter: Writing plot file /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/MAY2526A/30097/centroid_resids_scatter.png
2026-05-28 12:17:01,314 plot_n_kalman_delta_roll: Writing plot file /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/MAY2526A/30097/n_kalman_delta_roll.png
2026-05-28 12:17:01,469 write_index_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/MAY2526A/30097/index.html
2026-05-28 12:17:01,471 write_info_json: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/MAY2526A/30097/info.json
2026-05-28 12:17:01,473 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/30/30097/index.html
2026-05-28 12:17:01,474 main:
2026-05-28 12:17:01,474 main: ********************************************************************************
2026-05-28 12:17:01,474 main: Processing observation 30430
2026-05-28 12:17:01,474 main: ********************************************************************************
2026-05-28 12:17:01,476 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-05-28 12:17:01,480 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/MAY2526A/last.html
2026-05-28 12:17:04,961 get_centroid_resids: Computing centroid residuals from telemetry
2026-05-28 12:17:05,388 write_centroid_resids: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/MAY2526A/30430/centroid_resids.pkl with 5 slots
2026-05-28 12:17:05,442 plot_crs_time: Writing plot file /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/MAY2526A/30430/centroid_resids_time.png
2026-05-28 12:17:05,707 plot_crs_scatter: Writing plot file /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/MAY2526A/30430/centroid_resids_scatter.png
2026-05-28 12:17:06,782 plot_n_kalman_delta_roll: Writing plot file /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/MAY2526A/30430/n_kalman_delta_roll.png
2026-05-28 12:17:07,271 write_index_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/MAY2526A/30430/index.html
2026-05-28 12:17:07,275 write_info_json: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/MAY2526A/30430/info.json
2026-05-28 12:17:07,284 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/30/30430/index.html
2026-05-28 12:17:07,285 main:
2026-05-28 12:17:07,285 main: ********************************************************************************
2026-05-28 12:17:07,285 main: Processing observation 30437
2026-05-28 12:17:07,285 main: ********************************************************************************
2026-05-28 12:17:07,286 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-05-28 12:17:07,290 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/MAY2526A/last.html
2026-05-28 12:17:11,695 get_centroid_resids: Computing centroid residuals from telemetry
2026-05-28 12:17:12,225 write_centroid_resids: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/MAY2526A/30437/centroid_resids.pkl with 5 slots
2026-05-28 12:17:12,286 plot_crs_time: Writing plot file /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/MAY2526A/30437/centroid_resids_time.png
2026-05-28 12:17:12,603 plot_crs_scatter: Writing plot file /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/MAY2526A/30437/centroid_resids_scatter.png
2026-05-28 12:17:14,095 plot_n_kalman_delta_roll: Writing plot file /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/MAY2526A/30437/n_kalman_delta_roll.png
2026-05-28 12:17:14,234 write_index_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/MAY2526A/30437/index.html
2026-05-28 12:17:14,236 write_info_json: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/MAY2526A/30437/info.json
2026-05-28 12:17:14,239 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/30/30437/index.html
2026-05-28 12:17:14,240 main:
2026-05-28 12:17:14,240 main: ********************************************************************************
2026-05-28 12:17:14,240 main: Processing observation 41460
2026-05-28 12:17:14,240 main: ********************************************************************************
2026-05-28 12:17:14,242 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-05-28 12:17:14,244 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/MAY2526A/last.html
2026-05-28 12:17:16,688 get_centroid_resids: Computing centroid residuals from telemetry
2026-05-28 12:17:17,536 write_centroid_resids: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/MAY2526A/41460/centroid_resids.pkl with 8 slots
2026-05-28 12:17:17,608 plot_crs_time: Writing plot file /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/MAY2526A/41460/centroid_resids_time.png
2026-05-28 12:17:17,941 plot_crs_scatter: Writing plot file /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/MAY2526A/41460/centroid_resids_scatter.png
2026-05-28 12:17:18,049 plot_n_kalman_delta_roll: No OBC - GND deltas available, not plotting
2026-05-28 12:17:18,175 plot_n_kalman_delta_roll: Writing plot file /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/MAY2526A/41460/n_kalman_delta_roll.png
2026-05-28 12:17:18,339 write_index_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/MAY2526A/41460/index.html
2026-05-28 12:17:18,343 write_info_json: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/MAY2526A/41460/info.json
2026-05-28 12:17:18,346 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/41/41460/index.html
2026-05-28 12:17:18,347 main:
2026-05-28 12:17:18,347 main: ********************************************************************************
2026-05-28 12:17:18,347 main: Processing observation 41459
2026-05-28 12:17:18,347 main: ********************************************************************************
2026-05-28 12:17:18,349 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-05-28 12:17:18,353 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/MAY2526A/last.html
2026-05-28 12:17:20,800 __init__: ObsID 41459 has insufficient telemetry, skipping
2026-05-28 12:17:20,803 write_index_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/MAY2526A/41459/index.html
2026-05-28 12:17:20,811 write_info_json: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/MAY2526A/41459/info.json
2026-05-28 12:17:20,813 main:
2026-05-28 12:17:20,813 main: ********************************************************************************
2026-05-28 12:17:20,814 main: Processing observation 41458
2026-05-28 12:17:20,814 main: ********************************************************************************
2026-05-28 12:17:20,817 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-05-28 12:17:20,822 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/MAY2526A/last.html
2026-05-28 12:17:23,489 __init__: ObsID 41458 has insufficient telemetry, skipping
2026-05-28 12:17:23,490 write_index_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/MAY2526A/41458/index.html
2026-05-28 12:17:23,496 write_info_json: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/MAY2526A/41458/info.json
2026-05-28 12:17:23,498 main:
2026-05-28 12:17:23,498 main: ********************************************************************************
2026-05-28 12:17:23,498 main: Processing observation 41457
2026-05-28 12:17:23,498 main: ********************************************************************************
2026-05-28 12:17:23,501 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-05-28 12:17:23,503 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/MAY2526A/last.html
2026-05-28 12:17:26,205 __init__: ObsID 41457 has insufficient telemetry, skipping
2026-05-28 12:17:26,205 write_index_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/MAY2526A/41457/index.html
2026-05-28 12:17:26,215 write_info_json: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/MAY2526A/41457/info.json
2026-05-28 12:17:26,217 main:
2026-05-28 12:17:26,217 main: ********************************************************************************
2026-05-28 12:17:26,217 main: Processing observation 41456
2026-05-28 12:17:26,217 main: ********************************************************************************
2026-05-28 12:17:26,220 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-05-28 12:17:26,226 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/MAY2526A/last.html
2026-05-28 12:17:29,004 __init__: ObsID 41456 has insufficient telemetry, skipping
2026-05-28 12:17:29,004 write_index_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/MAY2526A/41456/index.html
2026-05-28 12:17:29,012 write_info_json: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/MAY2526A/41456/info.json
2026-05-28 12:17:29,014 main:
2026-05-28 12:17:29,014 main: ********************************************************************************
2026-05-28 12:17:29,014 main: Processing observation 41455
2026-05-28 12:17:29,014 main: ********************************************************************************
2026-05-28 12:17:29,016 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-05-28 12:17:29,019 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/MAY2526A/last.html
2026-05-28 12:17:31,787 __init__: ObsID 41455 has no maneuver event in telemetry, skipping
2026-05-28 12:17:31,789 write_index_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/MAY2526A/41455/index.html
2026-05-28 12:17:31,796 info: could not get attribute kalman_start for obsid 41455
2026-05-28 12:17:31,796 info: could not get attribute kalman_stop for obsid 41455
2026-05-28 12:17:31,796 write_info_json: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/MAY2526A/41455/info.json
2026-05-28 12:17:31,797 main:
2026-05-28 12:17:31,797 main: ********************************************************************************
2026-05-28 12:17:31,797 main: Processing observation 30202
2026-05-28 12:17:31,798 main: ********************************************************************************
2026-05-28 12:17:31,799 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-05-28 12:17:31,802 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/MAY2526A/last.html
2026-05-28 12:17:34,420 __init__: ObsID 30202 has no maneuver event in telemetry, skipping
2026-05-28 12:17:34,421 write_index_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/MAY2526A/30202/index.html
2026-05-28 12:17:34,427 info: could not get attribute kalman_start for obsid 30202
2026-05-28 12:17:34,427 info: could not get attribute kalman_stop for obsid 30202
2026-05-28 12:17:34,427 write_info_json: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/MAY2526A/30202/info.json
2026-05-28 12:17:34,428 main:
2026-05-28 12:17:34,428 main: ********************************************************************************
2026-05-28 12:17:34,428 main: Processing observation 30398
2026-05-28 12:17:34,428 main: ********************************************************************************
2026-05-28 12:17:34,429 obs_link_from_info: No /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/MAY2526A/30398/info.json file found, obs=None
2026-05-28 12:17:34,431 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-05-28 12:17:34,434 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/MAY2526A/last.html
2026-05-28 12:17:36,951 __init__: ObsID 30398 has no maneuver event in telemetry, skipping
2026-05-28 12:17:36,952 write_index_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/MAY2526A/30398/index.html
2026-05-28 12:17:36,959 info: could not get attribute kalman_start for obsid 30398
2026-05-28 12:17:36,959 info: could not get attribute kalman_stop for obsid 30398
2026-05-28 12:17:36,959 write_info_json: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/MAY2526A/30398/info.json
2026-05-28 12:17:36,961 main:
2026-05-28 13:17:39,988 run_check: Starting centroid dashboard continuity check
2026-05-28 13:17:39,989 collect_observations: Scanning observations in /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports
2026-05-28 13:17:39,994 collect_observations: Processing year 2020
2026-05-28 13:17:54,479 collect_observations: Processing year 2021
2026-05-28 13:18:16,700 collect_observations: Processing year 2022
2026-05-28 13:18:41,792 collect_observations: Processing year 2023
2026-05-28 13:19:07,590 collect_observations: Processing year 2024
2026-05-28 13:19:33,102 collect_observations: Processing year 2025
2026-05-28 13:19:52,803 collect_observations: Processing year 2026
2026-05-28 13:19:58,976 collect_observations: Collected 13936 observations from 4-digit years
2026-05-28 13:19:58,984 sort_observations_by_date: Sorted observations by date
2026-05-28 13:19:58,984 check_continuity: Starting continuity check from most recent: ObsID 30398 (MAY2526A) at 2026:148:13:56:30.207
2026-05-28 13:19:59,005 check_continuity: Reached chain start at ObsID 47337 (MAR3020A) at 2020:092:05:29:19.712
2026-05-28 13:19:59,005 check_continuity: Chain complete: visited 13936 observations out of 13936 total
2026-05-28 13:19:59,006 run_check: ✓ Observation chain continuity check PASSED
2026-05-28 13:19:59,007 run_check: Summary: 13936 observations from 4-digit years, all properly linked