Prev
Index
Next
centroid_dashboard: Mon Jun 29 13:17:43 2026
File: /proj/sot/ska/data/ska_trend/centroid_dashboard/logs/daily.0/centroid_dashboard.log
| Task | Status | Age | MaxAge | Last Date |
|---|
| centroid_dashboard |
OK
|
0.89 | 1.0 |
Mon Jun 29 13:17:43 2026 |
No errors
File contents:
2026-06-29 13:16:09,242 main: Processing from 2026:173:17:16:09.241 to 2026:180:17:16:09.241
2026-06-29 13:16:16,203 get_observations: Found observation 32405 at 2026:173:19:23:39.816 with 1 manvrs
2026-06-29 13:16:16,203 get_observations: Found observation 32407 at 2026:174:00:15:18.040 with 1 manvrs
2026-06-29 13:16:16,203 get_observations: Found observation 30134 at 2026:174:04:50:09.625 with 1 manvrs
2026-06-29 13:16:16,204 get_observations: Found observation 41394 at 2026:174:09:34:39.704 with 1 manvrs
2026-06-29 13:16:16,204 get_observations: Found observation 41393 at 2026:174:12:55:40.252 with 1 manvrs
2026-06-29 13:16:16,205 get_observations: Found observation 41392 at 2026:174:14:55:40.252 with 1 manvrs
2026-06-29 13:16:16,205 get_observations: Found observation 41391 at 2026:174:18:25:40.252 with 1 manvrs
2026-06-29 13:16:16,205 get_observations: Found observation 30061 at 2026:174:21:28:10.532 with 1 manvrs
2026-06-29 13:16:16,206 get_observations: Found observation 31402 at 2026:175:03:30:52.215 with 1 manvrs
2026-06-29 13:16:16,206 get_observations: Found observation 31403 at 2026:175:10:39:35.400 with 1 manvrs
2026-06-29 13:16:16,207 get_observations: Found observation 31405 at 2026:175:17:44:45.401 with 1 manvrs
2026-06-29 13:16:16,207 get_observations: Found observation 31406 at 2026:176:00:49:55.401 with 1 manvrs
2026-06-29 13:16:16,208 get_observations: Found observation 31611 at 2026:176:08:04:17.279 with 1 manvrs
2026-06-29 13:16:16,208 get_observations: Found observation 30512 at 2026:176:10:14:45.557 with 1 manvrs
2026-06-29 13:16:16,208 get_observations: Found observation 41390 at 2026:176:23:41:47.248 with 1 manvrs
2026-06-29 13:16:16,209 get_observations: Found observation 41389 at 2026:177:00:55:40.252 with 1 manvrs
2026-06-29 13:16:16,209 get_observations: Found observation 41388 at 2026:177:03:25:40.252 with 1 manvrs
2026-06-29 13:16:16,210 get_observations: Found observation 41387 at 2026:177:07:25:40.252 with 1 manvrs
2026-06-29 13:16:16,210 get_observations: Found observation 41386 at 2026:177:09:55:40.252 with 1 manvrs
2026-06-29 13:16:16,210 get_observations: Found observation 31483 at 2026:177:13:31:23.009 with 1 manvrs
2026-06-29 13:16:16,211 get_observations: Found observation 31384 at 2026:177:19:05:05.114 with 1 manvrs
2026-06-29 13:16:16,211 get_observations: Found observation 31385 at 2026:177:19:53:51.437 with 1 manvrs
2026-06-29 13:16:16,212 get_observations: Found observation 31596 at 2026:177:21:07:14.768 with 1 manvrs
2026-06-29 13:16:16,212 get_observations: Found observation 29829 at 2026:178:02:27:51.034 with 1 manvrs
2026-06-29 13:16:16,213 get_observations: Found observation 31597 at 2026:178:07:35:09.169 with 1 manvrs
2026-06-29 13:16:16,213 get_observations: Found observation 31408 at 2026:178:12:04:53.765 with 1 manvrs
2026-06-29 13:16:16,213 get_observations: Found observation 32335 at 2026:178:16:58:23.394 with 1 manvrs
2026-06-29 13:16:16,214 get_observations: Found observation 32403 at 2026:179:01:40:03.076 with 1 manvrs
2026-06-29 13:16:16,214 get_observations: Found observation 32404 at 2026:179:05:08:22.265 with 1 manvrs
2026-06-29 13:16:16,215 get_observations: Found observation 32406 at 2026:179:11:14:33.099 with 1 manvrs
2026-06-29 13:16:16,215 get_observations: Found observation 41385 at 2026:179:16:14:34.593 with 1 manvrs
2026-06-29 13:16:16,215 get_observations: Found observation 41384 at 2026:179:17:48:26.289 with 1 manvrs
2026-06-29 13:16:16,216 get_observations: Found observation 41383 at 2026:179:19:28:40.940 with 1 manvrs
2026-06-29 13:16:16,216 get_observations: Found observation 41382 at 2026:179:20:56:58.149 with 1 manvrs
2026-06-29 13:16:16,217 get_observations: Found observation 41381 at 2026:180:01:55:40.252 with 1 manvrs
2026-06-29 13:16:16,217 get_observations: Found observation 41380 at 2026:180:03:25:40.252 with 1 manvrs
2026-06-29 13:16:16,217 get_observations: Found observation 29996 at 2026:180:04:22:57.002 with 1 manvrs
2026-06-29 13:16:16,218 get_observations: Found observation 30123 at 2026:180:10:49:24.483 with 2 manvrs
2026-06-29 13:16:16,218 get_observations: Found observation 30118 at 2026:180:15:46:31.682 with 2 manvrs
2026-06-29 13:16:16,219 main: Found 39 observations
2026-06-29 13:16:16,219 main: ********************************************************************************
2026-06-29 13:16:16,219 main: Processing observation 32405
2026-06-29 13:16:16,219 main: ********************************************************************************
2026-06-29 13:16:16,222 obs_link_from_info: Found prev obsid 30249
2026-06-29 13:16:16,226 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-06-29 13:16:16,227 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2226A/last.html
2026-06-29 13:16:16,230 process_obs: ObsID 32405 already processed, skipping
2026-06-29 13:16:16,230 main:
2026-06-29 13:16:16,230 main: ********************************************************************************
2026-06-29 13:16:16,230 main: Processing observation 32407
2026-06-29 13:16:16,230 main: ********************************************************************************
2026-06-29 13:16:16,231 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-06-29 13:16:16,232 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2226A/last.html
2026-06-29 13:16:16,235 process_obs: ObsID 32407 already processed, skipping
2026-06-29 13:16:16,235 main:
2026-06-29 13:16:16,235 main: ********************************************************************************
2026-06-29 13:16:16,235 main: Processing observation 30134
2026-06-29 13:16:16,235 main: ********************************************************************************
2026-06-29 13:16:16,236 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-06-29 13:16:16,238 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2226A/last.html
2026-06-29 13:16:16,241 process_obs: ObsID 30134 already processed, skipping
2026-06-29 13:16:16,241 main:
2026-06-29 13:16:16,241 main: ********************************************************************************
2026-06-29 13:16:16,242 main: Processing observation 41394
2026-06-29 13:16:16,242 main: ********************************************************************************
2026-06-29 13:16:16,242 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-06-29 13:16:16,244 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2226A/last.html
2026-06-29 13:16:16,248 process_obs: ObsID 41394 already processed, skipping
2026-06-29 13:16:16,248 main:
2026-06-29 13:16:16,248 main: ********************************************************************************
2026-06-29 13:16:16,248 main: Processing observation 41393
2026-06-29 13:16:16,248 main: ********************************************************************************
2026-06-29 13:16:16,249 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-06-29 13:16:16,250 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2226A/last.html
2026-06-29 13:16:16,253 process_obs: ObsID 41393 already processed, skipping
2026-06-29 13:16:16,253 main:
2026-06-29 13:16:16,253 main: ********************************************************************************
2026-06-29 13:16:16,253 main: Processing observation 41392
2026-06-29 13:16:16,253 main: ********************************************************************************
2026-06-29 13:16:16,254 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-06-29 13:16:16,255 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2226A/last.html
2026-06-29 13:16:16,258 process_obs: ObsID 41392 already processed, skipping
2026-06-29 13:16:16,258 main:
2026-06-29 13:16:16,258 main: ********************************************************************************
2026-06-29 13:16:16,258 main: Processing observation 41391
2026-06-29 13:16:16,258 main: ********************************************************************************
2026-06-29 13:16:16,259 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-06-29 13:16:16,260 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2226A/last.html
2026-06-29 13:16:16,264 process_obs: ObsID 41391 already processed, skipping
2026-06-29 13:16:16,264 main:
2026-06-29 13:16:16,264 main: ********************************************************************************
2026-06-29 13:16:16,264 main: Processing observation 30061
2026-06-29 13:16:16,264 main: ********************************************************************************
2026-06-29 13:16:16,265 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-06-29 13:16:16,266 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2226A/last.html
2026-06-29 13:16:16,269 process_obs: ObsID 30061 already processed, skipping
2026-06-29 13:16:16,269 main:
2026-06-29 13:16:16,269 main: ********************************************************************************
2026-06-29 13:16:16,269 main: Processing observation 31402
2026-06-29 13:16:16,269 main: ********************************************************************************
2026-06-29 13:16:16,270 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-06-29 13:16:16,271 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2226A/last.html
2026-06-29 13:16:16,275 process_obs: ObsID 31402 already processed, skipping
2026-06-29 13:16:16,275 main:
2026-06-29 13:16:16,275 main: ********************************************************************************
2026-06-29 13:16:16,275 main: Processing observation 31403
2026-06-29 13:16:16,275 main: ********************************************************************************
2026-06-29 13:16:16,276 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-06-29 13:16:16,277 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2226A/last.html
2026-06-29 13:16:16,280 process_obs: ObsID 31403 already processed, skipping
2026-06-29 13:16:16,280 main:
2026-06-29 13:16:16,280 main: ********************************************************************************
2026-06-29 13:16:16,280 main: Processing observation 31405
2026-06-29 13:16:16,280 main: ********************************************************************************
2026-06-29 13:16:16,281 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-06-29 13:16:16,282 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2226A/last.html
2026-06-29 13:16:16,285 process_obs: ObsID 31405 already processed, skipping
2026-06-29 13:16:16,285 main:
2026-06-29 13:16:16,285 main: ********************************************************************************
2026-06-29 13:16:16,285 main: Processing observation 31406
2026-06-29 13:16:16,285 main: ********************************************************************************
2026-06-29 13:16:16,286 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-06-29 13:16:16,287 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2226A/last.html
2026-06-29 13:16:16,290 process_obs: ObsID 31406 already processed, skipping
2026-06-29 13:16:16,290 main:
2026-06-29 13:16:16,290 main: ********************************************************************************
2026-06-29 13:16:16,290 main: Processing observation 31611
2026-06-29 13:16:16,290 main: ********************************************************************************
2026-06-29 13:16:16,291 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-06-29 13:16:16,292 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2226A/last.html
2026-06-29 13:16:16,295 process_obs: ObsID 31611 already processed, skipping
2026-06-29 13:16:16,295 main:
2026-06-29 13:16:16,295 main: ********************************************************************************
2026-06-29 13:16:16,295 main: Processing observation 30512
2026-06-29 13:16:16,296 main: ********************************************************************************
2026-06-29 13:16:16,296 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-06-29 13:16:16,297 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2226A/last.html
2026-06-29 12:16:39,789 get_centroid_resids_from_file: Reading centroid residuals from /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2226A/30512/centroid_resids.pkl
2026-06-29 12:16:39,811 write_centroid_resids: Centroid residuals file exists, skipping
2026-06-29 12:16:39,811 plot_crs_time: Plot file exists, skipping
2026-06-29 12:16:39,811 plot_crs_scatter: Plot file exists, skipping
2026-06-29 12:16:39,909 plot_n_kalman_delta_roll: No OBC - GND deltas available, not plotting
2026-06-29 12:16:40,086 plot_n_kalman_delta_roll: Writing plot file /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2226A/30512/n_kalman_delta_roll.png
2026-06-29 12:16:42,420 write_index_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2226A/30512/index.html
2026-06-29 12:16:42,422 write_info_json: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2226A/30512/info.json
2026-06-29 12:16:42,426 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/30/30512/index.html
2026-06-29 12:16:42,430 main:
2026-06-29 12:16:42,430 main: ********************************************************************************
2026-06-29 12:16:42,430 main: Processing observation 41390
2026-06-29 12:16:42,430 main: ********************************************************************************
2026-06-29 12:16:42,432 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-06-29 12:16:42,434 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2226A/last.html
2026-06-29 12:16:42,438 process_obs: ObsID 41390 already processed, skipping
2026-06-29 12:16:42,438 main:
2026-06-29 12:16:42,438 main: ********************************************************************************
2026-06-29 12:16:42,438 main: Processing observation 41389
2026-06-29 12:16:42,438 main: ********************************************************************************
2026-06-29 12:16:42,439 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-06-29 12:16:42,441 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2226A/last.html
2026-06-29 12:16:42,445 process_obs: ObsID 41389 already processed, skipping
2026-06-29 12:16:42,445 main:
2026-06-29 12:16:42,445 main: ********************************************************************************
2026-06-29 12:16:42,445 main: Processing observation 41388
2026-06-29 12:16:42,445 main: ********************************************************************************
2026-06-29 12:16:42,446 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-06-29 12:16:42,447 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2226A/last.html
2026-06-29 12:16:42,450 process_obs: ObsID 41388 already processed, skipping
2026-06-29 12:16:42,451 main:
2026-06-29 12:16:42,451 main: ********************************************************************************
2026-06-29 12:16:42,451 main: Processing observation 41387
2026-06-29 12:16:42,451 main: ********************************************************************************
2026-06-29 12:16:42,452 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-06-29 12:16:42,453 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2226A/last.html
2026-06-29 12:16:42,457 process_obs: ObsID 41387 already processed, skipping
2026-06-29 12:16:42,457 main:
2026-06-29 12:16:42,457 main: ********************************************************************************
2026-06-29 12:16:42,457 main: Processing observation 41386
2026-06-29 12:16:42,457 main: ********************************************************************************
2026-06-29 12:16:42,458 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-06-29 12:16:42,459 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2226A/last.html
2026-06-29 12:16:42,463 process_obs: ObsID 41386 already processed, skipping
2026-06-29 12:16:42,463 main:
2026-06-29 12:16:42,463 main: ********************************************************************************
2026-06-29 12:16:42,463 main: Processing observation 31483
2026-06-29 12:16:42,463 main: ********************************************************************************
2026-06-29 12:16:42,464 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-06-29 12:16:42,467 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2226A/last.html
2026-06-29 12:16:42,474 process_obs: ObsID 31483 already processed, skipping
2026-06-29 12:16:42,474 main:
2026-06-29 12:16:42,474 main: ********************************************************************************
2026-06-29 12:16:42,474 main: Processing observation 31384
2026-06-29 12:16:42,474 main: ********************************************************************************
2026-06-29 12:16:42,475 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-06-29 12:16:42,476 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2226A/last.html
2026-06-29 12:16:42,479 process_obs: ObsID 31384 already processed, skipping
2026-06-29 12:16:42,479 main:
2026-06-29 12:16:42,479 main: ********************************************************************************
2026-06-29 12:16:42,479 main: Processing observation 31385
2026-06-29 12:16:42,480 main: ********************************************************************************
2026-06-29 12:16:42,480 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-06-29 12:16:42,482 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2226A/last.html
2026-06-29 12:16:42,488 process_obs: ObsID 31385 already processed, skipping
2026-06-29 12:16:42,488 main:
2026-06-29 12:16:42,488 main: ********************************************************************************
2026-06-29 12:16:42,488 main: Processing observation 31596
2026-06-29 12:16:42,488 main: ********************************************************************************
2026-06-29 12:16:42,490 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-06-29 12:16:42,491 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2226A/last.html
2026-06-29 12:16:42,494 process_obs: ObsID 31596 already processed, skipping
2026-06-29 12:16:42,495 main:
2026-06-29 12:16:42,495 main: ********************************************************************************
2026-06-29 12:16:42,495 main: Processing observation 29829
2026-06-29 12:16:42,495 main: ********************************************************************************
2026-06-29 12:16:42,496 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-06-29 12:16:42,497 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2226A/last.html
2026-06-29 12:16:42,503 process_obs: ObsID 29829 already processed, skipping
2026-06-29 12:16:42,503 main:
2026-06-29 12:16:42,503 main: ********************************************************************************
2026-06-29 12:16:42,503 main: Processing observation 31597
2026-06-29 12:16:42,503 main: ********************************************************************************
2026-06-29 12:16:42,504 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-06-29 12:16:42,505 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2226A/last.html
2026-06-29 12:16:42,512 process_obs: ObsID 31597 already processed, skipping
2026-06-29 12:16:42,513 main:
2026-06-29 12:16:42,513 main: ********************************************************************************
2026-06-29 12:16:42,513 main: Processing observation 31408
2026-06-29 12:16:42,513 main: ********************************************************************************
2026-06-29 12:16:42,514 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-06-29 12:16:42,518 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2226A/last.html
2026-06-29 12:16:42,524 process_obs: ObsID 31408 already processed, skipping
2026-06-29 12:16:42,524 main:
2026-06-29 12:16:42,524 main: ********************************************************************************
2026-06-29 12:16:42,524 main: Processing observation 32335
2026-06-29 12:16:42,524 main: ********************************************************************************
2026-06-29 12:16:42,525 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-06-29 12:16:42,526 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2226A/last.html
2026-06-29 12:16:44,745 get_centroid_resids_from_file: Reading centroid residuals from /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2226A/32335/centroid_resids.pkl
2026-06-29 12:16:44,752 write_centroid_resids: Centroid residuals file exists, skipping
2026-06-29 12:16:44,752 plot_crs_time: Plot file exists, skipping
2026-06-29 12:16:44,752 plot_crs_scatter: Plot file exists, skipping
2026-06-29 12:16:44,817 plot_n_kalman_delta_roll: No OBC - GND deltas available, not plotting
2026-06-29 12:16:44,979 plot_n_kalman_delta_roll: Writing plot file /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2226A/32335/n_kalman_delta_roll.png
2026-06-29 12:16:47,423 write_index_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2226A/32335/index.html
2026-06-29 12:16:47,425 write_info_json: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2226A/32335/info.json
2026-06-29 12:16:47,427 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/32/32335/index.html
2026-06-29 12:16:47,428 main:
2026-06-29 12:16:47,428 main: ********************************************************************************
2026-06-29 12:16:47,428 main: Processing observation 32403
2026-06-29 12:16:47,428 main: ********************************************************************************
2026-06-29 12:16:47,429 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-06-29 12:16:47,430 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2226A/last.html
2026-06-29 12:16:49,329 get_centroid_resids: Computing centroid residuals from telemetry
2026-06-29 12:16:49,656 write_centroid_resids: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2226A/32403/centroid_resids.pkl with 5 slots
2026-06-29 12:16:49,701 plot_crs_time: Writing plot file /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2226A/32403/centroid_resids_time.png
2026-06-29 12:16:50,221 plot_crs_scatter: Writing plot file /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2226A/32403/centroid_resids_scatter.png
2026-06-29 12:16:50,932 plot_n_kalman_delta_roll: Writing plot file /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2226A/32403/n_kalman_delta_roll.png
2026-06-29 12:16:51,074 write_index_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2226A/32403/index.html
2026-06-29 12:16:51,076 write_info_json: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2226A/32403/info.json
2026-06-29 12:16:51,079 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/32/32403/index.html
2026-06-29 12:16:51,079 main:
2026-06-29 12:16:51,079 main: ********************************************************************************
2026-06-29 12:16:51,079 main: Processing observation 32404
2026-06-29 12:16:51,079 main: ********************************************************************************
2026-06-29 12:16:51,081 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-06-29 12:16:51,082 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2226A/last.html
2026-06-29 12:16:52,916 get_centroid_resids: Computing centroid residuals from telemetry
2026-06-29 12:16:53,224 write_centroid_resids: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2226A/32404/centroid_resids.pkl with 5 slots
2026-06-29 12:16:53,274 plot_crs_time: Writing plot file /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2226A/32404/centroid_resids_time.png
2026-06-29 12:16:53,787 plot_crs_scatter: Writing plot file /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2226A/32404/centroid_resids_scatter.png
2026-06-29 12:16:54,551 plot_n_kalman_delta_roll: Writing plot file /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2226A/32404/n_kalman_delta_roll.png
2026-06-29 12:16:54,691 write_index_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2226A/32404/index.html
2026-06-29 12:16:54,692 write_info_json: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2226A/32404/info.json
2026-06-29 12:16:54,695 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/32/32404/index.html
2026-06-29 12:16:54,696 main:
2026-06-29 12:16:54,696 main: ********************************************************************************
2026-06-29 12:16:54,696 main: Processing observation 32406
2026-06-29 12:16:54,696 main: ********************************************************************************
2026-06-29 12:16:54,697 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-06-29 12:16:54,698 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2226A/last.html
2026-06-29 12:16:56,548 get_centroid_resids: Computing centroid residuals from telemetry
2026-06-29 12:16:56,848 write_centroid_resids: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2226A/32406/centroid_resids.pkl with 5 slots
2026-06-29 12:16:56,905 plot_crs_time: Writing plot file /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2226A/32406/centroid_resids_time.png
2026-06-29 12:16:57,192 plot_crs_scatter: Writing plot file /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2226A/32406/centroid_resids_scatter.png
2026-06-29 12:16:58,004 plot_n_kalman_delta_roll: Writing plot file /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2226A/32406/n_kalman_delta_roll.png
2026-06-29 12:16:58,161 write_index_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2226A/32406/index.html
2026-06-29 12:16:58,162 write_info_json: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2226A/32406/info.json
2026-06-29 12:16:58,165 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/32/32406/index.html
2026-06-29 12:16:58,166 main:
2026-06-29 12:16:58,166 main: ********************************************************************************
2026-06-29 12:16:58,166 main: Processing observation 41385
2026-06-29 12:16:58,166 main: ********************************************************************************
2026-06-29 12:16:58,167 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-06-29 12:16:58,168 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2226A/last.html
2026-06-29 12:17:00,024 get_centroid_resids: Computing centroid residuals from telemetry
2026-06-29 12:17:00,488 write_centroid_resids: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2226A/41385/centroid_resids.pkl with 8 slots
2026-06-29 12:17:00,556 plot_crs_time: Writing plot file /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2226A/41385/centroid_resids_time.png
2026-06-29 12:17:00,865 plot_crs_scatter: Writing plot file /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2226A/41385/centroid_resids_scatter.png
2026-06-29 12:17:00,941 plot_n_kalman_delta_roll: No OBC - GND deltas available, not plotting
2026-06-29 12:17:01,105 plot_n_kalman_delta_roll: Writing plot file /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2226A/41385/n_kalman_delta_roll.png
2026-06-29 12:17:01,273 write_index_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2226A/41385/index.html
2026-06-29 12:17:01,276 write_info_json: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2226A/41385/info.json
2026-06-29 12:17:01,280 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/41/41385/index.html
2026-06-29 12:17:01,281 main:
2026-06-29 12:17:01,281 main: ********************************************************************************
2026-06-29 12:17:01,282 main: Processing observation 41384
2026-06-29 12:17:01,282 main: ********************************************************************************
2026-06-29 12:17:01,283 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-06-29 12:17:01,285 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/last.html
2026-06-29 12:17:04,361 get_centroid_resids: Computing centroid residuals from telemetry
2026-06-29 12:17:04,957 write_centroid_resids: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/41384/centroid_resids.pkl with 8 slots
2026-06-29 12:17:05,020 plot_crs_time: Writing plot file /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/41384/centroid_resids_time.png
2026-06-29 12:17:05,330 plot_crs_scatter: Writing plot file /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/41384/centroid_resids_scatter.png
2026-06-29 12:17:05,408 plot_n_kalman_delta_roll: No OBC - GND deltas available, not plotting
2026-06-29 12:17:05,576 plot_n_kalman_delta_roll: Writing plot file /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/41384/n_kalman_delta_roll.png
2026-06-29 12:17:05,741 write_index_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/41384/index.html
2026-06-29 12:17:05,749 write_info_json: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/41384/info.json
2026-06-29 12:17:05,777 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/41/41384/index.html
2026-06-29 12:17:05,789 main:
2026-06-29 12:17:05,789 main: ********************************************************************************
2026-06-29 12:17:05,789 main: Processing observation 41383
2026-06-29 12:17:05,789 main: ********************************************************************************
2026-06-29 12:17:05,801 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-06-29 12:17:05,813 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/last.html
2026-06-29 12:17:09,982 get_centroid_resids: Computing centroid residuals from telemetry
2026-06-29 12:17:10,520 write_centroid_resids: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/41383/centroid_resids.pkl with 8 slots
2026-06-29 12:17:10,586 plot_crs_time: Writing plot file /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/41383/centroid_resids_time.png
2026-06-29 12:17:10,887 plot_crs_scatter: Writing plot file /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/41383/centroid_resids_scatter.png
2026-06-29 12:17:10,967 plot_n_kalman_delta_roll: No OBC - GND deltas available, not plotting
2026-06-29 12:17:11,131 plot_n_kalman_delta_roll: Writing plot file /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/41383/n_kalman_delta_roll.png
2026-06-29 12:17:11,285 write_index_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/41383/index.html
2026-06-29 12:17:11,286 write_info_json: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/41383/info.json
2026-06-29 12:17:11,289 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/41/41383/index.html
2026-06-29 12:17:11,291 main:
2026-06-29 12:17:11,291 main: ********************************************************************************
2026-06-29 12:17:11,291 main: Processing observation 41382
2026-06-29 12:17:11,291 main: ********************************************************************************
2026-06-29 12:17:11,292 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-06-29 12:17:11,293 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/last.html
2026-06-29 12:17:13,247 __init__: ObsID 41382 has insufficient telemetry, skipping
2026-06-29 12:17:13,248 write_index_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/41382/index.html
2026-06-29 12:17:13,252 write_info_json: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/41382/info.json
2026-06-29 12:17:13,253 main:
2026-06-29 12:17:13,253 main: ********************************************************************************
2026-06-29 12:17:13,253 main: Processing observation 41381
2026-06-29 12:17:13,253 main: ********************************************************************************
2026-06-29 12:17:13,255 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-06-29 12:17:13,256 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/last.html
2026-06-29 12:17:15,097 __init__: ObsID 41381 has insufficient telemetry, skipping
2026-06-29 12:17:15,097 write_index_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/41381/index.html
2026-06-29 12:17:15,102 write_info_json: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/41381/info.json
2026-06-29 12:17:15,103 main:
2026-06-29 12:17:15,103 main: ********************************************************************************
2026-06-29 12:17:15,103 main: Processing observation 41380
2026-06-29 12:17:15,103 main: ********************************************************************************
2026-06-29 12:17:15,104 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-06-29 12:17:15,105 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/last.html
2026-06-29 12:17:17,034 __init__: ObsID 41380 has insufficient telemetry, skipping
2026-06-29 12:17:17,035 write_index_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/41380/index.html
2026-06-29 12:17:17,039 write_info_json: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/41380/info.json
2026-06-29 12:17:17,040 main:
2026-06-29 12:17:17,040 main: ********************************************************************************
2026-06-29 12:17:17,040 main: Processing observation 29996
2026-06-29 12:17:17,040 main: ********************************************************************************
2026-06-29 12:17:17,041 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-06-29 12:17:17,043 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/last.html
2026-06-29 12:17:18,989 __init__: ObsID 29996 has insufficient telemetry, skipping
2026-06-29 12:17:18,990 write_index_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/29996/index.html
2026-06-29 12:17:18,993 get_aberration_correction: More than one entry for 29996 in /proj/sot/ska3/aca/data/mpcrit1/mplogs/2026/JUN2926/oflsa/output/JUN2926A_ManErr.txt. Skipping aber correction
2026-06-29 12:17:18,994 write_info_json: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/29996/info.json
2026-06-29 12:17:18,995 main:
2026-06-29 12:17:18,995 main: ********************************************************************************
2026-06-29 12:17:18,996 main: Processing observation 30123
2026-06-29 12:17:18,996 main: ********************************************************************************
2026-06-29 12:17:18,997 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-06-29 12:17:18,998 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/last.html
2026-06-29 12:17:20,891 __init__: ObsID 30123 has no maneuver event in telemetry, skipping
2026-06-29 12:17:20,891 write_index_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/30123/index.html
2026-06-29 12:17:20,895 get_aberration_correction: More than one entry for 30123 in /proj/sot/ska3/aca/data/mpcrit1/mplogs/2026/JUN2926/oflsa/output/JUN2926A_ManErr.txt. Skipping aber correction
2026-06-29 12:17:20,907 info: could not get attribute kalman_start for obsid 30123
2026-06-29 12:17:20,907 info: could not get attribute kalman_stop for obsid 30123
2026-06-29 12:17:20,907 write_info_json: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/30123/info.json
2026-06-29 12:17:20,908 main:
2026-06-29 12:17:20,908 main: ********************************************************************************
2026-06-29 12:17:20,908 main: Processing observation 30118
2026-06-29 12:17:20,908 main: ********************************************************************************
2026-06-29 12:17:20,908 obs_link_from_info: No /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/30118/info.json file found, obs=None
2026-06-29 12:17:20,909 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-06-29 12:17:20,910 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/last.html
2026-06-29 12:17:22,807 __init__: ObsID 30118 has no maneuver event in telemetry, skipping
2026-06-29 12:17:22,808 write_index_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/30118/index.html
2026-06-29 12:17:22,812 info: could not get attribute kalman_start for obsid 30118
2026-06-29 12:17:22,813 info: could not get attribute kalman_stop for obsid 30118
2026-06-29 12:17:22,813 write_info_json: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/JUN2926A/30118/info.json
2026-06-29 12:17:22,813 main:
2026-06-29 13:17:24,197 run_check: Starting centroid dashboard continuity check
2026-06-29 13:17:24,197 collect_observations: Scanning observations in /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports
2026-06-29 13:17:24,200 collect_observations: Processing year 2020
2026-06-29 13:17:26,283 collect_observations: Processing year 2021
2026-06-29 13:17:29,409 collect_observations: Processing year 2022
2026-06-29 13:17:32,603 collect_observations: Processing year 2023
2026-06-29 13:17:35,805 collect_observations: Processing year 2024
2026-06-29 13:17:38,943 collect_observations: Processing year 2025
2026-06-29 13:17:42,248 collect_observations: Processing year 2026
2026-06-29 13:17:43,527 collect_observations: Collected 14119 observations from 4-digit years
2026-06-29 13:17:43,531 sort_observations_by_date: Sorted observations by date
2026-06-29 13:17:43,531 check_continuity: Starting continuity check from most recent: ObsID 30118 (JUN2926A) at 2026:180:15:29:05.428
2026-06-29 13:17:43,551 check_continuity: Reached chain start at ObsID 47337 (MAR3020A) at 2020:092:05:29:19.712
2026-06-29 13:17:43,551 check_continuity: Chain complete: visited 14119 observations out of 14119 total
2026-06-29 13:17:43,551 run_check: ✓ Observation chain continuity check PASSED
2026-06-29 13:17:43,552 run_check: Summary: 14119 observations from 4-digit years, all properly linked