To be applied if the DAQ does not start and fds indicates that the board(s) are not synchronized.

NEVER DO daq_t0 or module_restart of the board(s) before trying other solutions (this would destroy the time alignment).

First, scroll back in fds to see the times of the boards printed when the DAQ attempted to start. Check the IP of the board(s) not synchronized or in time out.

Safety First

If FDS is running set remote control off to avoid DAQ start buy SAMURAI. This command make it non blocking with SAMURAI DAQ:

remote_control_suspend

Diagnostic

To test the link to board FPGA 0 (or “node 0”)

In a terminal, in the experiment directory, do :

ping <board IP>

To ask for the clock value of FPGA 0:

faster_ping <IP> 0

Node 0 should normally respond to faster_ping with clock : 0.0 s. If time out see corresponding procedure below.

To ask for the clock value of FPGA 1 (or 2):

faster_ping <IP> 1

Procedure if a board is in time out

If node 0 is in time out, in a terminal on expandacq:

  • Type faster_reset_nodes <IP> (this resets the FPGAs of the board without resetting their time stamps)
  • faster_ping <IP> 1 or faster_ping <IP> 2 -> if the time is close to the time of another board, then it is ok.
  • faster_ping <IP> 0 -> should respond clock : 0.0 s.
  • In addition, one can check that the time stamp is effectively aligned with the other ones by starting the DAQ (without saving). If the boards are synchronized, the DAQ should start. Stop the run.
  • If FASTER is in remote_control, ask the colleagues in charge of the main SAMURAI DAQ to stop the run and start a new run. The FASTER DAQ should then start.
  • Re-establish remote control:
remote_control_resume

Procedure if the offset is not reset

Symptoms:

  • faster_ping <IP> 0 returns clock=0.0 s,
  • and faster_ping <IP> 1 (or 2) returns a clock value shifted with respect to other channel clocks.

These symptoms probably indicate that the offset of the channel(s) was not reset at the end of the run.

In fds, type clock_offset_reset (command normally ran at the end of a run) then start (without saving) or faster_ping <IP> 1 (or 2).

If this does not solve the problem, try faster_reset_nodes <IP>, then start (without saving) or faster_ping <IP> 1 (or 2).

If the problem persists, in a terminal type gotoexp, then faster23-toolbox-menu-light.sh, and run Sync MSB T0.

Exit faster23-toolbox-menu-light.

Than, in fds try start (without saving) or faster_ping <IP> 1 (or 2).

If the DAQ starts, stop the run.

If FASTER is in remote_control, ask the colleagues in charge of the main SAMURAI DAQ to stop the run and start a new run. The FASTER DAQ should then start.

Re-establish remote control:

remote_control_resume

If fds crashes

If for some reason fds crashes (no access to the prompt)

In a terminal type:

pgrep -a python3

kill the fds processes

kill <ID process>

Do not start immediately fds in the related tmux, before check that the cards are synchronized in a terminal of the experiment directory

faster23-toolbox-menu-light.sh

and choose “Timing analysis” in the menu

If synchronized you can type fds in the tmux

If not synchronized do the procedure described above