\chapter{Network Interface}
\label{ni}

The Network Interface is designed to perform the Trigger and Raw Data Readout. In addition
it is possible to generate test patterns to the output port and to allow access to the ports
directly via the CPUs.

%-----------------------------------------------------------------------------
\section{Readout Modes}
\label{ni:rm}

Normally the Readout Mode is set by the Master State Machine (MSM) (Chapter~\ref{sm}). After a
Trigger or Raw Data Readout the Network Interface clears itself. Via the the \reg{NMOD}
register the Readout Mode of the MSM can be ignored and set by the signals {\ttfamily m}.

\register{NMOD}{0x0D40}{NI network mode control}
{Control network mode and clear of Network Interface}
{---- ---- ---- ---- ---- ---- --di cmmm}
{---- ---- ---- ---- ---- ---- --00 1000}
{d & disables the clear of the input fifos \\
 i & ignore readout mode of Master State Machine \\
 c & soft clear of Network Interface (low active) \\
 m & network mode: \\
   & {\ttfamily 000~} : Idle state, wait for Readout Mode \\
   & {\ttfamily 001~} : start Trigger Readout \\
   & {\ttfamily 010~} : start Raw Data Readout \\
   & {\ttfamily 011~} : start Network Mode \\
   & {\ttfamily 100~} : send Test Pattern \\
   & {\ttfamily else} : clear Network Interface \\
}

If \reg{NMOD.i} is set to 1 the Readout Mode can only be changed via the \reg{NMOD.m} number.
The low active \reg{NMOD.c} signal clears the Network Interface.

\begin{figure}[htb]
 \centering
 \includegraphics*[width=10cm]{chap_ni/eps/fsm.eps}
 \caption[Network Interface Readout modes state diagram]{State diagram for Readout modes.}
 \label{fig:fsm}
\end{figure}

As shown in Figure~\ref{fig:fsm} After a successful Trigger or Raw Data Readout the NI clears
itself and waits in the Idle state. Otherwise and in the Network Mode and Test Pattern Mode
the NI can only be resetted by a clear command.

%-----------------------------------------------------------------------------
\subsection{Idle Mode}
\label{ni:rm:idle}

After the NI is cleared it waits alsways in the Idle State for the Network Mode. All module, i.e.
the FIFOs and the input synchronization, are cleared during the Idle Mode.

%-----------------------------------------------------------------------------
\subsection{Trigger Readout}
\label{ni:rm:tr}

If the Trigger Readout is set the NI waits for data from the first data source and directly starts
to write resp.\ forward the data. The Trigger Readout order can be defined by the \reg{NTRO} register.
An entry, i.e.\ {\ttfamily f}, defines the subsequent data source, whereby port~0 ..\ port~3 corresponds
to 0..3 and the internal data is defined as 4. The number 7 is forseen to mark the last data source.

\register{NTRO}{0x0D43}{NI trigger readout order}
{Sets the order of readout for trigger readout more}
{---- ---- ---- --ii iddd cccb bbaa afff}
{---- ---- ---- --11 1111 1111 1111 1100}
{i & send after internal data \\
 d & send after port 3 \\
 c & send after port 2 \\
 b & send after port 1 \\
 a & send after port 0 \\
 f & send at first \\
}

If you set the register \reg{NTRO} i.e.\ to {\ttfamily 0x00014FFB} (or {\ttfamily 0247773} octal,
which is easier to read), the NI starts with port~3, then the internal data and afterwards the data
of port~2 will be sent. After port 2 the End Signatures will be put to the output.

\begin{figure}[htb]
 \centering
 \includegraphics*[width=10cm]{chap_ni/eps/fsm_tr.eps}
 \caption[Network Interface Trigger Readout mode state diagram]{State diagram for Trigger Readout mode.}
 \label{fig:fsmtr}
\end{figure}

After the Trigger Readout mode is set every CPU has to write a 32~Bit word to the Local I/O data input
of the NI at address {\ttfamily 0x0000}. If a CPU has no data to send it has to write the Trigger
Readout End Signature (at least the lower word). The data from the CPUs will be read by the NI in the
order CPU0 ..\ CPU3.

\register{NES}{0x0D45}{NI end signatures}
{End signatures for trigger and raw data readout modes}
{rrrr rrrr rrrr rrrr tttt tttt tttt tttt}
{0000 0000 0000 0000 0000 0000 0000 0000}
{r & low word end signature for raw data readout \\
 t & low word end signature for trigger readout \\
}

The Trigger Readout End Signature can be define via the \reg{NES.t} register. Only the low word
is used as End Signature and will be sent three times after the last data source is sent.

Since only $\sim$40 tracklets per module can be processed (incl. the Global Tracking Unit) in time,
the possibility to cut the incoming Trigger data stream is forseen.

\register{NCUT}{0x0D4C}{NI cut trigger stream}
{Cut incoming trigger readout stream for port 0..3 at word count a..d}
{dddd dddd cccc cccc bbbb bbbb aaaa aaaa}
{1111 1111 1111 1111 1111 1111 1111 1111}
{}

If you set the registers \reg{NCUT.a} ..\ \reg{NCUT.d} the data stream of port~0 ..\ port~3 will
be cutted after the number of words you set. This number should be even.

%-----------------------------------------------------------------------------
\subsection{Raw Data Readout}
\label{ni:rm:rr}

The Readout order of the Raw Data Readout can be set in the same manner as the Trigger Readout
order (\reg{NTRO}) by the \reg{NRRO} register.

\register{NRRO}{0x0D44}{NI raw data readout order}
{Sets the order of readout for raw data readout mode (see Reg.~{\sffamily NTRO} for details)}
{---- ---- ---- --ii iddd cccb bbaa afff}
{---- ---- ---- --11 1111 1111 1111 1100}
{}

In difference to the Trigger Readout the Raw Data Readout starts only if the control input
pin of the output port is set to 'low' or if the 'root' register \reg{NED.r} is set. After the
NI received the Raw Data Readout End Signature, it sets the control output of the
input port to 'low' which will be processed at next. The Signature is set via the \reg{NES} register.

\begin{figure}[htb]
 \centering
 \includegraphics*[width=10cm]{chap_ni/eps/fsm_rr.eps}
 \caption[Network Interface Raw Data Readout mode state diagram]{State diagram for Raw Data Readout mode.}
 \label{fig:fsmrr}
\end{figure}

In the Raw Data Readout mode all CPUs can write into the FIFOs. The last word has to be the
End Signature. If the sending order for the internal data is FIFO~0 .. FIFO3 which corresponds
to CPU~0 .. CPU~3.
%-----------------------------------------------------------------------------
\subsection{Network Mode}
\label{ni:rm:net}

In the Network Mode CPUs can write directly to the output port. The CPU which has access to the
output port is defined by the \reg{NTRO.f} register. All CPU can read from the FIFOs which work
as input buffers. The \reg{NLF.D} bit indicates if there is a valid 32~Bit word in the Local I/O
input.

%-----------------------------------------------------------------------------
\subsection{Test Pattern Mode}
\label{ni:rm:tp}

In the Test Pattern mode the 32~Bit word defined via the \reg{NTP} register is dumped continously
to the output port. To exit from this mode the NI has to be cleared.

\register{NTP}{0x0D46}{NI test pattern}
{Test patters which is sent in test pattern mode}
{pppp pppp pppp pppp pppp pppp pppp pppp}
{0000 0000 0000 0000 1111 1111 1111 1111}
{}

%-----------------------------------------------------------------------------
\section{Architecture}
\label{ni:arch}

The Network Interface is connected to the processor via four Local I/O busses and the Global
I/O bus. The Local I/O interfaces contain mapped status signals and allow to read from the input
ports and write to the output port. The internal data paths have a width of 16~Bit. The conversion
from 32~Bit of the I/O busses is also performed within the I/O interfaces. The Global I/O interface
which also can be accessed via the Configuration Unit contain the configuration registers and global
status signals.

\begin{figure}[htb]
 \centering
 \includegraphics*[angle=270,width=10cm]{chap_ni/eps/niblock.eps}
 \caption[Datapath of Network Interface]{Datapath diagram of the Network Interface. The
 I/O 0..3 and I/O G interfaces connect the NI to the CPUs resp.\ the global bus. Four input
 and one output port are the connection to the outer world. Up to 128 words can be buffered
 into the FIFOs via the input ports as well as via the CPUs.}
 \label{fig:niblock}
\end{figure}

In Trigger Readout and Network mode the CPUs write directly to the output port. The incoming
data is buffered in the FIFOs. In Network mode this data can be read by the CPUs, in Trigger Readout
the data is forwared to the output port depending on the readout order.

In Raw Data Readout mode the CPUs write data into the FIFOs and the complete Raw Data Readout
can be done without the CPUs running. In this mode the corresponding input port is directly
connected to the output.


%-----------------------------------------------------------------------------
\section{Input Ports}
\label{ni:in}

The input ports include the resynchronization from DDR using an strobe signal which is
uncorrelated in phase to the internal clock, the parity and spare bit extraction, the
parity generation and comparison and counters for incoming words and parity errors.

\register{NPn}{0x0D48+n}{NI input port n$\in$[0..3] control}
{Bit positions and control signal for input ports n$\in$[0..3]}
{---- ---- ---- ---- ---- -ppp pfff fecs}
{---- ---- ---- ---- ---- -100 0100 1100}
{p & position of parity bit (after spare) \\
 f & position of spare bit (extracted first) \\
 e & parity error counter enable \\
 c & set the control output via configuration (active low), \reg{NMOD}.m should be 011 (Network Mode)\\
 s & set control output (active low) of input port if c is set (low)\\
}

Figure~\ref{fig:excl} illustrates the functionality of the bit excluding. The extracted parity
bit is compared with the calculated parity for each byte and if the \reg{NPn.e} bit is set an
parity error increases the error counter \reg{NLP.H} resp.\ \reg{NLP.L}. In addition all incoming
word of a port a counted.

\register{NLP}{0x00C1}{NI (LIO) parity and word counter}
{Local I/O status of input port counters}
{---- ---- HHHH HHHH LLLL LLLL CCCC CCCC}
{---- ---- 0000 0000 0000 0000 0000 0000}
{H & input port parity error counter for high word \\
 L & input port parity error counter for low word \\
 C & input port word counter \\
}

%-----------------------------------------------------------------------------
\subsection{Data Resynchronization}
\label{ni:in:resync}

The incoming strobe signal (and the data) can be shifted to the internal clock. To compensate the
differenz in phase the incoming DDR data stream is resynchronized.

\begin{figure}[htb]
 \centering
 \includegraphics*[angle=270,width=14cm]{chap_ni/eps/resync_wave.eps}
 \caption[Waveform of data resynchronization]{Timing diagram of the resynchronization
 unit. At the point 50~ns and 80~ns the incoming strobe signal is shifted related to
 the internal clock signal.}
 \label{fig:resync}
\end{figure}

%-----------------------------------------------------------------------------
\section{Output Port}
\label{ni:out}

The output port has a width of 10~Bit plus the strobe and a control output and a control input.
The internal 16~Bit is splitted into a high- and a low-byte. Each byte is expanded by a parity and
a spare bit. The positions of these bits can be configured via the \reg{NED} register.
Figure~\ref{fig:excl} shows an example of bit insertion if \reg{NED.p} is set to 6 and \reg{NED.f}
is set to 5. Note that the position of the spare bit is relative to the 9~Bit word after the
parity bit was included.

\begin{figure}[htb]
 \centering
 \includegraphics*[width=10cm]{chap_ni/eps/excl.eps}
 \caption[Functionality of bit excluding]{The parity and a spare bit are added to the 10~Bit
 output whereby the position can be freely choosen via the \reg{NED.p} and \reg{NED.f} registers.
 The input port extracts the two bits. This allows a fault tolerance in static bit errors.}
 \label{fig:excl}
\end{figure}

The two 10~Bit words are sent (low..high word) as DDR to the output. To also connect an optical
transmitter there are two modes of output control signals. In default mode the strobe signal
indicates with each transition if a output word is valid. If the Bit \reg{NED.o} is set the
strobe output is used as a clock output and the control output is 'low' if a word is valid. If the
control output is set to 'high' all data bit are set to 'high'.

\register{NED}{0x0D42}{NI output excludes and ctrl delay}
{Set output exclude postitions and control delays}
{---- ---- ---- ---- orpp ppff ffcc csss}
{---- ---- ---- ---- 1010 0010 0100 0000}
{o & optical link mode (clk/ctrl output) \\
 r & root of readout tree \\
 p & position of parity bit (set first) \\
 f & position of spare bit (set after parity position) \\
 c & delay of control signal \\
 s & delay of strobe signal \\
}

%-----------------------------------------------------------------------------
\subsection{Programmable Delay Unit}
\label{ni:out:delay}

Each individual output bit (10~Bit data, strobe, control) can be delayed by the Programmable
Delay Unit (PDU) in steps of $\sim$1~ns. This allows to compensate routing delays on chip and
on board and to shift the data signal to an optimal position in relation to the strobe signal.

\begin{figure}[htb]
 \centering
 \includegraphics*[width=8cm]{chap_ni/eps/delay.eps}
 \caption[Block diagram of the Programmable Delay Unit]{Programmable Delay Unit
 and the cascading of several unit of different delay depths.}
 \label{fig:delay}
\end{figure}

The delays of the data signals can be set via the \reg{NDLY} register. The minimum delay (0) is
$\sim$1.7~ns and the maximum delay (7) is $\sim$8~ns. The Delays of the strobe and control output
can be set by the register \reg{NED.s} and \reg{NED.c}.

\register{NDLY}{0x0D41}{NI data delays}
{Delays of data signals (a/d(0) - j/d(9))}
{--jj jiii hhhg ggff feee dddc ccbb baaa}
{--10 0100 1001 0010 0100 1001 0010 0100}
{}

The first column in Table~\ref{tab:delay} is the selected delay which can be defined for each bit line.
If you take into account only the delay of the standard cells the 'stdc' column shows the corresponding
delay. The other columns show the delays for each signal regarding the extracted layout of
First Encounter.
\begin{table}[htb]
 \centering
 \scriptsize
 \begin{tabular}{|c|c||*{12}{r|}} \hline
  select & stdc & ctrl & strb &   d9 &   d8 &   d7 &   d6 &   d5 &   d4 &   d3 &   d2 &   d1 &   d0 \\ \hline \hline
       0 &  1.7 & 1.72 & 1.70 & 1.43 & 1.44 & 1.59 & 1.46 & 1.39 & 1.30 & 1.28 & 1.35 & 1.35 & 1.38 \\ \hline
       1 &  2.6 & 2.69 & 2.68 & 2.42 & 2.42 & 2.53 & 2.46 & 2.40 & 2.31 & 2.35 & 2.39 & 2.30 & 2.34 \\ \hline
       2 &  3.6 & 3.64 & 3.64 & 3.39 & 3.37 & 3.40 & 3.38 & 3.34 & 3.31 & 3.29 & 3.38 & 3.26 & 3.33 \\ \hline
       3 &  4.4 & 4.48 & 4.48 & 4.22 & 4.21 & 4.22 & 4.21 & 4.18 & 4.14 & 4.16 & 4.22 & 4.09 & 4.15 \\ \hline
       4 &  5.4 & 5.45 & 5.40 & 5.09 & 5.10 & 5.09 & 5.16 & 5.12 & 5.01 & 4.94 & 5.04 & 4.97 & 5.02 \\ \hline
       5 &  6.2 & 6.29 & 6.24 & 5.92 & 5.94 & 5.91 & 5.98 & 5.95 & 5.84 & 5.81 & 5.88 & 5.80 & 5.84 \\ \hline
       6 &  7.0 & 7.09 & 7.07 & 6.75 & 6.76 & 6.74 & 6.81 & 6.77 & 6.70 & 6.60 & 6.70 & 6.63 & 6.68 \\ \hline
       7 &  7.9 & 7.93 & 7.91 & 7.58 & 7.61 & 7.56 & 7.63 & 7.60 & 7.53 & 7.46 & 7.54 & 7.45 & 7.50 \\ \hline
 \end{tabular}
 \caption[Extracted delays of the Programmable Delay Unit]{The Programmable
 Delay Unit offers the shown delays [ns] for each individual Bit line including
 the control and strobe output. The row 'stdc' shows the delays of the standard
 cells only. The other results are calculated
 using the basic extraction of First Encounter and typical timing.}
 \label{tab:delay}
\end{table}

%-----------------------------------------------------------------------------
\subsection{Additional Status \& Control registers}
\label{ni:lgio:pm}

The low/high bounds are programmable boundaries for all FIFOs which affect the
status signals \reg{NLF.H} and \reg{NLF.L}.

\register{NBND}{0x0D47}{NI fifo bounds}
{Programmable FIFO bounds for low/high signals}
{---- ---- ---- ---- hhhh hhhh llll llll}
{---- ---- ---- ---- 0110 0000 0010 0000}
{h & high bound (for all fifos) \\
 l & low bound (for all fifos) \\
}

The following two status registers are mapped into Local I/O and each CPU can read
its correlated FIFO status.

\register{NLF}{0x00C0}{NI (LIO) FIFO states}
{Local I/O status for FIFOs}
{---- ---- ---- ---- ---- -DSS EHLZ YXWV}
{---- ---- ---- ---- ---- -000 0000 0001}
{D & data valid. A 32 bit word can be read from CPU \\
 S & parity status of input port \\
 E & fifo error signal \\
 H & more entries in fifo than programmable high bound \\
 L & less entries in fifo than programmable low bound \\
 Z & fifo full (128*16 bit)\\
 Y & fifo almost full (127*16 bit)\\
 X & fifo half full (64*16 bit) \\
 W & fifo almost empty (1*16 bit) \\
 V & fifo empty \\
}

\register{NLE}{0x00C2}{NI (LIO) FIFO entries}
{Local I/O number of FIFO entries}
{---- ---- ---- ---- ---- ---- EEEE EEEE}
{---- ---- ---- ---- ---- ---- 0000 0000}
{}

In addition the FIFO error flags are mapped into the Global I/O space.

\register{NFE}{0x0DC1}{NI fifo errors}
{FIFO 0..3 error signals mapped to GIO (a..d)}
{---- ---- ---- ---- ---- ---- ---- DCBA}
{---- ---- ---- ---- ---- ---- ---- 0000}
{}

The control input (inverted) of the output port, which is mainly used by the Raw Data Readout can
be read via the Global I/O.

\register{NCTRL}{0x0DC0}{NI control input}
{Control input of output port mapped to GIO}
{---- ---- ---- ---- ---- ---- ---- ---C}
{---- ---- ---- ---- ---- ---- ---- ---0}
{}

For debugging the NI (won't be necessary) the states of the FSM ({\ttfamily ni\_fsm.vhd})
can be read by the Slow Control Network via the Global I/O bus. The coding of the states
is listed in Table \ref{tab:statesfsm}.

\register{NFSM}{0x0DC2}{NI FSM state}
{FSM state of Network Interface mapped to GIO}
{---- ---- ---- ---- ---- ---- ---S SSSS}
{---- ---- ---- ---- ---- ---- ---0 0000}
{}

\begin{table}[htb]
 \centering
 \scriptsize
 \begin{tabular}{|c|c|l|} \hline
  state       &  Nr. & comment                                    \\ \hline \hline
  clear       &   0  & state after reset or sync. clear           \\ \hline
  idle        &   1  & wait for readout mode                      \\ \hline
  idle\_tr    &   2  & ready for trigger readout                  \\ \hline
  idle\_rr    &   3  & ready for raw data readout                 \\ \hline
  idle\_net   &   4  & ready for CPU network                      \\ \hline
  send\_tri0l &   5  & send tracklet data from cpu 0 (low word)   \\ \hline
  send\_tri0h &   6  & send tracklet data from cpu 0 (high word)  \\ \hline
  send\_tri1l &   7  & send tracklet data from cpu 1 (low word)   \\ \hline
  send\_tri1h &   8  & send tracklet data from cpu 1 (high word)  \\ \hline
  send\_tri2l &   9  & send tracklet data from cpu 2 (low word)   \\ \hline
  send\_tri2h &  10  & send tracklet data from cpu 2 (high word)  \\ \hline
  send\_tri3l &  11  & send tracklet data from cpu 3 (low word)   \\ \hline
  send\_tri3h &  12  & send tracklet data from cpu 3 (high word)  \\ \hline
  send\_trp0  &  13  & send tracklet data from port0              \\ \hline
  send\_trp1  &  14  & send tracklet data from port1              \\ \hline
  send\_trp2  &  15  & send tracklet data from port2              \\ \hline
  send\_trp3  &  16  & send tracklet data from port3              \\ \hline
  send\_rrf0  &  17  & send raw data from fifo 0 (cpu 0 data)     \\ \hline
  send\_rrf1  &  18  & send raw data from fifo 1 (cpu 1 data)     \\ \hline
  send\_rrf2  &  19  & send raw data from fifo 2 (cpu 2 data)     \\ \hline
  send\_rrf3  &  20  & send raw data from fifo 3 (cpu 3 data)     \\ \hline
  send\_rrp0  &  21  & send raw data from port 0                  \\ \hline
  send\_rrp1  &  22  & send raw data from port 1                  \\ \hline
  send\_rrp2  &  23  & send raw data from port 2                  \\ \hline
  send\_rrp3  &  24  & send raw data from port 3                  \\ \hline
  finished    &  25  & signal master state machine end of readout \\ \hline
  send\_tpl   &  26  & send low word of test pattern              \\ \hline
  send\_tph   &  27  & send high word of test pattern             \\ \hline

 \end{tabular}
 \caption[The state coding of the NI state machine]
 {The state coding of the NI state machine as can be read at NFSM status
 register}
 \label{tab:statesfsm}
\end{table}
