Two-terminal ferroelectric synaptic weights are fabricated on silicon. The active layers consist of a 2 nm thick WOx film and a 2.7 nm thick HfZrO4 (HZO) film grown by atomic layer deposition. The ultra-thin HZO layer is crystallized in the ferroelectric phase using a millisecond flash at a temperature of only 500 °C, evidenced by x-rays diffraction and electron microscopy. The current density is increased by four orders of magnitude compared to weights based on a 5 nm thick HZO film. Potentiation and depression (analog resistive switching) is demonstrated using either pulses of constant duration (as short as 20 nanoseconds) and increasing amplitude, or pulses of constant amplitude (+/−1 V) and increasing duration. The cycle-to-cycle variation is below 1%. Temperature dependent electrical characterisation is performed on a series of device cycled up to 108 times: they reveal that HZO possess semiconducting properties. The fatigue leads to a decrease, in the high resistive state only, of the conductivity and of the activation energy.