Sam Allardyce's men looked to have earned themselves a replay but Van Dijk - making his first appearance since a £75m move from Southampton - climbed high from a corner to head past Jordan Pickford.
Klopp - who said there was "no rush" to play Van Dijk - has already confirmed that Simon Mignolet will be rested entirely tonight, with deputy Loris Karius in goal and third-choice option Danny Ward likely to be on the bench. "I am really happy".
On a feisty night under the Anfield lights, the world's most expensive defender proved he can be effective at both ends of the pitch by nodding in an 84th-minute corner to make it 2-1.
"It was thoroughly deserved, it felt wonderful to be out there, the goal is very special for me and my family". The goal is very special for me and my family.
The Netherlands worldwide became the first player to score a victor at Anfield in an FA Cup tie against Everton, at the fifth time of asking.
Klopp said the match was not a showcase for the best football, describing it as a "proper fight" between the Merseyside rivals.
Adam Lallana then won the Reds a penalty in the 34th minute, with Mason Holgate adjudged to have fouled him.
"It was a similar situation (to the penalty against Liverpool for Everton last month) - the first one was no penalty in my opinion and this one wasn't also".