Crusaders goalkeeper Shea Callister was called into action early on to deny Nelson from breaking the deadlock, getting down to his left to prevent the Preston North End loanee from scoring his first goal in Linfield colours.
Callister once again came to the Crues rescue 10 minutes later, this time pulling off an even better save to prevent the Blues from hitting the front by tipping a curling Kyle McClean strike over the crossbar.
Just two minutes after the restart the home side hit the front. Ethan McGee's deflected cross made it all the way to the back post, where Nelson followed in and was on hand to fire home from close range.
The home side dominated proceedings after taking the lead, but were reduced to 10 men with 20 minutes remaining as Millar's late challenge on Kurtis Forsythe gave referee Tim Marshall no choice but to send him off.
Linfield held on despite their numerical disadvantage to secure all three points.