With Huddersfield winning 1-0 against Watford and 17th-placed Swansea drawing 1-1 with Everton, the Saints are in real danger of losing their five-year Premier League status.
Chelsea were back in the contest in the 70th minute, however, when Giroud, who had not always been on the field, headed a Marcos Alonso cross into the bottom corner of the net.
Eden Hazard levelled with an emphatic finish before Giroud - just eight minutes after his first - got a second to leave Saints deflated and defeated. He said he is the first person to take responsibility for that as the coach.
"In every game we must play with great passion, spirit".
The France striker, who joined from Arsenal in January, then completed the sensational comeback 12 minutes from time, finding the bottom-right corner from 14 yards to secure all three points, and he acknowledged he has been itching to get off the mark.
The win left Palace in 16th place, six points clear of Southampton. Although Seamus Coleman hit the bar for Everton late on, the visitors again failed to impress and some fans chanted for manager Sam Allardyce to be sacked even though its side sit comfortably in ninth place.
Burnley's unlikely pursuit of a Europa League spot continued with a 2-1 victory over Leicester thanks to goals from Chris Wood and Kevin Long.