Cabaye clipped Zaha's pass into the top corner before Gunners keeper Emiliano Martinez clumsily brought down Townsend, allowing Luka Milivojevic to confidently tuck in his first Palace goal.
In the second half at Selhurst Park, Arsenal fans were singing "You're not fit to wear the shirt", and there seemed to be a lack of fight and organization among Wenger's team.
Townsend said: "It was an incredible victory".
"We had to work extremely hard".
But Wenger's luck was out and Arsenal's creaky defence was almost breached again when a counter-attack ended with Benteke barging through for a shot that Martinez pushed away.
Locals look to help teams capture championship glory at Frozen Four
While Irish head coach Jeff Jackson was proud of his team's journey to the Frozen Four, he admitted the Irish were simply beaten. I give these kids a lot of credit because I don't think anybody expected them to be here. "It's gonna be a barnburner".
"The plan was about getting the ball to Christian Benteke and then getting the bits off him".
"We implemented Allardyce's tactics to a tee", he told Sky Sports.
"I think we need to get our heads back in the game and focus on Saturday".
"It was a great three points and a much needed three points". We didn't do that after we beat Chelsea, which cancelled out that result.
This was the 29th meeting between managers Arsene Wenger and Sam Allardyce, but despite the latter being widely regarded as such a threat to the Frenchman, this was only his fifth victory.
Agreement 'not possible' in Northern Ireland as government talks fail
Mr Brokenshire said there was "an overwhelming desire" among politicians and the public for "strong and stable devolved government".
If there's one thing Big Sam likes more than a big pint of wine at the end of a long day, it's getting a win over a manager perceived to be better him.
The same can not be said of Arsenal, with Alexis Sanchez the only Gunners player to earn a player rating in excess of 5/10.
'The first thing was to defend and frustrate them, keep them playing sideways, then use the space behind the full-backs.
"The weaknesses with Arsenal have been defensively because they leave (Shkodran) Mustafi and and Gabriel really exposed, Monreal and Bellerin play like right and left wingers, the wingers come inside with the centre forward and they're just left on their own".
Monday night's game drew similarities with the recent implosion against West Brom at the Hawthorns, where Arsenal dominated possession yet conceded clear-cut chances and were punished by the set piece prowess of their opponents.
Jays' offense looking better after Morales' slam
Toronto went up 5-0 on Barney's bunt that first baseman Logan Morrison was charged with an error for mishandling in the seventh. Souza, who has hit well this season, batted leadoff against Sabathia and had two infield hits in the Rays' 5-0 loss.
It did not have the desired effect, though, because Palace were two up soon after thanks to a world-class finish from one-time Arsenal target Cabaye, lifting the ball over the reach of Martinez when picked out by an in-form Zaha.