Wayne Rooney, Michael Carrick and Ashley Young failed to convert their spot-kicks for the home side, handing the Championship side a famous victory at Old Trafford.
United took the initiative early, keeping a majority of possession in the first half, but failed to create any clear scoring chances.
The home side heaped the pressure on in the second period, but it was Middlesbrough who nearly went ahead first and had a remarkable Daley Blind own goal disallowed for offside in the 58th minute.
United keeper Sergio Romero nearly steered a softly-hit Blind back-pass into his own net, just barely touching the ball wide of his post after an awful attempt at control in the 66th minute.
Two minutes later, Romero redeemed himself with a fine save from a lashed Stewart Downing shot from a tight angle to keep the game even.
Both clubs had chances in the final minutes to steal qualification. Tomas made a reaction save to keep Jesse Lingard’s well-hit shot out in the 90th minute. Romero then easily read and collected Downing’s’s one-on-one chip 30 seconds later, before saving another Downing pinged shot minute later.
Marouane Fellaini came close in first-half stoppage time, but he couldn’t get his head on a tantalising cross in front of a gaping goal mouth.
Minutes after a handball shout in the box, substitute Anthony missed a wide open net with diving header from just outside the six-yard box before Fellaini headed an uncontested attempt over the bar seconds before second-half stoppage time was whistled to an end.
Jurgen Klopp claimed his first win as Liverpool boss as his side saw off Bournemouth 1-0 to advance to the quarterfinals of the Capital One Cup.
Klopp had drawn all three of his previous games — against Tottenham, Rubin Kazan and Southampton — since being appointed by the Reds earlier this month, but got off the mark at Anfield on Wednesday thanks to a goal from Nathaniel Clyne.
Both sides had chances early on with Roberto Firmino slotting just wide inside the first 30 seconds, before Adam Bogdan was on hand to deny Junior Stanislas down the other end with a great save a few minutes later.
It was Liverpool, however, who broke the deadlock through Clyne — scoring his first goal for the club — on 17 minutes as the full-back lashed home the rebound after Adam Smith had cleared Joao Carlos Teixeira’s backheel off the line.
Bournemouth went in search of an equaliser with Bogdan being called into action by Stanislas on two more occasions before the break, although the home side came close to doubling their lead through a long-range Firmino effort, which was tipped over by Adam Federici.
Both sides continued to create chances in the second half, Shaun MacDonald sending a header from a corner agonisingly wide for the visitors and Firmino just missing the target for the hosts.
But Liverpool started to gain more control of the game as the half wore on, with strikes from Lucas Leiva and Teixeira both making Federici work.
And the Reds were eventually able to hold out for the victory in relatively comfortable fashion — bar a late stop from Bogdan to thwart Matt Ritchie — much to the delight of their manager.
Manchester City cruised into the quarterfinals with a 5-1 win over Crystal Palace at the Etihad.
City took the lead in the 22nd minute through Wilfried Bony. Aleksandar Kolarov delivered a corner into the box from the left and Bony met the ball with his head and powered it into the back of the net.
The second goal came a minute before the interval, created by excellent work from Kelechi Iheanacho. The Nigeria Under-20 international burst into the box before playing a perfect ball across the face of goal for Kevin De Bruyne to tap home.
City then lost defender Pablo Zabaleta to injury after a four-minute stoppage, and could face a long layoff.
The win was effectively sealed just before the hour as De Bruyne turned provider for Iheanacho to spin and fire home from ten yards out.
But City weren’t finished there. In the 76th minute, Damien Delaney hauled down Eliaquim Mangala in the area, and Yaya Toure made no mistake from the spot.
Delaney headed home a consolation goal for the Eagles in the 89th minute, and there was still time for City to net a fifth as debutant Manu Garcia slotted home in the 94th minute.
Managerless Aston Villa’s misery continued as they were dumped out of the competition after losing 2-1 to Southampton.
Villa looked most like scoring in the first half with Leandro Bacuna and Rudy Gestede both bringing out stops from Maarten Stekelenburg.
However, Yoshida grabbed Southampton’s opener shortly after the restart, picking out the bottom corner of the net with a low strike from the edge of the box.
And the Saints then doubled their lead through Graziano Pelle, who fired home after being found by a Dusan Tadic delivery.
Scott Sinclair did pull one back for Villa from the penalty spot at the death, but there was not enough time for the visitors to push for an equaliser.