WASHINGTON -- Sen. Joe Manchin (D-W.Va.) may blame the National Rifle Association for the failure of his background check legislation, but he has no plan to leave the group anytime soon.

"Why would I quit when I'm trying to change from within?" he said in response to a question from The Huffington Post at a breakfast discussion on Friday, sponsored by centrist think tank Third Way.

The NRA has for decades been the loudest, most aggressive and best-funded lobbying group working to block gun control legislation. The organization opposed the amendment strengthening and expanding background checks for gun sales that was worked out by Manchin and Sen. Pat Toomey (R-Pa.). It decided to include the vote on its scorecard ranking lawmakers on whether they support gun rights.

Manchin has said that the NRA's decision to score the vote was the main reason his compromise amendment on background checks failed on Wednesday. Without it, he argued, 70 senators -- well above the 60-vote threshold needed -- would have supported it.

The senator is a "life member" of the NRA. He argued on Friday that he and the vast majority of gun owners who support background checks can't just give up on the group; they need to keep pushing it to change.

"They're not going to run me off," he said.

To underscore his point, he pulled his membership card out of his wallet and showed it to The Huffington Post.

"I'm going to say there are more people like me who carry this card, and they're proud to have a card saying they support Second Amendment rights, who enjoy the rights I have as a United States citizen," he said. "I just want my other fellow members to know what we're trying to do. That brings credibility to our cause."

View Manchin's membership card:

"It's easy to quit," Manchin added. "There are two things that are easy to do in this town. Vote no and quit. Think about it. I can vote no and not have to explain to anybody. I can go home. Everybody says, I'm glad you voted that way because I didn't like it either. You follow me? It's the easiest vote I can take up here. But when the facts are supported by what we're doing, I've got to be ready to put it on the line."

Another prominent member of the NRA, Adolphus Busch IV, heir to the Busch family brewing fortune, decided to resign his membership on Thursday.

"The NRA I see today has undermined the values upon which it was established," he wrote in a letter to NRA President David Keene. "Your current strategic focus clearly places priority on the needs of gun and ammunition manufacturers while disregarding the opinions of your 4 million individual members."

"We disagree with his characterization, but we wish him all the best," responded NRA spokesman Andrew Arulanandam.

Pro-gun control groups are already trying to figure out how to turn this week's loss into a win, plotting how to kick out senators who voted against background checks and support those who voted for them.

Manchin, however, may not join in all these efforts. He said on Friday that he would not be campaigning against his fellow senators.

"I will not raise money or work against any colleague, whether I like them or not, whether they're Democrat or Republican," he said. "I am not going to be in an atmosphere in which I'm going to add to the toxicity. It's just wrong."

Also on HuffPost: