Brown withdrew his agreement to take part in the debate, sponsored by the Edward M. Kennedy Institute, because the widow of the late Sen. Ted Kennedy, Victoria Kennedy, would not agree to the Brown campaign's demand that she refrain from endorsing a candidate in the November senatorial election, the Boston Herald reported.
Jim Barnett, Brown's campaign manager, issued a statement saying the campaign "cannot accept a debate invitation from someone who plans to endorse Scott Brown's opponent."
"The Kennedy Institute cannot hold itself out as a non-partisan debate sponsor while the president of its board of trustees gets involved in the race on behalf of one of the candidates," the statement said.
Kennedy Institute officials said in a written statement a pledge by Victoria Kennedy not to endorse a candidate "is unprecedented and is not being required of any other persons or entities."
"To us, such a pledge seems inappropriate when a non-media sponsor issues a debate invitation," the officials said. "We can assure both campaigns that the debate will be fair -- just as the one we co-hosted between Senator Brown and [Massachusetts] Attorney General [Martha] Coakley ... was fair."
Brown defeated Coakley in 2010 in a special election to fill the seat left vacant by the death of Ted Kennedy.
Brown had accepted the Kennedy Institute invitation Monday to debate Warren, but only if local media sponsored the event, rather than MSNBC, and that Victoria Kennedy remain publicly neutral in the campaign.
MSNBC said Tuesday there had never been a plan for it to be a debate sponsor. The Herald did not indicate whether a date had been set for the debate.
Notable deaths of 2014 [PHOTOS]