Panetta told CNN the United States is "within an inch of war almost every day in that part of the world, and we just have to be very careful about what we say and what we do."
Since last week's failed rocket launch by Pyongyang there has been speculation that North Korea would carry out a nuclear test like it did after a previous launch failure.
Panetta said such a move would be considered a provocation and "worsen our relationship."
However, the defense secretary refused to discuss specific actions the United States would take in response.
He spoke from NATO headquarters in Brussels where he and Secretary of State Hillary Clinton are attending meetings to prepare for next month's NATO summit in Chicago.
Clinton said it was too early to determine if relations on the Korean peninsula will shift under the leadership of Kim Jong Un who took over as head of the secretive nation following the death of his father Kim Jong Il in December.