Pyongyang's call for talks comes after North Korean officials refused last week to take back 27 fishermen who wished to return, claiming the four not returning had been brainwashed by Seoul.
But Seoul says the four fishermen not wanting to return have asked to remain in the south.
North Korea also wants all 31 of the fishermen to be present at the proposed talks.
South Korea was ready to hand over the 27 at Panmunjom on the border between the two countries still technically at war since a 1953 cease-fire was signed, effectively creating the two counties.
A South Korean coast guard vessel picked up the ship carrying the 31 fishermen in early February. Their vessel had drifted into South Korean waters near the disputed island Yeonpyeong in the Yellow Sea and around 7 miles from North Korean shores.
The South Korean coast guard towed the vessel to the South Korean port of Incheon.
South Korean officials at first said none of the people on board -- 11 men and 20 women -- showed any wish to defect to the south. But after questioning, four requested to stay, South Korean officials said.
The North Korean government newspaper, Korean Central News, blasted the notion that any of its fishermen had requested to defect.
In an article titled "Verbal notice to the South side," Pyongyang said "if the South Korean authorities do not comply with the DPRK's just demand, it will seriously affect the North-South relations and the South side will be held wholly accountable for it."
The repatriation issue has particular significance because of the location in which South Korea seized the North Korean boat.
In November, North Korea unexpectedly shelled Yeonpyeong, damaging dozens of houses and several military buildings. The barrage killed two South Korean marines and two civilians and injured at least 20 people. South Korean forces returned fire but there were no known causalities on the North Korean side.
The shelling led to an aggressive war of words between the two countries. But even before the shelling, military relations had been tense.
In March last year, South Korea blamed North Korea for the sinking of the 1,200-ton South Korean patrol boat Cheonan and the loss of 46 sailors. North Korea repeatedly denied it had sunk the vessel, despite an international investigation that said it found strong evidence that the Cheonan had been hit by a torpedo of North Korean manufacture.
South Korea also carefully monitors Chinese vessels close to its waters and last week captured two Chinese vessels in the waters around 65 miles south west of Taean, south of Seoul on the west coast.
During the incident, a coast guard vessel fired at a Chinese vessel suspected of illegal fishing in South Korean waters, leaving one fisherman injured, South Korean police said. Two Chinese vessels are being held by the coast guard.
Tensions between the two Koreas, already high, further increased this month. Seoul suspects North Korea of trying to jam South Korean military equipment, including Global Positioning Systems, during major military exercises by the South Korean and U.S. troops.
Last week, many of South Korea's government departments, military systems and financial institutions were on alert after an attempted cyberattack. Suspicion fell on North Korea but an official investigation has been launched by South Korean authorities.