The Colombians opened the World Cup with a comfortable 3-0 rout of Greece on June 14 and followed it up with a 2-1 win against the Ivory Coast five days later, allowing the side to win the group on Tuesday with just a draw.
Jose Pekerman's team opened the scoring in the 17th minute through a penalty kick from Juan Carlos Cuadrado, but Japan pulled back the equalizer in first- half stoppage time through Shinji Okazaki.
Jackson Martinez netted the winner 10 minutes into the second half, before adding a second goal eight minutes from time, while Monaco midfielder James Rodriguez chipped in with two brilliant assists and a goal to hand the points and the group summit to Colombia, which will face Group D runners-up Uruguay on Saturday in Rio de Janeiro.
Japan, meanwhile, bows out of the World Cup on the bottom of the group with one point from its three matches.
The Blue Samurai entered Tuesday's contest needing a win and some help in order to reach the knockout stage, but they went behind after 17 minutes as Yasuyuki Konno hacked down Adrian Ramos in the box and referee Pedro Proenca pointed straight to the spot.
Cuadrado stepped up and blasted it past Japan goalkeeper Eiji Kawashima with pace to give the South American side the early lead.
Japan drew level in first-half stoppage time when Okazaki got on the end of a cross from Keisuke Honda to nod it past Colombia goalkeeper David Ospina and bring the sides level at the intermission.
Rodriguez was inserted at halftime and made a palpable impact in the second 45 minutes.
Rodriguez played provider in the 55th minute to give Colombia the lead for good when he slipped the ball to a wide-open Martinez, who collected and ripped it past Kawashima for his first goal in international competition since 2009.
Colombia continued to pour it on in the second half as Rodriguez again found Martinez with a brilliant through ball eight minutes from time, and the Porto man made no mistake with his left foot for his second goal of the night.
Rodriguez then decided to get into the scoring act for himself, finishing off the rout with a well-taken chipped effort in the final minute of regulation to seal the victory for Colombia.