London's Metropolitan Police said the 27-year-old man was arrested near the gates to Downing Street, where the prime minister's residence is located. Prime Minister Theresa May was not there at the time.
Police beefed up security around Parliament and sealed off the vicinity of May's residence.
The London Metropolitan police said there had been no injuries in the incident.
A House of Commons spokesman said the gates of the British Parliament have been closed, adding that Parliament was not in lockdown.
A Scotland Yard statement said, "The man, aged in his late twenties, was arrested on suspicion of possession of an offensive weapon and on suspicion of the commission, preparation and instigation of acts of terrorism. Knives have been recovered from him."
"He is being detained under the Terrorism Act and is in custody in a south London police station," it added.
Last month, five people were killed and 49 others were injured after an assailant plowed a car into pedestrians and stabbed a police officer near Parliament, an incident that has been declared a terrorist incident. The attacker was also shot dead by the police.