Officers were alerted after a man was spotted with the weapon on CCTV cameras.
He was then seen getting into a car.
The vehicle was later stopped by police and four people inside were arrested.
A knife was also seized from the scene. Officers have released a photograph of the weapon, below, on social media.
All four people arrested spent last night in police custody at Southampton Central police station while further investigations were carried out.
The exact location of the incident has not been revealed but police have said it happened in the St Mary’s area of the city.
A statement released by police in Southampton said: “The safety of our communities in paramount and the teams work around the clock to keep everyone safe and remove as many knives as possible from the streets.”