The 58-year-old Portuguese, who has won 25 trophies in his managerial career, was sacked by Premier League team Tottenham Hotspur in April after 17 months in charge.
Mourinho will take charge of the Italian side ahead of the 2021-22 campaign, with a deal that will run until 30 June 2024.
"The incredible passion of the Roma fans convinced me to accept the job and I cannot wait to start next season," Mourinho said in a statement.
Roma announced earlier on Tuesday that current coach Paulo Fonseca will leave his role at the end of the current campaign.
The club suffered a heavy 6-2 defeat against Manchester United in the Europa League semifinal first-leg last week and currently sits seventh in the league.
"We are thrilled and delighted to welcome José Mourinho into the AS Roma family," said Roma President Dan Friedkin and Vice-President Ryan Friedkin in a statement.
"A great champion who has won trophies at every level, José will provide tremendous leadership and experience to our ambitious project.
"The appointment of José is a huge step in building a long-term and consistent winning culture throughout the club."
Mourinho is one of the most well known and successful managers in the game and has previously coached the likes of Real Madrid, Manchester United, Inter Milan and Chelsea.
This is the second time that Mourinho has worked in Serie A. He was Inter Milan coach between 2008-2010, in his final season guiding the team to a remarkable treble - the Serie A title, the Coppa Italia and the Champions League.
