Abd al-Mahdi has said Iraq would maintain strong ties with Iran, but also with the United States and regional neighbors, many of which, like Saudi Arabia, consider Tehran a foe.
The premier went to Riyadh with a large delegation including officials and businessmen, with trade billed as a prime focus of the discussions between OPEC's two largest oil producers.
Abd al-Mahdi's office said the leaders had signed 13 agreements in areas such as trade, energy and political cooperation.
The countries had historically been at loggerheads since the Iraqi invasion of Kuwait in 1990, but Riyadh has wooed Baghdad as part of an effort to stem the growing influence of Tehran.
During his visit to Tehran, Abd al-Mahdi met with President Hassan Rouhani and Supreme Leader Ayatollah Ali Khamenei.
Many of Iraq's leaders, from its Shi'ite majority, have close ties with Iran, the main Shi'ite power in the Middle East.
Earlier this month, Saudi Arabia reopened a consulate in Baghdad which had been closed for 30 years.
King Salman also announced his country would provide Iraq $1 billion to build sport facilities, an announcement which kicked of a two-day visit to Iraq by high-level Saudi officials.