Abdel Shafi died at the Shifa Hospital following a two-year battle with cancer, medics and his family said.
After the funeral service, which was held in the cityâ€™s main Al Omari Mosque, his coffin was draped in a Palestinian flag and carried through the streets at the head of thousands of mourners.
In a rare show of unity after months of factional unrest, representatives of all Palestinian political factions and several organisations marched through the streets to a burial site east of the city waving Palestinian flags.
A medical doctor by training, Abdel Shafi was considered one of the leading secular Palestinian nationalist leaders.
He served as chairman of a first legislative council in Gaza in 1962, when it was under Egyptian administration, participated in the creation of the Palestine Liberation Organisation (PLO) in 1964 and became a member of its first executive committee.
Expelled from his native Gaza Strip by Israel several times following its capture in 1967, he eventually returned to the territory, where in 1972 he founded the Palestinian Red Crescent Society which he headed for several years.
After publicly opposing the 1978 Camp David peace accords between Egypt and Israel, he was temporarily prevented from leaving the Gaza Strip by Israel.
He came into the international spotlight when he headed the Palestinian delegation to the Madrid peace conference in 1991.
Abdel Shafi eventually left the negotiating team over his opposition to the Oslo accords, which grew out of secret negotiations between Israel and the Palestinians, and which he predicted would eventually collapse.
A critic of the concentration of power within the PLO, he led a Palestinian delegation to Tunis in 1994 to demand that veteran leader Yasser Arafat share power.
He was elected to the Palestine Legislative Council in 1996 but resigned a year later, saying that the body lacked power to improve the Palestiniansâ€™ lot.
In the final years of his life, Abdel Shafi was a tireless critic of Israeli settlements in the Palestinian territories and advocated unity among the many Palestinian warring factions.
Born in Gaza in 1919, and one of six children, Abdel Shafi studied medicine at the American University of Beirut, graduating in 1943. He later studied surgery at the Miami Valley Hospital in Dayton, Ohio, from 1949 to 1954, his family said.
In 1944, he joined the Jeish Al Badiah (Desert Army) of the British Jordanian Army as a medic. He returned to Gaza in 1945 and opened a private practice.
Abdel Shafi is survived by his wife, four children and seven grandchildren.