“All blacks in this country are condemned to be performers.” This is the thought, allegedly, of Ira Aldridge, a 19th-century black Shakespearean actor, who features in Ekow Eshun’s The Strangers. Why “allegedly”? Because there’s no record of Aldridge’s assertion. Eshun’s book, though, is not a novel; it’s “creative nonfiction”. This innovative approach, while successful, will nonetheless vex historians who wince at the use of speculation, rather than verifiable fact, in presenting the past.

Toni Morrison argued that black lives are “spoken of and written about as objects of history, not subjects within it”. How then to humanise and investigate the interior lives of historical figures in the absence of source material? It’s a dilemma addressed in Eshun’s hybrid of biography and memoir through an extraordinary feat of empathy. Along with Aldridge, Eshun takes four other black pioneers – Matthew Henson (an explorer), Frantz Fanon (a psychiatrist and thinker), Malcolm X (an activist) and Justin Fashanu (a footballer) – and writes about them using the second person. But he does it so intimately that “you” becomes a proxy for “I”.

In the chapter on Aldridge, his 1830s portrayal of Othello “causes only revulsion” in the mind of audiences who cannot accept “a full-blooded Negro, incarnating the profoundest creation of Shakespeare’s art”. But the actor is not enraged by detractors; his mournful response echoes his reflection on how it feels to play Othello in the light of Desdemona’s betrayal: “You feel only grief. The weight of this man’s story bears down on you.” The contrast between Aldridge’s sensitivity and how he is misperceived by the white British public is pitiable.

It’s surprising to learn that for much of his life Eshun, too, has felt misperceived. In decades past, as a cultural commentator – a favourite on the BBC’s Late Review, editor of Arena magazine and director of the Institute of Contemporary Arts – Eshun seemed to transcend race. But, lately, he has addressed it more directly as curator of exhibitions such as In the Black Fantastic and The Time Is Always Now. He is alive to the way in which every Black man is “heir to [a] legacy of caricature, which renders him simultaneously invisible and hypervisible”. Henson, for example, faithfully accompanies American explorer Robert Peary on a gruelling Arctic expedition, subsequently claimed to be the first to reach the north pole. But not only does Peary refuse to share the glory with his companion, critics such as congressman Robert Macon use Henson to cast doubt on the venture, labelling him a “useful Negro tool” and an untrustworthy witness. “The moral support of a white man,” laments the English reporter Henry Lewis, “would have done much toward establishing an unqualified claim of success.”

Fanon, Malcolm X and Fashanu are more widely known than Aldridge and Henson, but the same canopy of sadness and loneliness veils them, and Eshun captures it in his remarkable, imaginative writing. In 1955, working in sympathy with Algerian rebels in the midst of a brutal colonial war with France, Martinique-born psychiatrist Fanon is “stretched taut with the effort of maintaining a double life”. A decade later, ostracised by the Nation of Islam, Malcolm X catches sight of his mentee, Muhammad Ali in Ghana. The pugilist turns his back: “Watching Muhammad’s car roar away, you know it’s not possible for real love to disappear, even in the heat of betrayal.” Fashanu, threatened with lurid tabloid exposé, reluctantly comes out, becoming Britain’s first openly gay footballer before his suicide in 1998.

Stories from Eshun’s own life provide the thread that links his case studies, and include glimpses of unease over his inchoate racial identity. In one stark moment, he confesses that at the age of nine he took a Brillo pad to his face to rid him of its shameful blackness. There are vignettes of other trailblazers, such as the musician and producer Jay-Z. Eshun praises Jay-Z for his “final disavowal of the shell of unfeeling masculinity” that he has carried through life, in favour of “an open embrace of vulnerability, a reckoning with love as the signal marker of black humanness”. It’s a fitting description of the author, too. Ever since leaving home for university, dumping his past into a black bin liner bound for the incinerator, it seems Eshun has been a stranger to himself. His book is an act of rapprochement rendered with great emotional intelligence and tenderness.

The only way to get a look behind the scenes of the Saturday magazine. Sign up to get the inside story from our top writers as well as all the must-read articles and columns, delivered to your inbox every weekend.

after newsletter promotion

The Strangers: Five Extraordinary Black Men and the Worlds That Made Them by Ekow Eshun is published by Hamish Hamilton (£20). To support the Guardian and the Observer order your copy at guardianbookshop.com. Delivery charges may apply.