Forty years ago the topic of slums was a lot hotter in the United States than it is now. The Cold War was on, and Soviet propaganda could make hay with America’s urban riots in the mid-1960s. The Black Panthers organized armed patrols, set up free schoolchildren’s breakfast programmes and formed alliances with such urban gangs as Chicago’s Blackstone Rangers. American radicals started organizing in the ghettoes. In 1966 Malcolm X—the man who really frightened America’s ruling orders—was assassinated, probably with police connivance, in New York. Amid the uprisings that followed Martin Luther King’s murder on April 4, 1968 the young Panther leader Bobby Hutton was gunned down by Oakland cops, having surrendered after a police onslaught on the house he was living in. In December of the following year the Chicago cops, with fbi assistance, murdered Panther leader Fred Hampton in his bed. It was open season on the Black Panthers, many of whom were killed. Spiro Agnew, Nixon’s vice president, advised a sense of distance from urban policy: ‘If you’ve seen one city slum you’ve seen them all’, he nonchalantly declared. Enlightened opinion duly looked the other way, and that is how it has been ever since.

When slum dwellers finally did press upon public attention, as the flood waters in New Orleans forced them onto the roofs of their homes and then onto the elevated stretches of Interstate 10, armed police blocked the exit ramps. Plans rapidly unfurled to knock down the big public-housing projects, bulldoze the low-income neighbourhoods and disperse the poor blacks into the hinterland. In New York, as Forrest Hylton described in a 2007 CounterPunch article—and Deborah and Rodrick Wallace before him, in A Plague on Your Houses (1999)—city agencies, the rand corporation and the intellectual artificers of ‘planned shrinkage’ conspired together as ‘finishers’ of whole neighbourhoods, even boroughs. Between 1970 and 1980, 1.3 million white people left the city and some 600,000 blacks and Latinos were displaced within it, as thousands of homes were confiscated, flattened by bulldozers or burned down in huge gentrification programmes. If you add up the forcible clearances of poor people across the past forty years, from New York and the East Coast across the heartlands and on to San Francisco and Los Angeles, you have a chronicle of forced displacements, totting up to many millions of people, which still continues.

Today, in sync with this historical arc, the vast slum projects on Chicago’s South Side known as the Robert Taylor Homes, setting for Sudhir Venkatesh’s Gang Leader for a Day, no longer exist. The bulldozers started rolling in the early 1990s, only thirty years after the mini-city of 28 high rises went up. It was constructed on French modernist principles, a 2-mile by 2-block concrete desert in which the Chicago Housing Authority had very loose authority over 27,000 people: 99.9 per cent black, 95 per cent jobless and on welfare, over 40 per cent of the heads of household being single mothers. Venkatesh’s colourful and sympathetic memoir is a snapshot, like those you see stuck on posts alongside American highways where a car or truck took its human cargo into the hereafter. This is not the reminiscence of a denizen of the Projects, but of a sociologist who encountered the Homes at the start of the 1990s in their terminal stage: dangerous and filthy, controlled by drug gangs, the cops present mostly to accept bribes or extort levies from the gangs at gunpoint.

Born in Madras and raised in comfortable middle-class academic circumstances in southern California, Venkatesh embarked on his PhD in sociology at the University of Chicago in 1989. The dominant figure in the department at that time was William Julius Wilson, famous for arguing in such books as When Work Disappears and The Truly Disadvantaged that, contrary to depictions of ghetto blacks in right-wing bestiaries, which spoke of psychological, intellectual and even genetic deficits, the core problem was work. Without stable, well-paid jobs, any community will slide downhill, with blacks at the bottom of the heap.

Venkatesh soon got bored poring over data sets and yearned to scrutinize actual poor people. In the case of the University of Chicago, as with many other top-tier American campuses, desperately poor black people were available for scrutiny only a few blocks away. Wilson was embarking on a big new study of poverty and told Venkatesh to put together a questionnaire and start interviewing. To the homeboys lounging about in the stairwells of the Projects, selling crack and fending off competition, Venkatesh must have been an odd sight: a tall, dark-skinned fellow with a pony-tail and a tie-dye shirt, memento of his Deadhead cultural affiliations, flourishing a researcher’s clipboard and asking, ‘How does it feel to be black and poor?’ They figure him as a member of a Mexican gang, or an Arab, and hold him till the gang leader, J. T., a college dropout with a talent for organizing, assays Venkatesh’s academic credentials, his origins, and in short order says he can stay around—thus setting Venkatesh on a path that would eventually lead him to Harvard and then to Columbia University. His ten years of research in the Robert Taylor Homes have already yielded two formal academic works: in 2000, American Project: The Rise and Fall of a Modern Ghetto; and in 2006, Off the Books: the Underground Economy of the Urban Poor; as well as a documentary, DisLocation, in 2005. That same year, he made a much-noticed appearance in chapter 3 (‘Why Do Drug Dealers Still Live with Their Moms?’) of Steven Levitt and Stephen Dubner’s best-selling Freakonomics, about the economics of crack cocaine at the retail level.

Venkatesh does little more than gesture in a sentence or two about how exactly he earned the trust of J. T. and other powerful people in the Projects, such as the tenant leader, Ms Bailey. In keeping with this laconic, understated mode—one has the sense now and then of a book written in something of a hurry—he does not broach the subject of his own ethnic origins, but it obviously helped that he is not white. At all events, the laid-back personality that led J. T. and others to trust young Sudhir emerges clearly from his descriptions—at once sympathetic and detached—of slum life and the endless battles of the very poor to make it to the end of the day in one piece. His dry Indian eye allows him to sketch in vivid detail the entrepreneurial hive at the Robert Taylor Homes.

The Projects come alive in Venkatesh’s glancing descriptions: urine-soaked stairwells inhabited by squatters and cruised by hookers; the 16-storey buildings’ bleak outside corridors savaged by Chicago’s winter winds; welcoming apartments in which heroic mothers raise their kids and cram Sudhir’s plate with soul food as he writes up his notes. His posture is genuinely one of respect. The gang members are not the ‘superpredators’ demonized by the right-wing criminologists who dominated discussions of the ghetto and of the justice system’s stance toward gangs in the late 1980s and 90s. They are humans given scant choices. ‘You want to understand how black folks live in the Projects’, Ms Bailey tells Venkatesh. ‘Why we are poor. Why we have so much crime. Why we can’t feed our families. Why our kids can’t get work when they grow up. So will you be studying white people?’