The Derby is one of England's great horse races. MARK NICHOLLS spoke to Norwich-based author DJ Taylor about his latest book, Derby Day, and the painting which inspired it.

The painting is crowded, colourful, flamboyant and full of life… full of all Victorian life in its mid-19th century depiction of a day at the races.

For the artist, William Powell Frith, it made him a fortune, secured his reputation and attracted the attention of monarchs. The Derby Day, now in the Tate Collection, is a great Victorian panorama, painted to reflect one of the obsessions of the period – horse-racing.

All Victorian society is there, heading for the Epsom Downs in June… high life and low life, society beauties and Whitechapel street girls, bookmakers and gipsies, hawkers and acrobats, punters and thieves as whole families stream along the Surrey backroads, towards the greatest race of the year.

Yet for Norwich-born writer DJ Taylor, it has proved a fascinating inspiration for his latest novel.

Derby Day weaves an intriguing and imaginative plot around the characters in the scene, encased within the genre of the Victorian novel with Taylor drawing on this panorama to retell their stories in the Victorian vernacular in which he has become so accomplished.

Derby Day is not so much a sequel to his best-selling novel and another Victorian mystery Kept. Yet it does continue familiar themes and revives some of the characters.

'The villain of Kept, Mr Pardew, reappears in Derby Day,' Taylor reveals.

The wonder horse of the book is Tiberius, a dead cert. But those who can see money in the race have other ideas for the three-year-old. The villain believes he can make more by seeing its odds shortened and then backing a different horse at longer odds, even employing a jockey well past his prime to ride Tiberius.

While Frith wove together the layered fabric of Victorian society in his painting, Taylor moves the story forward.

'What interested me about Frith as a painter,' he said, 'was that he painted large groups of people against a recognisable backdrop, in this case of the Epsom Derby. The painting of is one of the great Victorian panoramas with something like 88 individual people portrayed in it.

'I have always known about Frith's painting. It was an absolute sensation when it came out and for me, the inspiration for the book. Setting the novel at a Victorian horse race, as Frith had his painting, offered the chance to bring large sectors of Victorian society together – all attracted by the magnet of the race.

'Horse-racing just seems to be one of the great Victorian subjects – if you want to establish a background, Victorian horse-racing is it. There are scams, atmosphere, it is the sport of kings but all parts of society take an interest in it.'

From there, Taylor introduces us to the old Mr Gresham and his daughter Rebecca, Mr Happerton and his crony Captain Raff, while in darkest Lincolnshire sad Mr Davenant – seemingly burdened with the potential of Tiberius – broods over his financial embarrassments.

In his memoirs, My Autobiography and Reminiscences (1887), WP Frith suggests he had been long searching for just such a scene.

He reflects: 'I felt sure that if I could find a theme capable of affording me the opportunity of showing an appreciation of the infinite variety of everyday life, I had confidence enough in my power of dealing with it successfully; but the subject – then, as now and ever, the chief difficulty – where was I to find a scene of such interest and importance as to warrant my spending months, perhaps a year or two, in representing it?'

Frith (1819-1909) found his answer on a visit to the Derby, discovering before his eyes the full scope and depth of Victorian society – gamblers, prostitutes, street-cleaners, rogues, lawyers, princes, and kings. Yet for many of those in the 1850s, the race – which lasted a mere two minutes – was almost incidental. It was the sideshows and socialising that attracted the masses.

'There are the gamblers playing thimble rig,' notes Taylor, 'there are acrobats, all different things going on and interaction between the characters. It is a painting full of stories with a huge assembly of characters connected to each other, which I realised I could transfer into a book via a series of narratives based on a horse race.'

Taylor re-creates a vivid picture of each character, their language, attire and surroundings and brings them together in a way that entrances the reader, building up to the climax of the finish of the race where the lives of each are affected – changed forever – by the moment the horses cross the line.

Their destinies depend on the champion horse Tiberius.

Taylor said: 'Everybody lives or falls by the outcome of the race. Some have their lives transformed for the better but the race is the catalyst that changes everybody's lives in one way or the other.'

The language of Derby Day is familiar – as in Kept – from a writer who through his novels is fluent in the tongue of Victorian England.

'I spent my 20s reading Victorian novels and the idea of writing one based on the premise is fascinating and beguiling. I have read 44 of the 48 Trollope novels,' said Taylor, who also wrote the biography of Thackeray.

'The thing I like about the Victorian novel is that they were written under conditions of censorship about what could and could not be said. The writer has to imply and infer. I think writers write better when they have limits on what they can say. The challenge is to find ways around these limits.'

To add to the depth of knowledge that appears throughout the book, Taylor researched in libraries, and examined papers and pamphlets of the time, unearthing works with titles such as How to Train Your Groom. He also went to the races to absorb the atmosphere and rekindle days from his childhood when his father John Taylor, an enthusiastic horse-racing fan, would take the family off to the course.

'We would always put a shilling on Wogan's Winner at Fakenham or Yarmouth,' recalls Taylor. 'Going to the races for this book was a case of two extremes – Ladies Day at Newmarket and a night meeting at Yarmouth where the East End had descended for a flutter.'

The Derby – to be run at Epsom later today – still stirs the imagination of punters as much as it ever did.

It dates back to 1780 and is still ranked the greatest flat race in the world with winnings of �1.25m. Run over a distance of one mile, four furlongs and 10 yards (2,423 metres), it is open to three-year-old thoroughbred colts and fillies.

Previous winners have included Seabird, Nijinsky, Mill Reef and Shergar, right up to 2010 victor Workforce.

DJ Taylor is married to novelist Rachel Hore and they have three boys – Felix, 18, Benjy, 15, and Leo, 11.

A novelist, critic and acclaimed biographer, whose Orwell: The Life won the Whitbread Biography prize in 2003, he is an ardent Norwich City fan. He was born and brought up in the city and later moved to London but has been back in Norwich for the last 10 years.

Along with Kept, his most recent books are Bright Young People: The Rise and Fall of a Generation 1918-1940, and the novels Ask Alice and At the Chime of a City Clock. He will follow Derby Day with a sequel to At the Chime of a City Clock next spring and at present is writing a novel set in the autumn of 1939.

t Derby Day is published by Chatto & Windus, priced �17.99.