Growing up in Belfast, Northern Ireland, he was surrounded by storytelling – books and tales played a major part in his life.
But one particular stood out and became his favourite book as a child.
He said: “The Bad Tempered Ladybird by Eric Carle because of the spread where the ladybird picked a fight with a whale.
“I was fascinated by how big the whale looked compared to the insect.”
And it was this fascination with the visual which eventually led to him to study illustration and visual communication at the University of Ulster.
Before graduating, Oliver had already made his name as an artist with several exhibitions in his home city of Belfast and a number of book jacket illustrations for local publishing houses.
Then during a year’s break from his studies in 1999 to 2000, Oliver travelled to Sydney, Australia where he got his first big break with the Lavazza Coffee Company.
They had spotted some of his off-beat illustrations and commissioned him to paint some pictures for their head office and illustrate their Christmas cards.
Back in Belfast in his final year at university, he held an exhibition entitled “Boys at the Bar” depicting the afternoon (and morning) drinking culture in Belfast.
All 16 portraits sold out on the opening night and he went on to graduate with first class honours. A stint window-dressing followed.
He said: “The second best job I ever had was doing the Waterstone’s windows in Belfast. That was a great laugh.”
But he finally turned to producing children’s books when as he puts it “people didn’t believe what I did with my time”.
In 2004, he sent his work to publisher HarperCollins who snapped it up and in July the same year his first children’s book How to Catch a Star was born.
The tale, featuring a small boy who wanted to own a star, captured the imagination of children and adults alike with the child-like logic which made the seemingly impossible possible.
In his follow-up book Lost and Found, published in 2005, the boy develops a friendship with a penguin which sees them set off on an intrepid journey to the South Pole.
It won the Nestle Children’s Book Prize Gold Award in 2005 and the Blue Peter Book of the Year in 2006.
Since then it has been made into an animated film for Channel 4 and is now a stage play currently being performed at the Polka Theatre in Wimbledon, south London.
Once again, the charm lies in the simplicity and the fantasy of the tale.
The boy rows the penguin across waves “as big as mountains” to get him to the South Pole.
For Oliver, the success spurred him on to produce five further books in as many years; The Incredible Book Eating Boy (2006), The Way Back Home (2007), The Great Paper Caper (2008), The Heart and the Bottle (2010) and his latest tale Up and Down which returns to the adventures of the small boy and his penguin friend.
In an age where books compete with television, Nintendos, and computers for a child’s attention, Oliver seems to have won through with his heart-warming stories and simple, colourful illustrations.
But despite this success, he still sees himself as an illustrator rather than a children’s author.
He said: “I suppose I’d consider myself more of an artist than a writer.
“I’ve met enough good writers to know I’m not one.
“For me the interest in picture books comes from how the words and pictures work together.”
Like his literary hero Roald Dahl, Oliver has a knack of making the unfeasible seem perfectly normal and always with a child’s eye view of the world.
In The Way Back Home, the boy jumps down to Earth from the moon to get the things he needs to help his Martian friend.
But when he gets indoors he’s distracted by his favourite programme on television so sits down to watch.
Oliver’s ability to connect with a child’s perspective on the world, where nothing is impossible, may just be the secret to his success.
It certainly keeps him young at heart.
Now based in Manhattan, New York and recently married, the 33-year-old has plans for more picture books and paintings.
But when asked what he is working on at the moment he replied: “My skate ramp technique”.