It was 9.30am on a bleak winter's day in 1966 when Joan Lindsay sat down to sketch out the plot of Picnic at Hanging Rock. It was the same time she normally started her writing day, but this day felt different. A sinister midnight gale had whipped the few remaining leaves off the old roses in Joan and her husband Daryl's side garden and a huddle of voluminous dark clouds had also moved in, crossing the countryside like black-coated villains in a Victorian Gothic novel. In the grim midwinter light, the landscape looked thoroughly dispiriting. It was, Joan mused to Rae Clements, their long-term, live-in housekeeper, the perfect day to stay inside and write.

Joan had woken that morning with the flecked remnants of a peculiar dream imprinted on the edges of her conscious. She knew immediately, she told Rae later that day, that it would make a good book. The dream had centred on a summer picnic at a place called Hanging Rock, which Joan knew well from her childhood holidays. Joan told Rae that the dream had felt so real that when she awoke at 7.30am, she could still feel the hot summer breeze blowing through the gum trees and she could still hear the peals of laughter and conversation of the people she'd imagined, and their gaiety and lightness of spirit as they set out on their joyful picnic expedition.

And so, wrapped up warm against the chill – for Mulberry Hill on Victoria's Mornington Peninsula was inadequately heated against the winter conditions – Joan pulled her favourite turquoise cardigan around her petite frame, walked up the staircase, turned left into her private writing room and quietly closed the door on the world. Her mind was focused on one thing: the strange picnic of her dream and getting it down on paper.

By midday Joan had already pencilled out the basic plot, according to Rae. The narrative revolved around a group of beautiful schoolgirls from an elite ladies' college in the Australian countryside who set out on a Valentine's Day picnic. She had decided to set the picnic, and the story, on Saint Valentine's Day, because it had always been her favourite day of the year – the day she and Daryl had eloped in London many years ago.

As the cold winter's day wore on, Joan continued to remember the dream of the picnic at Hanging Rock with unusual clarity. She spent most of that day trying to write it all down. That night she had the dream again, and the next day she rushed to write the narrative before it escaped her. This happened again the next night, and then every night for a week.

According to Rae, Joan didn't know why the story was coming to her in a series of very clear dreams, but she didn't care. The surge of words was invigorating her nearly 70-year-old body. Joan knew instinctively that she was onto a good thing. Tap, tap, tap went the typewriter keys. Morning, noon and night. Joan later told her literary agent, "Picnic at Hanging Rock" really was an experience to write, because I was just impossible when I was writing it. I just sort of thought about it all night and in the morning I would go straight up and sit on the floor, papers all around me, and just write like a demon!"

"She really did dream the sequence of chapters," recalls Rae, who also remembers Joan's excitement when she was writing the book. "She would come down from her study each day and say she'd had the dream again. Then she'd discuss the characters and what they were up to. She loved Miranda and the French mistress. Miranda was her favourite character. She was also fond of Albert. She often said, 'Poor Albert! Poor little Sara!' She definitely had her favourites."

While Joan credited her curious dreams for the plot of Picnic at Hanging Rock, she had also envisaged writing such a story for some time, although she may not have thought of a defined outline. In 1963, she had told her good friend Colin Caldwell that she wanted to write a novel about a place that had always fascinated her. She then produced a print of the 1875 painting of Hanging Rock by William Ford (At the Hanging Rock, sometimes referred to as Picnic at Hanging Rock.) Caldwell asked when she'd last seen Hanging Rock. She couldn't remember, she said. So off the two of them went in Caldwell's car, for an afternoon picnic. "We took cold duck and a bottle of wine," wrote Joan in her unpublished memoirs.

Joan and Caldwell spent the morning eating and drinking at Hanging Rock's picnic grounds, just as picnickers had been doing since the 1850s. When they'd finished their little picnic, the two began to climb the trail, up to the higher slopes. Halfway up the path, Caldwell decided to leave her alone – "to feel that haunted thing", he later said.

Some of Joan's friends called her a "mystic". According to those who were close to her, she had certain abilities, sensitivities. She could "see" things that others couldn't, especially in the bush landscape. She knew things without being told. She could not only tell what had happened in the past, but also predict events in the future, without knowing why or how. And she could communicate with those who live in that grey space between life and the world beyond it. Those friends feel that Joan's novel is the result of this curious ability, which she'd had ever since she was three.

"Oh yes, she was very much a mystic," confirms Caldwell, who was convinced of her ability after their Hanging Rock expedition. "She could sense things in the landscape that others couldn't."

Melbourne academic Terence O'Neill, who befriended Joan in her later years, believes that she always tried to hide this mystical side from her husband, Daryl, because she knew he was deeply cynical of anything of that nature. "It was clear that she was interested in Spiritualism, and longed for some spiritual dimension in her life, but she didn't feel safe bringing that side of her out in front of her husband," he says. "So I think she channelled it into her writing. I know she was very interested in Arthur Conan Doyle and his belief in and theories about Spiritualism, nature and the existence of spirits."

Less than two weeks was all it took to write the book. When Joan had finished writing, she thought about a title. Then she remembered William Ford's painting. The title was perfect. It was simple and pretty, and belied the horrors hidden within. A paradox, really. Joan had always adored those.

F. W. Cheshire Melbourne publishing director Andrew Fabinyi thought it had promise, and passed it to senior editor John Hooker and junior editor Sandra Forbes. Forbes, who went on to a stellar career that included several years as executive officer of the Australia Council's literature unit, wrote the following to Joan: "I really enjoyed reading this, it seems to have the right blend of 'truth' and fiction. Given that the actual disappearance of the girls is a fact, it is a fascinating problem, well presented in a style very much in keeping with the period and personages involved." There was just one thing, added Forbes. Might the story benefit from a little more ambiguity? She suggested deleting the final chapter, which delved into the spiritual realm.

The suggestion, to which Joan agreed, proved to be a prudent one. Without the final chapter, in which the missing schoolgirls seem to disappear into time, the story ends with a question mark, and it's this mystery that lingers in readers' minds.

Production on the novel, in preparation for a 1967 publication date, took several months, and the two women got to know each other over Joan's favourite tipple, gin. "Joan was Lady Lindsay by this point, and a prominent figure in Melbourne society," remembers Forbes. "She was very intelligent and very elegant. She took me to lunch one day at her club, the Lyceum Club in Melbourne. We talked a lot about the Australian bush and Joan's feeling for the bush. She said she'd grown up in the Hanging Rock area or had holidayed there with friends as a child. We talked about the myth and mystery of the bush. She also talked about how children, or people, could disappear into it at any time. I think Picnic was ultimately about the portrait of the rich history of Australia, and the myths of the bush. She was fascinated by patterns, by things rippling out from a centre and influencing other things, which is definitely a theme of Picnic.

"Did I think the story was true? We did talk about this. But the truth for Joan was different to the rest of us. She was never straightforward about it. I think I decided in the end that it was a great work of the imagination. I see it as a book of place; a painterly book that captures the atmosphere of the Australian bush."

The novel was launched in Melbourne on November 1, 1967 – All Saints Day – by Robert Menzies, who had twice been prime minister of Australia and was a longtime friend of the Lindsays. There were some notable reviews. A critic called Vintner called it "mythopoeic"; and a year later another, called Lemming, likened it to a "faded watercolour". But the highly regarded, highly influential Bulletin wasn't as convinced. Picnic at Hanging Rock, it said, was "too sunlit to be called Gothic".

The novel languished for a year or so – then came a stroke of luck. In 1969 John Hooker, who had been working as a publisher at Cheshire since 1964, was headhunted by Penguin. He stayed there for a 10-year stint that saw him contribute a great deal to the Australian publishing scene. Picnic at Hanging Rock was part of his master plan for a distinctly Australian front list. Little did he know how large a part this little novel would play in the landscape and history of Australian literature – and, indeed, of Australian film.

The late Australian television presenter Patricia Lovell first read Picnic at Hanging Rock in 1971, several years after it was first published. She found it in a discounted pile of books in a newsagent and thought it looked interesting. She took it home and read it in one sitting. She thought it was an extraordinary story. She didn't realise, she admitted later to the writer Cathy Peake, that it would "change her life".

After she put it down, she couldn't get it out of her mind for several days. She wondered if it would make a good film. Inspired by the idea, Lovell wrote to Joan Lindsay to see if anybody had bought the rights. No, said Joan. And then, no doubt sensing a good business opportunity, Joan immediately asked if Lovell was interested and promptly invited her down to Mulberry Hill for a meeting.

Lovell knew she needed to beef up the artillery before she could fly down to meet Joan. After all, her film CV wasn't exactly extensive, and although her television career was impressive (she'd been a host on the ABC children's show Mr Squiggle, a presenter for the current affairs program The Today Show and an actress on Skippy and Homicide), it didn't include producer duties. She decided to take a director with her to show she was a serious contender. But where would she find one? And who would do it?

During her time as a television presenter on The Today Show, Lovell had interviewed a young guy called Peter Weir, whom she'd continued to admire as he established his film career. She decided he would be perfect for the project, even though he was only 27.

"Reading the book for the first time," he would later recall in another interview with Lovell for a mini-documentary produced by Lovell called Recollections – Hanging Rock 1900 (recently included in a special issue of the director's cut of the film), "it was irresistible reading. It was the tremendous unease … I couldn't wait to get to the rock to see if it was as good as it read."

And so, in April 1973, Patricia Lovell and Peter Weir travelled down to Victoria to meet with Joan at her home. Joan's agent at the time, John Taylor, went with them. Although nervous at the prospect of meeting the author of the book that he now desperately wanted to film, Weir charmed Joan from the first hello. As he remembers it, there was an instant bond between them, created over their love for the novel.

Unfortunately, Lovell only had a small amount in her bank account when she met with Joan. The most she could offer for a holding option on the screen rights was $100. Was that acceptable to Joan, her agent and her publishers? To her relief, they all said yes. It was, Lovell recalled later, the "best two days of my life".

The next day, Lovell and Weir travelled up to Hanging Rock. On the way they got lost and ended up approaching the rock from the wrong direction. They turned in from the Mount Macedon side, and saw the rock straightaway, with a little cloud sitting on top of it. Immediately, they sensed the eeriness of the place.

"We'd thought that we'd save money by maybe filming the story in the Blue Mountains," Lovell has said. "I mean, we thought a rock is a rock, you know. Every mountain's the same. It was only when we drove over from Mount Macedon and we saw it, this extraordinary eruption of rock and trees … all on its own. We went completely silent. We knew then that we could never film it anywhere else."

Lovell was immediately uneasy. The rock seemed "so alien to the rest of the countryside". Her feeling worsened as the day went on. When the group arrived at the picnic grounds at the base of the rock, her watch inexplicably stopped. It was the first of many times this would happen, either at Hanging Rock or around Joan herself.

Forty years later, scriptwriter Cliff Green remembers Picnic at Hanging Rock's production team as one of the best he's ever worked on. It was, he says, a period of absolute joy. "I really enjoyed working on Picnic at Hanging Rock," he recalled in an interview for my biography of Joan Lindsay. "The whole thing was a magical experience. The strength of the book really stood out in its own quiet, poetic way.

"I did ask [Joan] if the story was true. I'd been warned early on not to ask, but I did anyway. Her stock answer was, 'Some of it is true and some of it isn't.' In the end, I decided that fiction and facts had been woven so inextricably together that it was impossible, even for her, to distinguish the difference. Writers use a multitude of threads of reality and fiction to create their stories. As I read the novel, I saw the film unfold; I saw the look of the film immediately. The novel is an incredible filmic piece of work in itself.

"The first 20 minutes of the film were easy to write because it's a straight line, chronologically. But the moment when Edith comes screaming down the hill, it becomes more of a complex story. So much of it is atmosphere and setting. It's really a book about the atmosphere of Australia."

After the main players signed up, Patricia Lovell began to seek funding of $440,000, which took her two years. The money finally came from a combination of players, including the Australian Film Development Corporation and the South Australian Film Corporation, whose investment was conditional upon the film being substantially shot in South Australia. It was the first time a feature film had received such multifaceted support.

With the principal team in place, Lovell, Weir and the other co-producers, Hal and Jim McElroy, got on with the job of auditioning for the cast. Some of the characters were easy to identify and those roles filled first. Helen Morse was the perfect French teacher, Mademoiselle de Poitiers. Vivean Gray proved superb as the repressed spinster and mathematics mistress, Miss McCraw. Her performance was so strong that in 1995 her role in the film was commemorated with a stamp by Australia Post. Jack Fegan, the original star of Homicide, was endearing as the kindly Doc McKenzie. Jacki Weaver was delightful as the irrepressible Minnie, one of the school's bestloved staff members. A young and handsome John Jarratt was moving and memorable as the lively stableboy-with-a-heart Albert. And Tony Llewellyn-Jones was specifically praised in Variety for his role as Tom, the gardener. Other actors included Dominic Guard as the main male character, Michael Fitzhubert, and Garry McDonald as Constable Jones.

Curiously, the roles of the schoolgirls proved to be the most difficult to fill. Searching for these girls became a long and complex process because Peter Weir wanted them to have a distinct sense of innocence about them. It was only when the team went to South Australia that they finally found girls who had that dreamy, innocent, 19th-century look that Weir wanted.

Cliff Green recalls that working with a cast that was composed predominantly of teenage girls came with its own set of unique challenges. "Peter Weir had recruited these lovely, innocent 16-year-old girls out of very expensive schools in Adelaide. But it wasn't long before many of the girls identified with their roles. I mean, they were hysterical most of the time! But Peter purposely generated that – that level of excitement that those girls showed on the screen. It was genius of Peter Weir to use real schoolgirls – to pick that up and translate it into a film. It was a highly charged film in lots of ways. Film communities live in very close-knit environments, and everybody does bounce off each other – sometimes literally. But I've never struck that as strongly as I did in Picnic."

For the main role of Miranda, Weir had cast a lively and fresh-faced young girl called Ingrid Mason, but then realised after several weeks of rehearsals that it wasn't working. (Ingrid Mason still appeared in the film as one of the other schoolgirls, Rosamund.) When he was left unsure about his leading lady, someone suggested that he consider a young actress called Anne Louise (Anne) Lambert.

"I was 19 and was working in the [TV soap opera] Class of '74 at the time," recalls Lambert. "We met over coffee one day. Peter brought with him a coffee-table book of David Hamilton photographs featuring girls in white muslin tops, back-lit in gardens. That was the quality and light he wanted to achieve. It sounded really interesting. I wanted the role. I really did. I thought I was made for the part. But like Miranda, I'd grown up in Queensland and was just a country girl and painfully shy. I had to overcome a lot of nerves to do the role."

For the first few weeks, Lambert remained uncertain as to whether she was playing the part correctly. There were lots of takes and retakes and more takes.

The turning point, she says, came in the form of someone unexpected: Joan Lindsay. Lambert says Joan really brought her out of her shell and inspired her to deliver a remarkable performance. "One day we were shooting a particular scene – the one where I say to the other three girls: 'Look up there, up there in the sky!' I couldn't seem to get the scene right and Peter would just say, 'Do it again!' Finally, he told us all to take a rest. While the cast and crew went to get coffee, I wandered off into the bush, still dressed in costume, to try to pull myself together. I was very emotional: it had all been too much, and I was ready to cry.

"At that moment, in the corner of my eye, I could see a lady making her way towards me. She was walking across these rough rocks, so I waited for her to navigate them. I realised that it was the author, Joan Lindsay. I went to hold out my hand, but she walked straight up to me, put her arms around me, and said in a very emotional way: 'Oh Miranda, it's been so long!' She was shaking like a leaf.

"I wasn't sure what to do, so I said very politely, 'It's me, Joan; it's Anne. It's so nice to meet you.' But she dismissed this with a wave of her hand. She just said 'Miranda' again and clung to me, so I embraced her back. I think we both started to cry. It was very moving. And it was clear she'd regressed into some part of her past. To her, I really was someone she had known, somewhere in time. Right then, I felt that if Joan Lindsay believed I was Miranda, I must be doing okay. I felt that if she believed in me, I would be okay."

Despite the angst and worry, Lambert delivered one of the most spellbinding performances in Australian film history. And even she agrees that it was all worth it, because the film is truly beautiful. "There was a sense, even then, that it was a special film," she says. "We all felt it. It was a remarkable piece of cinema."

Filming commenced at Hanging Rock on February 4, 1975. The shoot took a total of six weeks. When it came to the visual elements, both Green and Weir took a lot of their inspiration from Joan's home, Mulberry Hill, with its walls of art, as well as from the rich, painterly layers of imagery in her novel. "Without question, we were both influenced by Joan," says Green. "Knowing she herself had been a painter until she was in her mid-20s and then going in and out of that house, which is like a museum of Australian paintings, affected us all deeply. Unquestionably, that atmosphere affected the writing of the screenplay in terms of its visual Australian Impressionist feel."

But as the medium that Weir was using was film rather than paint, it took time to work out how to create the Impressionistic look and feel for the screen. Finally, he found the solution in the work of French photographer Henri Cartier-Bresson, who is now considered to be the father of modern photojournalism. Cartier Bresson had experimented with placing various veils over the lenses of his cameras, and had discovered that they created an unusually diffused, soft-focus quality.

It was exactly what Weir was after. He and cinematographer Russell Boyd walked into a bridal shop and ordered various pieces of wedding veil.

Placing these bridal veils over the lenses, using various thickness for different effects, created a palpably dense, dream-like atmosphere that significantly added to the mood of the film. The scenes almost glowed, as if candlelit. And when the dialogue – which in the first part of the film was mostly a series of gentle whispers – was added, the characters seemed to exist as hazy, evocative figures in a painting, rather than real, three-dimensional people.

Before filming began, Wendy Stites, Weir's wife, spent a great deal of time contemplating Edwardian dresses. The tea dresses were originally going to be in pastel colours, but after she studied photographs of the era, she changed them to white. There was just one problem: the fabrics appeared a little too white on film, so Wendy soaked them in tea to soften the brightness.

Patricia Lovell also pitched in to help with the costumes. "I remember turning up to set at Martindale Hall [the setting for the novel's Appleyard College] in South Australia early one morning and seeing Pat there, at the ironing board, carefully ironing all the starched white cap sleeves," recalls Anne Lambert. "She ironed most of the costumes every day. She was incredible!" Lovell spent nearly 12 hours ironing petticoats, camisoles, frocks and sashes for one scene alone.

Peter Weir was a stickler for detail. For example, the daisy pattern in the lace trim of Miranda's dress was included because daisies were Miranda's favourite flower. The butterfly buckle was selected because butterflies' lives are beautiful and brief, as is Miranda's. The muslin fabric was intended to signify lightness of spirit. And in the scene where Irma returns to the school to say goodbye, Irma's crimson velvet cape – a rich blood-red – was intended to symbolise not just blood but also sex, shame and heated passions.

Likewise, Stites' attention to detail was so fine that in the pivotal scene where Michael (Dominic Guard) clutches a piece of lace that he finds while searching for the girls, the fabric was cut from the hem of Miranda's petticoat.

While Lambert, as the central character of Miranda, was in many scenes, she wore only one dress for the entire filming period. Amazingly, it survived the rough bush landscape – and, afterwards, she was allowed to keep it. Later, she kindly donated the famous picnic dress to the National Film and Sound Archives in Canberra.

Australians loved Picnic at Hanging Rock. It wasn't just an Australian product through and through – a film that had been set in Australia, and produced with Australian funding and an Australian cast and crew – but its subject was also uniquely Australian. It represented the country's landscape, its people and the problems that faced its early settlers in a way no other film had done before. Even the characters seemed recognisable to many people, from the ocker stable boy Albert to the pretentious British headmistress. Everybody adored Miranda, of course.

The film premiered at the then new Hindley Cinema Complex in Adelaide on August 8, 1975, and almost immediately was hailed as a success. The film seemed to achieve the impossible: both a commercial and critical success; a highbrow art-house film and a spectacular Hollywood production, destined for both the serious art cinemas and general release. The critics adored it, lauding its delicacy, its light and shade, and its artful construction.

At the time, Australians were hungering for something that was quintessentially Australian. They wanted books and films that held up a mirror to the nation. It was perfect timing.

The Original Clapperboard used when they filmed the last take