Prologue

In Long Island, a teenage Minae Mizumura meets Taro Azuma, another Japanese immigrant who works as the chauffeur of one of her father's associates who, unlike Mizumura and the other immigrants she knows, is poor. Mizumura's father finds him studious and hardworking and hires him as a technician for the camera company he works for.

Over the years, Mizumura and Azuma encounter each other briefly at company outings and parties. Right before she leaves high school, Mizumura has a moment where she reflects that her life is full of possibility and Azuma's is not. However, Azuma does eventually become extremely successful. After becoming a technician who specializes in endoscopes, Azuma begins to sell them and receives increasingly large commissions. When the company tries to cut his commission percentage, Azuma leaves and creates his own company.

Mizumura's family meanwhile enters a period of decline after her father is demoted and her mother leaves him for another man. She and her sister squander their education and youth. Later, Mizumura manages to begin a career for herself as an academic and begins writing novels. She returns to America occasionally where she catches up with old friends and learns of Azuma's rise to success. One year, she returns and discovers that he has disappeared and no one has heard anything of him.

In 1998, while teaching at Stanford University, Mizumura is approached by a young man, Yusuke Kato, who tells her that he met Azuma a few years before he disappeared in Karuizawa, Nagano, where he mentioned he knew the novelist Minae Mizumura. Yusuke and Mizumura spend the night discussing Azuma and Mizumura, who had been struggling with her third novel, decides that Azuma's story would make a perfect novel. She explains to the reader that while she had originally intended to write an "I Novel", her work is a "True Novel", i.e., a novel that is pure fiction.

Volume I

Yusuke Kato is persuaded by a wealthy friend to spend time with him in his vacation cottage in Karuizawa. While traveling through Oiwake at night, he has a bicycle accident in front of an old, run-down house. The house is owned by an unfriendly middle-aged man, Taro Azuma, but Kato is persuaded to stay the night by the maid of the house, Fumiko Tsuchiya. The following day, he encounters Fumiko again and is taken to the house of three snobbish elderly sisters, Harue, Natsue, and Fuyue. The sisters are there to scatter the ashes of Yoko, Natsue's daughter. They hint at their relationship to Taro, who they dismiss as the relation of one of the sister's rickshaw drivers. Afterwards, Fumiko invites him back to Taro's house in order to tell him about Taro and his relationship to the sisters.

Fumiko begins the story by telling how she was born in 1937 to poor farmers in Saku, Nagano. Her early memories are of the extreme poverty suffered during World War II. As a teenager, she moved to Tokyo with her uncle Genji or "George", who secured her a position as a maid on an American army base. When Fumiko was 17, he contacted wealthy acquaintances in order to try to find her work as a maid in a private household. She was eventually sent to work as a maid and nanny for Natsue and her two young daughters, Yuko and Yoko.

Natsue is the middle daughter of the Saegusa family, a snobbish clan descended from wealthy landowners. The family is close to the Shigemitsu family who, despite being financially poorer, are descended from nobility. The two families share adjoining plots of land in Tokyo as well as in Karuizawa, where they both vacation. The families celebrate Western ideals over Japanese ones, with each family having spent significant time in Europe. Each Saegusa sister had once held aspirations of marrying into the Shigemitsu family, but the only son, Noriyuki, died during the war.

Fumiko spends her time taking care of Yoko, Natsue's youngest child, who is kept close to home and doesn't fit in with the rest of the Saegusa and Shugemitsu members of her generation. Shortly after Yoko begins to attend school, Roku, an old servant of Yoko's grandfather, invites his nephew and his family to stay with him. The youngest of the nephew's children is Taro. Taro is routinely abused by his family, and Fumiko later learns that he is actually the illegitimate child of one of Roku's nieces and was raised by his uncle after his mother died. When local children begin to bully him, Yoko joins in, only to be discouraged from doing so by her father. She eventually shows him small kindnesses and he becomes enraptured by her. At the same time, Fumiko and Yoko's step-grandmother, Mrs. Utagawa, conspire to have him work in the Utagawa household, so as to spare him the abuse from his aunt.

Volume II

Yoko and Taro grow increasingly close throughout the years, encouraged by Mrs. Utagawa, who hides their relationship from Yoko's parents. The only time they spend apart is her vacation months in Karuizawa. Mrs. Utagawa eventually persuades her step-son to build her a villa in Oiwake. She takes Taro with her to help around the house; however, he is hurt when he visits Yoko in Karuizawa, and the Saegusas and Shigemitsus treat him as a handyman.

Eventually Mrs. Utagawa dies. Her final wish being that her step-son provide enough money for Taro to go to high school. The family complies with her wishes; however, shortly after, Yoko's father transfers to a university in Sapporo. Yoko and Taro stay in sporadic contact throughout the years, aided by Fumiko. When Yoko is eighteen, she contacts Taro again, and she and Fumiko are surprised to discover that his adoptive family used the money meant for his education as a down payment for a small business. When they finally meet, chaperoned by Fumiko, Yoko is humiliated by Taro's sullen behaviour but arranges a more clandestine meeting with him. After a few days, she is discovered naked in the summer home in Oiwake, sick with pneumonia. Yoko's family is humiliated, believing that Taro and Yoko had sex; Yoko later reveals to Fumiko that Taro refused to have sex with her after she told him she would never marry him.

Taro runs away from his adoptive parents to live with Fumiko. He drinks heavily and gives up his plans for education instead focusing his sights on moving to America. Fumiko's uncle Genji helps to secure him a position as a chauffeur in New York City and Taro leaves, disappearing for fifteen years.

After seven years, Yoko marries Masayuki, the only son in the Shigetmitsu family, finally fulfilling the desire of the Shigemitsus and Saegusas to unite through marriage.

Taro eventually comes back into Fumiko and Yoko's lives when he anonymously buys the old Oiwake villa and enlists Fumiko to renovate it. Spending a week there on vacation, he is hesitant to contact Yoko, but Fumiko intervenes and the two are briefly reunited. Their first meeting is interrupted by a weeping Masayuki. Yoko pledges to stay married to him.

After her second husband dies, Fumiko is persuaded by Taro to work for him in Tokyo and, in her late forties, eventually becomes the career woman she longed to be. Yoko and Taro become lovers with Masayuki's blessing. Their unconventional relationship works for seven years until the Saegusa sisters stumble upon Taro and Yoko in public. Hatsue confronts Masayuki who in turn fights with Yoko to the point where he says he no longer wishes to be married to her.

Unable to bear the thought of her husband leaving her, Yoko runs away. She is eventually found days later in one of the Karuizawa homes that Fumiko failed to search properly. She develops pneumonia and dies, although not before forcing Taro to promise her that he will not kill himself before Masayuki dies. A few years later, Masayuki develops cancer and dies, and he and Yoko's remains are meant to be scattered around the Karuizawa, which is why Fumiko is there in the present.

As Fumiko's story ends, Fuyue arrives at the Oiwake villa and informs Fumiko that now that both Yoko and Masayuki are dead, Taro is leaving her with both the Oiwake and Karuizawa homes.

Fuyue then drives Yusuke home, but not before revealing that, during the period where Taro lived with Fumiko in Tokyo, the two became lovers.

Before his vacation ends, Yusuke returns to the Oiwake villa in secret where he watches Fumiko and Taro scatter Yoko and Masayuki's ashes.

The Saegusa Family

• Harue Saegusa: The eldest of the original three Saegusa sisters. Despite the fact that she married well, she is always resentful that Noriyuki Shigemitsu died before she could marry him. She is ambitious and plans for her own daughter to marry into the Shigemisu family, only to be disappointed when her niece Yoko does instead.
• Natsue Utagawa (née Saegusa): Yoko's mother who is kind but naive and is influenced by her older sister Harue. She favours her other daughter Yuko over Yoko.
• Fuyue Saegusa: The youngest of the sisters. She never marries after her first crush, Noriyuki, dies in the war. She becomes a pianist and is much kinder and less formal than her older sisters.
• Yoko Utagawa: The younger of Natsue's daughters. She is considered an outsider by her family, not getting along with her aunts or her cousins. She is close to Mrs. Utagawa and later Taro Azuma. Loosely based on Catherine Earnshaw.

The Shigemitsu Family

• Noriyuki Shigemitsu: The only son in the Shigemitsu family. He dies in combat during the war, before Fumiko goes to work for the Saegusa family, but his presence is still felt since each Saegusa sister once longed to marry him.
• Yayoi Shigemitsu: The only surviving family member of her generation. She is good friends with the Saegusa sisters.
• Masayuki Shigemitsu: The only child of Yayoi Shigemitsu. As a child he is close to Harue's children Mari and Eri and Natsue's eldest daughter Yuko, but as an adult he falls in love with and marries Yoko. Loosely based on Edgar Linton.