One of the strangest things an actor has to do for a movie is hand over the past. Family snapshots, school portraits, wedding photos, baby pictures—once they get the Photoshop treatment, an alterna-history shared by costars is born.

Altered images figure heavily in the background of This Is Where I Leave You, the bittersweet family comedy that’s (now in theaters) based on Jonathan Tropper’s 2009 best-seller. Directed by Shawn Levy with a screenplay by Tropper, the movie stars Jason Bateman, Tina Fey, Corey Stoll, and Adam Driver as four siblings who reluctantly reunite in their childhood home after the death of their father. Their newly widowed mother (Jane Fonda) is pushing her fractious brood to remember happier times, and every shelf, end table, and naked space on the wall of the Long Island home where the movie was shot last year is covered with these framed fauxtographs in which cast members’ faces appear on somebody else’s body.

On the day of EW’s set visit (in June 2013), Fey and Driver are studying the photos adorning Fey’s character’s upstairs bedroom. “It’s a fun game to try and figure out whose picture it is originally,” says the actress, who plays Wendy, a mother of two stuck in a stultifying marriage. “I was asked to bring in a picture of me holding a newborn, some other specific ones. We just brought in pictures of us from different ages.”

“They tell you to pick things that are from celebrations mostly, graduations and things like that,” adds Driver, who costars as youngest brother Phillip, a wild child desperate for his siblings to think he has finally grown up. Earlier, Bateman was mocking Driver’s dinner-plate-size ears in an elementary school portrait, which is proof these pics really bring out the big brother in people.

“Everyone has felt entertained-slash-weird about them,” Fey says. “Jane Fonda’s daughter was here visiting, and she was like, ‘That’s your head on me as a child!’ There’s a picture on the stairwell right now that’s me and Jane, but originally it’s me and Amy Poehler at an improv festival. I sent that to Amy…” Fey tightens her mouth. “She needs to know that I can do that at any time.”

In that spirit, we put together our own scrapbook of memorable moments from our exclusive set visit.

Filming is just getting started for the day, and Fey has already made two babies scream.

Kids melting down are an occupational hazard in movies about families, but this scene is supposed to focus on the adults losing it. Driver’s character has just had a bedroom fight with his cougarish girlfriend (Connie Britton), and the argument has exploded into the living room, where his mother, child-juggling sister, and two brothers are scrambling to stifle the deluge of profanity and hostility—at least in front of the kids.

And all the while, the 8-month-old in Fey’s arms won’t stop improvising her own tantrum.

The baby’s twin is brought in…and promptly freaks out too. “How many babies do we have?” the actress wonders as the second wailing infant is taken away.

“That was it,” Levy says. They don’t even have a fake one handy. Levy asks the camera operator to zoom in and frame Fey from the shoulders up, turning a wide shot into a close-up. “Tina, can you…pretend you’re holding a baby?” the director asks.

Fey slowly cradles her arms and begins rocking an invisible infant back and forth. “I continue to plead my case that it’s not me, but every time we put a baby in my hands, it’s terrified and it screams and cries,” she says. There’s a lesson here for screenwriters, she adds: If you need a baby in your story, “do it with a 6-month-old or a 15-month-old.”

She shakes her head. “Eight months, 9 months—that’s where it happens: It’s called stranger danger, and I’m the new face of it.”

Not making the children weep? That’d be Driver, who, at 30, still feels at home at the kiddie table, and he takes it upon himself to entertain the child actors when they get tired or fussy. He’s the young, cool uncle.

“I always feel bad for, like, the parents who come in for the crying babies because they’re always like, ‘Sorry, sorry.’ But obviously the child is doing something that’s totally natural,” Driver says.

He’s happy to take them under his wing—sometimes literally, in the case of Cade Lappin, the 3-year-old who plays Fey’s older child. “Adam was really great with Cade,” Fey says. “He’s quite tiny. You had a day where you just carried him around for the whole scene.”

“Kids, they’re awesome actors,” Driver says. “They’re totally free. We were doing one scene where [Cade] comes in and he’s supposed to use the restroom, and he did. He came in and pissed right in the little potty. He wasn’t scared of anything.”

“Method actor,” Fey says.

If there’s a central character in This Is Where I Leave You, it’s Bateman’s sad-sack Judd Altman. He’s the narrator of Tropper’s novel and the anchor of Levy’s film, the fractured lens through which the audience sees his equally damaged family.

“What’s fortunate about playing the Everyman or the straight man, the guy in the middle or the relatable one, is that what you are asked to do runs very close to the middle line, so the comedy that you play is not superbroad, the drama that you play, it’s not My Left Foot,” Bateman says. “You’re able to, oftentimes on the same page, do something funny and something dramatic.”

Judd’s wife has left him and is possibly pregnant with his boss’ child. He has lost his job (another consequence of her having an affair with his employer), and there doesn’t seem to be much hope of pulling out of his tailspin. All he wants is a second chance. Coincidentally, that’s all Bateman wanted too.

The Arrested Development actor was the only holdover from an earlier effort to adapt the book with Hairspray director Adam Shankman. That version would have starred Jason Sudeikis, Zac Efron, and Leslie Mann as the other siblings, and Goldie Hawn as the mother.

“It didn’t really come together in the way a lot of people were thinking or hoping that it might. As a result, it kind of fell apart,” Bateman says. “That whole package went away, then Shawn took over. I was sort of crossing my fingers, ‘Oh God, I hope they come back to me for Judd.’”

Tropper, who also had written the earlier project’s script, and Levy wanted the same thing. “Tropper will tell you, I will tell you, Jason Bateman was always the perfect Judd,” Levy says. “Tropper heard that voice in his head for years, so I rebuilt the movie for Warner Brothers keeping one piece, the critical piece.”

“I felt very lucky to have made that cut,” says Bateman. So…he's sort of the lone wolf of the pack, right? “The lone teen wolf,” the Teen Wolf Too star deadpans.

Sex scenes are awkward under any circumstance, but there’s one way to make them even weirder—record only the audio.

“It’s hard to do without actually physically doing a little air humping,” says Stoll, whose character’s ovulating wife (Kathryn Hahn) is so desperate to become pregnant she makes him sneak away for a little shivah delight, not realizing there’s a baby monitor that broadcasts their lovemaking to a living room full of mourners.

Hahn and Stoll aren’t on screen during the sequence, but did record audio of the raunchy sex on set. “The whole cast had to be quiet, and we just stood there,” Hahn says. “What made me giggle about it was how embarrassed and shy Shawn got when we were doing it. We couldn’t even look at each other.”

It wasn’t all improvisation. “There were some lines, which I thought was weird,” Stoll says. “They’re actually really insulting to my character. She says, ‘Is it in?’”

“‘Are you hard yet?’” Hahn says, citing another. “I think it’s because there’s a lot of pressure on him right now. It’s got to be difficult for a man to perform under that.”

“What better way to relax a man than to say, ‘Is it hard?’” Stoll says.

Turns out verbal sex is more exhausting than you’d think. “I felt like I was going to pass out afterward,” Hahn says. “We were just heavy breathing for a long time.”

One of the sadder story lines in This Is Where I Leave You involves Timothy Olyphant’s character, Horry, the boy next door who never quite grew up. Once upon a time, he was dating Fey’s character, but suffered a traumatic brain injury in a car accident. His girlfriend left him behind, and so did life.

“I find even just his wardrobe to be quite heartbreaking,” Fey says, comparing it to a little boy’s clothes. “It’s like a guy who never has to be anywhere. He never has to be dressed up. For the rest of his life he will live in his mother’s basement.”

The funeral brings her back together with her slow-but-sweet former love, and when Olyphant stops by the set later that day, he’s sporting flip-flops, a faded T-shirt, and baggy shorts. Asked about his character’s go-nowhere outfits, the Justified actor admits these are his own clothes.

“Shawn said, ‘Is that your wardrobe?’ And I said, ‘No, I just came straight from the hotel,’” says Olyphant.

He doesn’t disagree with Fey’s assessment, though. “Which is sad because I think of myself as a very busy guy,” he says. “Some of this is what I packed!”

Even before she landed the role of Hillary Altman, Fonda was already part of This Is Where I Leave You.

In the novel, the character is exercising to a Jane Fonda workout tape, and is obsessively comparing herself to the Oscar-winning actress. Judd tries to reassure her by telling his mother she’s prettier than Fonda. “Of course I am, sugar,” Hillary responds. “But she has a better butt.”

That scene isn’t in the movie, and Fonda seems only mildly amused when reminded of it. “I get referenced all the time,” she says. “I’m used to it. But I think it’s funny.”

She relates to Hillary’s boldness and sexuality, but Fonda doesn’t think they have much else in common, except for their dysfunctional families. “She’s a bit of a narcissist, and she doesn’t really step into [her kids’] shoes,” Fonda says. “I don’t know if my children would say that I have some narcissism. I hope not. I’m conscious, my parents were both narcissists, so I try to pay close attention to that. But I don’t think Hillary is really aware of the effects that she’s had [on them]. If she really thinks about it, she would be, but she tries not to think about it too much, you know?

If there was a presence on set more intimidating than Fonda, it was Fonda’s “breasts”— the fake ones created for her character’s recent augmentation.

“It was prosthetics, but to Jane it felt like she was wearing armor,” Levy says. “To the rest of us, it looked like a topless woman. She called me into the hair-and-makeup trailer and greeted me with just pants on. I was like, ‘Whoa! What? Why?’ She’s like, ‘Don’t you love my boobs?’ She’s fondling them and shoving them at me. I was like, ‘Jane, this is really troubling.’ And on the walk from hair and makeup to the set, she was flashing drivers. To her, it was just superfun.”

Surprisingly, no one ever thought to employ Fonda’s fake mammaries to distract the squawking babies.

Which is probably for the best.