Jim Keltner is best known for his session work on solo recordings by three of The Beatles, working often with George Harrison, John Lennon (including Lennon solo albums, as well as albums released both by the Plastic Ono Band and Yoko Ono), and Ringo Starr.

He and Starr were the drummers on the Concert for Bangladesh, rock music's first charity benefit, initiated by George Harrison and Ravi Shankar, in August 1971 at Madison Square Garden in New York; he also performed at the Garden in 1972 for John Lennon's "One To One" benefit for the Willowbrook State School.

Jim Keltner started out in jazz, although his first session was recording "She's Just My Style" for the pop group Gary Lewis and the Playboys.

In addition to his work with three of the Beatles, Keltner, as a free-lance drummer, has also worked with Leon Russell, Gabor Szabo, Delaney Bramlett, Roy Orbison, Harry Nilsson, Jerry Garcia, Eric Clapton, Steely Dan, Joe Cocker, Van Dyke Parks, the Rolling Stones, Ronnie Wood, Bill Wyman, Mick Jagger, Keith Richards, Bob Dylan, Joni Mitchell, Brian Wilson, Seals and Crofts, The Ramones, Bill Frisell, Neil Young, Crosby, Stills, Nash and Young, Phil Keaggy, Steve Jones, Crowded House, Fiona Apple, Elvis Costello, The Bee Gees, Jackson Browne, The Manhattan Transfer, Randy Newman, Ry Cooder, Sam Phillips, Los Lobos, Pink Floyd, Warren Zevon, Rufus Wainwright, Tom Petty, Gillian Welch, the Steve Miller Band, Alice Cooper, Sheryl Crow and Lucinda Williams among many others.

He is featured on Carly Simon's 1971 album, Anticipation. In 1973, Keltner was the session drummer on Bob Dylan's Pat Garrett & Billy the Kid, which includes the hit "Knockin' on Heaven's Door". Keltner says that session "was a monumental session for me because it was such a touching song, it was the first time I actually cried when I was playing".

In the late 1980s, Keltner toured with Ringo Starr's All-Starr Band. He also played drums on both albums released by the 1980s supergroup, the Traveling Wilburys, playing under the pseudonym Buster Sidebury.

He has specialized in R&B, and is said to have influenced Jeff Porcaro and Danny Seraphine of Chicago.

His drumming style typically melds deceptively simple drum patterns and a casual, loose feel with extraordinary precision. Demonstrations of his style and range can be found from "Jealous Guy" on John Lennon's Imagine, the hit single "Dream Weaver" by Gary Wright, "Josie" on Aja by Steely Dan, "Watching the River Flow" by Bob Dylan and The Thorns' debut, in which he sensitively accompanies Matthew Sweet, Pete Droge and Shawn Mullins. Keltner performed on many classic recordings by J. J. Cale, and often worked with bassist Tim Drummond.

In 1987, Keltner, along with guitarist Ry Cooder, and bassist Nick Lowe played on John Hiatt's Bring the Family. Four years later, the four musicians reunited as the band Little Village, recording an eponymously-named album.